YREA-SLS/SLS_Tests/interpreter_tester/README.md

Interpreter Test Runner

Requires pip install pyyaml

Test SLS (UNIX):
python3 test_runner.py ../../SLS_C/bin/sls tests.yaml
python3 test_runner.py "python3 -m ../../SLS_Python/sls" tests.yaml
python3 test_runner.py ../../SLS_Rust/target/debug/sls tests.yaml

Test SLS (Windows):
python test_runner.py ..\..\SLS_C\bin\sls.exe tests.yaml
python test_runner.py "python -m ..\..\SLS_Python\sls" tests.yaml
python test_runner.py ..\..\SLS_Rust\target\debug\sls.exe tests.yaml

Python Examples:
python test_runner.py python python_tests.yaml
python test_runner.py pypy python_tests.yaml

>python test_runner.py python python_tests.yaml
✓ PASS: Simple print statement

✓ PASS: Basic arithmetic

✓ PASS: Variable assignment and usage

✗ FAIL: Syntax error detection
  Reason: stderr mismatch
Expected:
SyntaxError: unterminated string literal (detected at line 1)

Got:
  File "<stdin>", line 1
    print("missing closing quote)
          ^
SyntaxError: unterminated string literal (detected at line 1)


✓ PASS: Import and use module

✓ PASS: List operations

✓ PASS: Execute Python file with args

✓ PASS: Division by zero error

✗ FAIL: Check Python version (using args instead of stdin)
  Reason: stdout mismatch
Expected:
Python 3
Got:
Python 3.14.0


✓ PASS: Multi-line function definition

✓ PASS: Long running operation with custom timeout

============================================================
TEST SUMMARY
============================================================
Total tests: 11
Passed: 9
Failed: 2

Failed tests:
  - Syntax error detection
  - Check Python version (using args instead of stdin)
============================================================

>python test_runner.py pypy python_tests.yaml
✓ PASS: Simple print statement

✓ PASS: Basic arithmetic

✓ PASS: Variable assignment and usage

✗ FAIL: Syntax error detection
  Reason: stderr mismatch
Expected:
SyntaxError: unterminated string literal (detected at line 1)

Got:
  File "<stdin>", line 1
    print("missing closing quote)
          ^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: unterminated string literal (detected at line 1)


✓ PASS: Import and use module

✓ PASS: List operations

✓ PASS: Execute Python file with args

✓ PASS: Division by zero error

✗ FAIL: Check Python version (using args instead of stdin)
  Reason: stdout mismatch
Expected:
Python 3
Got:
Python 3.10.13 (f1607341da97ff5a1e93430b6e8c4af0ad1aa019, Sep 28 2023, 05:42:24)
[PyPy 7.3.13 with MSC v.1929 64 bit (AMD64)]


✓ PASS: Multi-line function definition

✓ PASS: Long running operation with custom timeout

============================================================
TEST SUMMARY
============================================================
Total tests: 11
Passed: 9
Failed: 2

Failed tests:
  - Syntax error detection
  - Check Python version (using args instead of stdin)
============================================================