125 lines
2.9 KiB
Markdown
125 lines
2.9 KiB
Markdown
# Interpreter Test Runner
|
|
|
|
Requires `pip install pyyaml`
|
|
|
|
Test SLS (UNIX):
|
|
`python3 test_runner.py ../../SLS_C/bin/sls tests.yaml`
|
|
`python3 test_runner.py "python3 -m ../../SLS_Python/sls" tests.yaml`
|
|
`python3 test_runner.py ../../SLS_Rust/target/debug/sls tests.yaml`
|
|
|
|
Test SLS (Windows):
|
|
`python test_runner.py ..\..\SLS_C\bin\sls.exe tests.yaml`
|
|
`python test_runner.py "python -m ..\..\SLS_Python\sls" tests.yaml`
|
|
`python test_runner.py ..\..\SLS_Rust\target\debug\sls.exe tests.yaml`
|
|
|
|
Python Examples:
|
|
`python test_runner.py python python_tests.yaml`
|
|
`python test_runner.py pypy python_tests.yaml`
|
|
|
|
```
|
|
>python test_runner.py python python_tests.yaml
|
|
✓ PASS: Simple print statement
|
|
|
|
✓ PASS: Basic arithmetic
|
|
|
|
✓ PASS: Variable assignment and usage
|
|
|
|
✗ FAIL: Syntax error detection
|
|
Reason: stderr mismatch
|
|
Expected:
|
|
SyntaxError: unterminated string literal (detected at line 1)
|
|
|
|
Got:
|
|
File "<stdin>", line 1
|
|
print("missing closing quote)
|
|
^
|
|
SyntaxError: unterminated string literal (detected at line 1)
|
|
|
|
|
|
✓ PASS: Import and use module
|
|
|
|
✓ PASS: List operations
|
|
|
|
✓ PASS: Execute Python file with args
|
|
|
|
✓ PASS: Division by zero error
|
|
|
|
✗ FAIL: Check Python version (using args instead of stdin)
|
|
Reason: stdout mismatch
|
|
Expected:
|
|
Python 3
|
|
Got:
|
|
Python 3.14.0
|
|
|
|
|
|
✓ PASS: Multi-line function definition
|
|
|
|
✓ PASS: Long running operation with custom timeout
|
|
|
|
============================================================
|
|
TEST SUMMARY
|
|
============================================================
|
|
Total tests: 11
|
|
Passed: 9
|
|
Failed: 2
|
|
|
|
Failed tests:
|
|
- Syntax error detection
|
|
- Check Python version (using args instead of stdin)
|
|
============================================================
|
|
```
|
|
|
|
```
|
|
>python test_runner.py pypy python_tests.yaml
|
|
✓ PASS: Simple print statement
|
|
|
|
✓ PASS: Basic arithmetic
|
|
|
|
✓ PASS: Variable assignment and usage
|
|
|
|
✗ FAIL: Syntax error detection
|
|
Reason: stderr mismatch
|
|
Expected:
|
|
SyntaxError: unterminated string literal (detected at line 1)
|
|
|
|
Got:
|
|
File "<stdin>", line 1
|
|
print("missing closing quote)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
SyntaxError: unterminated string literal (detected at line 1)
|
|
|
|
|
|
✓ PASS: Import and use module
|
|
|
|
✓ PASS: List operations
|
|
|
|
✓ PASS: Execute Python file with args
|
|
|
|
✓ PASS: Division by zero error
|
|
|
|
✗ FAIL: Check Python version (using args instead of stdin)
|
|
Reason: stdout mismatch
|
|
Expected:
|
|
Python 3
|
|
Got:
|
|
Python 3.10.13 (f1607341da97ff5a1e93430b6e8c4af0ad1aa019, Sep 28 2023, 05:42:24)
|
|
[PyPy 7.3.13 with MSC v.1929 64 bit (AMD64)]
|
|
|
|
|
|
✓ PASS: Multi-line function definition
|
|
|
|
✓ PASS: Long running operation with custom timeout
|
|
|
|
============================================================
|
|
TEST SUMMARY
|
|
============================================================
|
|
Total tests: 11
|
|
Passed: 9
|
|
Failed: 2
|
|
|
|
Failed tests:
|
|
- Syntax error detection
|
|
- Check Python version (using args instead of stdin)
|
|
============================================================
|
|
```
|