# Stack Language Specification v0.8 Reorganization Plan ## Overview This document provides a detailed plan for reorganizing the Stack Language Specification from v0.7 to v0.8. The reorganization focuses on: - **Audience**: Language learners/users (while retaining implementation details) - **Duplication**: Minimal (use cross-references) - **Section Length**: Shorter sections with more granularity - **Structure**: Clear progression from basics to advanced topics --- ## High-Level Structure ``` 1. Overview 2. Lexical Structure 3. Primitive Types 4. Basic Operations 5. Functions 6. Control Flow 7. Data Structures 8. Type System 9. Trait System 10. Generic Programming 11. Advanced Topics Appendices: A. Standard Library B. Complete Trait Reference C. Complete Operator Reference D. Grammar Summary E. Module System (Future) F. Memory Management (Future) G. Examples & Tutorials ``` --- ## Detailed Section Breakdown ### 1. Overview **Content from v0.7**: Section 1 "Overview" **Changes**: - Keep "Design Principles" subsection - Keep "Operators vs Functions" subsection - Add brief "Document Organization" subsection explaining structure - Add "Reading Guide" (e.g., "New users start at sections 1-7, implementers should also read sections 8-11") **Length**: ~2 pages --- ### 2. Lexical Structure **Content from v0.7**: Section 2 "Lexical Structure" **Changes**: - Keep all subsections: - 2.1 Comments - 2.2 Identifiers - 2.3 Literals (all sub-parts) - Move "Token Strings" subsection to new Section 5.5 (after function definition) - Move "Lambda Operator" subsection to new Section 5.6 - Keep cross-references clear (e.g., "Token Strings are explained in Section 5.5") **Length**: ~3 pages **Rationale**: Token strings and lambdas are more advanced topics related to functions, so they fit better there. --- ### 3. Primitive Types **Content from v0.7**: Section 3.1 "Primitive Types" **Changes**: - Extract from current Section 3 - Make this a standalone section - List all primitive types with brief descriptions - Add examples of each type - Add "Type Annotations" subsection (from current 2.3 Integer Literals, Floating Point Literals) - Remove `ptr` from main list, add note: "See Appendix F for pointer types (future feature)" **Length**: ~1 page **Rationale**: Users need to understand primitive types before learning about operations, but don't need the full type system yet. --- ### 4. Basic Operations **Content from v0.7**: Sections 5 & 6 **Changes**: - **4.1 Stack Operations** - Extract from current Section 5 - Start with usage examples (dup, drop, swap, over, rot) - Include stack effect notation explanation: `// ( a b -- b a )` - Add "Stack Inspection" subsection (depth, pick, roll) - Add cross-reference: "Stack operations implement the Stackable trait (see Section 9.1)" - Remove trait definition (move to Appendix B) - **4.2 Arithmetic Operators** - Extract from current Section 6.1 - Show usage examples only: `3 4 +`, `10 3 -`, etc. - Add cross-reference: "Arithmetic operators implement Addable, Multiplyable, Exponentiable, and Logarithmic traits (see Appendix B)" - Remove trait definitions - **4.3 Comparison Operators** - Extract from current Section 6.2 - Usage examples only - Cross-reference to Appendix B for trait details - **4.4 Logical Operators** - Extract from current Section 6.3 - Usage examples only - Cross-reference to Appendix B - **4.5 Bitwise Operators** - Extract from current Section 6.4 - Usage examples only - Cross-reference to Appendix B **Length**: ~3-4 pages **Rationale**: Users need to use operators before they understand the trait system. Implementation details go in the appendix. --- ### 5. Functions **Content from v0.7**: Section 7 **Changes**: - **5.1 What are Functions** - Brief explanation of functions vs operators - Cross-reference to Section 1 "Operators vs Functions" - **5.2 Defining Functions** - Extract from current 7.1 - Basic function definition syntax - Simple examples (no generics yet) - Stack effect signatures explained - **5.3 Calling Functions** - Simple examples of function invocation - Chaining functions - **5.4 Recursion** - Examples of recursive functions - Stack considerations - **5.5 Token Strings** - Moved from Section 2.3 - Explain what they are - List all operators that use TokenStrings - How functions use TokenStrings for bodies - **5.6 Lambda Functions** - Moved from Section 2.3 - Lambda operator - First-class functions - Examples **Length**: ~4 pages **Rationale**: Functions are the primary abstraction users write, so this should come early. --- ### 6. Control Flow **Content from v0.7**: Section 8 **Changes**: - **6.1 Conditionals** - Extract from 8.1 - Keep all examples - Emphasize TokenString usage - **6.2 While Loops** - Extract from 8.2 - Keep examples - **6.3 For Loops** - Extract from 8.2 - Keep examples - **6.4 Loop Control** - Extract from 8.3 - break and continue - **6.5 Pattern Matching** - Extract from 8.4 - Match expressions - Pattern syntax **Length**: ~3 pages **Rationale**: Control flow is fundamental and should come before data structures. --- ### 7. Data Structures **Content from v0.7**: Sections 9 & 10 **Changes**: - **7.1 Structs** - Extract from 9.1 & 9.2 - Definition syntax - Field access (get/set) - Examples - Note: "Structs can be generic (see Section 10)" - **7.2 Unions** - Extract from 9.3 - Definition syntax - Creating union values - Examples - Note about generics - **7.3 Enums** - Extract from 9.4 - Definition syntax - Usage examples - **7.4 Arrays** - Extract from Section 10 - Array literals - Basic operations (at, slice, length) - Array combinators (map, filter, reduce, each) - Array arithmetic - Array manipulation **Length**: ~4 pages **Rationale**: Users need to work with data before understanding the type system deeply. --- ### 8. Type System **Content from v0.7**: Section 3 **Changes**: - **8.1 Types vs Traits** - Extract from 3.2 - Clear explanation of distinction - Examples - **8.2 Type Inference** - New subsection - How the compiler infers types - When annotations are needed - Examples - **8.3 Type Tuples** - Extract from 2.3 - Stack effect signatures - How to read them - Variable arguments with Iterable - **8.4 Type Composition** - How types can contain other types - Nested structures - Arrays of types **Length**: ~2-3 pages **Rationale**: By now users have seen types in action, this section formalizes the concepts. --- ### 9. Trait System **Content from v0.7**: Section 4 **Changes**: - **9.1 What are Traits** - Extract from 4.1 - Behavioral contracts - Why traits exist - Brief overview of standard traits (with cross-ref to Appendix B) - **9.2 Defining Traits** - Extract from 4.2 - Syntax - Method signatures - Examples (simple traits only) - **9.3 Implementing Traits** - Extract from 4.3 - Implementation syntax - Implementation rules (1-4 from v0.7) - Simple examples - Note: Human-verified code blocks should be moved to Appendix G as examples - **9.4 Trait Inheritance** - Extract from 4.4 - Inheritance syntax - Multiple inheritance - Examples - **9.5 Using Traits in Functions** - Extract from 4.5 - Trait bounds - Generic functions with trait constraints - **9.6 Standard Traits Overview** - Brief list of all standard traits - One-line description of each - Cross-reference: "See Appendix B for complete trait definitions" - Remove all trait definitions from main body **Length**: ~4-5 pages **Rationale**: Traits are advanced but essential for understanding how operators work. --- ### 10. Generic Programming **Content from v0.7**: Section 3.3 & scattered examples **Changes**: - **10.1 Type Parameters** - Extract from 3.3 - Generic syntax `` - Type parameter constraints - **10.2 Generic Functions** - Extract from 7.2 - How to write generic functions - Trait constraints in practice - Examples - **10.3 Generic Data Structures** - Extract generic examples from Section 9 - Point, Option, Result - How to use generic types - **10.4 Generic Traits** - Extract from 3.3 "Generic Trait Syntax" - Container, Map, etc. - Generic trait inheritance - **10.5 Type Parameter Enforcement** - Note about current parse-time behavior - Cross-reference to Appendix F for future enhancements **Length**: ~3 pages **Rationale**: Generics are an advanced topic that builds on functions, data structures, and traits. --- ### 11. Advanced Topics **Content from v0.7**: Sections 11 & 12, scattered advanced topics **Changes**: - **11.1 Dynamic Code Evaluation** - Extract from Section 11 (Eval Operator) - Eval operator - Building code at runtime - Use cases and warnings - **11.2 Identifier Literals** - Extract from 2.2 - The `::` prefix - When to use identifier literals - Context-dependent rules - **11.3 Standard Library** - Extract from Section 12 - Brief overview of categories - I/O examples - String operations - Type conversion - Cross-reference: "See Appendix A for complete standard library reference" **Length**: ~2-3 pages **Rationale**: These are advanced features users encounter after mastering basics. --- ## Appendix Reorganization > **FOR AI AGENTS:** Do NOT complete full entries for the lists of items in the appendix. Just create a comprehensive list and an example of an entry. You will use up all usage and fill up your context otherwise if you do all the entries. ### Appendix A: Standard Library **Content Source**: Section 12 + scattered standard library mentions **Organization**: - Alphabetically organized by function/operator name - Each entry includes: - Name - Type signature - Trait requirements - Description - Examples - Related functions **Categories** (with alphabetical entries within): - Array Operations (at, concat, filter, length, map, reduce, reverse, slice, transpose, window) - I/O Operations (input, print, read, write) - String Operations (concat, join, length, split, substr) - Type Conversion (parse, to_i8, to_i16, ..., to_f32, to_f64, to_str) - Utility Operations (eval, lambda) **Format Example**: ``` ### at **Signature**: `(Selectable Size -- T)` **Trait**: Selectable **Description**: Access element at the given index. **Example**: [10 20 30] 1 at // Returns 20 **See Also**: slice, length ``` --- ### Appendix B: Complete Trait Reference **Content Source**: Section 4.1 + Appendix A from v0.7 **Organization**: - Alphabetically organized by trait name - Each entry includes: - Trait name - Generic parameters (if any) - Inheritance (if any) - Complete trait definition (code block) - Method descriptions - Standard implementations - Usage examples - Related traits **All Traits** (alphabetical): - Addable - ArrayOf - Bitwise - Comparable - Concatenable - Convertible - Equatable - Exponentiable - Identifier - Implementable - Iterable - Logarithmic - Logical - Multiplyable - Number - Orderable - Parseable - Selectable - Size - Sized - Sliceable - Stackable - String - Stringifiable **Format Example**: ``` ### Stackable **Generic Parameters**: None **Inherits**: None **Definition**: { (Self -- Self Self) dup: (Self -- ) drop: (Self Self -- Self Self) swap: (Self Self -- Self Self Self) over: (Self Self Self -- Self Self Self) rot: (Size -- Self) pick: (Size Size -- ) roll: (-- Size) depth: } ::Stackable trait **Methods**: #### dup: **Signature**: `(Self -- Self Self)` **Description**: Duplicate the top item on the stack. **Example**: `5 dup // => 5 5` #### drop: **Signature**: `(Self -- )` **Description**: Remove and discard the top item. **Example**: `5 10 drop // => 5` [... continue for all methods ...] **Standard Implementations**: All primitive types implement Stackable. **Related Traits**: None **See Also**: Section 4.1 (Stack Operations) ``` --- ### Appendix C: Complete Operator Reference **Content Source**: Section 6 from v0.7 + scattered operator mentions **Organization**: - Alphabetically organized by operator symbol/name - Each entry includes: - Operator symbol/name - Type signature - Trait requirements - Description - Examples - Related operators **All Operators** (alphabetical): - `!=` (not equal) - `%` (modulo) - `&` (this is bitand) - `*` (multiply) - `+` (add) - `-` (subtract) - `/` (divide) - `<` (less than) - `<=` (less or equal) - `==` (equal) - `>` (greater than) - `>=` (greater or equal) - `^` (exponentiate) - `|` (this is bitor) - `and` (logical and) - `at` (array access) - `bitand` (bitwise and) - `bitnot` (bitwise not) - `bitor` (bitwise or) - `bitxor` (bitwise xor) - `break` (exit loop) - `concat` (concatenate) - `continue` (skip iteration) - `depth` (stack depth) - `drop` (remove top) - `dup` (duplicate top) - `each` (iterate) - `enum` (define enum) - `eval` (execute code) - `filter` (filter array) - `fn` (define function) - `for` (for loop) - `get` (field access) - `if` (conditional) - `impl` (implement trait) - `inher` (trait inheritance) - `lambda` (create lambda) - `length` (container length) - `ln` (natural log) - `log` (log base 10) - `logb` (log with base) - `map` (transform array) - `match` (pattern match) - `not` (logical not) - `or` (logical or) - `over` (copy second) - `pick` (copy nth) - `reduce` (fold array) - `reverse` (reverse array) - `roll` (rotate n items) - `rot` (rotate three) - `set` (field assignment) - `shl` (shift left) - `shr` (shift right) - `slice` (extract slice) - `struct` (define struct) - `substr` (substring) - `swap` (swap two) - `trait` (define trait) - `transpose` (transpose array) - `union` (define union) - `while` (while loop) - `window` (sliding window) **Format Example**: ``` ### dup **Operator Type**: Stack Manipulation **Signature**: `(T -- T T)` **Trait**: Stackable **Description**: Duplicate the top item on the stack. **Example**: 5 dup // => 5 5 "hello" dup // => "hello" "hello" **See Also**: drop, swap, over **Section**: 4.1 (Stack Operations) ``` --- ### Appendix D: Grammar Summary **Content Source**: Appendix E from v0.7 **Changes**: - Remove duplication of struct, trait, fn, for, etc. grammars - Instead, reference the traits that define them - Focus on lexical grammar and expression grammar - Add note: "High-level constructs (struct, trait, fn, etc.) are defined by the Implementable trait operators. See Appendix B for their specifications." **Sections**: - D.1 Lexical Grammar (comments, identifiers, literals, token strings) - D.2 Expression Grammar (postfix expressions, operators precedence note) - D.3 Type Grammar (type tuples, generic types) - D.4 Construct Grammar (brief reference to Implementable trait operators) **Format**: ``` D.1 Lexical Grammar [Keep current lexical elements section] D.2 Expression Grammar expression ::= literal | identifier | identifier_literal | expression operator | ... D.3 Type Grammar [Keep current type expression section] D.4 Construct Grammar Note: Language constructs (fn, struct, trait, impl, enum, union, inher) are defined by operators in the Implementable trait. See Appendix B: Implementable for their complete specifications. ``` --- ### Appendix E: Module System (Future) **Content Source**: Appendix C from v0.7 **Changes**: - Keep all content as-is - This is already well-organized --- ### Appendix F: Memory Management (Future) **Content Source**: Appendix B from v0.7 **Changes**: - Keep all content as-is - Add pointer type details here (currently in Section 3.1) - This is already well-organized --- ### Appendix G: Examples & Tutorials **Content Source**: Appendix D from v0.7 ~~+ human-verified code blocks scattered throughout~~ **Organization**: - G.1 Tutorial: First Steps - G.2 Tutorial: Working with Traits - G.3 Tutorial: Generic Programming - G.4 Complete Examples (from current Appendix D) - ~~G.5 Verified Code Examples (human-reviewed blocks from throughout v0.7)~~ These were removed between v0.7 and v0.7.2. **Changes**: - Move all "Example - Actual Language Definitions" and "Example - Using Actual Definitions" blocks here - Organize by topic/difficulty - Add tutorial-style explanations - ~~Keep human verification notes with blocks~~ **New Tutorial Sections**: **G.1 Tutorial: First Steps** - Hello World - Basic arithmetic - Simple functions - Stack manipulation practice **G.2 Tutorial: Working with Traits** - Implementing a trait for a custom type - Using trait bounds - Understanding standard traits **G.3 Tutorial: Generic Programming** - Writing generic functions - Generic data structures - Trait constraints in generics **G.4 Complete Examples** (keep from v0.7 Appendix D): - D.1 Trait Implementation Example → G.4.1 - D.2 Trait Inheritance Example → G.4.2 - D.3 Logarithm Usage → G.4.3 - D.4 Factorial → G.4.4 - D.5 FizzBuzz → G.4.5 - D.6 Using Roll → G.4.6 - D.7 Array Processing → G.4.7 - D.8 Identifier Literals in Practice → G.4.8 ~~**G.5 Verified Code Examples**:~~ - ~~All human-verified code blocks from v0.7~~ - ~~Organized by topic~~ - ~~Include verification notes~~ These were removed between v0.7 and v0.7.2. --- ## Content Movement Matrix | v0.7 Location | v0.8 Location | Notes | |---------------|---------------|-------| | Section 1 | Section 1 | Keep, add reading guide | | Section 2.1-2.2 | Section 2.1-2.2 | Keep as-is | | Section 2.3 (most) | Section 2.3 | Keep literals | | Section 2.3 (Token Strings) | Section 5.5 | Move to Functions | | Section 2.3 (Lambda) | Section 5.6 | Move to Functions | | Section 3.1 | Section 3 | Expand to full section | | Section 3.2 | Section 8.1 | Move to Type System | | Section 3.3 (basics) | Section 10.1-10.2 | Expand into Generic Programming | | Section 3.3 (traits) | Section 10.4 | Generic traits section | | Section 4.1 (trait defs) | Appendix B | Move all trait definitions | | Section 4.1 (overview) | Section 9.6 | Brief overview only | | Section 4.2 | Section 9.2 | Keep | | Section 4.3 | Section 9.3 | Keep, move examples to Appendix G | | Section 4.4 | Section 9.4 | Keep | | Section 4.5 | Section 9.5 | Keep | | Section 5.1 | Section 4.1 | Move to Basic Operations | | Section 5.2 | Section 4.1 | Merge with 5.1 | | Section 6.1 | Section 4.2 | Move to Basic Operations | | Section 6.2 | Section 4.3 | Move to Basic Operations | | Section 6.3 | Section 4.4 | Move to Basic Operations | | Section 6.4 | Section 4.5 | Move to Basic Operations | | Section 7.1 | Section 5.2 | Keep in Functions | | Section 7.2 | Section 5.2 + 10.2 | Split generic parts | | Section 8.1 | Section 6.1 | Keep | | Section 8.2 | Section 6.2-6.3 | Split while/for | | Section 8.3 | Section 6.4 | Keep | | Section 8.4 | Section 6.5 | Keep | | Section 9.1 | Section 7.1 | Keep | | Section 9.2 | Section 7.1 | Merge with 9.1 | | Section 9.3 | Section 7.2 | Keep | | Section 9.4 | Section 7.3 | Keep | | Section 10 | Section 7.4 | Move to Data Structures | | Section 11 | Section 11.1 | Move to Advanced Topics | | Section 12 | Section 11.3 + Appendix A | Brief overview + full reference | | Appendix A | Appendix B | Rename to Trait Reference | | Appendix B | Appendix F | Keep | | Appendix C | Appendix E | Keep | | Appendix D | Appendix G.4 | Move to Examples | | Appendix E | Appendix D | Simplify | | New | Section 8.2 | Type Inference | | New | Section 8.4 | Type Composition | | New | Section 11.2 | Identifier Literals (advanced) | | New | Appendix A | Standard Library Reference | | New | Appendix C | Operator Reference | | New | Appendix G.1-G.3 | Tutorials | | ~~New~~ | ~~Appendix G.5~~ | ~~Verified Examples~~ These were removed between v0.7 and v0.7.2. | --- ## Cross-Reference Strategy To maintain minimal duplication, use consistent cross-references: **Format**: "See Section X.Y for details" or "See Appendix X for complete specification" **Key Cross-References to Add**: 1. **Section 4 (Basic Operations)** → Appendix B for trait definitions 2. **Section 5 (Functions)** → Section 10 for generic functions 3. **Section 7 (Data Structures)** → Section 10 for generic types 4. **Section 9 (Trait System)** → Appendix B for complete trait specs 5. **Section 11.3 (Standard Library)** → Appendix A for full reference 6. **All sections** → Appendix G for examples **Cross-Reference Template**: ``` > **Implementation Details**: See Appendix B (Stackable Trait) for the complete trait definition and implementation requirements. > **Examples**: See Appendix G.4.6 for advanced stack manipulation examples. > **Related**: This concept is explained further in Section 10.2 (Generic Functions). ``` --- ## Style and Formatting Guidelines ### Code Block Annotations ~~Preserve all human verification notes:~~ ``` > **For AI Reviewers @ 0.5:** The following block has been human verified... ``` These were removed between v0.7 and v0.7.2. Remove "Example - Actual Language Definitions" labels - replace with clearer context: - "Defining a trait:" (instead of "Example - Actual Language Definitions:") - "Using the operator:" (instead of "Example - Using Actual Definitions:") - "Complete implementation:" (for full examples) Where these weren't fixed between v0.6 and v0.7. Be careful not to confuse examples (including examples using *language definitions*/*appendix reference segments*; and examples that use code that isn't actually part of any build-in or standard library code) and the actual language definitions that are in the appendix sections A, B, and C. ### Section Headers Use consistent header style: - **Section Number. Section Name** (e.g., "4. Basic Operations") - **Subsection Number.Subsection Name** (e.g., "4.1 Stack Operations") - **Sub-subsection Name** (no numbering for third level) ### Consistency Checks 1. **Trait names**: Always use :: prefix in text: "the `::Stackable` trait" 2. **Operators**: Use backticks: "the `dup` operator" 3. **Functions**: Use backticks: "the `factorial` function" 4. **Types**: Use backticks: "`i32`", "`Point`" 5. **Stack effects**: Always use format: `// ( before -- after )` --- ## Version Control Update version header: ```markdown **Version**: 0.8 **Status**: Draft Specification **Changes**: [Keep all previous version history] - 0.8 (AI) 1. **Document reorganization** - Restructured for user-focused learning 2. **Section consolidation** - Moved implementation details to appendices 3. **Minimal duplication** - Added cross-references instead of repeating content 4. **Shorter sections** - Split large sections into focused subsections 5. **New appendices** - Added Standard Library and Operator references 6. **Tutorial content** - Added beginner tutorials to Appendix G 7. **Trait reference** - Consolidated all trait definitions in Appendix B 8. **Grammar simplification** - Referenced Implementable trait instead of repeating ``` --- ## Implementation Checklist For the AI agent implementing v0.8: - [ ] Update version number and changelog - [ ] Create all 11 main sections with proper numbering - [ ] Move content according to Content Movement Matrix - [ ] Create Appendix A (Standard Library) - alphabetical - [ ] Create Appendix B (Complete Trait Reference) - alphabetical - [ ] Create Appendix C (Complete Operator Reference) - alphabetical - [ ] Simplify Appendix D (Grammar Summary) - [ ] Keep Appendix E (Module System) as-is - [ ] Keep Appendix F (Memory Management) as-is, add pointer types - [ ] Create Appendix G (Examples & Tutorials) - [ ] Add cross-references throughout - [ ] Remove duplicate content (trait definitions from main sections) - [ ] Update all internal section references - [ ] ~~Preserve all human-verified code blocks with notes~~ These were removed between v0.7 and v0.7.2. - [ ] Remove "Example - Actual Language Definitions" labels - [ ] Add reading guide to Section 1 - [ ] Verify all cross-references are valid - [ ] Check consistency of terminology - [ ] Ensure no broken references to moved content --- ## Success Criteria The reorganization is successful if: 1. **User-focused**: Sections 1-7 can be read linearly by a beginner 2. **No duplication**: Each concept explained once, cross-referenced elsewhere 3. **Implementation preserved**: All trait definitions in Appendix B 4. **Easy lookup**: Appendices A, B, C are alphabetically searchable 5. **Shorter sections**: No section exceeds 5 pages 6. **Clear progression**: Basic → Intermediate → Advanced → Reference 7. **All content preserved**: Nothing from v0.7 is lost 8. **Cross-references work**: All "See Section X" references are valid --- ## Notes for Implementation Agent 1. ~~**Preserve all human-verified blocks** - These have "For AI Reviewers @ 0.5" notes~~ These were removed between v0.7 and v0.7.2. 2. **Don't paraphrase trait definitions** - Copy them exactly to Appendix B 3. **Keep code examples** - Don't summarize, show actual code 4. **Maintain postfix consistency** - All examples should be valid postfix notation 5. **Cross-reference liberally** - Better to over-reference than under-reference 6. **Check operator list completeness** - Appendix C must have ALL operators 7. **Alphabetize carefully** - Appendices A, B, C are reference material 8. ~~**Preserve TODO comments** - Keep the one remaining human TODO~~ This was removed between v0.7 and v0.7.2. 9. **Grammar simplification** - Don't remove grammar, just avoid duplication 10. **Tutorial quality** - Appendix G.1-G.3 should be beginner-friendly