updated readme, updated tests

fddl.kdev4

@@ -1,4 +0,0 @@
-[Project]
-CreatedFrom=
-Manager=KDevGenericManager
-Name=fddl

readme.md

@@ -18,36 +18,66 @@ Well, Ferris, the mascot of Rust, is a crab. There's a type of crab that exists
 
 ## Current Features
 
-- **Custom Syntax**: fddl introduces unique operators and keywords to make programming more intuitive and fun.
-- **Lexer**: A working lexer that tokenizes fddl scripts into understandable pieces of the language.
-- **Tilde Operator**: Includes a custom `~` and `~=` operator for creative syntax possibilities.
-  
+-   **Custom Syntax**: `fddl` introduces unique operators and keywords.
+-   **Lexer**:
+    -   Tokenizes `fddl` scripts, handling various operators, literals (numbers, strings, booleans, nil), and keywords.
+    -   Supports single-line comments (`//`, `#`) and multi-line block comments (`/* ... */`).
+    -   Keywords include `let`, `func`, `return`, `if`, `else`, `while`, `for`, `print`, `true`, `false`, `nil`, `and`, `or`, `some`, `not`, and more.
+-   **Parser**:
+    -   Builds an Abstract Syntax Tree (AST) from the token stream.
+    -   **Comprehensive Expression Parsing**:
+        -   Literals: Numbers, strings, booleans, `nil`.
+        -   Unary Operations: `-` (negation), `~` (almost), `some`, `not`.
+        -   Binary Operations: Handles arithmetic (`+`, `-`, `*`, `/`, `%`), comparisons (`<`, `<=`, `>`, `>=`), equality (`==`, `!=`), and logical (`and`, `or`) operators with correct precedence and associativity.
+        -   Grouping: Parenthesized expressions `(...)`.
+        -   Function Calls: Parses `function_name(arg1, arg2, ...)` with complex expressions as arguments.
+    -   **Statement Parsing**:
+        -   `print` statements.
+        -   `let` variable declaration statements.
+        -   Assignment statements (`identifier = expression;`).
+        -   Block statements (`{ ... }`) for grouping multiple statements.
+        -   `if-else if-else` control flow statements with block bodies.
+        -   `while` loop statements with block bodies.
+        -   `for` loop statements (C-style: `for (initializer; condition; increment) { body }`, including `let` initializers) with block bodies.
+        -   `func` function declaration statements (name, parameters, block body).
+        -   `return` statements (with optional expression).
+        -   Expression statements.
+    -   Skips comment tokens during parsing.
+-   **Basic Interpreter (Ongoing)**:
+    -   Tree-walking interpreter for executing ASTs.
+    -   Evaluates literal expressions (numbers, strings, booleans, nil).
+    -   Evaluates unary minus (`-`) expressions.
+    -   Evaluates binary arithmetic expressions (`+`, `-`, `*`, `/`, `%`) including division-by-zero checks.
+    -   Evaluates grouping expressions `()`.
+    -   Executes `PrintStatement` and `ExpressionStatement`.
+-   **Tilde Operator**: Includes a custom `~` (unary "Almost") and `~=` (binary "AlmostEqual" - lexed, parser TBD) operator.
+
 ---
 
+
 ## Getting Started
 
-To start experimenting with fddl, you can run it in two ways:
+To get started with `fddl`, you'll first need to have the Rust programming language and its package manager, Cargo, installed on your system. If you don't have them yet, you can find installation instructions on the official Rust website: [rust-lang.org](https://www.rust-lang.org/tools/install).
 
-### Run the REPL (Interactive Mode)
-```sh
-cargo run
-```
+Once Rust and Cargo are set up:
 
-### Parse a fddl Script
-```sh
-cargo run path/to/script.fddl
-```
+1.  **Clone the Repository** (if you haven't already):
+    ```sh
+    # Replace <repository_url> with the actual URL of your fddl repository
+    git clone <repository_url>
+    cd fddl
+    ```
 
-## Running the Project
+2.  **Build the Project**:
+    You can build the project using Cargo:
+    ```sh
+    cargo build
+    ```
+    This will compile `fddl` and place the executable in the `target/debug/` directory.
 
-Make sure your project compiles and the tests pass:
+After building, you can run `fddl` using `cargo run` (which compiles and then runs).
 
-```bash
-cargo build
-cargo test
-```
-
----
+Mind you, there isn't much there currently. The REPL only returns minimal information currently.
 
 ## Examples
 
@@ -81,36 +111,41 @@ print(`The square of $number is ${math.square($number)}`);
 
 ## Roadmap and Next Steps
 
-fddl is very much a work in progress, with lots of planned improvements and additions. Here's a breakdown of what’s done and what’s coming up:
+`fddl` is very much a work in progress.
 
-- **Lexer**: 
-  - [x] Built and tested for basic syntax and operators.
-  - [x] Supports single-line and documentation comments.
-  - [ ] Add support for more complex syntax and features.
-
-- **Parser**: 
-  - [x] Parser parsing tilde and minus successfully
-  - [x] Parser parsing function calls
-  - [x] Parser parsing the rest of the operators (mostly complete)
-  - [ ] Working on building out functions to parse simple functionality in the language (if, while, for), and to read their expressions and values
-  - [ ] Implement parsing for ~~function calls,~~ expressions, checks, literally everything.
-  - [ ] L & R Values
-
-- **Compiler**: 
-  - [ ] Currently a placeholder. Implement the compiler to compile parsed code.
-
-- **Comments**:
-  - [x] Added support for single-line and documentation comments.
-  - [ ] Implement multi-line comments.
-  - [ ] Implement document building comments.
-
-- **Error Handling**: 
-  - [ ] Replace `stderr` with a more robust error handling mechanism.
-  - [ ] Errors as values
-
-- **Testing**: 
-  - [x] Added initial `lexer` tests. 
-  - [ ] Expand tests to cover more syntax and edge cases.
+-   **Lexer**:
+    -   [x] Core functionality built and tested.
+    -   [x] Supports single-line (`//`, `#`) and multi-line block comments (`/* ... */`).
+    -   [ ] Consider advanced features like escape sequences in strings more thoroughly.
+-   **Parser**:
+    -   [x] Comprehensive expression parsing (primary, unary (`-`, `~`, `some`, `not`), binary with precedence (arithmetic, comparison, equality, logical), grouping, function calls).
+    -   [x] Core statement parsing (`print`, `let`, assignment, `if/else`, `while`, `for` (with `let` initializers), blocks (`{...}`), `func` declaration, `return`).
+    -   [ ] L & R Values: Formalize for assignment and other contexts (more a semantic/compiler concern).
+    -   [ ] Potentially parse types for type checking later if `fddl` becomes statically typed.
+-   **Interpreter (Current Focus)**:
+    -   [x] Basic tree-walking framework for AST evaluation.
+    -   [x] Evaluation of literal expressions (numbers, strings, booleans, `nil`).
+    -   [x] Evaluation of unary minus (`-`) expressions.
+    -   [x] Evaluation of binary arithmetic expressions (`+`, `-`, `*`, `/`, `%`) including division-by-zero checks.
+    -   [x] Evaluation of grouping expressions `()`.
+    -   [x] Execution of `PrintStatement` and `ExpressionStatement`.
+    -   [ ] Implement evaluation for remaining unary operators (`not`, `some`, `~`).
+    -   [ ] Implement evaluation for binary comparison (`<`, `<=`, `>`, `>=`), equality (`==`, `!=`), and logical (`and`, `or`) operators.
+    -   [ ] **Environment for Variables**: Implement variable declaration (`let`), assignment (`=`), and lookup (`identifier`).
+    -   [ ] **Control Flow Execution**: `if/else`, `while`, `for`.
+    -   [ ] **Function Execution**: Handling function calls, parameter passing, environments/scopes, and `return` statements.
+-   **Compiler**:
+    -   [ ] Currently a placeholder. Future goal: Implement a compiler (e.g., to bytecode or another target).
+-   **Error Handling**:
+    -   [ ] Improve error reporting with more precise location information (line/column) consistently across lexer, parser, and interpreter.
+    -   [ ] Consider "errors as values" as a language feature (currently a back-burner idea).
+-   **Testing**:
+    -   [x] Added initial `lexer` tests.
+    -   [ ] Expand tests to cover parser AST output more systematically.
+    -   [ ] Add tests for interpreter behavior as features are implemented.
+-   **REPL/Tooling**:
+    -   [ ] Enhance REPL for multi-line input.
+    -   [ ] Ensure robust file input processing in `main.rs`.
 
 ---
 

tests/lexer_tests.rs

@@ -134,4 +134,197 @@ fn test_tilde_operator() {
             Token::EOF
         ]
     );
+}
+
+#[test]
+fn test_modulus_operator() {
+    let source = String::from("10 % 3;");
+    let mut lexer = Lexer::new(source.clone()); // Assuming Lexer::new takes String or &str
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Number(10.0),
+            Token::Percent,
+            Token::Number(3.0),
+            Token::Semicolon,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_nil_keyword() {
+    let source = String::from("let a = nil;");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Let,
+            Token::Identifier("a".to_string()),
+            Token::Equal,
+            Token::Nil,
+            Token::Semicolon,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_some_keyword() {
+    let source = String::from("if some value { }"); // Assuming 'value' is an identifier
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::If,
+            Token::Some,
+            Token::Identifier("value".to_string()),
+            Token::LeftBrace,
+            Token::RightBrace,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_not_keyword() {
+    let source = String::from("not true");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Not,
+            Token::True,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_block_comments() {
+    let source = String::from("/* this is a block comment */ let x = 1; /* another one */");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    // Assuming your lexer includes comments as Token::Comment in the stream
+    // And that consume_block_comment() correctly extracts the inner text.
+    // Adjust the expected comment string based on your lexer's behavior (e.g., if it includes spaces).
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Comment(" this is a block comment ".to_string()), // Content might vary based on trimming
+            Token::Let,
+            Token::Identifier("x".to_string()),
+            Token::Equal,
+            Token::Number(1.0),
+            Token::Semicolon,
+            Token::Comment(" another one ".to_string()), // Content might vary
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_multiline_block_comment() {
+    let source = String::from("let y; /* hello\n world \n spanned */ print y;");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Let,
+            Token::Identifier("y".to_string()),
+            Token::Semicolon,
+            Token::Comment(" hello\n world \n spanned ".to_string()), // Verify exact content including newlines
+            Token::Print,
+            Token::Identifier("y".to_string()),
+            Token::Semicolon,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_empty_block_comment() {
+    let source = String::from("/**/print 1;");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Comment("".to_string()), // Empty content
+            Token::Print,
+            Token::Number(1.0),
+            Token::Semicolon,
+            Token::EOF
+        ]
+    );
+}
+
+#[test]
+fn test_block_comment_at_eof() {
+    let source = String::from("let z = 10; /* block comment at eof"); // Unterminated
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    // This depends on how your lexer handles unterminated block comments.
+    // The consume_block_comment function we discussed would try to form a comment
+    // with what it has or an error.
+    // If it forms a comment with the partial content:
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Let,
+            Token::Identifier("z".to_string()),
+            Token::Equal,
+            Token::Number(10.0),
+            Token::Semicolon,
+            Token::Comment(" block comment at eof".to_string()), // Or whatever your lexer emits for unterminated
+            Token::EOF                                          // EOF is always last
+        ]
+        // Alternative if it produces an error token for unterminated comments:
+        // vec![
+        //     ...,
+        //     Token::Semicolon,
+        //     Token::Error("Unterminated block comment".to_string()), // Or similar
+        //     Token::EOF
+        // ]
+    );
+}
+
+#[test]
+fn test_combined_new_tokens() {
+    let source = String::from("let result = some value % 2 == 0 and not nil;");
+    let mut lexer = Lexer::new(source.clone());
+    let tokens = lexer.scan_tokens();
+
+    assert_eq!(
+        tokens,
+        vec![
+            Token::Let,
+            Token::Identifier("result".to_string()),
+            Token::Equal,
+            Token::Some,
+            Token::Identifier("value".to_string()),
+            Token::Percent,
+            Token::Number(2.0),
+            Token::EqualEqual,
+            Token::Number(0.0),
+            Token::And,
+            Token::Not,
+            Token::Nil,
+            Token::Semicolon,
+            Token::EOF
+        ]
+    );
 }