$ cargo add clap --features deriveIf you are seeing this, that means your Extraterrestrial Vegetation Evaluator, or EVE probe, has returned from Earth.
We are almost at the end of the first cycle in the creation of our interpreter. After having learned how to scan and parse tokens, building a syntax tree that represents the code we wrote, it's time to evaluate it. For the time being, we are doing it only with expressions, so our interpreter is currently nothing more than a slightly advanced calculator. Nevertheless, it's important to see the full cycle and to create something that actually works end-to-end.
When it comes to evaluating expressions, the main problem is type correctness. Evaluating an operation like 5 * 4 (or, if you want, (* 5 4)) is simple, but problems start when the user inputs "astring" * 4. Will our programming language support * for strings? And if so, what would the outcome of that operation be? And what about +, -, /, and so on?
As always, the business logic has to cover some happy paths, while error management has to cover all remaining conditions, which are usually many more. Rust is not an object-oriented programming language, so techniques shown by Robert Nystrom in his book must be adapted or replaced by other strategies. In particular, we will not implement the visitor pattern, but I highly recommend reading the discussion about it as it is one of the best takes on one of the most misunderstood design patterns.
In this chapter, we are going to add a new command to the CLI of our interpreter: evaluate. The current code in src/main.rs has been created on top of the basic template provided by CodeCrafters in the first stage of the whole challenge, and it can be radically improved. The first improvement is to use a CLI management library, and in this case we are going to use clap. The second improvement is to reduce code duplication across stages: when we execute the parse command, the code goes through the stages of tokenize almost verbatim, so it is worth trying to find a way to isolate and reuse that code.
The changes to the file are however pretty radical, so instead of showing a complicated diff I will as much as possible show the new code directly with an explanation of the most important points.
First of all, we need to install clap, following the official instructions:
$ cargo add clap --features deriveThis will automatically change the files Cargo.lock and Cargo.toml and make clap available. Still following the official documentation, we can define a parser and commands:
src/main.rs use clap::{Parser as CParser, Subcommand as CSubcommand};
#[derive(CSubcommand)]
enum Commands {}
#[derive(CParser)]
#[command(version, about, long_about = None)]
struct Args {
#[command(subcommand)]
command: Commands,
}
fn main() {}Please note the aliases CParser and CSubcommand used to avoid clashes with the naming of structures that belong to our interpreter. For the time being we don't need to define any variant of CSubcommand.
The first command we want to define is the tokenisation of the input:
src/main.rs use std::fs;
use std::path::PathBuf;
use crate::token::TokenResult;
use crate::tokenizer::tokenize;
use clap::{Parser as CParser, Subcommand as CSubcommand};
mod token;
mod tokenizer;
#[derive(CSubcommand)]
enum Commands {}
enum Commands {
Tokenize { filename: PathBuf },
}
#[derive(CParser)]
#[command(version, about, long_about = None)]
struct Args {
#[command(subcommand)]
command: Commands,
}
fn tokenize_source(filename: &PathBuf) -> Vec<TokenResult> {
// Open the input file and read the content.
let file_contents = fs::read_to_string(filename).unwrap_or_else(|_| {
eprintln!("Failed to read file {}", filename.display());
String::new()
});
// Process the file and extract the token results.
let results: Vec<TokenResult> = tokenize(file_contents.chars().peekable());
results
}
fn main() {}
fn main() {
let args = Args::parse();
match args.command {
Commands::Tokenize { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
}
}
}The function tokenize_source comes directly from the previous version of the code, but has been isolated in a function so that we can later call it both in the command tokenize and in the command parse.
We are missing the error management part of the command tokenize, but it is worth implementing part of the command parse so that we can see where they overlap and develop a proper solution to deal with tokenisation errors.
src/main.rs #[derive(CSubcommand)]
enum Commands {
Parse { filename: PathBuf },
Tokenize { filename: PathBuf },
}
fn main() {
let args = Args::parse();
match args.command {
Commands::Parse { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
// COLLECT TOKENS AND REPORT ERRORS
}
Commands::Tokenize { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
// ANALYSE TOKENS AND REPORT ERRORS
}
}
}While the two commands perform the same operation to tokenise the source, the command tokenize ends with a report of invalid tokens, while parse needs to collect the valid ones and process them. We can therefore create a function that loops over the Vec<TokenResult> returned by tokenize_source, and that performs customisable operations on valid and invalid tokens:
src/main.rs use std::fs;
use std::path::PathBuf;
use std::process;
use crate::token::TokenResult;
use crate::token::{Token, TokenError, TokenResult};
use crate::tokenizer::tokenize;
fn tokenize_process_results(
results: Vec<TokenResult>,
ok: impl Fn(Token),
err: impl Fn(TokenError),
) {
// Initialise the error flag.
let mut errors: bool = false;
// Iterate through the results and call
// the provided functions according
// to their variant.
// Record if there are errors.
for result in results {
match result {
Ok(token) => ok(token),
Err(e) => {
err(e);
errors = true
}
}
}
// In case there were errors
// exit with a specific code.
if errors {
process::exit(65);
}
}The function receives two other functions, defined through impl Fn(Token) and impl Fn(TokenError) [docs] that will provide the logic for valid and invalid tokens. We can call the function tokenize_process_results in tokenize, performing different operations through closures:
src/main.rs fn main() {
let args = Args::parse();
match args.command {
Commands::Parse { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
}
Commands::Tokenize { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
let results = tokenize_source(&filename);
// Process all results, print
// tokens on stdout and error
// on the stderr.
tokenize_process_results(
results,
|token| println!("{}", token),
|error| eprintln!("{}", error),
);
}
}
}This version of the code passes all the CodeCrafters tests related to scanning.
We can now call the function tokenize_process_results in parse, passing a closure that collects the valid tokens into a vector:
src/main.rs fn main() {
match args.command {
Commands::Parse { filename } => {
// Process the source and
// extract Tokens or errors.
tokenize_source(&filename);
let results = tokenize_source(&filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
}Unfortunately, this doesn't work straight away. The closure |token| tokens.push(token) passed to the function to process valid tokens borrows tokens mutably, and thus it doesn't match the definition of the trait Fn, which is a "call operator that takes an immutable receiver". The solution is to change the definition of the parameter ok in tokenize_process_results:
src/main.rs fn tokenize_process_results(
results: Vec<TokenResult>,
ok: impl Fn(Token),
mut ok: impl FnMut(Token),
err: impl Fn(TokenError),
) {
...
}The real reason why the function must be FnMut instead of Fn is not trivial, and for now it suffices to connect it with the fact that it mutates a captured state. See the aside for an in-depth explanation of this subtle problem.
The real reason why we need FnMut lies in the fact that we are passing a closure, and that is implied by the definition of the parameter ok of tokenize_process_results. Let's walk through the problem step by step.
A closure is a function that captures the environment, so the function itself holds data. This means that if a closure changes the captured state the closure itself must be mutable. This is the reason why we need FnMut: the definition mentions call operator that takes a mutable receiver and here the receiver is the closure itself. The type definition is obviously not enough: once we declare that we accept a closure that mutates its environment we need to actually pass it in a mutable fashion, hence the mut ok.
If we wanted to pass a normal function defined with fn we wouldn't need the FnMut, as a normal function doesn't need to be mutable as it doesn't capture the environment. However, in that case we would have to use a function that borrows a mutable vector, and the definition of tokenize_process_results would have to reflect that. For example:
src/main.rs fn tokenize_process_results(
results: Vec<TokenResult>,
ok: impl Fn(&mut Vec<Token>, Token),
err: impl Fn(TokenError),
) {
...
}The final part of the command parse is straightforward:
src/main.rs use crate::parser::Parser;
use crate::token::{Token, TokenError, TokenResult};
use crate::tokenizer::tokenize;
use clap::{Parser as CParser, Subcommand as CSubcommand};
mod parser;
mod token;
mod tokenizer;
fn main() {
match args.command {
Commands::Parse { filename } => {
// Process the source and
// extract Tokens or errors.
let results = tokenize_source(&filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
let mut p = Parser::new(tokens);
match p.expression() {
Ok(v) => println!("{}", v),
Err(e) => {
eprintln!("{}", e);
process::exit(65);
}
}
}This version of the code passes all the CodeCrafters tests related to parsing.
Now that the main function is in better shape we can focus our attention on the main topic of this chapter, which is expression evaluation. The problem is less complicated than it looks, at least from the point of view of the business logic. We need a function that transforms an Expression into a value that can be printed on screen.
Before we do that, it is worth isolating the data structures created for Expression. They have been created in src/parser.rs but the file is becoming a little bit too crowded.
To isolate the structures related to Expression we can just move the code to a new file src/expression.rs and adjust module definitions and imports. There are two tests in src/parser.rs that are checking the rendering of Literal::Number, so we need to move those as well.
src/expression.rs use std::fmt;
use crate::token::Token;
#[derive(Debug, PartialEq)]
pub enum Literal {
Boolean(bool),
Nil,
Number(f64),
String(String),
}
impl fmt::Display for Literal {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Self::Boolean(b) => write!(f, "{b}"),
Self::Nil => f.write_str("nil"),
Self::Number(n) if n.fract() == 0.0 => write!(f, "{n:.1}"),
Self::Number(n) => write!(f, "{n}"),
Self::String(s) => f.write_str(s),
}
}
}
#[derive(Debug, PartialEq)]
pub enum Expression {
Binary {
left: Box<Self>,
operator: Token,
right: Box<Self>,
},
Group(Box<Self>),
Literal(Literal),
Unary {
operator: Token,
right: Box<Self>,
},
}
impl fmt::Display for Expression {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let name: String = match self {
Self::Binary {
left,
operator,
right,
} => format!("({} {} {})", operator.lexeme, left, right),
Self::Group(p) => format!("(group {})", p),
Self::Literal(s) => s.to_string(),
Self::Unary { operator, right } => format!("({} {})", operator.lexeme, right),
};
write!(f, "{}", name)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
// Test that integer numbers are displayed
// correctly with a decimal part of `.0`.
fn test_literal_display_integer_number() {
assert_eq!(Literal::Number(42.0).to_string(), "42.0");
}
#[test]
// Test that float numbers are displayed
// correctly with a decimal part.
fn test_literal_display_decimal_number() {
assert_eq!(Literal::Number(42.5).to_string(), "42.5");
}The code comes directly from src/parser.rs. It must be removed and replaced with the correct import:
src/parser.rs use std::fmt;
use crate::expression::{Expression, Literal};
use crate::token::{Keyword, Token, TokenLiteral, TokenType};
#[derive(Debug, PartialEq)]
pub enum Literal {
...
}
impl fmt::Display for Literal {
...
}
#[derive(Debug, PartialEq)]
pub enum Expression {
...
}
impl fmt::Display for Expression {
...
}
mod tests {
#[test]
// Test that integer numbers are displayed
// correctly with a decimal part of `.0`.
fn test_literal_display_integer_number() {
assert_eq!(Literal::Number(42.0).to_string(), "42.0");
}
#[test]
// Test that float numbers are displayed
// correctly with a decimal part.
fn test_literal_display_decimal_number() {
assert_eq!(Literal::Number(42.5).to_string(), "42.5");
}Finally, the new module must be declared in src/main.rs:
src/main.rs mod expression;
mod parser;
mod token;
mod tokenizer;To evaluate expressions, we can start defining a type for the result:
src/evaluate.rs use std::fmt;
#[derive(Debug, PartialEq, Eq)]
pub enum Value {
Boolean(bool),
Nil,
}
impl fmt::Display for Value {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Self::Boolean(b) => write!(f, "{b}"),
Self::Nil => f.write_str("nil"),
}
}
}The two variants Value::Boolean and Value::Nil will be used to implement the requirements for this step, but other variants will be added later. The type Value is remarkably similar to expression::Literal, but it is important to highlight that they serve different purposes and might evolve in different directions in the future.
Having defined a type for evaluated values, it is reasonable to also create a type for evaluation errors and a type for evaluation results:
src/evaluate.rs #[derive(Debug, PartialEq, Eq)]
pub struct EvaluationError {
pub message: String,
pub line: u16,
}
impl fmt::Display for EvaluationError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}\n[line {}]", self.message, self.line)
}
}
impl EvaluationError {
fn new(line: u16, message: impl Into<String>) -> Self {
Self {
message: message.into(),
line,
}
}
}
pub type EvaluationResult<T> = Result<T, EvaluationError>;The struct EvaluationError has been designed with Lox errors in mind, as they are defined in Robert Nystrom's book. They include a message and the line of code that contains the error, displayed on two separate lines as can be seen in the implementation of fmt.
With these structures in place, creating the base implementation of evaluate is a breeze:
src/evaluate.rs use crate::expression::{Expression, Literal};
pub fn evaluate(expression: &Expression) -> EvaluationResult<Value> {
match expression {
Expression::Literal(Literal::Boolean(b)) => Ok(Value::Boolean(*b)),
Expression::Literal(Literal::Nil) => Ok(Value::Nil),
_ => Err(EvaluationError::new(1, expression.to_string())),
}
}Please note that the function borrows the passed Expression instead of taking ownership. The reason is partly related to performance, but it's mostly a matter of ownership.
Performance-wise, the types Expression and &Expression have considerably different sizes. An enum that contains data needs a place to store it, so Rust enums compile to a block of memory big enough to host the largest variant and an integer that marks which variant is actually stored.
This means that if we pass Expression by value, every call of evaluate will copy as many bytes as the size of that enum. While the type &Expression is always fixed in size (8 bytes on a 64-bit machine), Expression is already bigger, and can grow as we add variants. The variant Expression::Binary is 72 bytes (see the calculations below).
The real advantage of borrowing Expression, however, is that the caller keeps the ownership of the syntax tree, and in this case it is the right choice as the function is only reading the expression.
+-----------------------------------------------------------------+
| Type | Size | Reason |
+-----------------+------+----------------------------------------|
| Box<Expression> | 8 | One pointer |
+-----------------+------+----------------------------------------|
| String | 24 | Capacity + pointer + length |
+-----------------+------+----------------------------------------|
| Literal | 24 | Largest variant is String(String) |
+-----------------+------+----------------------------------------|
| TokenLiteral | 24 | Largest variant is String(String) |
+-----------------+------+----------------------------------------|
| Keyword | 1 | 16 variants without data |
+-----------------+------+----------------------------------------|
| TokenType | 1 | Largest variant is Reserved(Keyword) |
+-----------------+------+----------------------------------------|
| Token | 56 | 24 (String) + |
| | | 24 (TokenLiteral) + |
| | | 2 (u16) + |
| | | 1 (TokenType) + |
| | | 5 (padding to 8-byte alignment) |
+-----------------+------+----------------------------------------|
| Expression | 72 | Largest variant is Binary |
| | | 56 (Token) |
| | | 8 (Box<Expression>) |
| | | 8 (Box<Expression>) |
+-----------------+------+----------------------------------------|
(Generated with rustc 1.97.0 on 64-bit Linux)At the moment, the error returned by the catch-all in evaluate is initialised with the line number 1. This is however just a placeholder, as the match will eventually cover all variants and there won't be any catch-all when the code is completed.
We can write tests for the business logic in evaluate:
src/evaluate.rs #[cfg(test)]
mod tests {
use super::*;
#[test]
// Test that the literal `true`
// evaluates to a boolean value.
fn test_evaluate_literal_true() {
let expression = Expression::Literal(Literal::Boolean(true));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that the literal `false`
// evaluates to a boolean value.
fn test_evaluate_literal_false() {
let expression = Expression::Literal(Literal::Boolean(false));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that the literal `nil`
// evaluates to the value `Nil`.
fn test_evaluate_literal_nil() {
let expression = Expression::Literal(Literal::Nil);
assert_eq!(evaluate(&expression), Ok(Value::Nil));
}
}Finally, we can introduce the new command evaluate in main:
src/main.rs use std::fs;
use std::path::PathBuf;
use std::process;
use crate::evaluate::evaluate;
use crate::parser::Parser;
use crate::token::{Token, TokenError, TokenResult};
use crate::tokenizer::tokenize;
use clap::{Parser as CParser, Subcommand as CSubcommand};
mod evaluate;
mod expression;
mod parser;
mod token;
mod tokenizer;
#[derive(CSubcommand)]
enum Commands {
Evaluate { filename: PathBuf },
Parse { filename: PathBuf },
Tokenize { filename: PathBuf },
}
fn main() {
match args.command {
Commands::Evaluate { filename } => {
// Process the source and
// extract Tokens or errors.
let results = tokenize_source(&filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
let mut p = Parser::new(tokens);
let parsed = p.expression().unwrap_or_else(|e| {
eprintln!("{}", e);
process::exit(65);
});
let result = evaluate(&parsed);
match result {
Ok(v) => println!("{}", v),
Err(e) => {
eprintln!("{}", e);
process::exit(65);
}
}
}However, since there is clearly a good amount of duplication between parse and evaluate, it makes sense to extract part of the logic into a function:
src/main.rs fn parse_source(filename: &PathBuf) -> Parser {
// Process the source and
// extract Tokens or errors.
let results = tokenize_source(filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
Parser::new(tokens)
}
fn main() {
match args.command {
Commands::Evaluate { filename } => {
// Process the source and
// extract Tokens or errors.
let results = tokenize_source(&filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
let mut p = Parser::new(tokens);
// Process the source and
// extract a parser.
let mut p = parse_source(&filename);
let parsed = p.expression().unwrap_or_else(|e| {
eprintln!("{}", e);
process::exit(65);
});
let result = evaluate(&parsed);
match result {
Ok(v) => println!("{}", v),
Err(e) => {
eprintln!("{}", e);
process::exit(65);
}
}
}
Commands::Parse { filename } => {
// Process the source and
// extract Tokens or errors.
let results = tokenize_source(&filename);
// A vector of tokens extracted from the results.
let mut tokens: Vec<Token> = Vec::new();
tokenize_process_results(
results,
|token| tokens.push(token),
|error| eprintln!("{}", error),
);
let mut p = Parser::new(tokens);
// Process the source and
// extract a parser.
let mut p = parse_source(&filename);
match p.expression() {
Ok(v) => println!("{}", v),
Err(e) => {
eprintln!("{}", e);
process::exit(65);
}
}
}Stage 1 - Literals: Booleans & Nil
This version of the code passes Stage 1 of the extension "Evaluating expressions".
The next feature we want to add is the evaluation of numbers and strings. Both are straightforward:
src/evaluate.rs #[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq)]
pub enum Value {
Boolean(bool),
Nil,
Number(f64),
String(String),
}
impl fmt::Display for Value {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Self::Boolean(b) => write!(f, "{b}"),
Self::Nil => f.write_str("nil"),
Self::Number(n) => write!(f, "{n}"),
Self::String(s) => f.write_str(s),
}
}
}
pub fn evaluate(expression: &Expression) -> EvaluationResult<Value> {
match expression {
Expression::Literal(Literal::Boolean(b)) => Ok(Value::Boolean(*b)),
Expression::Literal(Literal::Nil) => Ok(Value::Nil),
Expression::Literal(Literal::Number(n)) => Ok(Value::Number(*n)),
Expression::Literal(Literal::String(s)) => Ok(Value::String(s.clone())),
_ => Err(EvaluationError::new(1, expression.to_string())),
}
}In the arm Expression::Literal(Literal::String(s)) we use s.clone() to build the Value, while for booleans and numbers it is sufficient to dereference the variable.
The reason is that bool and f64 are Copy (that is, they implement the trait), and dereferencing a reference to a Copy value actually copies the bits. String is not Copy, so we cannot move it out of a reference with *s and we need to clone it into a new one.
This can be done in several, equivalent ways: s.clone(), s.to_string(), and s.to_owned() are the most common.
Please note that, as happened before for TokenLiteral and for Literal, the introduction of an f64 makes it impossible for Value to derive Eq. We can write tests for the new logic:
src/evaluate.rs mod tests {
#[test]
// Test that an integer number literal
// evaluates to a number value.
fn test_evaluate_literal_number_integer() {
let expression = Expression::Literal(Literal::Number(42.0));
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that a float number literal
// evaluates to a number value.
fn test_evaluate_literal_number_float() {
let expression = Expression::Literal(Literal::Number(42.5));
assert_eq!(evaluate(&expression), Ok(Value::Number(42.5)));
}
#[test]
// Test that a string literal
// evaluates to a string value.
fn test_evaluate_literal_string() {
let expression = Expression::Literal(Literal::String(String::from("Some string")));
assert_eq!(
evaluate(&expression),
Ok(Value::String(String::from("Some string")))
);
}Stage 2 - Literals: Strings & Numbers
This version of the code passes Stage 2 of the extension "Evaluating expressions".
Evaluation of groups is even simpler than strings and numbers as it doesn't require any additional variants in Value. Groups are nodes that contain another expression, so we just need to extract that and evaluate it recursively:
src/evaluate.rs pub fn evaluate(expression: &Expression) -> EvaluationResult<Value> {
match expression {
Expression::Group(g) => evaluate(g),
Expression::Literal(Literal::Boolean(b)) => Ok(Value::Boolean(*b)),
Expression::Literal(Literal::Nil) => Ok(Value::Nil),
Expression::Literal(Literal::Number(n)) => Ok(Value::Number(*n)),
Expression::Literal(Literal::String(s)) => Ok(Value::String(s.clone())),
_ => Err(EvaluationError::new(1, expression.to_string())),
}
}As usual, we can write tests for the new logic:
src/evaluate.rs #[test]
// Test that a group evaluates
// to the value of the expression
// it contains.
fn test_evaluate_group() {
let expression = Expression::Group(Box::new(Expression::Literal(Literal::Number(42.0))));
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that a nested group evaluates
// to the value of the innermost
// expression.
fn test_evaluate_group_nested() {
let expression = Expression::Group(Box::new(Expression::Group(Box::new(
Expression::Literal(Literal::Number(42.0)),
))));
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}Stage 3 - Parentheses
This version of the code passes Stage 3 of the extension "Evaluating expressions".
Evaluating unary expressions is very similar to evaluating groups, but with the additional complexity of having different use cases according to the operator and the type of the operand. Indeed, this is the first time we need to check the consistency of operations and data types: for example the operator Minus can be applied only to numbers, as -true or -"somestring" do not make sense in Lox.
src/evaluate.rs use std::fmt;
use crate::expression::{Expression, Literal};
use crate::token::{Token, TokenType};
pub fn evaluate(expression: &Expression) -> EvaluationResult<Value> {
match expression {
Expression::Group(g) => evaluate(g),
Expression::Literal(Literal::Boolean(b)) => Ok(Value::Boolean(*b)),
Expression::Literal(Literal::Nil) => Ok(Value::Nil),
Expression::Literal(Literal::Number(n)) => Ok(Value::Number(*n)),
Expression::Literal(Literal::String(s)) => Ok(Value::String(s.clone())),
Expression::Unary { operator, right } => evaluate_unary(operator, right),
_ => Err(EvaluationError::new(1, expression.to_string())),
}
}
fn evaluate_unary(operator: &Token, right: &Expression) -> EvaluationResult<Value> {
let value = evaluate(right)?;
match operator.ttype {
TokenType::Bang => match value {
Value::Boolean(b) => Ok(Value::Boolean(!b)),
Value::Number(_) => Ok(Value::Boolean(false)),
Value::Nil => Ok(Value::Boolean(true)),
Value::String(_) => Ok(Value::Boolean(false)),
},
TokenType::Minus => match value {
Value::Number(n) => Ok(Value::Number(-n)),
_ => Err(EvaluationError::new(1, "Operand must be a number.")),
},
_ => unreachable!("Invalid unary operator {:?}", operator.ttype),
}
}The helper evaluate_unary is useful to isolate the rich logic and keep the code of evaluate simple. The error messages have been written following the convention used by Robert Nystrom in "Crafting Interpreters".
The macro unreachable! [docs] is useful as in this case the compiler cannot determine that the arm _ is actually impossible to reach. Indeed, that depends on the logic we implemented and is not enforced by the type, as in theory we might create a unary operation with any variant of TokenType.
Given the number of arms in evaluate_unary, it is no surprise that we have several use cases to test. First of all we can create a couple of helper functions:
src/evaluate.rs mod tests {
use crate::token::TokenLiteral;
use super::*;
fn dummy_token(ttype: TokenType) -> Token {
Token {
ttype,
lexeme: String::new(),
literal: TokenLiteral::Null,
line: 1,
}
}
// Build a unary expression with the
// given operator and right operand.
fn dummy_unary_expression(ttype: TokenType, right: Expression) -> Expression {
Expression::Unary {
operator: dummy_token(ttype),
right: Box::new(right),
}
}Using them we can test the behaviour of unary Bang:
src/evaluate.rs mod tests {
#[test]
// Test that `Bang` in front of
// `true` returns `false`.
fn test_evaluate_unary_bang_true() {
let expression =
dummy_unary_expression(TokenType::Bang, Expression::Literal(Literal::Boolean(true)));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Bang` in front of
// `false` returns `true`.
fn test_evaluate_unary_bang_false() {
let expression = dummy_unary_expression(
TokenType::Bang,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `Bang` in front of
// a number returns `false`.
fn test_evaluate_unary_bang_number() {
let expression =
dummy_unary_expression(TokenType::Bang, Expression::Literal(Literal::Number(42.0)));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Bang` in front of
// the number zero returns `false`.
fn test_evaluate_unary_bang_zero() {
let expression =
dummy_unary_expression(TokenType::Bang, Expression::Literal(Literal::Number(0.0)));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Bang` in front of
// `nil` returns `true`.
fn test_evaluate_unary_bang_nil() {
let expression = dummy_unary_expression(TokenType::Bang, Expression::Literal(Literal::Nil));
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that a double `Bang` in front
// of a boolean returns the original boolean.
fn test_evaluate_unary_double_bang_boolean() {
let expression = dummy_unary_expression(
TokenType::Bang,
dummy_unary_expression(TokenType::Bang, Expression::Literal(Literal::Boolean(true))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}And in the same way test the behaviour of unary Minus:
src/evaluate.rs mod tests {
#[test]
// Test that `Minus` in front of
// a number negates it.
fn test_evaluate_unary_minus_number() {
let expression =
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Number(42.0)));
assert_eq!(evaluate(&expression), Ok(Value::Number(-42.0)));
}
#[test]
// Test that a double `Minus` in front
// of a number returns the original number.
fn test_evaluate_unary_double_minus_number() {
let expression = dummy_unary_expression(
TokenType::Minus,
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Number(42.0))),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that `Minus` in front of
// a group negates the value of the group.
fn test_evaluate_unary_minus_group() {
let expression = dummy_unary_expression(
TokenType::Minus,
Expression::Group(Box::new(Expression::Literal(Literal::Number(42.0)))),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(-42.0)));
}
#[test]
// Test that `Minus` in front of
// a boolean returns an error.
fn test_evaluate_unary_minus_boolean() {
let expression = dummy_unary_expression(
TokenType::Minus,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Minus` in front of
// a string returns an error.
fn test_evaluate_unary_minus_string() {
let expression = dummy_unary_expression(
TokenType::Minus,
Expression::Literal(Literal::String(String::from("Some string"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Minus` in front of
// `nil` returns an error.
fn test_evaluate_unary_minus_nil() {
let expression =
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil));
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that an error in the right
// operand is returned by the
// outer unary expression.
fn test_evaluate_unary_propagates_error() {
let expression = dummy_unary_expression(
TokenType::Bang,
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Please note that we are not testing the error messages. This is not merely a matter of style, as there are two underlying considerations:
Arguably, one might consider error messages for an interpreter part of the API and test them. However, in that case I would suggest returning enum variants and associating the string to them.
Stage 4 - Unary Operators: Negation & Not
This version of the code passes Stage 4 of the extension "Evaluating expressions".
Evaluating binary expressions is similar to evaluating unary ones, and the number of test cases increases, given there are more dimensions to cover. In essence, however, we have the same structure:
src/evaluate.rs pub fn evaluate(expression: &Expression) -> EvaluationResult<Value> {
match expression {
Expression::Binary {
left,
operator,
right,
} => evaluate_binary(operator, left, right),
Expression::Group(g) => evaluate(g),
Expression::Literal(Literal::Boolean(b)) => Ok(Value::Boolean(*b)),
Expression::Literal(Literal::Nil) => Ok(Value::Nil),
Expression::Literal(Literal::Number(n)) => Ok(Value::Number(*n)),
Expression::Literal(Literal::String(s)) => Ok(Value::String(s.clone())),
Expression::Unary { operator, right } => evaluate_unary(operator, right),
_ => Err(EvaluationError::new(1, expression.to_string())),
}
}With Expression::Binary we cover all possible variants of Expression, so there is no need for the catch-all arm any more. The helper function evaluate_binary follows what we did for evaluate_unary:
src/evaluate.rs fn evaluate_binary(
operator: &Token,
left: &Expression,
right: &Expression,
) -> EvaluationResult<Value> {
let lvalue = evaluate(left)?;
let rvalue = evaluate(right)?;
match (&operator.ttype, lvalue, rvalue) {
(TokenType::Slash, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a / b)),
(TokenType::Star, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a * b)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
}
}In this step, we cover only Star and Slash, leaving Plus and Minus to the next one. We can obviously write tests for the new logic. We first need a helper function to create binary expressions:
src/evaluate.rs mod tests {
// Build a binary expression with the
// given operator and operands.
fn dummy_binary_expression(
left: Expression,
ttype: TokenType,
right: Expression,
) -> Expression {
Expression::Binary {
left: Box::new(left),
operator: dummy_token(ttype),
right: Box::new(right),
}
}With that we can cover several use cases for both Star and Slash:
src/evaluate.rs mod tests {
#[test]
// Test that `Star` works correctly
// with numbers that have no decimals.
// A float with no decimal part is
// the way we represent an integer.
fn test_evaluate_binary_star_integer() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(6.0)),
TokenType::Star,
Expression::Literal(Literal::Number(7.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that `Star` works correctly
// with float numbers.
fn test_evaluate_binary_star_float() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(2.5)),
TokenType::Star,
Expression::Literal(Literal::Number(4.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(10.0)));
}
#[test]
// Test that `Slash` between two
// numbers divides them.
fn test_evaluate_binary_slash() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Slash,
Expression::Literal(Literal::Number(6.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(7.0)));
}
#[test]
// Test that `Slash` returns a float
// when the division is not exact.
fn test_evaluate_binary_slash_float_result() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(7.0)),
TokenType::Slash,
Expression::Literal(Literal::Number(2.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(3.5)));
}We can also test the behaviour of binary expressions combined with other types of expression:
src/evaluate.rs mod tests {
#[test]
// Test that a chain of binary expressions
// is evaluated following the structure
// of the tree, e.g. (42 / 6) * 2.
fn test_evaluate_binary_chain() {
let expression = dummy_binary_expression(
dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Slash,
Expression::Literal(Literal::Number(6.0)),
),
TokenType::Star,
Expression::Literal(Literal::Number(2.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(14.0)));
}
#[test]
// Test that a group used as an operand
// is evaluated before the binary
// expression, e.g. 42 / (6 * 2).
fn test_evaluate_binary_group_operand() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Slash,
Expression::Group(Box::new(dummy_binary_expression(
Expression::Literal(Literal::Number(6.0)),
TokenType::Star,
Expression::Literal(Literal::Number(2.0)),
))),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(3.5)));
}
#[test]
// Test that a unary expression used
// as an operand is evaluated before
// the binary expression, e.g. -6 * 7.
fn test_evaluate_binary_unary_operand() {
let expression = dummy_binary_expression(
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Number(6.0))),
TokenType::Star,
Expression::Literal(Literal::Number(7.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(-42.0)));
}And finally the long list of cases that should return an error:
src/evaluate.rs mod tests {
#[test]
// Test that an error in the left
// operand is returned by the
// binary expression.
fn test_evaluate_binary_propagates_error_left() {
let expression = dummy_binary_expression(
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil)),
TokenType::Star,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that an error in the right
// operand is returned by the
// binary expression.
fn test_evaluate_binary_propagates_error_right() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Star,
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Star` with a boolean
// operand returns an error.
fn test_evaluate_binary_star_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Star,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Star` with a string
// operand returns an error.
fn test_evaluate_binary_star_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("Some string"))),
TokenType::Star,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Star` with a `nil`
// operand returns an error.
fn test_evaluate_binary_star_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Star,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Slash` with a boolean
// operand returns an error.
fn test_evaluate_binary_slash_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Slash,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Slash` with a string
// operand returns an error.
fn test_evaluate_binary_slash_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("Some string"))),
TokenType::Slash,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Slash` with a `nil`
// operand returns an error.
fn test_evaluate_binary_slash_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Slash,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Stage 5 - Arithmetic Operators (1/2)
This version of the code passes Stage 5 of the extension "Evaluating expressions".
The additional operators Plus and Minus can be covered by adding two arms to the helper function:
src/evaluate.rs fn evaluate_binary(
operator: &Token,
left: &Expression,
right: &Expression,
) -> EvaluationResult<Value> {
let lvalue = evaluate(left)?;
let rvalue = evaluate(right)?;
match (&operator.ttype, lvalue, rvalue) {
(TokenType::Minus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a - b)),
(TokenType::Plus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a + b)),
(TokenType::Slash, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a / b)),
(TokenType::Star, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a * b)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
}
}Please note that in this stage we need to provide code to subtract and to add numbers. String concatenation through the operator Plus will be implemented in the next stage.
The tests for Minus are:
src/evaluate.rs mod tests {
#[test]
// Test that `Minus` works correctly
// with numbers that have no decimals.
fn test_evaluate_binary_minus_integer() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(50.0)),
TokenType::Minus,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that `Minus` works correctly
// with float numbers.
fn test_evaluate_binary_minus_float() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(5.5)),
TokenType::Minus,
Expression::Literal(Literal::Number(2.25)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(3.25)));
}
#[test]
// Test that `Minus` returns a negative
// number when the right operand is
// greater than the left one.
fn test_evaluate_binary_minus_negative_result() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(8.0)),
TokenType::Minus,
Expression::Literal(Literal::Number(50.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(-42.0)));
}
#[test]
// Test that `Minus` with a boolean
// operand returns an error.
fn test_evaluate_binary_minus_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Minus,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Minus` with a `nil`
// operand returns an error.
fn test_evaluate_binary_minus_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Minus,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Minus` cannot be
// used between strings.
fn test_evaluate_binary_minus_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Minus,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}The tests for Plus omit the check with strings on purpose. We will add that functionality in the next stage.
src/evaluate.rs mod tests {
#[test]
// Test that `Plus` works correctly
// with numbers that have no decimals.
fn test_evaluate_binary_plus_integer() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(34.0)),
TokenType::Plus,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(42.0)));
}
#[test]
// Test that `Plus` works correctly
// with float numbers.
fn test_evaluate_binary_plus_float() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(1.25)),
TokenType::Plus,
Expression::Literal(Literal::Number(2.5)),
);
assert_eq!(evaluate(&expression), Ok(Value::Number(3.75)));
}
#[test]
// Test that `Plus` with a boolean
// operand returns an error.
fn test_evaluate_binary_plus_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Plus,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Plus` with a `nil`
// operand returns an error.
fn test_evaluate_binary_plus_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Plus,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Stage 6 - Arithmetic Operators (2/2)
This version of the code passes Stage 6 of the extension "Evaluating expressions".
String concatenation is nothing more than one of the cases of binary evaluation, where we pin the type of the operands to Value::String. As numbers and strings are currently the only types that Plus supports, we can also add an error message for the remaining cases:
src/evaluate.rs fn evaluate_binary(
operator: &Token,
left: &Expression,
right: &Expression,
) -> EvaluationResult<Value> {
let lvalue = evaluate(left)?;
let rvalue = evaluate(right)?;
match (&operator.ttype, lvalue, rvalue) {
// Arithmetic
(TokenType::Minus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a - b)),
(TokenType::Plus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a + b)),
(TokenType::Slash, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a / b)),
(TokenType::Star, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a * b)),
// Strings
(TokenType::Plus, Value::String(a), Value::String(b)) => Ok(Value::String(a + &b)),
(TokenType::Plus, _, _) => Err(EvaluationError::new(
1,
"Operands must be two numbers or two strings.",
)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
}
}The number of combinations that are not number + number or string + string is huge, so it doesn't make sense to cover all of them with tests.
src/evaluate.rs mod tests {
#[test]
// Test that `Plus` can be
// used between strings.
fn test_evaluate_binary_plus_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Plus,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert_eq!(
evaluate(&expression),
Ok(Value::String(String::from("StringAStringB")))
);
}Stage 7 - String Concatenation
This version of the code passes Stage 7 of the extension "Evaluating expressions".
Comparisons are simply another binary use case, so we can tackle them by adding the necessary arms to evaluate_binary:
src/evaluate.rs fn evaluate_binary(
operator: &Token,
left: &Expression,
right: &Expression,
) -> EvaluationResult<Value> {
let lvalue = evaluate(left)?;
let rvalue = evaluate(right)?;
match (&operator.ttype, lvalue, rvalue) {
// Arithmetic
(TokenType::Minus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a - b)),
(TokenType::Plus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a + b)),
(TokenType::Slash, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a / b)),
(TokenType::Star, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a * b)),
// Comparisons
(TokenType::Greater, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a > b)),
(TokenType::GreaterEqual, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a >= b)),
(TokenType::Less, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a < b)),
(TokenType::LessEqual, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a <= b)),
// Strings
(TokenType::Plus, Value::String(a), Value::String(b)) => Ok(Value::String(a + &b)),
(TokenType::Plus, _, _) => Err(EvaluationError::new(
1,
"Operands must be two numbers or two strings.",
)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
}
}Once again, the number of tests is rather high. We have tests for Greater:
src/evaluate.rs mod tests {
#[test]
// Test that `Greater` returns `true`
// when the comparison holds.
fn test_evaluate_binary_greater_true() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(50.0)),
TokenType::Greater,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `Greater` returns `false`
// when the comparison does not hold.
fn test_evaluate_binary_greater_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(8.0)),
TokenType::Greater,
Expression::Literal(Literal::Number(50.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Greater` between two
// equal numbers returns `false`.
fn test_evaluate_binary_greater_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Greater,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Greater` with a boolean
// operand returns an error.
fn test_evaluate_binary_greater_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Greater,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Greater` with a `nil`
// operand returns an error.
fn test_evaluate_binary_greater_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Greater,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Greater` with two string
// operands returns an error.
fn test_evaluate_binary_greater_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Greater,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Tests for GreaterEqual:
src/evaluate.rs mod tests {
#[test]
// Test that `GreaterEqual` returns `true`
// when the comparison holds.
fn test_evaluate_binary_greater_equal_true() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(50.0)),
TokenType::GreaterEqual,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `GreaterEqual` returns `false`
// when the comparison does not hold.
fn test_evaluate_binary_greater_equal_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(8.0)),
TokenType::GreaterEqual,
Expression::Literal(Literal::Number(50.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `GreaterEqual` between two
// equal numbers returns `true`.
fn test_evaluate_binary_greater_equal_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::GreaterEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `GreaterEqual` with a boolean
// operand returns an error.
fn test_evaluate_binary_greater_equal_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::GreaterEqual,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `GreaterEqual` with a `nil`
// operand returns an error.
fn test_evaluate_binary_greater_equal_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::GreaterEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `GreaterEqual` with two string
// operands returns an error.
fn test_evaluate_binary_greater_equal_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::GreaterEqual,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Tests for Less:
src/evaluate.rs mod tests {
#[test]
// Test that `Less` returns `true`
// when the comparison holds.
fn test_evaluate_binary_less_true() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(8.0)),
TokenType::Less,
Expression::Literal(Literal::Number(50.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `Less` returns `false`
// when the comparison does not hold.
fn test_evaluate_binary_less_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(50.0)),
TokenType::Less,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Less` between two
// equal numbers returns `false`.
fn test_evaluate_binary_less_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Less,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `Less` with a boolean
// operand returns an error.
fn test_evaluate_binary_less_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::Less,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Less` with a `nil`
// operand returns an error.
fn test_evaluate_binary_less_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::Less,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Less` with two string
// operands returns an error.
fn test_evaluate_binary_less_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Less,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}And finally tests for LessEqual:
src/evaluate.rs mod tests {
#[test]
// Test that `LessEqual` returns `true`
// when the comparison holds.
fn test_evaluate_binary_less_equal_true() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(8.0)),
TokenType::LessEqual,
Expression::Literal(Literal::Number(50.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `LessEqual` returns `false`
// when the comparison does not hold.
fn test_evaluate_binary_less_equal_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(50.0)),
TokenType::LessEqual,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `LessEqual` between two
// equal numbers returns `true`.
fn test_evaluate_binary_less_equal_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::LessEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `LessEqual` with a boolean
// operand returns an error.
fn test_evaluate_binary_less_equal_boolean() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::LessEqual,
Expression::Literal(Literal::Boolean(true)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `LessEqual` with a `nil`
// operand returns an error.
fn test_evaluate_binary_less_equal_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::LessEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `LessEqual` with two string
// operands returns an error.
fn test_evaluate_binary_less_equal_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::LessEqual,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Stage 8 - Relational Operators
This version of the code passes Stage 8 of the extension "Evaluating expressions".
In the previous 8 steps we wrote a lot of tests, the number of which is a silent testimony to the richness of use cases that can be created by a handful of operators and types. However, the sheer amount of tests present in src/evaluate.rs makes it difficult to find and maintain them. It is high time we found a better setup for them.
In Rust, tests are typically added in the same file that contains the business logic, but nothing stops us from isolating them into submodules. Ultimately, the only requirement is that they should be available when the configuration option test is turned on.
In this step we are going to split tests into several submodules, trying to keep each module to a manageable size.
First of all, let's isolate some of the helper functions. Create the file src/test_utils.rs:
src/test_utils.rs use crate::token::{Token, TokenLiteral, TokenType};
// Build a token of the given type
// with no lexeme and no literal.
pub fn dummy_token(ttype: TokenType) -> Token {
Token {
ttype,
lexeme: String::new(),
literal: TokenLiteral::Null,
line: 1,
}
}The file must be added to the modules in src/main.rs, but as it is used for tests only it will be marked with the annotation #[cfg(test)]:
src/main.rs mod evaluate;
mod expression;
mod parser;
mod token;
mod tokenizer;
#[cfg(test)]
mod test_utils;
#[derive(CSubcommand)]
enum Commands {
Evaluate { filename: PathBuf },
Parse { filename: PathBuf },
Tokenize { filename: PathBuf },
}We can now create the directory src/evaluate and the file src/evaluate/tests.rs:
src/evaluate/tests.rs use super::*;
use crate::test_utils::dummy_token;
// Build a unary expression with the
// given operator and right operand.
fn dummy_unary_expression(ttype: TokenType, right: Expression) -> Expression {
Expression::Unary {
operator: dummy_token(ttype),
right: Box::new(right),
}
}
// Build a binary expression with the
// given operator and operands.
fn dummy_binary_expression(left: Expression, ttype: TokenType, right: Expression) -> Expression {
Expression::Binary {
left: Box::new(left),
operator: dummy_token(ttype),
right: Box::new(right),
}
}At this point, all the tests in src/evaluate.rs can be moved into files created inside src/evaluate/tests, e.g.:
src/evaluate/tests/binary_strings.rs use super::*;
#[test]
// Test that `Star` with a string
// operand returns an error.
fn test_star_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("Some string"))),
TokenType::Star,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Slash` with a string
// operand returns an error.
fn test_slash_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("Some string"))),
TokenType::Slash,
Expression::Literal(Literal::Number(42.0)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Minus` cannot be
// used between strings.
fn test_minus_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Minus,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}
#[test]
// Test that `Plus` can be
// used between strings.
fn test_plus_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::Plus,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert_eq!(
evaluate(&expression),
Ok(Value::String(String::from("StringAStringB")))
);
}As you can see, each test has a simpler name, given that it belongs to a specific file. You can see the list of files and the content of each one of them in the Git diff, but you are free to rearrange tests as you see fit. After all, this refactoring is done solely to make the code more manageable.
Each file that contains tests must be added to src/evaluate/tests.rs:
src/evaluate/tests.rs use super::*;
use crate::test_utils::dummy_token;
mod binary_strings;When all tests have been moved, the only test-related content in src/evaluate.rs should be the declaration of the module tests, which can be put at the beginning of the file for convenience:
src/evaluate.rs use std::fmt;
use crate::expression::{Expression, Literal};
use crate::token::{Token, TokenType};
#[cfg(test)]
mod tests;
#[derive(Debug, PartialEq)]
pub enum Value {
Boolean(bool),
Nil,
Number(f64),
String(String),
}
...
#[cfg(test)]
mod tests {
...
}The new module test_utils can be used in the tests for src/parser.rs:
src/parser.rs mod tests {
use crate::test_utils::dummy_token;
use crate::token::TokenLiteral;
use super::*;
fn dummy_token(ttype: TokenType) -> Token {
Token {
ttype,
lexeme: String::new(),
literal: TokenLiteral::Null,
line: 1,
}
}The final shape of the directory src is:
src/
+-- evaluate
| +-- tests
| | +-- binary_arithmetic.rs <--- Tests for this component
| | +-- binary_comparisons.rs
| | +-- binary_strings.rs
| | +-- literal_group.rs
| | +-- unary.rs
| +-- tests.rs <--- dummy_unary_expression()
| dummy_binary_expression()
+-- evaluate.rs
+-- expression.rs
+-- main.rs
+-- parser.rs
+-- test_utils.rs <--- dummy_token()
+-- tokenizer.rs
+-- token.rsTo evaluate equality we need to add new arms to evaluate_binary. The only difference from what we have done so far is the fact that we do not need to pin any variant of Value, as equality is valid (for the time being) for every type:
src/evaluate.rs fn evaluate_binary(
operator: &Token,
left: &Expression,
right: &Expression,
) -> EvaluationResult<Value> {
let lvalue = evaluate(left)?;
let rvalue = evaluate(right)?;
match (&operator.ttype, lvalue, rvalue) {
// Arithmetic
(TokenType::Minus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a - b)),
(TokenType::Plus, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a + b)),
(TokenType::Slash, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a / b)),
(TokenType::Star, Value::Number(a), Value::Number(b)) => Ok(Value::Number(a * b)),
// Comparisons
(TokenType::Greater, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a > b)),
(TokenType::GreaterEqual, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a >= b)),
(TokenType::Less, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a < b)),
(TokenType::LessEqual, Value::Number(a), Value::Number(b)) => Ok(Value::Boolean(a <= b)),
// Equality
(TokenType::BangEqual, l, r) => Ok(Value::Boolean(l != r)),
(TokenType::EqualEqual, l, r) => Ok(Value::Boolean(l == r)),
// Strings
(TokenType::Plus, Value::String(a), Value::String(b)) => Ok(Value::String(a + &b)),
(TokenType::Plus, _, _) => Err(EvaluationError::new(
1,
"Operands must be two numbers or two strings.",
)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
}
}The tests for the new logic can be added to a new file. There are tests for EqualEqual:
src/evaluate/tests/binary_equality.rs use super::*;
#[test]
// Test that `EqualEqual` returns `true`
// for two equal numbers.
fn test_equal_equal_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::EqualEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for two different numbers.
fn test_equal_equal_different_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::EqualEqual,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` returns `true`
// for two equal strings.
fn test_equal_equal_same_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::EqualEqual,
Expression::Literal(Literal::String(String::from("StringA"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for two different strings.
fn test_equal_equal_different_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::EqualEqual,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` returns `true`
// for two equal booleans.
fn test_equal_equal_same_booleans() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Boolean(true)),
TokenType::EqualEqual,
Expression::Literal(Literal::Boolean(true)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for two different booleans.
fn test_equal_equal_different_booleans() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Boolean(true)),
TokenType::EqualEqual,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` returns `true`
// when both operands are `nil`.
fn test_equal_equal_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::EqualEqual,
Expression::Literal(Literal::Nil),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for a number and a string with the
// same content.
fn test_equal_equal_number_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::EqualEqual,
Expression::Literal(Literal::String(String::from("42"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for `nil` and `false`, even though
// both are falsey.
fn test_equal_equal_nil_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::EqualEqual,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` returns `false`
// for zero and `false`.
fn test_equal_equal_zero_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(0.0)),
TokenType::EqualEqual,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `EqualEqual` compares the
// values of its operands and not their
// structure.
fn test_equal_equal_expression_operand() {
let expression = dummy_binary_expression(
dummy_binary_expression(
Expression::Literal(Literal::Number(6.0)),
TokenType::Star,
Expression::Literal(Literal::Number(7.0)),
),
TokenType::EqualEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that an error in an operand
// is returned by `EqualEqual`.
fn test_equal_equal_propagates_error() {
let expression = dummy_binary_expression(
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil)),
TokenType::EqualEqual,
Expression::Literal(Literal::Nil),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}And tests for BangEqual:
src/evaluate/tests/binary_equality.rs #[test]
// Test that `BangEqual` returns `false`
// for two equal numbers.
fn test_bang_equal_same_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::BangEqual,
Expression::Literal(Literal::Number(42.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `BangEqual` returns `true`
// for two different numbers.
fn test_bang_equal_different_numbers() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::BangEqual,
Expression::Literal(Literal::Number(8.0)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `BangEqual` returns `false`
// for two equal strings.
fn test_bang_equal_same_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::BangEqual,
Expression::Literal(Literal::String(String::from("StringA"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `BangEqual` returns `true`
// for two different strings.
fn test_bang_equal_different_strings() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::String(String::from("StringA"))),
TokenType::BangEqual,
Expression::Literal(Literal::String(String::from("StringB"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `BangEqual` returns `false`
// for two equal booleans.
fn test_bang_equal_same_booleans() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Boolean(true)),
TokenType::BangEqual,
Expression::Literal(Literal::Boolean(true)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `BangEqual` returns `true`
// for two different booleans.
fn test_bang_equal_different_booleans() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Boolean(true)),
TokenType::BangEqual,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `BangEqual` returns `false`
// when both operands are `nil`.
fn test_bang_equal_nil() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::BangEqual,
Expression::Literal(Literal::Nil),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(false)));
}
#[test]
// Test that `BangEqual` returns `true`
// for a number and a string with the
// same content.
fn test_bang_equal_number_string() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Number(42.0)),
TokenType::BangEqual,
Expression::Literal(Literal::String(String::from("42"))),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that `BangEqual` returns `true`
// for `nil` and `false`.
fn test_bang_equal_nil_false() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::BangEqual,
Expression::Literal(Literal::Boolean(false)),
);
assert_eq!(evaluate(&expression), Ok(Value::Boolean(true)));
}
#[test]
// Test that an error in an operand
// is returned by `BangEqual`.
fn test_bang_equal_propagates_error() {
let expression = dummy_binary_expression(
Expression::Literal(Literal::Nil),
TokenType::BangEqual,
dummy_unary_expression(TokenType::Minus, Expression::Literal(Literal::Nil)),
);
assert!(matches!(evaluate(&expression), Err(EvaluationError { .. })));
}Finally, the new module binary_equality must be added to the list of test modules:
src/evaluate/tests.rs mod binary_arithmetic;
mod binary_comparisons;
mod binary_equality;
mod binary_strings;
mod literal_group;
mod unary;Stage 9 - Equality Operators
This version of the code passes Stage 9 of the extension "Evaluating expressions".
The final stages of the challenge are almost already completed, as all error messages have been designed with the convention of "Crafting Interpreters" in mind.
Actually, to pass them it is sufficient to exit with the right error code when the evaluation runs into an error:
src/main.rs fn main() {
match args.command {
Commands::Evaluate { filename } => {
match result {
Ok(v) => println!("{}", v),
Err(e) => {
eprintln!("{}", e);
process::exit(65);
process::exit(70);
}
}However, we can also take advantage of the fact that the type Token contains a line number to output more useful error messages:
src/evaluate.rs fn evaluate_unary(operator: &Token, right: &Expression) -> EvaluationResult<Value> {
match operator.ttype {
TokenType::Minus => match value {
Value::Number(n) => Ok(Value::Number(-n)),
_ => Err(EvaluationError::new(1, "Operand must be a number.")),
_ => Err(EvaluationError::new(
operator.line,
"Operand must be a number.",
)),
},
fn evaluate_binary(
match (&operator.ttype, lvalue, rvalue) {
// Strings
(TokenType::Plus, Value::String(a), Value::String(b)) => Ok(Value::String(a + &b)),
(TokenType::Plus, _, _) => Err(EvaluationError::new(
1,
operator.line,
"Operands must be two numbers or two strings.",
)),
_ => Err(EvaluationError::new(1, "Operands must be numbers.")),
_ => Err(EvaluationError::new(
operator.line,
"Operands must be numbers.",
)),Stages 10-13 - Runtime Errors
This version of the code passes Stage 10, Stage 11, Stage 12, and Stage 13 of the extension "Evaluating expressions".
Before we tackle the next extension of the CodeCrafters challenge, we can restructure the tests we wrote for Parser. It's never a good idea to keep too many tests in the same file, as it can easily become too difficult to see what has been tested and where.
Therefore, we can follow the same strategy used to refactor the evaluation tests. Let's create the directories src/parser and src/parser/tests to host the new module. The changes to src/parser.rs are:
src/parser.rs #[cfg(test)]
mod tests {
...
}
mod tests;Test imports are moved into src/parser/tests.rs together with submodule declarations:
src/parser/tests.rs use super::*;
use crate::test_utils::dummy_token;
use crate::token::TokenLiteral;
mod comparison;
mod equality;
mod factor;
mod parser;
mod primary;
mod term;
mod unary;
mod utils;The single tests are moved into files that roughly correspond to the parsing rules. The code is copied verbatim and only the function names are changed: you can see the full diff using the link at the beginning of the section. I show below an example of what has been done, in this case with the tests for the method Parser::comparison.
src/parser/tests/comparison.rs use super::*;
#[test]
// Test that the method `comparison` works
// correctly with two primaries and
// a token `Less`.
fn test_less() {
...
}
#[test]
// Test that the method `comparison` works
// correctly with two primaries and
// a token `LessEqual`.
fn test_less_equal() {
...
}
#[test]
// Test that the method `comparison` works
// correctly with two primaries and
// a token `Greater`.
fn test_greater() {
...
}
#[test]
// Test that the method `comparison` works
// correctly with two primaries and
// a token `GreaterEqual`.
fn test_greater_equal() {
...
}
#[test]
// Test that the method `comparison` works
// correctly with multiple primaries
// chained by comparison
// tokens.
fn test_chain() {
...
}