ast.litkal

Jump To … +

ast.litkal command.litkal generator.litkal grammar.litkal interactive.litkal kal.litkal lexer.litkal literate.litkal parser.litkal sugar.litkal

ast.litkal
¶

The Abstract Syntax Tree (AST)
¶

This module defines error classes and the base abstract syntax tree class used by the parser and grammar. It's main purpose is to provide utility methods to parse the token array into classes that inherit from ASTBase.

Errors
¶

There are two types of errors that can be reported during parsing. ParseFailed is less serious indicating that the class failed to parse the upcoming tokens in the array, however the syntax might still be valid if parsed as another type of AST node. This generally will not abort compilation as the parent node will try other AST classes against the token stream before failing. not_compiler_issue indicates to the compiler that this is a problem with the input code, not a bug in the compiler itself. This error is used internally only.
```
class ParseFailed
  method initialize(message)
    me.message = message
    me.not_compiler_issue = yes
```
¶

SyntaxError is more serious indicating that the parser is using the correct class to parse the stream, but the code has invalid syntax. For example, if for x would throw this error because the IfStatement class is sure that this is an if statement, but if for is definitely invalid. This always aborts compilation and gives the user an error. This error will get thrown to the top level module.
```
class SyntaxError
  method initialize(message)
    me.message = message
    me.locked = yes
    me.not_compiler_issue = yes

exports.SyntaxError = SyntaxError
```
¶

ASTBase Class
¶

This class serves as a base class on top of which AST nodes are defined. It contains utility functions to parse the token stream.
```
class ASTBase

  method initialize(ts, parent)
```
¶

The object is initially marked as “unlocked”, indicating that we are not sure that this is the right node to parse this segment of the token stream. We can't declare syntax errors until we are sure this is the right class.
```
    me.locked = no
```
¶

Record the line and link to the token stream. Also note the parent object for code generation
```
    me.ts = ts
    me.line = ts.line
    me.ast_parent = parent
```
¶

The parse method, which is overriden by child classes, attempts to parse the current spot in the token stream. It will fail with a ParseFailed error or SyntaxError if parsing fails.
```
    me.parse()
```
¶

Record the last line of the node. This is used to place comments correctly in the output Javascript.
```
    me.endline = ts.line
```
¶

opt attempts to parse the token stream using one of the classes or token types specified. This method takes a variable number of arguments. For example, calling me.opt IfStatement, Expression, 'IDENTIFIER' would attempt to parse the token stream first as an IfStatement. If that fails, it would attempt to use the Expression class. If that fails, it will accept a token of type IDENTIFIER. If all of those fail, it will return none.

Note that SyntaxErrors will still be thrown, but not ParseFailed errors.
```
  method opt()
```
¶

If opt fails, it will reset the current index of the token stream.
```
    start_index = me.ts.index
```
¶

For each argument, which can be a class or a string, it will attempt to parse the token stream with that class or match the token type to the string.
```
    for cls in arguments
```

For strings it just checks the token type. And returns the token (incrementing the token stream index) if the type matches.

      if typeof(cls) is 'string'
        if me.ts.type is cls
          token = me.ts.current
          me.ts.next()
          return token

¶

For classes, it attempts to instantiate the class (which calls the class's parse). new will fail with either a ParseFailed, SyntaxError, or other exception (if the compiler has a bug) if parsing fails.
```
      else
        try
          return new cls me.ts, me
```
¶

If parsing does fail, we want to rewind the token stream. If the error has the locked flag or the error came from the compiler, we do want to abort and throw.
```
        catch err
          me.ts.goto_token start_index
          fail with err when err.locked or not err.not_compiler_issue
```
¶

opt returns nothing if none of the arguments could be use to parse the stream.
```
    return nothing
```
¶

req works the same way as opt except that it throws an error if none of the arguments can be used to parse the stream.
```
  method req()
```
¶

We first call opt to see what we get. If a value is returned, the function was successful, so we just return the node that opt found.
```
    rv = me.opt.apply this, arguments
    return rv if rv exists
```

If opt returned nothing, we want to give the user a useful error.

    list = [cls.name or cls for cls in Array.prototype.slice.call(arguments)]
    if list.length is 1
      message = "Expected #{list[0]}"
    else
      message = "Expected one of #{list.join(', ')}"
    me.error "#{message}"

¶

opt_val checks if the next token has a semantic value that matches one of the arguments provided. If so it returns that token and advances the stream. Otherwise, it returns nothing.
```
  method opt_val()
    if me.ts.value in arguments
      token = me.ts.current
      me.ts.next()
      return token
    else
      return nothing
```

req_val is the same as opt_val except that it throws an error if the semantic value could not be matched.

  method req_val()
    rv = me.opt_val.apply this, arguments
    return rv if rv exists

¶

The JavaScript (node) arguments structure is a little weird in that it‘s not strictly an array. It’s actually an object mapping with some extra properties. In order to produce a useful error message we make a copy of the values as an array.
```
    args = [v for v in Array.prototype.slice.call(arguments)]
    me.error "Expected '#{args.join('\' or \'')}'"
```
¶

opt_multi this method is like opt except that it will return zero or more of the requested class or token type. If it can't match any tokens it returns an empty array. It is “greedy” in that it will try to match as many occurances of a token or class as possible. opt_multi will only return objects/tokens from the first argument that matches the token stream. This method always returns an array.
```
  method opt_multi()
    cls = me.opt.apply this, arguments
    return [] unless cls exists
    rv = [cls]
    while cls exists
      cls = me.opt.apply this, arguments
      rv.push(cls) if cls exists
    return rv
```
¶

req_multi this method is like req except that it will return one or more of the requested class or token type. If it can't match any tokens it does throw an error. It is “greedy” in that it will try to match as many occurances as possible. req_multi will only return objects/tokens from the first argument that matches the token stream. This method always returns an array.
```
  method req_multi()
    rv = me.opt_multi.apply this, arguments
    return rv if rv.length > 0
```

Create a useful error message for the user.

    list = [cls.name or cls for cls in Array.prototype.slice.call(arguments)]
    me.error "Expected one of #{list.join(', ')}"

parse and js are abstract methods defined here to catch missing implementations. Child classes must override these methods, otherwise compilation will fail.

  method parse()
    me.lock()
    me.error 'Parser Not Implemented: ' + me.constructor.name

  method js()
    me.error 'Javascript Generator Not Implemented: ' + me.constructor.name

¶

error throws a ParseFailed error if the object is unlocked (syntax does not match this class, but may still be valid for another node) or a SyntaxError if the object is locked (syntax matches this node but is invalid).
```
  method error(msg)
    if me.locked
      full_msg = msg + ' on line ' + me.line
```

We try to report the filename if possible.

      if me.ts.options?['filename'] exists
        full_msg += ' in file ' + me.ts.options['filename']
      throw new SyntaxError full_msg
    else
      throw new ParseFailed msg

¶

lock marks this class as locked, meaning we are certain this is the correct class for the given syntax. For example, if the FunctionExpression class sees the IDENTIFIER function, we are certain this is the correct class to use. Once locked, any invalid syntax causes compilation to fail.

lock can be called multiple times to update the line number. If a node spans multiple lines, this is useful because the line number is reported in the error message.
```
  method lock()
    me.locked = yes
    me.line = me.ts.line
```
¶

ASTBase is the base class for all AST nodes.
```
exports.ASTBase = ASTBase
```

ast.litkal

The Abstract Syntax Tree (AST)

Errors

ASTBase Class