Solution 2 for https://stackoverflow.com/questions/58891186/custom-location-tracking-in-jison-gho#58891186

100%

Solution 2 for https://stackoverflow.com/questions/58891186/custom-location-tracking-in-jison-gho#58891186

✓ Published0🌍 Public

Last edited Nov 17, 2019

Created on Nov 17, 2019

This example demonstrates a Jison grammar and lexer for parsing a custom "gifcode" language, with a focus on tracking token positions. The `post_lex` hook in the lexer overrides `first_column` and `last_column` to store an incrementing token counter, enabling the parser to track which token range corresponds to each grammar rule, while the underlying `range` member still tracks raw character positions. The lexer uses start conditions (`%x delimit`, `%x string_literal`) to manage delimiter and string states, and its error handling throws descriptions for invalid characters. The visible output is the parsed abstract syntax tree, built with the `Stmt.ProgramStmt` node, where location information is embedded in the `@$` attribute.

AI-generated description

Similar vizzes

Loading thumbnail…

Working implementation 1 for https://stackoverflow.com/questions/58891186/custom-location-tracking-in-jison-gho#58891186

This example demonstrates a custom lexer and parser grammar for the Jison parser generator, as discussed in a Stack Overflow question about tracking custom source locations. The Jison lexer configuration defines tokens like `LT`, `PLUS`, and `DIGIT`, with special states such as `delimit` and `string_literal` to handle token boundaries and quoted strings. The grammar rules then assemble these tokens into program structures, while extra lexer code maintains a token counter that is attached to each token’s location object. The example shows how Jison’s `yylloc` and custom `post_lex` hooks can be used to annotate parsed tokens with additional positional data.

GGerHobbelt

88% match

Loading thumbnail…

OP file, compilable by jison-gho. SO: https://stackoverflow.com/posts/58902159

This example demonstrates a Jison-gho lexer and parser grammar for a custom GIF-like code language, converting token sequences into an abstract syntax tree. The lexer uses state management (`begin`, `popState`, `delimit` and `string_literal` states) to enforce semicolon delimiters between tokens and handle quoted strings. The parser defines operator precedence with `%left` declarations and builds AST nodes via `new yy.Stmt` and `new yy.Expr` constructors, processing number literals, identifiers, and control flow keywords. The grammar handles numeric formats and string concatenation rules visible in rules like `UFloat` and `AlfanumAtom`.

GGerHobbelt

71% match

Loading thumbnail…

snip&snap extracts from our major JISON grammar file, showcasing 'code sections' a la BISON plus a few other bits & tricks. Note the %{ ... %} sections which are JISON's 'code sections'. Also note the code following that last '%%' marker: that is another 'code section' - and the most important one.

This example presents a JISON grammar file for a formula parser, highlighting its use of embedded code sections marked by `%{ ... %}`, a feature inherited from BISON. It demonstrates a lexical hack where a dotted identifier regex includes a look-ahead for an open parenthesis to recognize function tokens, using the parser's symbol table lookup. The code also documents how JISON transforms lexer regex macros and how the `this.unput` and `this.matches` APIs manage pushed-back input. The grammar is rendered as a raw source file from a gist.

GGerHobbelt

64% match

Loading thumbnail…

grammar pseudo language

This visualization shows the grammar rules of a custom programming pseudo-language as defined in a Jison specification file. It uses Jison’s `%lex` and `%ebnf` sections to declare lexical tokens (e.g., `NUMBER`, `PRINTLN`) and a grammar for mathematical expressions, functions, assignments, and print statements. The example includes semantic actions in JavaScript that return computed values or generated code strings, and a custom `parser.main` function tests two sample inputs by calling `parser.parse()`. The source is a GitHub gist by GerHobbelt.

GGerHobbelt

62% match

Loading thumbnail…

JISON sample wrapper showcasing custom error handlers (stripped JavaScript, ... where more must be done)

This example demonstrates a JavaScript wrapper around a JISON-generated parser, showcasing custom error handling and event-driven integration. The code overrides `parserWithTokens.yy.parseError` to distinguish and dispatch parser versus lexer errors, and hooks `lexer.post_lex` to log token strings and monitor input consumption. It also registers a `post_parse` extension to ensure full lexing even after errors, and uses a simple event system to allow user action code inside the lexer/parser to trigger handlers dynamically. The implementation relies on the JISON parser generator’s terminal identifier table and its generated parser class.

GGerHobbelt

62% match

Loading thumbnail…

This does not lead to a good design. BAD BAD BAD

This example contrasts rule-based token transformation with PEG parsing logic, showing how history-dependent lookbehind rules produce different outputs than future-looking parsing. The code defines transformation rules using a custom `history` operator, replacing characters like ‘x’ after ‘zz’ and ‘z’ after single ‘z’, iterating through the input stepwise. It demonstrates the distinction between processing tokens based on preceding context versus PEG’s lookahead predicates, highlighting limitations in handling left recursion and ambiguous histories. The visualization uses a simple grammar specification with action blocks like `replace_with` and `copy`, rendered as text-based pseudocode in the README.

GGerHobbelt

54% match

Loading thumbnail…

Lexing and Parsing in JavaScript

This example demonstrates a live lexer and recursive descent parser for a custom wiki-like markup language, converting user input from a textarea into HTML output. The code uses d3.v2 to manage DOM selection and event handling, registering callbacks that recompile the input on changes. It showcases the parsing pipeline by processing the typed text in real time, with a table tracking events fired on the html, body, and textarea elements. The parser applies grammar rules in precedence order, prioritizing robustness over performance by assuming limited input size.

GGerHobbelt

48% match

Loading thumbnail…

Gist 4905f6e091305c6a3ff3c1fe2c0a50b2

This example visualizes the structure of a DAX query plan by parsing a text-based log file into a hierarchical tree. It uses the Moo lexer to tokenize lines, identifiers, column references, and operator types, then builds a nested representation of the plan. The visualization likely renders this tree to show the flow of logical and physical operations, with each node displaying its operator type and associated properties such as filters, joins, or spooling details. The code defines custom token patterns for DAX-specific syntax, including column references like `'Table'[Column]` and measure references like `[Measure]`, and parses them into structured objects for display.

HHugoberry

45% match