Skip to main content
SimilarExport ZIP
100%

Working implementation 1 for https://stackoverflow.com/questions/58891186/custom-location-tracking-in-jison-gho#58891186

✓ Published0🌍 Public
GGerHobbelt
Last edited Nov 17, 2019
Created on Nov 17, 2019

This example demonstrates a custom lexer and parser grammar for the Jison parser generator, as discussed in a Stack Overflow question about tracking custom source locations. The Jison lexer configuration defines tokens like `LT`, `PLUS`, and `DIGIT`, with special states such as `delimit` and `string_literal` to handle token boundaries and quoted strings. The grammar rules then assemble these tokens into program structures, while extra lexer code maintains a token counter that is attached to each token’s location object. The example shows how Jison’s `yylloc` and custom `post_lex` hooks can be used to annotate parsed tokens with additional positional data.

AI-generated description

Similar vizzes

Loading thumbnail…

Solution 2 for https://stackoverflow.com/questions/58891186/custom-location-tracking-in-jison-gho#58891186

This example demonstrates a Jison grammar and lexer for parsing a custom "gifcode" language, with a focus on tracking token positions. The `post_lex` hook in the lexer overrides `first_column` and `last_column` to store an incrementing token counter, enabling the parser to track which token range corresponds to each grammar rule, while the underlying `range` member still tracks raw character positions. The lexer uses start conditions (`%x delimit`, `%x string_literal`) to manage delimiter and string states, and its error handling throws descriptions for invalid characters. The visible output is the parsed abstract syntax tree, built with the `Stmt.ProgramStmt` node, where location information is embedded in the `@$` attribute.

GGerHobbelt
88% match
Loading thumbnail…

JISON sample wrapper showcasing custom error handlers (stripped JavaScript, ... where more must be done)

This example demonstrates a JavaScript wrapper around a JISON-generated parser, showcasing custom error handling and event-driven integration. The code overrides `parserWithTokens.yy.parseError` to distinguish and dispatch parser versus lexer errors, and hooks `lexer.post_lex` to log token strings and monitor input consumption. It also registers a `post_parse` extension to ensure full lexing even after errors, and uses a simple event system to allow user action code inside the lexer/parser to trigger handlers dynamically. The implementation relies on the JISON parser generator’s terminal identifier table and its generated parser class.

GGerHobbelt
69% match
Loading thumbnail…

OP file, compilable by jison-gho. SO: https://stackoverflow.com/posts/58902159

This example demonstrates a Jison-gho lexer and parser grammar for a custom GIF-like code language, converting token sequences into an abstract syntax tree. The lexer uses state management (`begin`, `popState`, `delimit` and `string_literal` states) to enforce semicolon delimiters between tokens and handle quoted strings. The parser defines operator precedence with `%left` declarations and builds AST nodes via `new yy.Stmt` and `new yy.Expr` constructors, processing number literals, identifiers, and control flow keywords. The grammar handles numeric formats and string concatenation rules visible in rules like `UFloat` and `AlfanumAtom`.

GGerHobbelt
66% match
Loading thumbnail…

snip&snap extracts from our major JISON grammar file, showcasing 'code sections' a la BISON plus a few other bits & tricks. Note the %{ ... %} sections which are JISON's 'code sections'. Also note the code following that last '%%' marker: that is another 'code section' - and the most important one.

This example presents a JISON grammar file for a formula parser, highlighting its use of embedded code sections marked by `%{ ... %}`, a feature inherited from BISON. It demonstrates a lexical hack where a dotted identifier regex includes a look-ahead for an open parenthesis to recognize function tokens, using the parser's symbol table lookup. The code also documents how JISON transforms lexer regex macros and how the `this.unput` and `this.matches` APIs manage pushed-back input. The grammar is rendered as a raw source file from a gist.

GGerHobbelt
65% match
Loading thumbnail…

grammar pseudo language

This visualization shows the grammar rules of a custom programming pseudo-language as defined in a Jison specification file. It uses Jison’s `%lex` and `%ebnf` sections to declare lexical tokens (e.g., `NUMBER`, `PRINTLN`) and a grammar for mathematical expressions, functions, assignments, and print statements. The example includes semantic actions in JavaScript that return computed values or generated code strings, and a custom `parser.main` function tests two sample inputs by calling `parser.parse()`. The source is a GitHub gist by GerHobbelt.

GGerHobbelt
64% match
Loading thumbnail…

Lexing and Parsing in JavaScript

This example demonstrates a live lexer and recursive descent parser for a custom wiki-like markup language, converting user input from a textarea into HTML output. The code uses d3.v2 to manage DOM selection and event handling, registering callbacks that recompile the input on changes. It showcases the parsing pipeline by processing the typed text in real time, with a table tracking events fired on the html, body, and textarea elements. The parser applies grammar rules in precedence order, prioritizing robustness over performance by assuming limited input size.

GGerHobbelt
52% match
Loading thumbnail…

This does not lead to a good design. BAD BAD BAD

This example contrasts rule-based token transformation with PEG parsing logic, showing how history-dependent lookbehind rules produce different outputs than future-looking parsing. The code defines transformation rules using a custom `history` operator, replacing characters like ‘x’ after ‘zz’ and ‘z’ after single ‘z’, iterating through the input stepwise. It demonstrates the distinction between processing tokens based on preceding context versus PEG’s lookahead predicates, highlighting limitations in handling left recursion and ambiguous histories. The visualization uses a simple grammar specification with action blocks like `replace_with` and `copy`, rendered as text-based pseudocode in the README.

GGerHobbelt
51% match
Loading thumbnail…

Parser

This example presents a recursive descent parser for a simple grammar of arithmetic-like expressions using the operators "+" and "/" to represent horizontal and vertical composition. The code defines a tokenizer and parsing functions for factors, terms, and expressions, building a nested tree structure with orientation and children properties. It includes a comprehensive test suite that verifies parsing results for various input strings, including precedence, parentheses, and multi-character names. The parser runs entirely in the browser console, using no external libraries.

CCurran Kelleher
44% match