Source Code Reading: OQL Parser — From Text to Execution Plan

“

Series: S9 Source Code Reading · Article 6 | Level: Advanced | Reading Time: 25 min

#TL;DR

OQL (Ontology Query Language) is coomia-dip's custom query language, with a fully hand-written parser — no ANTLR or JavaCC. The parsing pipeline has three layers: OQLParserService (entry facade) -> OQLLexer (lexical analysis) -> OQLParser (recursive descent syntax analysis), outputting an OQLQuery AST. The lexer supports 30+ keywords, 6 token types (identifier, string, integer, float, operator, parameter placeholder), and the parser implements a complete SELECT-FROM-WHERE-GROUP BY-ORDER BY-LIMIT grammar extended with three domain-specific functions: SIMILAR_TO (vector search), CONNECTED_TO (graph traversal), and METRIC (metric reference). This article dissects the lexer state machine, recursive descent production rules, AST Record model design, and error recovery strategy.

#Table of Contents

Three-Layer Parsing Architecture
Entry Facade: OQLParserService
Lexical Analyzer: OQLLexer
Token Types and Keyword Table
String Literals and Escape Handling
Number Recognition: Integer vs Float
Multi-Character Lookahead for Operators
Comment Skipping: Line and Block Comments
Recursive Descent Parser: OQLParser
AST Model: OQLQuery Record
Key Takeaways

#1. Three-Layer Parsing Architecture

Code

OQL Text -> OQLParserService -> OQLLexer -> List<TokenInfo>
                                              |
                               OQLParser -> OQLQuery (AST)
                                              |
                               QueryOptimizer -> PhysicalPlan

Code

data-Layer/src/main/java/com/onto/data/query/
├── parser/
│   ├── OQLParserService.java    # Entry facade: validation + assembly
│   ├── OQLLexer.java            # Hand-written lexer
│   ├── OQLParser.java           # Recursive descent parser
│   └── OQLToken.java            # Token enum
├── ast/
│   ├── OQLQuery.java            # AST root (Record)
│   ├── SelectClause.java        # SELECT clause
│   ├── FromClause.java          # FROM clause
│   ├── Projection.java          # Projection (field/aggregate/similarity/metric)
│   └── ...
├── ast/condition/
│   ├── Condition.java           # Condition interface
│   ├── ComparisonCondition.java # Comparison (=, !=, >, <, >=, <=)
│   ├── LogicalCondition.java    # Logical (AND, OR, NOT)
│   ├── GraphCondition.java      # CONNECTED_TO graph condition
│   └── VectorCondition.java     # SIMILAR_TO vector condition
└── exception/
    ├── OQLSyntaxException.java  # Syntax error (with line/column)
    └── OQLSemanticException.java # Semantic error

#2. Entry Facade: OQLParserService

Java

@ApplicationScoped
public class OQLParserService {
    private static final int MAX_QUERY_LENGTH = 10000;

    public OQLQuery parse(String oql) {
        validateInput(oql);                   // 1. Input validation
        var lexer = new OQLLexer(oql);
        var tokens = lexer.tokenize();         // 2. Lexical analysis
        var parser = new OQLParser(tokens);
        return parser.parse();                 // 3. Syntax analysis
    }
}

Facade pattern value: OQLParserService hides the two-step "first Lexer, then Parser" detail. Callers need only one line: parser.parse(oql). MAX_QUERY_LENGTH = 10000 prevents OOM and ReDoS attacks from excessively long queries.

#3. Lexical Analyzer: OQLLexer

Java

public class OQLLexer {
    private final String input;
    private int pos;
    private int line;
    private int column;

    public List<TokenInfo> tokenize() {
        var tokens = new ArrayList<TokenInfo>();
        while (pos < input.length()) {
            skipWhitespaceAndComments();
            if (pos >= input.length()) break;
            var token = nextToken();
            if (token != null) tokens.add(token);
        }
        tokens.add(new TokenInfo(OQLToken.EOF, "", line, column));
        return tokens;
    }
}

State machine design: Three state variables — pos (position), line (line number), column (column number) — track scanning progress. Every token records its precise line and column position, enabling precise syntax error location.

#3.1 Token Dispatcher

Java

private TokenInfo nextToken() {
    char c = current();
    if (c == '\'')                              return readString(...);
    if (Character.isDigit(c) || (c == '-'...))  return readNumber(...);
    if (Character.isLetter(c) || c == '_')      return readIdentifier(...);
    if (c == '$')                               return readParameter(...);
    return readOperatorOrSymbol(...);
}

Five-way dispatch based on first character type: single quote -> string, digit/minus -> number, letter/underscore -> identifier/keyword, $ -> parameter placeholder, other -> operator/symbol.

#4. Token Types and Keyword Table

Java

private static final Map<String, OQLToken> KEYWORDS = Map.ofEntries(
    Map.entry("SELECT", OQLToken.SELECT),
    Map.entry("FROM", OQLToken.FROM),
    Map.entry("WHERE", OQLToken.WHERE),
    Map.entry("AND", OQLToken.AND),
    Map.entry("OR", OQLToken.OR),
    // ... 30+ keywords
    Map.entry("CONNECTED_TO", OQLToken.CONNECTED_TO),
    Map.entry("SIMILARITY", OQLToken.SIMILARITY),
    Map.entry("METRIC", OQLToken.METRIC),
    Map.entry("GRAPH_TABLE", OQLToken.GRAPH_TABLE)
);

Keyword recognition strategy: Read the complete identifier first, then look up the uppercase form in the KEYWORDS Map. This means OQL is case-insensitive — select, SELECT, and Select are all valid.

Three domain extension keyword groups:

Vector search: SIMILARITY
Graph traversal: CONNECTED_TO, MATCH, SHORTEST_PATH, GRAPH_TABLE
Metric reference: METRIC

#5. String Literals and Escape Handling

Java

private TokenInfo readString(int startLine, int startColumn) {
    advance(); // skip opening quote
    var sb = new StringBuilder();
    while (pos < input.length()) {
        char c = current();
        if (c == '\'') {
            if (pos + 1 < input.length() && input.charAt(pos + 1) == '\'') {
                sb.append('\'');  // Escaped quote: '' -> '
                advance(); advance();
            } else {
                advance(); break;  // End of string
            }
        } else {
            sb.append(c); advance();
        }
    }
    return new TokenInfo(OQLToken.STRING, sb.toString(), startLine, startColumn);
}

SQL-style escaping: OQL uses '' (two single quotes) to represent a literal single quote, consistent with the SQL standard. For example, 'O''Brien' parses to the string O'Brien.

#6. Number Recognition: Integer vs Float

Java

private TokenInfo readNumber(int startLine, int startColumn) {
    // ... read digits
    if (current() == '.' && Character.isDigit(input.charAt(pos + 1))) {
        // Has decimal point followed by digit -> NUMBER (float)
        return new TokenInfo(OQLToken.NUMBER, ...);
    }
    return new TokenInfo(OQLToken.INTEGER, ...);
}

INTEGER vs NUMBER: The lexer distinguishes integers from floating-point numbers at the token level — 42 is INTEGER, 3.14 is NUMBER. This provides a foundation for type inference in subsequent parsing.

Lookahead check: The condition current() == '.' && Character.isDigit(input.charAt(pos + 1)) ensures a . must be followed by a digit to count as a decimal point — otherwise . in 42.field would be misidentified.

#7. Multi-Character Lookahead for Operators

Java

private TokenInfo readOperatorOrSymbol(int startLine, int startColumn) {
    char c = current();
    switch (c) {
        case '<':
            if (peek() == '=') { return tokenInfo(OQLToken.LTE, "<="); }
            if (peek() == '>') { return tokenInfo(OQLToken.NEQ, "<>"); }
            return tokenInfo(OQLToken.LT, "<");
        case '.':
            if (peek() == '.') { return tokenInfo(OQLToken.DOTDOT, ".."); }
            return tokenInfo(OQLToken.DOT, ".");
    }
}

One-character lookahead: < could be < (LT), <= (LTE), or <> (NEQ), requiring one character of lookahead to determine. Similarly . could be property access (DOT) or range operator (DOTDOT ..).

#8. Comment Skipping: Line and Block Comments

Java

private void skipWhitespaceAndComments() {
    // Line comment: -- to end of line
    if (c == '-' && input.charAt(pos + 1) == '-') {
        while (pos < input.length() && current() != '\n') pos++;
        continue;
    }
    // Block comment: /* ... */
    if (c == '/' && input.charAt(pos + 1) == '*') {
        pos += 2;
        while (pos + 1 < input.length()) {
            if (current() == '*' && input.charAt(pos + 1) == '/') { pos += 2; break; }
            pos++;
        }
        continue;
    }
}

OQL supports SQL-style comments: -- (line) and /* */ (block).

#9. Recursive Descent Parser: OQLParser

#9.1 Top-Level Grammar

Java

public OQLQuery parse() {
    var builder = OQLQuery.builder();
    builder.select(parseSelectClause());    // SELECT (required)
    builder.from(parseFromClause());        // FROM (required)
    if (check(OQLToken.WHERE))   builder.where(parseWhereClause());
    if (check(OQLToken.GROUP))   builder.groupBy(parseGroupByClause());
    if (check(OQLToken.ORDER))   builder.orderBy(parseOrderByClause());
    if (check(OQLToken.LIMIT))   builder.limit(parseLimitClause());
    expect(OQLToken.EOF, "end of query");
    return builder.build();
}

#9.2 Condition Parsing: Priority Hierarchy

Code

condition       -> parseOrCondition()
orCondition     -> andCondition (OR andCondition)*
andCondition    -> notCondition (AND notCondition)*
notCondition    -> NOT? primaryCondition
primaryCondition -> LPAREN condition RPAREN
                  | HAS relationCondition
                  | CONNECTED_TO graphCondition
                  | SIMILARITY vectorCondition
                  | field IS [NOT] NULL
                  | field [NOT] IN (values)
                  | field LIKE pattern
                  | field op value

Priority through call hierarchy: OR < AND < NOT < primary conditions. For example, a = 1 AND b = 2 OR c = 3 is parsed as (a = 1 AND b = 2) OR c = 3.

#9.3 Projection Parsing: Four Types

Java

private Projection parseProjection() {
    if (check(OQLToken.STAR))           return Projection.wildcard();
    if (isAggregateFunction(current())) return parseAggregateProjection();
    if (check(OQLToken.SIMILARITY))     return parseSimilarityProjection();
    if (check(OQLToken.METRIC))         return parseMetricProjection();
    return Projection.field(parseFieldPath(), alias);
}

Type	Example	Method
Wildcard	`*`	`Projection.wildcard()`
Aggregate	`COUNT(*)`, `SUM(amount)`	`parseAggregateProjection()`
Similarity	`SIMILARITY(embedding, [0.1, 0.2])`	`parseSimilarityProjection()`
Metric	`METRIC('revenue') AS rev`	`parseMetricProjection()`
Field	`name`, `attributes.role`	`Projection.field()`

#9.4 Parameterized Queries

Java

case PARAMETER -> new ParameterPlaceholder(token.value());

public record ParameterPlaceholder(String name) {}

$paramName syntax supports parameterized queries, preventing OQL injection. ParameterPlaceholder is replaced with actual values during execution.

#10. AST Model: OQLQuery Record

Java

public record OQLQuery(
    SelectClause select,
    FromClause from,
    Optional<WhereClause> where,
    Optional<GroupByClause> groupBy,
    Optional<OrderByClause> orderBy,
    Optional<LimitClause> limit
) {
    public boolean isAggregateQuery() {
        return select.hasAggregation() || groupBy.isPresent();
    }

    public OQLQuery withSelect(SelectClause newSelect) {
        return new OQLQuery(newSelect, from, where, groupBy, orderBy, limit);
    }
}

Record immutability: Java Record ensures the AST is immutable once constructed. withSelect() returns a new AST instance — this is functional immutable update pattern, used by MetricProjectionRewriter and other query rewriting scenarios.

Aggregate query detection: isAggregateQuery() checks for aggregate functions in SELECT or GROUP BY presence — this affects downstream optimization and caching strategies.

#11. Key Takeaways

Hand-written lexer: No ANTLR/JavaCC dependency; OQLLexer uses state machine pattern scanning character-by-character with 30+ keywords and precise line/column tracking
Recursive descent parser: OQLParser implements operator precedence through call hierarchy (OR < AND < NOT < primary), no precedence table needed
Case insensitive: Identifiers are uppercased before keyword table lookup; select and SELECT are equivalent
INTEGER vs NUMBER: Token-level distinction between integers and floats, providing foundation for type inference
SQL-style comments: Supports -- (line) and /* */ (block) comments
Parameterized queries: $paramName syntax generates ParameterPlaceholder, preventing OQL injection
Immutable AST: Java Record + Optional combination; withSelect() enables functional updates
Domain extensions: SIMILAR_TO, CONNECTED_TO, METRIC — three domain-specific functions extending standard SQL grammar

#Next Article

S9-07: AnalyticsQueryService — 14 Aggregation Implementations. We will dive into Data Layer's analytics query service, examining how it implements grouped aggregation, time-bucket aggregation, TopN, and distribution queries through the SQL Builder pattern.

Tags: #coomia-dip #source-code-reading #data-Layer #oql #lexer #parser #ast #recursive-descent #query-language