Skip to content

Fix parsing of ClickHouse transformers (APPLY/EXCEPT/EXCLUDE/REPLACE) after '*' - #2653

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
kyy-logs:fix-2636-allcolumns-transformers
Sep 20, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
kyy-logs:fix-2636-allcolumns-transformers

Conversation

@kyy-logs

Copy link
Copy Markdown
Contributor

Fixes #2636.

What

* is parsed by AllColumns, which modelled the trailing modifiers itself (exceptColumns / replaceExpressions / exceptKeyword) and accepted only a single EXCEPT/EXCLUDE followed by a single REPLACE. APPLY was not accepted at all, so

SELECT * APPLY(sum) FROM t

parsed as AllColumns with the alias "APPLY(sum)" — syntactically fine, round-trips fine, wrong AST. Anything building on the AST sees a wildcard with an alias instead of a transformer.

How

AllColumns now holds a List<ColumnsTransformer>, mirroring ColumnsExpression from #2635, and the production loops over ColumnsTransformer(). Chains and any keyword order come for free.

ColumnsTransformerType gains EXCLUDE (DuckDB), previously carried as a plain string in AllColumns.exceptKeyword.

Regression risk I checked

The transformer lookahead could not be reused verbatim. ColumnsTransformerAhead() returns false for (<K_APPLY>|<K_REPLACE>) AliasColumns(), which is right after COLUMNS(...)COLUMNS('m') APPLY(a, b) is an alias — but wrong after *, where it would turn * REPLACE (x) into an alias. AllColumnsTransformerAhead() keeps the alias reading for APPLY only. ClickHouseTest does not cover * REPLACE (col), so this was found by reading the AliasColumns() production rather than by a failing test.

Deliberately unchanged: * APPLY and * APPLY(a, b) stay aliases.

This also widens the shared ColumnsTransformer() production: COLUMNS(...) EXCLUDE (...) now parses. That is beyond the issue, tell me if you'd rather gate it.

Relation to #2652

#2652 patches AllColumns.appendTo() for a null exceptKeyword and adds addExceptColumn / setExceptColumns tests. Those accessors are the ones this change removes, so the two cannot both land as-is. Under the transformers model the null-keyword state cannot arise — a transformer always knows its own keyword — so #2652's fix is subsumed. Happy to rebase on it if it lands first.

Verification

mvn clean test

7457 tests, 0 failures, 0 errors, 25 skipped. spotless:check, checkstyle and PMD pass.

New tests in ClickHouseTest: testAllColumnsApplyIssue2636 (fails before the change with expected: <null> but was: < APPLY(sum)>) and testAllColumnsChainedTransformersIssue2636 (fails before with a parse error at EXCEPT (a)).

AI disclosure: I use an AI coding assistant; I reviewed and tested everything here and take responsibility for it.

AllColumns modelled the trailing modifiers itself and only accepted a single
EXCEPT/EXCLUDE followed by a single REPLACE, so '* APPLY(sum)' was parsed as a
wildcard carrying the alias "APPLY(sum)" — a wrong AST that round-trips without
error.

AllColumns now holds a List<ColumnsTransformer>, mirroring ColumnsExpression,
and the production loops over ColumnsTransformer(), so the modifiers may repeat
and combine in any order. ColumnsTransformerType gains EXCLUDE (DuckDB), which
AllColumns previously carried as a plain string.

The transformer lookahead is position specific: reusing ColumnsTransformerAhead()
verbatim would turn '* REPLACE (x)' into an alias.
@manticore-projects
manticore-projects merged commit 956e109 into JSQLParser:master Sep 20, 2026
10 checks passed
@manticore-projects

Copy link
Copy Markdown
Contributor

Thank you very much and welcome to this project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] 5.4-SNAPSHOT : ClickHouse : asterisk column modifiers fail or mis-parse (APPLY as alias, docs chain, STRICT, EXCEPT regex)

2 participants