Fix parsing of ClickHouse transformers (APPLY/EXCEPT/EXCLUDE/REPLACE) after '*' - #2653
Merged
manticore-projects merged 1 commit intoSep 20, 2026
Conversation
AllColumns modelled the trailing modifiers itself and only accepted a single EXCEPT/EXCLUDE followed by a single REPLACE, so '* APPLY(sum)' was parsed as a wildcard carrying the alias "APPLY(sum)" — a wrong AST that round-trips without error. AllColumns now holds a List<ColumnsTransformer>, mirroring ColumnsExpression, and the production loops over ColumnsTransformer(), so the modifiers may repeat and combine in any order. ColumnsTransformerType gains EXCLUDE (DuckDB), which AllColumns previously carried as a plain string. The transformer lookahead is position specific: reusing ColumnsTransformerAhead() verbatim would turn '* REPLACE (x)' into an alias.
Contributor
|
Thank you very much and welcome to this project! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2636.
What
*is parsed byAllColumns, which modelled the trailing modifiers itself (exceptColumns/replaceExpressions/exceptKeyword) and accepted only a singleEXCEPT/EXCLUDEfollowed by a singleREPLACE.APPLYwas not accepted at all, soparsed as
AllColumnswith the alias"APPLY(sum)"— syntactically fine, round-trips fine, wrong AST. Anything building on the AST sees a wildcard with an alias instead of a transformer.How
AllColumnsnow holds aList<ColumnsTransformer>, mirroringColumnsExpressionfrom #2635, and the production loops overColumnsTransformer(). Chains and any keyword order come for free.ColumnsTransformerTypegainsEXCLUDE(DuckDB), previously carried as a plain string inAllColumns.exceptKeyword.Regression risk I checked
The transformer lookahead could not be reused verbatim.
ColumnsTransformerAhead()returns false for(<K_APPLY>|<K_REPLACE>) AliasColumns(), which is right afterCOLUMNS(...)—COLUMNS('m') APPLY(a, b)is an alias — but wrong after*, where it would turn* REPLACE (x)into an alias.AllColumnsTransformerAhead()keeps the alias reading forAPPLYonly.ClickHouseTestdoes not cover* REPLACE (col), so this was found by reading theAliasColumns()production rather than by a failing test.Deliberately unchanged:
* APPLYand* APPLY(a, b)stay aliases.This also widens the shared
ColumnsTransformer()production:COLUMNS(...) EXCLUDE (...)now parses. That is beyond the issue, tell me if you'd rather gate it.Relation to #2652
#2652 patches
AllColumns.appendTo()for a nullexceptKeywordand addsaddExceptColumn/setExceptColumnstests. Those accessors are the ones this change removes, so the two cannot both land as-is. Under the transformers model the null-keyword state cannot arise — a transformer always knows its own keyword — so #2652's fix is subsumed. Happy to rebase on it if it lands first.Verification
7457 tests, 0 failures, 0 errors, 25 skipped.
spotless:check, checkstyle and PMD pass.New tests in
ClickHouseTest:testAllColumnsApplyIssue2636(fails before the change withexpected: <null> but was: < APPLY(sum)>) andtestAllColumnsChainedTransformersIssue2636(fails before with a parse error atEXCEPT (a)).AI disclosure: I use an AI coding assistant; I reviewed and tested everything here and take responsibility for it.