Skip to content

fix(data issue): fix dictionary encoded binary read as empty - #373

Open
zhangweilst wants to merge 1 commit into
apache:mainfrom
zhangweilst:binary-dict
Open

zhangweilst wants to merge 1 commit into
apache:mainfrom
zhangweilst:binary-dict

Conversation

@zhangweilst

Copy link
Copy Markdown
Contributor

Purpose

Dictionary encoded binary will be read out empty in that ColumnarUtils::GetView didn't handle Binary type correctly.
This patch fix this data error issue and add tests.

Linked issue: close #xxx

Fix dictionary encoded binary read data issus.

Tests

Added UT in the patch.

API and Format

No

Documentation

No

Generative AI tooling

@lxy-9602

Copy link
Copy Markdown
Member

Thanks a lot for the fix. Could you also create a corresponding issue for this PR to describe the problem?

I’d also like to better understand your use case: are you using your own Parquet reader in your environment? It seems the default Parquet reader would not return a dictionary for the binary type.

Also, there appear to be several places in the current codebase that forcibly assume a string dictionary, which may have similar issues. Would you prefer to address those in this PR as well, or leave them for follow-up PRs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants