GH-48072: [C++][Acero] fix a bug of materialize boolean - #48073
Conversation
|
|
|
@westonpace PTAL |
kou
left a comment
There was a problem hiding this comment.
+1
Could you rebase on main to run CI?
d2cda53 to
a1a6b03
Compare
Rebase done. |
|
Sorry for my late check... There are CI failures. Could you rebase on main again? |
a1a6b03 to
49a539e
Compare
|
I've rebased this, but it was failing so I had Claude fix up the remaining things, my changes are detailed below. @kou - any more changes required for this? 🤖 : The new test in Changes to the test:
The other failures in the previous CI run ( |
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The offset calculation is correct and the regression test covers values, validity, and non-byte-aligned slices.
Review effort: Balanced
Findings: None
What changed in this PR
Fixes Boolean column materialization for arrays with non-byte-aligned offsets.
Changes:
- Correctly calculates byte and bit offsets for Boolean values.
- Adds regression coverage for sliced, nullable Boolean columns during sorted merge.
| File | Description |
|---|---|
cpp/src/arrow/acero/unmaterialized_table_internal.h |
Corrects bit-packed Boolean access. |
cpp/src/arrow/acero/sorted_merge_node_test.cc |
Tests Boolean materialization across unaligned batches. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
kou
left a comment
There was a problem hiding this comment.
@zanmato1984 Do you want to review this before we merge this?
|
Hi @kou , I'm taking a look. Thanks for letting me know. |
zanmato1984
left a comment
There was a problem hiding this comment.
+1
Thanks for working on this.
Co-authored-by: Rossi Sun <zanmato1984@gmail.com>
974eea2 to
a559083
Compare
|
CI failures are ones on main, so I'll merge |
|
@zanmato1984 Thanks for reviewing this! @thisisnic Thanks for completing this! |
|
Thanks both for looking at this! I'm going to see if there are any other similar PRs that are almost complete but look like they have been forgotten that I can complete with very minimal AI effort to get them over the line. If it starts getting annoying, given I don't quite have the C++ skills to totally understand what I'm producing, just let me know and I can change approach. Will try not to take on any which need more than minor changes anyway. |
Rationale for this change
Use the correct offset when accessing boolean values from the buffer.
What changes are included in this PR?
In
unmaterialized_table_internal.h, use the correct offset when accessing boolean values from the buffer. Also, add a unit test to regress this bug.Are these changes tested?
Yes
Are there any user-facing changes?
No
This PR contains a "Critical Fix".
Yes, fix a bug that caused incorrect data