Skip to content

GH-48072: [C++][Acero] fix a bug of materialize boolean - #48073

Merged
thisisnic merged 4 commits into
apache:mainfrom
waruto210:fix/materialize_boolean
Sep 30, 2026
Merged

thisisnic merged 4 commits into
apache:mainfrom
waruto210:fix/materialize_boolean

Conversation

@waruto210

@waruto210 waruto210 commented Nov 6, 2025 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Use the correct offset when accessing boolean values from the buffer.

What changes are included in this PR?

In unmaterialized_table_internal.h, use the correct offset when accessing boolean values from the buffer. Also, add a unit test to regress this bug.

Are these changes tested?

Yes

Are there any user-facing changes?

No

This PR contains a "Critical Fix".

Yes, fix a bug that caused incorrect data

@waruto210
waruto210 requested a review from westonpace as a code owner November 6, 2025 05:43
@github-actions github-actions Bot added the awaiting review Awaiting review label Nov 6, 2025
@github-actions

github-actions Bot commented Nov 6, 2025

Copy link
Copy Markdown

⚠️ GitHub issue #48072 has been automatically assigned in GitHub to PR creator.

@waruto210

Copy link
Copy Markdown
Contributor Author

@westonpace PTAL

@kou kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Could you rebase on main to run CI?

@waruto210
waruto210 force-pushed the fix/materialize_boolean branch from d2cda53 to a1a6b03 Compare January 15, 2026 10:12
@waruto210

Copy link
Copy Markdown
Contributor Author

+1

Could you rebase on main to run CI?

Rebase done.

@kou

kou commented Feb 12, 2026

Copy link
Copy Markdown
Member

Sorry for my late check... There are CI failures. Could you rebase on main again?

@thisisnic

thisisnic commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

I've rebased this, but it was failing so I had Claude fix up the remaining things, my changes are detailed below.

@kou - any more changes required for this?


🤖 :

The new test in sorted_merge_node_test.cc didn't compile on builds using -Werror, because of an implicit int64_t to int32_t conversion and the unused true_count and false_count variables. The fix in unmaterialized_table_internal.h is unchanged.

Changes to the test:

  • Added a static_cast and removed the unused counters
  • Changed the batch size from 1024 to 1001, so that batches start at offsets which aren't byte-aligned
  • Added nulls to the boolean column, and a check that they come through the merge correctly
  • Removed the unused arrow/dataset includes and added <limits>
  • Fixed a stale comment and failure message, and renamed the helper lambda to make_source

The other failures in the previous CI run (parquet-reader-test on MinGW, an S3 test timeout on macOS, and the macOS GLib build) were also failing on main, so they aren't caused by this PR.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The offset calculation is correct and the regression test covers values, validity, and non-byte-aligned slices.

Review effort: Balanced
Findings: None

What changed in this PR

Fixes Boolean column materialization for arrays with non-byte-aligned offsets.

Changes:

  • Correctly calculates byte and bit offsets for Boolean values.
  • Adds regression coverage for sliced, nullable Boolean columns during sorted merge.
File Description
cpp/​src/​arrow/​acero/​unmaterialized_table_internal.h Corrects bit-packed Boolean access.
cpp/​src/​arrow/​acero/​sorted_merge_node_test.cc Tests Boolean materialization across unaligned batches.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@kou kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zanmato1984 Do you want to review this before we merge this?

@zanmato1984

Copy link
Copy Markdown
Contributor

Hi @kou , I'm taking a look. Thanks for letting me know.

Comment thread cpp/src/arrow/acero/unmaterialized_table_internal.h Outdated

@zanmato1984 zanmato1984 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Thanks for working on this.

@github-actions github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Sep 30, 2026
@thisisnic
thisisnic force-pushed the fix/materialize_boolean branch from 974eea2 to a559083 Compare September 30, 2026 15:33
@thisisnic

Copy link
Copy Markdown
Member

CI failures are ones on main, so I'll merge

@thisisnic
thisisnic merged commit fcac3df into apache:main Sep 30, 2026
57 of 59 checks passed
@thisisnic thisisnic removed the awaiting committer review Awaiting committer review label Sep 30, 2026
@kou

kou commented Sep 30, 2026

Copy link
Copy Markdown
Member

@zanmato1984 Thanks for reviewing this!

@thisisnic Thanks for completing this!

@thisisnic

Copy link
Copy Markdown
Member

Thanks both for looking at this! I'm going to see if there are any other similar PRs that are almost complete but look like they have been forgotten that I can complete with very minimal AI effort to get them over the line. If it starts getting annoying, given I don't quite have the C++ skills to totally understand what I'm producing, just let me know and I can change approach. Will try not to take on any which need more than minor changes anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants