Skip to content

[C++][Parquet] int32 offset overflow in dictionary SetDict causes heap out-of-bounds write #51641

Description

@Arawoof06

Describe the bug, including details regarding any error messages, version, and platform.

DictDecoderImpl<ByteArrayType>::SetDict and DictDecoderImpl<FLBAType>::SetDict in cpp/src/parquet/decoder.cc concatenate the decoded dictionary values into byte_array_data_, which is resized to a 64-bit total_size, but they walk it with a 32-bit offset accumulator:

// ByteArray
int32_t offset = 0;
for (int i = 0; i < dictionary_length_; ++i) {
  memcpy(bytes_data + offset, dict_values[i].ptr, dict_values[i].len);
  ...
  offset += dict_values[i].len;
}

// FLBA
for (int32_t i = 0, offset = 0; i < dictionary_length_; ++i, offset += fixed_len) {
  memcpy(bytes_data + offset, dict_values[i].ptr, fixed_len);
  ...
}

When the concatenated dictionary exceeds INT32_MAX bytes, offset wraps negative and memcpy(bytes_data + offset, ...) writes outside the allocated buffer (heap out-of-bounds write). The size is driven entirely by the dictionary page of an untrusted Parquet file: a DICTIONARY_PAGE whose decoded values total more than 2 GB reaches SetDict and triggers the wrap.

total_size is already computed as int64_t, so the accumulator was simply left too narrow. For FLBA the concatenation is addressed directly by index * type_length and can legitimately be larger than 2 GB, so the offset should be 64-bit. For BYTE_ARRAY the values are exposed through int32 offsets (byte_array_offsets_), so a concatenation past INT32_MAX is unrepresentable and should be rejected rather than silently wrapped.

Component(s)

C++, Parquet

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions