Describe the bug, including details regarding any error messages, version, and platform.
DictDecoderImpl<ByteArrayType>::SetDict and DictDecoderImpl<FLBAType>::SetDict in cpp/src/parquet/decoder.cc concatenate the decoded dictionary values into byte_array_data_, which is resized to a 64-bit total_size, but they walk it with a 32-bit offset accumulator:
// ByteArray
int32_t offset = 0;
for (int i = 0; i < dictionary_length_; ++i) {
memcpy(bytes_data + offset, dict_values[i].ptr, dict_values[i].len);
...
offset += dict_values[i].len;
}
// FLBA
for (int32_t i = 0, offset = 0; i < dictionary_length_; ++i, offset += fixed_len) {
memcpy(bytes_data + offset, dict_values[i].ptr, fixed_len);
...
}
When the concatenated dictionary exceeds INT32_MAX bytes, offset wraps negative and memcpy(bytes_data + offset, ...) writes outside the allocated buffer (heap out-of-bounds write). The size is driven entirely by the dictionary page of an untrusted Parquet file: a DICTIONARY_PAGE whose decoded values total more than 2 GB reaches SetDict and triggers the wrap.
total_size is already computed as int64_t, so the accumulator was simply left too narrow. For FLBA the concatenation is addressed directly by index * type_length and can legitimately be larger than 2 GB, so the offset should be 64-bit. For BYTE_ARRAY the values are exposed through int32 offsets (byte_array_offsets_), so a concatenation past INT32_MAX is unrepresentable and should be rejected rather than silently wrapped.
Component(s)
C++, Parquet
Describe the bug, including details regarding any error messages, version, and platform.
DictDecoderImpl<ByteArrayType>::SetDictandDictDecoderImpl<FLBAType>::SetDictincpp/src/parquet/decoder.ccconcatenate the decoded dictionary values intobyte_array_data_, which is resized to a 64-bittotal_size, but they walk it with a 32-bitoffsetaccumulator:When the concatenated dictionary exceeds
INT32_MAXbytes,offsetwraps negative andmemcpy(bytes_data + offset, ...)writes outside the allocated buffer (heap out-of-bounds write). The size is driven entirely by the dictionary page of an untrusted Parquet file: aDICTIONARY_PAGEwhose decoded values total more than 2 GB reachesSetDictand triggers the wrap.total_sizeis already computed asint64_t, so the accumulator was simply left too narrow. For FLBA the concatenation is addressed directly byindex * type_lengthand can legitimately be larger than 2 GB, so the offset should be 64-bit. ForBYTE_ARRAYthe values are exposed through int32 offsets (byte_array_offsets_), so a concatenation pastINT32_MAXis unrepresentable and should be rejected rather than silently wrapped.Component(s)
C++, Parquet