Skip to content

refactor(serde): move DataFile JSON handling from REST to core - #974

Open
kamcheungting-db wants to merge 3 commits into
apache:mainfrom
kamcheungting-db:datafile-serde-core
Open

kamcheungting-db wants to merge 3 commits into
apache:mainfrom
kamcheungting-db:datafile-serde-core

Conversation

@kamcheungting-db

@kamcheungting-db kamcheungting-db commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Move DataFile JSON serialization and deserialization from the REST catalog into core serde.

This is a mechanical move. It does not intentionally change the JSON format or validation behavior. REST keeps forwarding overloads, so existing REST callers continue to use the same API.

Why

Core FileScanTask serde needs to read and write data files without depending on the REST catalog.

Before:

iceberg::rest::DataFileFromJson(json, specs, schema);

After:

iceberg::DataFileFromJson(json, specs, schema);

Stack

This is 1 of 4:

  1. This PR: move DataFile serde into core
  2. Java-compatible DataFile JSON (fix(serde): make DataFile JSON compatible with Java Iceberg #975)
  3. Core FileScanTask JSON serde (feat(serde): support standalone FileScanTask JSON in core #939)
  4. Strict schema and partition ID parsing (fix(serde): reject invalid schema and partition IDs in JSON #940)

Each PR targets main; later PRs contain the commits from earlier PRs.

Testing

  • Built json_serde_test and rest_catalog_test on the complete stack.
  • Full stack: 142 JSON serde tests passed.
  • Full stack: 332 REST tests passed; 1 existing test skipped.

@kamcheungting-db kamcheungting-db changed the title refactor(serde): move DataFile JSON serde into core refactor(serde): move DataFile JSON handling from REST to core Sep 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant