get_metadata() drops the pii flag the API returns for each field

[anil-bd]

Summary

get_metadata() discards the pii flag the API returns for each field. DatasetMetadata.from_dict builds every field as a DatasetField with four keys, type, active, required and description, and drops everything else. On datasets of personal data, pii is the field a caller needs most.

Environment

• brightdata-sdk 2.5.2 (latest on PyPI)
• Python 3.12.13
• Verified 18 Sep 2026

Reproduction

from brightdata import SyncBrightDataClient

with SyncBrightDataClient(auto_create_zones=False) as c:
    fields = c.datasets.linkedin_profiles.get_metadata().fields

print([n for n, s in fields.items() if getattr(s, "pii", False)])
print(hasattr(next(iter(fields.values())), "pii"))

[]
False

The same endpoint, read directly:

GET https://api.brightdata.com/datasets/gd_l1viktl72bvl7bjuj0/metadata

pii fields: ['id', 'name', 'about', 'url', 'input_url', 'linkedin_id', 'first_name', 'last_name']

Why it happens

datasets/models.py, DatasetMetadata.from_dict:

fields[name] = DatasetField(
    type=field_data.get("type", "text"),
    active=field_data.get("active", True),
    required=field_data.get("required", False),
    description=field_data.get("description"),
)

pii is never read. Neither are normalize and validation_rules, which the API also returns.

Why it matters

A caller building a data-retention policy, a redaction step, or a field allowlist would reasonably read pii from the schema instead of hard-coding a list. Through the SDK they get no field marked, which reads as "no personal data here" on a dataset of people.

The JavaScript SDK passes the raw field object through, so the flag survives there. The two SDKs disagree on the same endpoint.

Expected

Add pii: bool = False to DatasetField and read it in from_dict. Keeping the raw dict on the model, for keys the SDK does not model yet, would stop the next new key being dropped the same way.