Report on order items, JSON totals and computed group keys

[roncodes]

Why

Reports on orders couldn't answer basic questions like "Which products sold the most this month?" or "What did orders total this month?". The reporting framework got in the way in several places:

• Every root column ending in _id was hidden as a foreign key, including public_id and internal_id. As a result, Order ID never appeared in the column picker.
• Schema summary columns (Column::count/sum/avg) resolved to a physical column that doesn't exist, e.g. orders.total_orders, so selecting them failed.
• There was no way to declare a column read out of JSON (storefront totals live in orders.meta.total). Custom expressions also rejected JSON_EXTRACT, JSON_UNQUOTE and CAST(… AS DECIMAL(15,2)), and misread DAY, SIGNED and DISTINCT as column names.
• Computed columns couldn't be group keys, filters or sort columns, and a grouped report couldn't be sorted by its aggregates.
• Soft-deleted rows were always counted.
• Grouped reports used a client-supplied aggregateBy.computation without validating it.

What changed

Schema API

• Column::expression($name, $sql, $type): a row-level expression column. Bare names resolve against the table or relationship that declares it. For example, JSON_EXTRACT(meta, '$.quantity') declared on payload.entities reads the joined entity's meta. An expression column can be selected, filtered, sorted, grouped by and aggregated.
• Aggregate computed columns are flagged aggregate (also exposed in toArray()) and resolve to their computation:
  • without grouping, they return a single summary row;
  • with grouping, they sit beside the group keys.
• Table::softDeletes() / Relationship::softDeletes() leave out rows whose deleted_at is set. On joins the filter goes in the ON clause, so LEFT joins keep the parent row.
• public_id and internal_id are no longer treated as foreign keys.
• _key and _import_id are internal bookkeeping columns. They are never listed or selectable, on a table or through a relationship, whatever a schema declares.
• Relationship columns are labelled with the whole relationship name, not just its first word. For example, "Order Config Namespace" used to be "Order Namespace", and "Customer Vendor Name" used to be "Customer Name". The name isn't repeated when the column label already starts with it: "Transaction ID", not "Transaction Transaction ID".

Queries

• Computed columns can be group keys, conditions and sort columns.

• A grouped report can be sorted by an aggregate's alias (sum_payload_entities_quantity).

• New count_distinct aggregate.

• Expressions may use:

  • the JSON functions, DATE() and the CAST target types;
  • DISTINCT, IN, GROUP_CONCAT(… ORDER BY … SEPARATOR …) and ->/->>;
  • INTERVAL n UNIT.

  Cast types and interval units are keywords only where they can't be a column, so orders.time still resolves.

• Aggregate labels use the column label, e.g. "Sum (Quantity)" rather than "Sum (payload.entities.quantity)".

Hardening

• Every computed column is validated up front, including in grouped reports. Names must be safe identifiers, and SELECT is forbidden.
• Schema-declared columns always take their SQL from the registry, never from the request.
• Group keys, aggregate columns, sort columns and condition fields must be allowed columns or computed columns.
• Sort direction is normalised to asc/desc, because it is interpolated into the raw ORDER BY.
• Invalid report shapes fail with a clear message instead of an SQL error, for example a summary column next to per-row columns without grouping, aggregating a summary column, or filtering on one.

Verification

• Registry tests cover the relationship label rules and the system-column exclusion, on tables, relationships, isColumnAllowed and the converter.
• New ReportQueryConverterOrderReportingTest models orders → payload → entities, with JSON totals, soft-deleted rows and a second tenant. It asserts the top-products and monthly-total reports end to end, plus 15 refused report shapes.
• Full suite: 1,817 passing. Every file under src/Support/Reporting is at 100% line coverage. php-cs-fixer is clean. phpstan reports 261 errors for src/Support/Reporting, against 264 on main.
• Ran order reports against the dev MySQL in strict mode (ONLY_FULL_GROUP_BY) with 76 real storefront orders, reading item quantities and order totals out of meta with computed columns: top products, monthly totals, weekly revenue per storefront (a computed group key), and an order list with tracking/customer and a filtered computed total.
• Fliit's existing row-level Column::computed(...) columns, such as hire_days_current_month, now resolve to their expressions. Before this change, selecting them produced a reference to a column that doesn't exist.

Behaviour changes to note

• In a grouped report, a selected column that is neither a group key nor aggregated is now an error. It used to be dropped silently, or fail the same check.
• Condition, group and sort fields that aren't declared in the schema are now rejected by the converter itself as well as by ReportQueryValidator.

Related

• fleetbase/fleetops feature/ember-ui-coverage-campaign uses this in the order report schema (items, tracking, summary columns, soft deletes). It deliberately declares no columns that read keys out of meta, since meta has no fixed shape; reports read those keys with computed columns.
• fleetbase/ember-ui feature/code-coverage-campaign adds computed group keys and sorts, sorting by aggregates, and Count Distinct to the report builder.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (3e1e14a) to head (fd32df4).

Additional details and impacted files

@@             Coverage Diff              @@
##                main      #269    +/-   ##
============================================
  Coverage     100.00%   100.00%            
- Complexity      7410      7492    +82     
============================================
  Files            430       430            
  Lines          24357     24471   +114     
============================================
+ Hits           24357     24471   +114     

Flag	Coverage Δ
backend	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.