fix(traffic): correct stale telemt metric names

[rvalitov]

Summary

Two independent upstream telemt changes left this script scraping counter names the engine no longer publishes. The manager therefore displayed zeros where it should have displayed traffic, with no error anywhere — an awk sum over a pattern that matches nothing yields 0, not a failure. A stale metric name is indistinguishable from a genuinely idle proxy.

This PR corrects both, and adds tests/test_metric_names.sh which fails 5 of its 6 assertions before the change.

How these were found

By cross-checking every telemt_* name this script matches against every telemt_* name the engine actually emits (grep -rhoE '"telemt_[a-zA-Z_]+' src/ in the engine repo), rather than by reading the scrape code and assuming it was current.

That sweep is what surfaced change 2, which the more obvious approach misses: change 1 involves names matched as /^name\{/, but the gauges in change 2 are matched as /^name / — with a trailing space. Any audit that greps for telemt_*{ patterns will not see them.

---

Change 1 — the conventional _total suffix on the user octet counters

Commit	ede3314bee356339dca4ca1b378c251ebac15358
Date	2026-08-01
Subject	Fix name metric counter
Link	telemt/telemt@ede3314
Location	src/metrics/render/users.rs
First release	3.5.4

The upstream change, verbatim:

-        "# HELP telemt_user_octets_from_client Per-user bytes received"
+        "# HELP telemt_user_octets_from_client_total Per-user total bytes received"
-    let _ = writeln!(out, "# TYPE telemt_user_octets_from_client counter");
+    let _ = writeln!(out, "# TYPE telemt_user_octets_from_client_total counter");
-                "telemt_user_octets_from_client{{user=\"{}\"}} {}",
+                "telemt_user_octets_from_client_total{{user=\"{}\"}} {}",

The same commit renamed the to_client pair, and added telemt_user_msgs_from_client_total / _to_client_total. This script references no msgs counter, so those need nothing.

Release evidence

git grep at each tag:

tag	..._from_client{	..._from_client_total{
3.5.0	present	–
3.5.1	present	–
3.5.2	present	–
3.5.3	present	–
3.5.4	–	present
3.5.5	–	present
3.5.6	–	present
3.5.7	–	present

A clean switch with no overlap. Note this means the affected range starts at 3.5.4, not 3.5.6 as first reported — the counter has been unreadable since 3.5.4.

The scrape-side defect

The patterns required { immediately after client:

awk '/^telemt_user_octets_from_client\{/{s+=$NF}END{printf "%.0f",s}'

and the per-user form:

awk -v u="$user" '$0 ~ "^telemt_user_octets_from_client\\{.*user=\"" u "\"" {print $NF}'

With _total inserted between client and {, neither can match. Measured on a live engine, seconds apart:

script pattern  in  = 0
script pattern  out = 0
ACTUAL metric   in  = 801502131
ACTUAL metric   out = 39774071544

Consequences: secret list showed TRAFFIC IN/OUT = 0 B for every secret; metrics, traffic, status --json and the per-user breakdowns all reported zero.

Fix

All 34 sites across both syntactic forms now match the _total spelling.

Why the exact _total spelling, not (_total)?

Because the two spellings never coexisted — the same commit removed the old names, so no telemt release ever published both. A tolerant pattern would therefore work today, but it would silently double-count if a future release ever emitted both during a transition. The test pins the exclusive behaviour with a payload containing a legacy-named line and asserts it is not counted.

---

Change 2 — the aggregate connection gauges were deleted

Commit	c07b600acb6bb59762bd96af6ce5b7fa90ec9de1
Date	2026-03-19
Subject	Integration hardening: reconcile main+flow-sec API drift and restore green suite
Link	telemt/telemt@c07b600
First release	predates 3.5.0 — absent from every 3.5.x tag

The upstream change, verbatim (removals only — nothing replaced them):

-    let _ = writeln!(out, "# HELP telemt_connections_current Current active connections");
-    let _ = writeln!(out, "# TYPE telemt_connections_current gauge");
-        "telemt_connections_current {}",
-    let _ = writeln!(out, "# HELP telemt_connections_direct_current Current active direct connections");
-    let _ = writeln!(out, "# TYPE telemt_connections_direct_current gauge");
-        "telemt_connections_direct_current {}",
-    let _ = writeln!(out, "# HELP telemt_connections_me_current Current active middle-end connections");
-    let _ = writeln!(out, "# TYPE telemt_connections_me_current gauge");
-        "telemt_connections_me_current {}",
-        assert!(output.contains("telemt_connections_current 2"));
-        assert!(output.contains("telemt_connections_direct_current 1"));
-        assert!(output.contains("telemt_connections_me_current 1"));

Release evidence

telemt_connections_current is absent at every tag from 3.5.0 through 3.5.7. Whatever engine version this script supports, these three have never existed for it.

The scrape-side defect

show_connections() and show_metrics() read them with a trailing space:

/^telemt_connections_current /         { total=$NF }
/^telemt_connections_me_current /      { c_me  = $NF }
/^telemt_connections_direct_current /  { c_dir = $NF }

Because the removed names are matched by space rather than {, a sweep of telemt_*{ patterns does not find them — which is why an earlier audit of this same code reported only the two octet counters as drifted.

Consequences: mtproxymax connections printed Total active: 0 regardless of load, and the metrics view carried a permanent (ME: 0 direct: 0).

Fix

• The aggregate is now summed from telemt_user_connections_current, which the engine still publishes. This needs no assumption about a replacement name: it is the same quantity (active connections), just summed per user.
• The ME/direct breakdown is dropped. telemt removed those gauges outright and exposes no per-transport current-connection metric, so continuing to print (ME: 0 direct: 0) would assert a number that was never measured. Happy to keep a placeholder instead if you would rather the line stay visually stable.

---

Testing

New tests/test_metric_names.sh drives get_proxy_stats, get_user_stats and show_connections against a payload shaped like a real scrape.

Before this change — 5 of 6 assertions fail:

FAIL  global totals come from the _total counters (got=0 0 12 want=801502231 39774071744 12)
FAIL  per-user totals come from the _total counters (got=0 0 7 want=801502131 39774071544 7)
FAIL  active connection total is summed from the per-user gauge (got=0 want=12)
FAIL  no pattern still reads the deleted aggregate gauges (got=4 want=0)
FAIL  a legacy-named line is not also counted (got=999999 want=801502231)

After — 6 of 6 pass. Note the first line: the traffic columns are 0 0, which is exactly the production symptom.

The test also carries both upstream commits as comments with links and dates, so whoever next sees a metric name go stale has the history in front of them instead of rediscovering it.

Verified on Debian 12, Ubuntu 22.04, Ubuntu 24.04, Alpine 3.20 and Fedora 41.