Skip to content

feat: codeceptjs lint command with Claude Code hook - #5737

Open
DavertMik wants to merge 1 commit into
4.xfrom
feat/lint-command
Open

DavertMik wants to merge 1 commit into
4.xfrom
feat/lint-command

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

Base branch: 4.x, not 3.x. The spec (plan 016) targets the ESM codebase: 3.x is CommonJS, has no docs/agents.md and no acorn-based lib/command/list.js to mirror, and the sibling init:agent PR (#5735) also targets 4.x.

Summary

Adds codeceptjs lint, a deterministic AST checker (acorn + acorn-walk) for CodeceptJS anti-patterns in tests, page objects and helpers. One engine, two entry points in v1:

npx codeceptjs lint [paths...]      # tests + local include + local helpers from config by default
npx codeceptjs lint --json
npx codeceptjs lint -c path/to/codecept.conf.js
npx codeceptjs lint --hook claude   # Claude Code PreToolUse hook (payload on stdin)
  • lib/lint.js: rule table, parsing (module, retried as script for CommonJS), TypeScript handling, suppression comments, file collection, new-error diff.
  • lib/command/lint.js: CLI output, JSON, exit codes, hook mode.
  • acorn-walk@8.3.5 becomes a direct dependency (it was only transitive).
  • Config: optional lint: { rules, ignore } in codecept.conf.js. Inline: // codeceptjs-lint-disable-line <rule>, // codeceptjs-lint-disable-next-line <rule>. The rule id is required.
  • Docs: docs/lint.md and a short paragraph in docs/agents.md (Essential Setup).

CLI exit codes: 0 no errors (warnings allowed), 1 errors, 2 bad input / parse failure.

Rules

Rule Default Detects
no-fixed-wait error I.wait(<number literal>)
no-sleep error setTimeout / new Promise(r => setTimeout(...)) in a Scenario, hook or page object method
no-only error on CI, warn locally Scenario.only, Feature.only, Data(...).only.Scenario
no-pause error on CI, warn locally pause()
secret-credentials error I.fillField('<password/token/secret/api key>', value) without secret(); process.env.<CREDENTIAL> passed to any I.* call without secret()
await-grab error I.grab*() assigned, returned or passed on without await
no-actor-in-helper error I (including const { I } = inject() and inject().I) inside a class extending Helper
raw-browser-in-test warn I.use*To (matched by method-name pattern) / I.executeScript in a Scenario body

Hook behaviour

  • Only Write, Edit, MultiEdit on .js/.ts/.mjs/.cjs files inside the project (CLAUDE_PROJECT_DIR, else payload cwd) are checked. Files under node_modules and lint.ignore matches are allowed.
  • The resulting file is built in memory (content, or replacements applied to the current file, replace_all honoured, MultiEdit edits in order).
  • The file is linted before and after; the edit is blocked (exit 2, findings on stderr, nothing on stdout) only when there are new error findings. Identity is rule id + whitespace-normalized node source, compared as a multiset, so a second identical I.wait(5) is new.
  • Warnings never block, so the agent can still write a pause() stub.
  • Any internal failure (bad JSON, unparseable result, config load error, unknown --hook agent) exits 0. A parse error is printed as a notice on stderr. Config loading in hook mode captures stdout so e.g. dotenv logs don't leak.

Decisions on the spec's open points

  • Severities exactly as in the rules table; no-pause / no-only read process.env.CI at lint time.
  • no-actor-in-helper flags only I references inside extends Helper (or extends x.Helper) classes; other inject() usage is not flagged.
  • TypeScript: module.stripTypeScriptTypes (strip mode, positions preserved). If unavailable or it throws (enum, namespace, parameter properties), transpile with typescript when installed (dynamic import) and map positions back through its source map, so line numbers stay correct in both paths. Otherwise the file is skipped with a notice. engines is unchanged. Node experimental/deprecation warnings are silenced only for the duration of that call.
  • Additions beyond the spec text:
    • no-sleep in object/class methods only fires when the file is a CodeceptJS file (calls inject(), actor(), Feature or a Scenario/hook). Without this, vendored bundles and app code light up; test/data/app/js/codemirror6.js produced 14 false positives before this guard. Scenario and hook bodies are always checked. Helper classes are excluded.
    • await-grab does not flag I.grab*().then(...), yield, bare statements, or grabs inside Promise.all/allSettled/race/any([...]).
  • Not implemented (per spec): candidate rules from the "left out of v1" list, MCP tool, check integration, running lint in run, Codex/Cursor/OpenCode hooks. init:agent is untouched.

Known false positives

  • await-grab flags return I.grabX() (spec says "returned"). It's legitimate inside a session() callback (test/acceptance/session_test.js:199,210) and in page-object getters whose caller awaits.
  • secret-credentials matches labels like "Password hint" (spec review point 4).

Test plan

  • npx mocha test/unit/command --timeout 20000: 60 passing (34 in lint_test.js): passing + failing fixture per rule asserting rule id and line, suppression, config levels, TS stripping keeps line/column, TS transpile fallback maps lines back, CommonJS retry, file collection from config with ignore, CLI exit codes 0/1/2 and JSON, hook Write/Edit/MultiEdit, pre-existing violations allowed, second identical I.wait(5) blocked, project config respected, unparseable result allowed, stdin end-to-end via the binary, invalid stdin exits 0, CI env toggling.
  • npx eslint and prettier --check clean on changed files.
  • ./bin/codecept.js lint --help; linted test/acceptance, examples, test/data (316 files, no parse failures).
  • TS fallback verified manually with module.stripTypeScriptTypes removed: identical positions.

🤖 Generated with Claude Code

AST-based checker for CodeceptJS anti-patterns in tests, page objects
and helpers: no-fixed-wait, no-sleep, no-only, no-pause,
secret-credentials, await-grab, no-actor-in-helper, raw-browser-in-test.

`--hook claude` reads a PreToolUse payload, lints the file before and
after the edit and blocks only on new error findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant