diff --git a/CHANGELOG.md b/CHANGELOG.md index 317ce7c..c22a108 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,41 +8,78 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +## [3.4.0] - 2026-09-18 + +Findings for the hardcoded-credential rules now report where a credential is +without reproducing what it is. Minor rather than patch: the snippet a finding +carries changes for every consumer that reads it, and the release adds a new +rule-metadata key. + +### Upgrade notes + +No configuration change is required, but findings differ on the first run after +upgrading. + +- **`codeSnippet` content changes for the credential rules.** The field keeps + the assignment target and surrounding syntax where that is unambiguous, as + well as the file and line, and masks the literal. A baseline keyed on exact + snippet text will not match; key on rule ID plus location instead. Rules whose + match is not a credential are unaffected. (#119) +- **The same applies to `detailedReport.content` and `dataflowTrace`.** Both + quote source lines and both are masked on the same terms. (#119) +- **A finding's `description` can also change.** OpenGrep expands metavariables + into a rule's message before returning a result, so a message quoting the + matched value carried it too. Expanded metavariables are masked for the + credential rules. (#119) +- **Masking is deliberately conservative in several visible places.** + `define('SECRET', '...')` masks the constant name along with the value, and + `password: "admin"` hides which default was used. Ambiguous unquoted values + can also mask the rest of a statement or line rather than risk treating part + of the credential as source syntax. Rule ID, file and line still identify the + finding in each case. (#119) + ### Fixed - **A finding's snippet no longer reproduces the value it reports.** A SAST finding's `codeSnippet` is the source line the rule matched. For nearly every rule that line is the code the finding is about; for the hardcoded-credential rules it contains the credential, so the finding carried the value into `.socket.facts.json`, the uploaded facts and the configured notifiers. - Snippets for those rules now keep the assignment target, the syntax, the file - and the line, and mask the literal's contents. This covers 20 rules across all - fifteen bundled language rule sets, not only the Python and JavaScript ones: + Snippets for those rules now keep the assignment target and syntax when safe, + keep the file and line, and mask the literal's contents. This covers 20 rules + across all fifteen bundled language rule sets, not only the Python and + JavaScript ones: `*-hardcoded-secret(s)`, `*-hardcoded-credentials`, `*-hardcoded-password-default`, `*-default-credentials`, `*-plain-text-password`, `*-weak-jwt-secret` and `*-empty-password`. Rules - whose match is not a credential keep their snippets verbatim. -- Every snippet, dataflow-trace step and detailed report, whatever rule produced - it, is now masked of values matching a well-known credential format: AWS key + whose match is logic keep their snippets verbatim, and a complete assigned + call with a literal argument is treated as code, so + `user.password = request.form.get('password')` keeps its expression while the + quoted argument is masked. (#119) +- Every snippet, dataflow-trace step, rule message and detailed report, whatever + rule produced it, is now masked of values matching a well-known credential + format: AWS key IDs, GitHub tokens, Stripe keys, Slack tokens, Google API keys, npm and PyPI tokens, JWTs, PEM private key bodies, and credentials in a URL authority. A - rule unrelated to secrets can still match a line that carries one. + rule unrelated to secrets can still match a line that carries one. (#119) - TruffleHog's `redactedValue` kept the first and last four characters of any value longer than eight, which left most of a short password readable. Values - under sixteen characters are now masked in full. + under sixteen characters are now masked in full. (#119) - TruffleHog no longer scans the facts file the run writes. That file lands inside the scan target, so a previous run's output was on disk during the walk and its contents were reported as findings of their own, pointing at the - output file rather than the source line. + output file rather than the source line. (#119) ### Changed +- socketdev 3.5.0 -> 3.6.0 in the lockfile. The `>=3.5.0` floor in + `pyproject.toml` is unchanged. (#117) - `load_explicit_env_config` builds its "API key sources detected" debug line by iterating a tuple of variable names rather than a dict of presence booleans. The line is unchanged, including the exclusion of an exported-but-empty - variable. + variable. (#119) ### Added - A `redact` rule-metadata key. Set it on a custom SAST rule to mark the match - as a credential, or to opt a rule out; without it, the rule name decides. + as a credential, or to opt a rule out; without it, the rule name decides. (#119) ## [3.3.0] - 2026-09-15 diff --git a/README.md b/README.md index ee1ca05..8e45e3c 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ jobs: - name: Run Socket Basics # Pin to a commit SHA for supply-chain safety. # Dependabot will keep this up to date automatically — see docs/github-action.md. - uses: SocketDev/socket-basics@ # v3.3.0 + uses: SocketDev/socket-basics@ # v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -179,10 +179,10 @@ For GitHub Actions, see the [Quick Start](#-quick-start---github-actions) above ```bash # Pull the pre-built image (recommended — no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # Run scan -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0 \ +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ diff --git a/action.yml b/action.yml index 626ac6f..0c69b8b 100644 --- a/action.yml +++ b/action.yml @@ -4,7 +4,7 @@ author: "Socket" runs: using: "docker" - image: "docker://ghcr.io/socketdev/socket-basics:3.3.0" + image: "docker://ghcr.io/socketdev/socket-basics:3.4.0" env: # Core GitHub variables (these are automatically available, but we explicitly pass GITHUB_TOKEN) GITHUB_TOKEN: ${{ inputs.github_token }} diff --git a/docs/github-action.md b/docs/github-action.md index ae051aa..3eb44a2 100644 --- a/docs/github-action.md +++ b/docs/github-action.md @@ -43,7 +43,7 @@ jobs: steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -57,7 +57,7 @@ With just your `SOCKET_SECURITY_API_KEY`, all scanning configurations are manage ### How the action is currently built -When you reference `uses: SocketDev/socket-basics@v3.3.0`, GitHub Actions pulls the +When you reference `uses: SocketDev/socket-basics@v3.4.0`, GitHub Actions pulls the pre-built image referenced by [`action.yml`](../action.yml). The historical multi-stage Docker build still matters for maintainers because it determines what lands in the published image: @@ -75,7 +75,7 @@ Socket Basics from source in every workflow run. ### Pre-built image Starting with v2, the action pulls a pre-built image from GHCR rather than -building from source on every run. Pinning to a specific version tag (e.g. `@v3.3.0`) +building from source on every run. Pinning to a specific version tag (e.g. `@v3.4.0`) means the action starts in seconds — the image is built, integration-tested, and published before the release tag is ever created. @@ -85,7 +85,7 @@ If you run socket-basics in other CI systems (Jenkins, GitLab, CircleCI, etc.) o as a standalone `docker run`, pull the pre-built image directly: ```bash -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 ``` See [Local Docker Installation](local-install-docker.md) for usage examples. @@ -101,7 +101,7 @@ is immediately affected. We've seen this happen across the ecosystem: publish `:latest`/`:latest-heavy` Docker aliases as an onboarding convenience, but treat them as exactly that — production pipelines should pin an exact version or digest.) -- **Version tags** (`@v3.3.0`) are better, but tags are mutable by default. +- **Version tags** (`@v3.4.0`) are better, but tags are mutable by default. A tag can be deleted and recreated pointing at a different commit. There are documented cases of this happening — maliciously and accidentally. - **Commit SHAs** are the only truly immutable reference. A SHA cannot be @@ -126,14 +126,14 @@ The only truly immutable reference. Dependabot keeps it current automatically. ```yaml - name: Run Socket Basics # Dependabot keeps this SHA up to date — see .github/dependabot.yml setup below. - uses: SocketDev/socket-basics@ # v3.3.0 + uses: SocketDev/socket-basics@ # v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} ``` Get the SHA for any release: ```bash -git ls-remote https://github.com/SocketDev/socket-basics refs/tags/v3.3.0 +git ls-remote https://github.com/SocketDev/socket-basics refs/tags/v3.4.0 ``` --- @@ -145,7 +145,7 @@ enforces tag protection rules). SHA pinning is still preferable for defence in depth. ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} ``` @@ -166,7 +166,7 @@ updates: ``` Dependabot opens a PR for each new release, updating the SHA or version tag -and keeping the `# v3.3.0` comment in sync. You review, approve, and merge +and keeping the `# v3.4.0` comment in sync. You review, approve, and merge on your own schedule — automated upgrades with a human gate. --- @@ -176,7 +176,7 @@ on your own schedule — automated upgrades with a human gate. | Strategy | Immutable? | Auto-updates | Review gate | |---|---|---|---| | `@v2` floating tag | ❌ (not published) | — | — | -| `@v3.3.0` + Dependabot | ✅ (tag protection enforced) | Yes (weekly PR) | Yes | +| `@v3.4.0` + Dependabot | ✅ (tag protection enforced) | Yes (weekly PR) | Yes | | `@` + Dependabot | ✅ always | Yes (weekly PR) | Yes | ## Basic Configuration @@ -217,7 +217,7 @@ Include these in your workflow's `jobs..permissions` section. **SAST (Static Analysis):** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} # Enable SAST for specific languages @@ -231,7 +231,7 @@ Include these in your workflow's `jobs..permissions` section. **Secret Scanning:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} secret_scanning_enabled: 'true' @@ -251,7 +251,7 @@ Include these in your workflow's `jobs..permissions` section. **Container Scanning:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} # Listing images or Dockerfiles auto-enables the matching Trivy scan. @@ -271,7 +271,7 @@ Include these in your workflow's `jobs..permissions` section. **Socket Tier 1 Reachability:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_tier_1_enabled: 'true' @@ -280,7 +280,7 @@ Include these in your workflow's `jobs..permissions` section. ### Output Configuration ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} python_sast_enabled: 'true' @@ -318,7 +318,7 @@ jobs: fetch-depth: 0 - name: Run Socket Basics (changed files only) - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }} with: @@ -447,7 +447,7 @@ Configure Socket Basics centrally from the [Socket Dashboard](https://socket.dev **Enable in workflow:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -460,7 +460,7 @@ Configure Socket Basics centrally from the [Socket Dashboard](https://socket.dev > [!NOTE] > You can also pass credentials using environment variables instead of the `with:` section: > ```yaml -> - uses: SocketDev/socket-basics@v3.3.0 +> - uses: SocketDev/socket-basics@v3.4.0 > env: > SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_SECURITY_API_KEY }} > with: @@ -478,7 +478,7 @@ All notification integrations require Socket Enterprise. **Slack Notifications:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -490,7 +490,7 @@ All notification integrations require Socket Enterprise. **Jira Issue Creation:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -505,7 +505,7 @@ All notification integrations require Socket Enterprise. **Microsoft Teams:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -517,7 +517,7 @@ All notification integrations require Socket Enterprise. **Generic Webhook:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -529,7 +529,7 @@ All notification integrations require Socket Enterprise. **SIEM Integration:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -565,7 +565,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -607,7 +607,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Full Security Scan - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -666,7 +666,7 @@ jobs: run: docker build -t myapp:${{ github.sha }} . - name: Run Socket Basics (image + Dockerfile scan) - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }} with: @@ -727,7 +727,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -785,7 +785,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -946,7 +946,7 @@ in the [name mapping](parameters.md#name-mapping). ```yaml steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - Must be first - - uses: SocketDev/socket-basics@v3.3.0 + - uses: SocketDev/socket-basics@v3.4.0 ``` ### PR Comments Not Appearing diff --git a/docs/github-pr-comment-guide.md b/docs/github-pr-comment-guide.md index 66a241e..914bb7b 100644 --- a/docs/github-pr-comment-guide.md +++ b/docs/github-pr-comment-guide.md @@ -315,7 +315,7 @@ PR. This is for teams who want to review finding quality in the Socket dashboard first, without every PR growing a comment that developers have to scroll past. ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} github_token: ${{ secrets.GITHUB_TOKEN }} diff --git a/docs/local-install-docker.md b/docs/local-install-docker.md index 30c301c..570aa1f 100644 --- a/docs/local-install-docker.md +++ b/docs/local-install-docker.md @@ -16,7 +16,7 @@ Run Socket Basics locally using Docker without installing security tools on your ```bash # 1. Pull a pinned release from GHCR (no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # 2. Create .env file with your credentials (the API key is environment-only; # the organization can also be passed per run with --socket-org) @@ -29,14 +29,14 @@ EOF docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ --console-tabular-enabled ``` -The Docker image should always be pinned to an exact version such as `3.3.0`. Avoid +The Docker image should always be pinned to an exact version such as `3.4.0`. Avoid floating tags like `:latest` in CI/CD. ## Using Pre-built Images @@ -46,13 +46,13 @@ The baked-in security tool versions are recorded in the image labels so you can inspect exactly what's inside: ```bash -docker inspect ghcr.io/socketdev/socket-basics:3.3.0 \ +docker inspect ghcr.io/socketdev/socket-basics:3.4.0 \ | jq '.[0].Config.Labels' # { # "com.socket.trivy-version": "0.73.0", # "com.socket.trufflehog-version": "3.96.0", # "com.socket.opengrep-version": "v1.26.0", -# "org.opencontainers.image.version": "3.3.0", +# "org.opencontainers.image.version": "3.4.0", # ... # } ``` @@ -90,8 +90,8 @@ tool: `socketcli` runs the Python CLI; `socket-basics`, or any other argument, runs Socket Basics: ```bash -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0-heavy socketcli --help -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0-heavy --workspace /workspace --python +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0-heavy socketcli --help +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0-heavy --workspace /workspace --python ``` `latest` and `latest-heavy` are floating aliases published for onboarding @@ -109,7 +109,7 @@ comments and labels. If you must run the image directly, pin the exact version: -v "$GITHUB_WORKSPACE:/workspace" \ -e SOCKET_SECURITY_API_KEY=${{ secrets.SOCKET_SECURITY_API_KEY }} \ -e SOCKET_ORG=${{ secrets.SOCKET_ORG }} \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -122,7 +122,7 @@ comments and labels. If you must run the image directly, pin the exact version: ```yaml security-scan: image: - name: ghcr.io/socketdev/socket-basics:3.3.0 + name: ghcr.io/socketdev/socket-basics:3.4.0 entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: @@ -141,7 +141,7 @@ security-scan: ```dockerfile # Pin socket-basics and let Dependabot send upgrade PRs automatically -FROM ghcr.io/socketdev/socket-basics:3.3.0 +FROM ghcr.io/socketdev/socket-basics:3.4.0 ``` ### Staying Up to Date with Dependabot @@ -159,7 +159,7 @@ updates: interval: "weekly" ``` -Dependabot will detect the `FROM ghcr.io/socketdev/socket-basics:3.3.0` reference +Dependabot will detect the `FROM ghcr.io/socketdev/socket-basics:3.4.0` reference and open a PR with the version bump when a new release is available. ## Building the Docker Image @@ -170,10 +170,10 @@ Pull a specific release without building locally: ```bash # GHCR (preferred) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # Docker Hub -docker pull socketdev/socket-basics:3.3.0 +docker pull socketdev/socket-basics:3.4.0 ``` ### Build from Source @@ -186,7 +186,7 @@ git clone https://github.com/SocketDev/socket-basics.git cd socket-basics # Build with version tag (multi-stage; first build is slower, subsequent ones are fast) -docker build -t socket-basics:3.3.0 . +docker build -t socket-basics:3.4.0 . # Verify the build docker images | grep socket-basics @@ -195,7 +195,7 @@ docker images | grep socket-basics ### Build for a Specific Platform (M1/M2 Macs) ```bash -docker build --platform linux/amd64 -t socket-basics:3.3.0 . +docker build --platform linux/amd64 -t socket-basics:3.4.0 . ``` ### Build with Custom Tool Versions @@ -206,7 +206,7 @@ The image pins the bundled tools to specific versions. You can override them at docker build \ --build-arg TRUFFLEHOG_VERSION=3.96.0 \ --build-arg OPENGREP_VERSION=v1.26.0 \ - -t socket-basics:3.3.0 . + -t socket-basics:3.4.0 . ``` Trivy comes from a Socket-built image pinned by digest via the `TRIVY_IMAGE` @@ -219,13 +219,13 @@ tests image, build from the `app_tests` directory and use the same build args. ```bash # The image's entrypoint is `socket-basics`, so its own flags need no prefix -docker run --rm socket-basics:3.3.0 --version +docker run --rm socket-basics:3.4.0 --version # Other bundled tools need --entrypoint -docker run --rm --entrypoint socket socket-basics:3.3.0 --version -docker run --rm --entrypoint opengrep socket-basics:3.3.0 --version -docker run --rm --entrypoint trufflehog socket-basics:3.3.0 --version -docker run --rm --entrypoint trivy socket-basics:3.3.0 --version +docker run --rm --entrypoint socket socket-basics:3.4.0 --version +docker run --rm --entrypoint opengrep socket-basics:3.4.0 --version +docker run --rm --entrypoint trufflehog socket-basics:3.4.0 --version +docker run --rm --entrypoint trivy socket-basics:3.4.0 --version ``` ### Smoke Test @@ -260,7 +260,7 @@ Mount your project directory into the container: # Scan current directory docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -277,7 +277,7 @@ docker run --rm \ # Scan a specific project directory docker run --rm \ -v "/path/to/your/project:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --javascript \ --secrets @@ -288,7 +288,7 @@ docker run --rm \ ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages \ --secrets \ @@ -341,7 +341,7 @@ INPUT_VERBOSE=false docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets @@ -356,7 +356,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -e "SOCKET_SECURITY_API_KEY=scrt_your_api_key" \ -e "SOCKET_ORG=your-org-slug" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -378,7 +378,7 @@ docker run --rm \ --env-file .env.socket \ --env-file .env.notifiers \ --env-file .env.scanning \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages ``` @@ -397,7 +397,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -e "SOCKET_SECURITY_API_KEY=$SOCKET_SECURITY_API_KEY" \ -e "SOCKET_ORG=$SOCKET_ORG" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python ``` @@ -424,7 +424,7 @@ docker run --rm \ -e GITHUB_TOKEN \ -e GITHUB_REPOSITORY=owner/repo \ -e GITHUB_PR_NUMBER=123 \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python --javascript --secrets ``` @@ -463,7 +463,7 @@ mkdir -p ./scan-results docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -488,7 +488,7 @@ docker run --rm -it \ -v "$PWD:/workspace" \ --env-file .env \ --entrypoint /bin/bash \ - socket-basics:3.3.0 + socket-basics:3.4.0 # Inside container, run commands manually: # cd /workspace @@ -517,7 +517,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -v "$PWD/socket-config.json:/config.json" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --config /config.json ``` @@ -541,7 +541,7 @@ for PROJECT in "${PROJECTS[@]}"; do docker run --rm \ -v "$PROJECT:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages \ --secrets \ @@ -584,7 +584,7 @@ pipeline { script { // --entrypoint='' is required: Jenkins runs `cat` to keep the // container alive, and the image's entrypoint is socket-basics. - docker.image('ghcr.io/socketdev/socket-basics:3.3.0').inside( + docker.image('ghcr.io/socketdev/socket-basics:3.4.0').inside( "--entrypoint='' -v ${WORKSPACE}:/workspace --env-file .env" ) { sh ''' @@ -607,7 +607,7 @@ pipeline { ```yaml security-scan: image: - name: ghcr.io/socketdev/socket-basics:3.3.0 + name: ghcr.io/socketdev/socket-basics:3.4.0 entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: @@ -650,7 +650,7 @@ security-scan: ```bash docker run --rm \ -v "$(pwd):/workspace" \ # Use $(pwd) instead of $PWD - socket-basics:3.3.0 + socket-basics:3.4.0 ``` 2. Verify mount (the entrypoint is `socket-basics`, so override it to run `ls`): @@ -658,7 +658,7 @@ security-scan: docker run --rm \ -v "$PWD:/workspace" \ --entrypoint ls \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ -la /workspace ``` @@ -688,7 +688,7 @@ security-scan: docker run --rm \ -v "$PWD:/workspace" \ --env-file "$(pwd)/.env" \ - socket-basics:3.3.0 + socket-basics:3.4.0 ``` ### Container Image Too Large @@ -719,7 +719,7 @@ security-scan: ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -740,7 +740,7 @@ security-scan: ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --output /workspace/results.json # Save to mounted directory ``` @@ -751,7 +751,7 @@ security-scan: mkdir -p ./scan-results docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --output /workspace/scan-results/scan.json ``` @@ -784,7 +784,7 @@ Add these to your `~/.bashrc` or `~/.zshrc` for quick access: ```bash # Socket Basics Docker aliases -alias sb-docker='docker run --rm -v "$PWD:/workspace" --env-file .env ghcr.io/socketdev/socket-basics:3.3.0 --workspace /workspace' +alias sb-docker='docker run --rm -v "$PWD:/workspace" --env-file .env ghcr.io/socketdev/socket-basics:3.4.0 --workspace /workspace' alias sb-quick='sb-docker --secrets --console-tabular-enabled' alias sb-python='sb-docker --python --secrets --console-tabular-enabled' alias sb-js='sb-docker --javascript --secrets --console-tabular-enabled' @@ -810,7 +810,7 @@ sb-all 1. **Use pre-built images** — Pull `ghcr.io/socketdev/socket-basics:` instead of building locally 2. **Use the standard image** — `-heavy` exists for one deployment constraint (see [Image Variants](#image-variants)); it adds nothing to Socket Basics -3. **Pin to a specific version** — Avoid `:latest` in production CI; pin to `3.3.0` and upgrade deliberately +3. **Pin to a specific version** — Avoid `:latest` in production CI; pin to `3.4.0` and upgrade deliberately 4. **Use Dependabot** — Reference the image in your Dockerfile/Compose to get automatic upgrade PRs 5. **Inspect baked-in labels** — Run `docker inspect | jq '.[0].Config.Labels'` to verify tool versions 6. **Use .env files** — Keep credentials out of command history @@ -830,7 +830,7 @@ set -e # Configuration PROJECT_DIR="$(pwd)" RESULTS_DIR="scan-results" # relative to the project: it must stay inside the workspace -IMAGE_NAME="ghcr.io/socketdev/socket-basics:3.3.0" +IMAGE_NAME="ghcr.io/socketdev/socket-basics:3.4.0" ENV_FILE=".env" # Create results directory (add it to .gitignore) diff --git a/docs/parameters.md b/docs/parameters.md index 6e011cf..b06bb13 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -310,7 +310,7 @@ workflow and pass it in yourself: run: echo "ref=$(gh pr view ${{ github.event.issue.number }} --json baseRefName -q .baseRefName)" >> "$GITHUB_OUTPUT" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_BASE_REF: ${{ steps.prbase.outputs.ref }} with: diff --git a/docs/pre-commit-hook.md b/docs/pre-commit-hook.md index c6d5fdd..496ffbe 100644 --- a/docs/pre-commit-hook.md +++ b/docs/pre-commit-hook.md @@ -41,7 +41,7 @@ Best for: Teams wanting consistent environments without installing security tool ```bash # Pull the pre-built image (no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 ``` **2. Create pre-commit hook:** @@ -64,7 +64,7 @@ fi # Run Socket Basics in Docker docker run --rm \ -v "$PWD:/workspace" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -118,7 +118,7 @@ fi # Scope the scan to the staged changes docker run --rm \ -v "$PWD:/workspace" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --changed-files auto \ --python \ @@ -153,7 +153,7 @@ docker run --rm \ -e SOCKET_ORG="$SOCKET_ORG" \ -e SOCKET_SECURITY_API_KEY="$SOCKET_SECURITY_API_KEY" \ -e SLACK_WEBHOOK_URL="$SLACK_WEBHOOK_URL" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -504,7 +504,7 @@ repos: hooks: - id: socket-basics name: Socket Basics Security Scan - entry: docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0 --workspace /workspace --changed-files auto --python --secrets + entry: docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0 --workspace /workspace --changed-files auto --python --secrets language: system pass_filenames: false ``` diff --git a/pyproject.toml b/pyproject.toml index c75006a..9a6f9b7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "socket_basics" -version = "3.3.0" +version = "3.4.0" description = "Socket Basics with integrated SAST, secret scanning, and container analysis" readme = "README.md" requires-python = ">=3.10" diff --git a/socket_basics/__init__.py b/socket_basics/__init__.py index b768586..aa00ec2 100644 --- a/socket_basics/__init__.py +++ b/socket_basics/__init__.py @@ -12,7 +12,7 @@ from .socket_basics import SecurityScanner, main from .core.config import load_config_from_env, Config -__version__ = "3.3.0" +__version__ = "3.4.0" __author__ = "Socket.dev" __email__ = "support@socket.dev" diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 6f69474..47b2f2e 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -104,30 +104,84 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, re.DOTALL, ) -# Quoted string literals, including escaped quotes. Covers the single, double -# and backtick forms the bundled rules match across languages. +# Quoted string literals, including escaped quotes. Covers the single, double, +# backtick and triple-quoted forms the bundled rules match across languages. +# +# The triple-quoted alternatives come first so a terminated block matches as one +# literal with its real body. They do not rescue the unterminated case: the +# engine backtracks to the single-quote alternative, which matches the first two +# quotes of ``\"\"\"`` as an empty string. ``_mask_statement`` guards against that +# spurious match rather than the pattern. _STRING_LITERAL = re.compile( - r"""(?P["'`])(?P(?:\\.|(?!(?P=quote))[^\\])*)(?P=quote)""", + r'(?P"""|\'\'\'|["\'`])(?P(?:\\.|(?!(?P=quote))[^\\])*)(?P=quote)', re.DOTALL, ) # Fallback for unquoted forms such as ``password: hunter2`` in config-style # sources, used only when a credential finding has no string literal to mask. # -# The operator alternation is what keeps a quoted value out of this branch. A -# bare ``[=:]`` stops on the first character of ``:=`` or ``==`` and leaves the -# rest of the operator at the head of the value, which no longer looks quoted, -# so a Go short declaration or a comparison would be starred out whole instead -# of going to the literal pass. ``=`` and ``:`` are matched only where they are -# not part of a longer operator, and a comparison assigns nothing, so it does -# not match here at all. -_UNQUOTED_ASSIGNMENT = re.compile( - r'^(?P[^=:]*(?::=|=(?!=)|:(?!:))\s*)(?P\S.*?)(?P\s*)$' +# Which operator binds decides whether a value reaches the literal pass, and +# three things have to hold at once: +# +# - Only a whole operator counts. Stopping on the first character of ``:=`` or +# ``==`` leaves the rest of it heading the value, which then does not look +# quoted, so a Go short declaration would be starred out whole. A comparison +# assigns nothing and does not match here at all. +# - The last operator before a real literal binds. Taking the colon of +# ``password: str = "..."`` leaves ``str = "..."`` as the value, so an +# annotated declaration -- ordinary Python and TypeScript -- would never +# reach the literal pass. Without a literal, the first operator wins so an +# ``=`` or ``:`` inside an unquoted credential stays in the masked value. +# - An operator inside a string literal is not an operator. The ``:`` in +# ``url = "https://..."`` would otherwise bind and star out the URL. +# +# A single regex cannot express the third, so ``_split_assignment`` walks the +# matches and skips the ones a literal covers. +_ASSIGNMENT_OPERATOR = re.compile(r':=|(?:])=(?!=)|(?"""|\'\'\'|["\'`])', re.IGNORECASE ) +# Interpolation placeholders: f-strings, template literals, shell-style. +_INTERPOLATION = re.compile(r'\$?\{[^}]*\}') + +# Any quote character. Used to find the opening delimiter of a literal that +# never closes, which by definition no literal match can cover. +_QUOTE = re.compile(r'["\'`]') + +# Triple-quote delimiters, counted to detect a block that never closes. +TRIPLE_DOUBLE = chr(34) * 3 +TRIPLE_SINGLE = chr(39) * 3 +_TRIPLE_QUOTE = re.compile(r'\"\"\"|\'\'\'') + # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and # the password-policy rules deliberately do not appear: their snippets are # logic, and masking them would remove the reason the finding was raised. +# +# ``plain-text-password`` does stay, even though the rule it names mostly +# matches password *handling* rather than a literal. One of its patterns is a +# comparison against a hardcoded string, and no ``hardcoded-*`` rule covers that +# shape, so dropping it here is the difference between masking a password and +# publishing one. Keeping the handling snippets readable is the job of the +# expression carve-out in ``redact_literals``, not of this list. _CREDENTIAL_RULE_FRAGMENTS = ( 'hardcoded-secret', 'hardcoded-credential', @@ -169,12 +223,170 @@ def scrub_tokens(text: Any) -> str: return scrubbed +def _split_assignment(code: str, offset: int, in_literal, + literal_start_within) -> 'tuple[str, str, str, str] | None': + """Split a statement at the assignment operator that binds, if it has one. + + Returns ``(head, value, trailing_whitespace, operator)``, where ``head`` + runs through the operator and any space after it. Operators covered by a + string literal are skipped. The last operator before the first real literal + wins; with no literal, the first operator wins so punctuation inside an + unquoted credential cannot be mistaken for syntax. + """ + operators = [] + for match in _ASSIGNMENT_OPERATOR.finditer(code): + if in_literal(offset + match.start()): + continue + operators.append(match) + if not operators: + return None + + literal_at = literal_start_within(offset, offset + len(code)) + before_literal = [ + match for match in operators + if literal_at is not None and offset + match.end() <= literal_at + ] + chosen = before_literal[-1] if before_literal else operators[0] + + rest = code[chosen.end():] + value = rest.lstrip() + if not value: + return None + head = code[:chosen.end()] + rest[:len(rest) - len(value)] + stripped = value.rstrip() + return head, stripped, value[len(stripped):], chosen.group(0) + + +def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, int]]': + """Split code on statement separators, keeping each separator as a piece. + + One operator binds per statement, so a line carrying more than one has to be + handled a statement at a time: in ``a = hunter2; password = x`` the last + operator is the second, which would leave the first value in the head. + """ + pieces: 'list[tuple[str, int]]' = [] + start = 0 + for separator in _STATEMENT_SEPARATOR.finditer(code): + if in_literal(offset + separator.start()): + continue + pieces.append((code[start:separator.start()], offset + start)) + pieces.append((separator.group(0), offset + separator.start())) + start = separator.end() + pieces.append((code[start:], offset + start)) + return pieces + + +def _mask_outside_literals(text: str, offset: int, in_literal, + prefix: int = 0) -> str: + """Mask text except a trusted prefix and recognized literal spans.""" + return ''.join( + char if index < prefix or in_literal(offset + index) else '*' + for index, char in enumerate(text) + ) + + +def _mask_statement(code: str, offset: int, in_literal, literal_opens_at, + literal_start_within) -> 'tuple[str, bool]': + """Mask one statement and report whether it contained an assignment.""" + split = _split_assignment(code, offset, in_literal, literal_start_within) + if not split: + return code, False + head, value, trailing, operator = split + + value_start = offset + len(head) + literal_at = literal_start_within(value_start, value_start + len(value)) + + # A complete code assignment that calls something with a literal argument + # remains readable; the literal pass masks the argument. The literal is + # required because an unquoted config value can otherwise look like a call. + # Colon-delimited values stay conservative for the same reason. + call_with_literal = all(( + operator in ('=', ':='), + literal_at is not None, + _CALL_EXPRESSION.match(value), + )) + if call_with_literal: + return code, True + + # Preserve real literal spans for the literal pass, while masking every + # adjacent character. Keeping the suffix wholesale would expose values such + # as ``abc"decoy"hunter2``; masking the opening quote would instead break a + # multiline literal and leave its continuation unprotected. + opener = _LITERAL_OPENER.match(value) + opens_literal = bool(opener) and literal_opens_at( + value_start + opener.start('quote'), len(opener.group('quote')) + ) + if literal_at is not None: + prefix = opener.start('quote') if opens_literal else 0 + masked_value = _mask_outside_literals( + value, value_start, in_literal, prefix + ) + return f"{head}{masked_value}{trailing}", True + + # With no literal there is no syntax worth guessing at. Mask the whole value + # so embedded operators and call-shaped config values cannot escape. + return f"{head}{'*' * len(value)}{trailing}", True + + +def _mask_code(code: str, offset: int, in_literal, literal_opens_at, + literal_start_within) -> str: + """Mask a line of code without trusting ambiguous statement boundaries.""" + handled = [] + has_assignment = False + for piece, piece_offset in _split_statements(code, offset, in_literal): + if piece == ';': + handled.append((piece, piece_offset, piece, False, True)) + continue + masked, assigned = _mask_statement( + piece, piece_offset, in_literal, literal_opens_at, + literal_start_within + ) + handled.append((piece, piece_offset, masked, assigned, False)) + has_assignment = has_assignment or assigned + + # Once a line contains an assignment, every other semicolon fragment is + # ambiguous: it may be another statement, or it may be part of an unquoted + # config value. Keep the first assignment readable, preserve literal spans + # for the later masking pass, and fail closed on every other fragment. + if has_assignment: + first_assignment = next( + index for index, item in enumerate(handled) if item[3] + ) + return ''.join( + original if separator + else masked if index == first_assignment or not original.strip() + else _mask_outside_literals(original, piece_offset, in_literal) + for index, (original, piece_offset, masked, _, separator) + in enumerate(handled) + ) + return ''.join(masked for _, _, masked, _, _ in handled) + + +def _split_comment(line: str, offset: int, in_literal) -> 'tuple[str, str, str]': + """Split a line into code, comment marker and comment text. + + The marker is kept so the masked line still reads as commented. A marker + inside a string literal is part of the value, not a comment. + """ + for marker in _COMMENT_MARKER.finditer(line): + if in_literal(offset + marker.start()): + continue + return line[:marker.start()], marker.group(0), line[marker.end():] + return line, '', '' + + def redact_literals(text: Any) -> str: """Mask the body of every string literal, keeping the surrounding syntax. ``API_KEY = "sk_live_abc123"`` becomes ``API_KEY = "****************"``: the name, the operator and the line all survive, which is what makes the finding actionable, while the value does not. + + Where the shape of the value is not recognized the whole value is masked + rather than guessed at, so a subscript, a ternary or a prefixed literal + (``f"..."``, ``r'...'``) loses more of the line than a plain assignment + does. That direction is deliberate: the rule ID, file and line still + identify the finding, and the alternative is leaving a credential in place. """ if not isinstance(text, str) or not text: return text if isinstance(text, str) else '' @@ -184,28 +396,98 @@ def _mask_literal(match: 're.Match[str]') -> str: if not body: return match.group(0) quote = match.group('quote') - return f'{quote}{mask_value(body)}{quote}' + if '\n' in body or _INTERPOLATION.search(body): + # A body that spans lines, or that interpolates, is a block of + # content rather than one opaque value. The head-and-tail reveal + # measures the whole thing, so the literal text around a placeholder + # inflates the length and buys a reveal the bare value would not get + # -- ``f"{b}_SuperSecret123!"`` would show ``123!``. Mask every line + # and keep the line breaks, so the snippet still shows where the + # literal starts and ends. + masked = '\n'.join('*' * len(segment) for segment in body.split('\n')) + else: + masked = mask_value(body) + return f'{quote}{masked}{quote}' + + # Literal spans are measured over the whole snippet, not line by line. A + # literal can span lines, and a ``#`` or ``--`` on its second line is part + # of the value; reading it as a comment and starring the rest of that line + # can drop the closing quote, after which the literal pass no longer matches + # and the opening line's credential survives. + # A ``"`` or ``'`` literal cannot hold a raw newline in any language these + # rules cover, so a match that does is not a literal -- it is an unclosed + # quote that paired with a stray one further down, and the span between them + # would hide whatever it covers, including a real assignment on a later + # line. Backticks and triple quotes span lines legitimately and are kept. + spans = [ + match.span() for match in _STRING_LITERAL.finditer(text) + if len(match.group('quote')) > 1 + or match.group('quote') == '`' + or '\n' not in match.group(0) + ] + + def in_literal(position: int) -> bool: + return any(start <= position < end for start, end in spans) + + def literal_start_within(start: int, stop: int): + """Return the first literal opening inside ``[start, stop)``, if any.""" + found = [s for s, _ in spans if start <= s < stop] + return min(found) if found else None + + def literal_opens_at(position: int, delimiter: int) -> bool: + """Report whether a real literal starts at ``position``. + + A value is only deferred to the masking pass when that pass will cover + it. ``\"\"\"secret`` with no closing delimiter still produces a match -- + the first two quotes read as an empty string -- so requiring the span to + hold an opening and a closing delimiter is what separates a literal the + pass can mask from an artifact of the unterminated one. + """ + return any( + start == position and end - start >= 2 * delimiter + for start, end in spans + ) - # Mask unquoted assignments line by line before processing literals. A - # quoted value is left for the literal pass, while an unquoted value is - # masked even if another line or a trailing comment contains quoted text. masked_lines = [] + offset = 0 for line in text.split('\n'): - match = _UNQUOTED_ASSIGNMENT.match(line) - body = match.group('body') if match else '' - if match and not body.lstrip().startswith(('"', "'", '`')): - # If quoted text appears later in an unquoted value, mask the whole - # body. Measuring that combined text could otherwise make a short - # credential eligible for a partial reveal. - masked_body = ( - '*' * len(body) if _STRING_LITERAL.search(body) else mask_value(body) - ) - masked_lines.append( - f"{match.group('head')}{masked_body}{match.group('tail')}" - ) - else: - masked_lines.append(line) - return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) + # The comment comes off first. Everything below reasons about where the + # value ends, and a comment can hold anything the value can -- including + # an operator later in the line than the real one, which would bind and + # leave the credential sitting in the head. + code, marker, comment = _split_comment(line, offset, in_literal) + masked_lines.append( + f"{_mask_code(code, offset, in_literal, literal_opens_at, literal_start_within)}" + f"{marker}{'*' * len(comment)}" + ) + offset += len(line) + 1 + masked = _STRING_LITERAL.sub(_mask_literal, "\n".join(masked_lines)) + + # Everything above depends on knowing where literals begin and end. A + # snippet is a slice of a file, so that is sometimes unknowable, and the two + # ways it happens both read as "a literal opened and never closed": + # + # - a quote no surviving span covers, and + # - an odd number of triple delimiters, whose first two quotes match as an + # empty string while the third pairs with any stray quote further on. + # + # Past such a point every character is inside that literal as far as any + # reader can tell, so it is masked to the end of the snippet rather than to + # the end of its line: the credential is often on a continuation line. + # Every pass above preserves length, so positions still line up. + unreliable = [] + for quote in _QUOTE.finditer(text): + if not in_literal(quote.start()): + unreliable.append(quote.end()) + break + for delimiter in (TRIPLE_DOUBLE, TRIPLE_SINGLE): + found = [m.start() for m in re.finditer(re.escape(delimiter), text)] + if len(found) % 2: + unreliable.append(found[-1] + len(delimiter)) + if unreliable: + kept = min(unreliable) + masked = masked[:kept] + re.sub(r"[^\n]", "*", masked[kept:]) + return masked def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: diff --git a/socket_basics/version.py b/socket_basics/version.py index 88c513e..903a158 100644 --- a/socket_basics/version.py +++ b/socket_basics/version.py @@ -1 +1 @@ -__version__ = "3.3.0" +__version__ = "3.4.0" diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 2862897..07df5e1 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -205,6 +205,179 @@ def test_empty_literals_are_left_as_they_are(self): assert redact_literals('password = ""') == 'password = ""' +class TestOperatorBinding: + """Which operator binds decides whether a value reaches the literal pass.""" + + def test_an_annotated_declaration_keeps_its_syntax(self): + # The colon of the annotation must not win over the assignment: binding + # it leaves `str = "..."` as the value, which is not quoted. + for line, prefix in ( + ('password: str = "SuperSecret123!"', 'password: str = "'), + ('const password: string = "SuperSecret123!";', 'const password: string = "'), + ): + redacted = redact_literals(line) + assert "SuperSecret123!" not in redacted + assert redacted.startswith(prefix), redacted + + def test_an_operator_inside_a_literal_does_not_bind(self): + # The `:` in the URL scheme would otherwise split the line and star out + # the value instead of masking it as a literal. + redacted = redact_literals('url = "https://example.com/a"') + assert redacted.startswith('url = "') + assert redacted.endswith('"') + + def test_a_value_containing_an_operator_is_masked(self): + assert "hunter2" not in redact_literals('password = "a=b:c hunter2"') + + def test_operators_inside_an_unquoted_value_do_not_rebind(self): + for line in ( + "password: hunter2=foo", + "password: hunter2:foo", + "password = hunter2=foo", + ): + redacted = redact_literals(line) + assert "hunter2" not in redacted, redacted + + def test_a_line_with_no_value_after_the_operator_is_left_alone(self): + assert redact_literals("password =") == "password =" + + +class TestTrailingComments: + def test_a_comment_beside_a_masked_value_is_masked_too(self): + """Masking the value alone leaves the plaintext sitting next to it.""" + for line in ( + 'DB_PASSWORD = "x" # real one is hunter2', + "password = get_secret() # real value is hunter2", + "user.password = request.form.get('pw') // was hunter2", + 'password = "SuperSecret123!" -- legacy hunter2', + ): + assert "hunter2" not in redact_literals(line), line + + def test_a_marker_inside_a_literal_is_not_a_comment(self): + # The `#` is part of the URL, so the line is not truncated there. + redacted = redact_literals('password = "a # b"') + assert redacted == 'password = "*****"' + + +class TestStatementsAndComments: + """A line can carry more than the one value the operator search finds.""" + + def test_each_statement_on_a_line_is_masked(self): + # One operator binds per statement. Searching the whole line finds the + # last one and leaves every earlier value sitting in the head. + for line in ( + "a = hunter2; password = x", + "password = hunter2; b = 1", + "user = admin; pwd = hunter2", + ): + assert "hunter2" not in redact_literals(line), line + + def test_a_separator_inside_a_literal_is_part_of_the_value(self): + assert redact_literals('password = "a;b"') == 'password = "***"' + + def test_a_separator_inside_an_unquoted_value_masks_every_fragment(self): + for line in ("password: abc;hunter2", "password: abc;hunter2=foo"): + redacted = redact_literals(line) + assert "hunter2" not in redacted + assert redacted.startswith("password: ***;") + + def test_an_operator_in_a_comment_does_not_bind(self): + # The comment's `=` is later in the line than the real one, so binding + # it would leave the credential in the head. + for line in ( + "password = hunter2 # see x = y", + "password: hunter2 # ratio a:b", + "token = SuperSecret123! # cf. k=v", + ): + redacted = redact_literals(line) + assert "hunter2" not in redacted and "SuperSecret123!" not in redacted, line + + def test_the_comment_marker_survives_so_the_line_still_reads(self): + assert redact_literals("password = hunter2 # note").startswith( + "password = ******* #" + ) + + def test_a_call_shaped_unquoted_value_is_masked(self): + for line in ("password: hunter2(foo)", "password = hunter2(foo)"): + assert "hunter2" not in redact_literals(line), line + + +class TestMultilineAndUnterminatedLiterals: + """Literal spans are a property of the snippet, not of one line. + + A snippet is a slice of a file, so a literal can open on one line and close + on another, or never close at all. + """ + + TRIPLE_DOUBLE = 'PASSWORD = ' + '"' * 3 + 'hunter2\n# not a comment\n' + '"' * 3 + TRIPLE_SINGLE = 'SQL = ' + "'" * 3 + '\nSELECT hunter2 -- inline\n' + "'" * 3 + BACKTICK = 'password = `hunter2\n// js template\n`' + + def test_a_marker_on_a_later_line_of_a_literal_is_not_a_comment(self): + for snippet in (self.TRIPLE_DOUBLE, self.TRIPLE_SINGLE, self.BACKTICK): + assert "hunter2" not in redact_literals(snippet), snippet + + def test_an_unterminated_triple_quote_is_masked(self): + """The spurious empty match must not read as a literal worth deferring. + + ``\"\"\"secret`` with no closing delimiter still produces a match: the + first two quotes parse as an empty string. Treating that as "a literal + the masking pass will cover" leaves the value untouched, because the + pass covers only the two quotes. + """ + for opener in ('"' * 3, "'" * 3, 'r' + '"' * 3, "f" + "'" * 3): + snippet = f"password = {opener}hunter2\nunterminated" + assert "hunter2" not in redact_literals(snippet), snippet + + def test_an_unterminated_literal_is_masked_rather_than_deferred(self): + # A snippet cut mid-string has an opening quote and no closing one, so + # the literal pass never matches it. Deferring would leave it untouched. + for snippet in ('password = "hunter2\n# broken', "password = 'hunter2\n-- sql"): + assert "hunter2" not in redact_literals(snippet), snippet + + def test_a_literal_spanning_lines_keeps_the_line_structure(self): + redacted = redact_literals(self.TRIPLE_DOUBLE) + assert redacted.count("\n") == 2 + assert "hunter2" not in redacted + + +class TestPrefixedAndInterpolatedLiterals: + """A prefix still opens a literal, and an interpolated body is not one value.""" + + Q3 = '"' * 3 + + def test_a_prefixed_multiline_literal_is_masked(self): + # Treating the prefix as unquoted stars the opening line, which removes + # the quotes the rest of the snippet is measured against: later lines + # still read as inside a literal, so nothing masks them. + for prefix in ("r", "f", "rb", "R"): + snippet = f"password = {prefix}{self.Q3}hunter2\nmore SuperSecret123!\n{self.Q3}" + redacted = redact_literals(snippet) + assert "hunter2" not in redacted, snippet + assert "SuperSecret123!" not in redacted, snippet + + def test_a_prefixed_single_line_literal_keeps_its_syntax(self): + redacted = redact_literals("password = r'SuperSecret123!'") + assert "SuperSecret123!" not in redacted + assert redacted.startswith("password = r'") + + def test_text_adjacent_to_a_literal_is_masked_too(self): + for line in ('password: r""hunter2', 'password: abc"decoy"hunter2'): + redacted = redact_literals(line) + assert "hunter2" not in redacted, redacted + + def test_an_interpolated_body_is_masked_whole(self): + # The literal text around a placeholder inflates the body past the + # partial-reveal threshold, which would expose the tail of the value. + for snippet in ( + 'password = f"{b}_SuperSecret123!"', + "password = `${b}_SuperSecret123!`", + ): + redacted = redact_literals(snippet) + assert "SuperSecret123!" not in redacted + assert "123!" not in redacted, redacted + + class TestCredentialRuleSelection: @pytest.mark.parametrize( "rule_id", @@ -217,6 +390,7 @@ class TestCredentialRuleSelection: "python-hardcoded-password-default", "js-default-credentials", "js-weak-jwt-secret", + "python-plain-text-password", ], ) def test_hardcoded_credential_rules_are_selected(self, rule_id): @@ -300,8 +474,6 @@ def test_interpolated_metavariables_are_masked_for_credential_findings(self): def test_known_tokens_are_scrubbed_from_every_message(self): assert AWS_KEY_ID not in redact_message(f"Logged value: {AWS_KEY_ID}") - -class TestRedactMessage: def test_a_short_bound_value_does_not_mangle_the_rest_of_the_message(self): """The replace is by value, so a short one is also an ordinary substring. @@ -343,6 +515,44 @@ def test_a_non_credential_finding_keeps_its_message(self): assert redact_message(message, {"$X": {"abstract_content": "eval"}}) == message +class TestPasswordLogicRules: + """``python-plain-text-password`` matches two shapes and needs both served. + + Its handling patterns assign request input to a password field, where the + expression is the finding. Its comparison pattern can bind a hardcoded + string, and no ``hardcoded-*`` rule covers that shape, so the value has to + be masked here or it is not masked at all. + """ + + def test_a_password_handling_snippet_keeps_its_expression(self): + redacted = redact_snippet( + "user.password = request.form.get('password')", credential_finding=True + ) + assert redacted.startswith("user.password = request.form.get(") + assert redacted.endswith(")") + + def test_a_hardcoded_comparison_value_is_masked(self): + redacted = redact_snippet( + 'if user.password == "hunter2":', credential_finding=True + ) + assert "hunter2" not in redacted + assert redacted.startswith('if user.password == "') + + def test_a_bare_value_with_a_trailing_comment_is_masked_whole(self): + # No call, so this stays on the unquoted path: measuring the value plus + # the comment could otherwise partially reveal a short credential. + redacted = redact_snippet('password: hunter2 # see "notes"', credential_finding=True) + assert "hunter2" not in redacted + assert "notes" not in redacted + + def test_a_literal_argument_to_a_call_is_still_masked(self): + redacted = redact_snippet( + "password = get_secret('default_pw')", credential_finding=True + ) + assert "default_pw" not in redacted + assert redacted.startswith("password = get_secret(") + + class TestRedactDataflowTrace: def test_a_credential_finding_masks_literals_in_its_trace(self): """A trace step is a source line, so it gets the snippet's treatment. diff --git a/tests/test_secret_redaction_fuzz.py b/tests/test_secret_redaction_fuzz.py new file mode 100644 index 0000000..adbdcda --- /dev/null +++ b/tests/test_secret_redaction_fuzz.py @@ -0,0 +1,122 @@ +"""Generated-snippet coverage for credential masking. + +The hand-written cases in ``test_secret_redaction`` assert the shapes someone +thought of. Every gap found in review was a shape nobody thought of, so this +generates them instead: assignment operators, quote styles and prefixes, +terminated and unterminated literals, comments, statement separators, and +nesting, combined at random and checked against one invariant. + + A credential in the input never survives to the output. + +The seed is fixed so a failure is reproducible. Raise ``ITERATIONS`` locally, or +run ``.context/fuzz_redaction.py`` for a longer sweep, when changing masking. +""" + +import random + +from socket_basics.core.utils.redaction import redact_snippet + +ITERATIONS = 20_000 +SEED = 1337 + +SHORT_SECRET = "hunter2" +LONG_SECRET = "Sup3rS3cr3tValue99xyz" + +TARGETS = [ + "password", "api_key", "DB_PASSWORD", "user.password", "apiKey", + "self.password", "PASSWORD", "password: str", "const password: string", + "creds['db']", "x", +] +OPERATORS = ["=", ":", ":=", "==", "!=", " = ", ": "] +QUOTES = ['"', "'", "`", '"""', "'''"] +PREFIXES = ["", "r", "f", "b", "u", "rb", "R", "F"] +COMMENT_MARKERS = ["#", "//", "--"] +COMMENT_BODIES = [ + "note", "see x = y", "ratio a:b", 'a "quoted" note', 'unbalanced " quote', + "cf. k=v", "see get_secret()", "", +] +WRAPPERS = [ + "{value}", "get_secret({value})", "a if b else {value}", + "{value} + other", "other + {value}", "[{value}]", "f({value}, x)", +] +BARE_VALUES = [ + "{secret}", "{secret}=suffix", "prefix:{secret}", "prefix;{secret}", + "prefix;{secret}=suffix", "{secret}(arg)", + 'prefix"decoy"{secret}', 'r""{secret}', +] + + +def _value(rng, secret, allow_bare=True): + # A bare token inside a call argument or a comparison is a variable + # reference in any real language, not a credential, so it is not generated + # there -- it would report a leak that cannot occur in a real snippet. + choices = ["quoted", "quoted", "quoted", "wrapped"] + if allow_bare: + choices.append("bare") + style = rng.choice(choices) + if style == "bare": + return rng.choice(BARE_VALUES).format(secret=secret) + if style == "wrapped": + return rng.choice(WRAPPERS).format(value=_value(rng, secret, allow_bare=False)) + body = secret + if rng.random() < 0.25: + body = "{x}_" + secret # interpolation inflates the length + if rng.random() < 0.2: + # The secret goes on either side of the break: a continuation line is + # exactly where masking that stops at the opener's line loses it. + body = (secret + "\nsecond line" if rng.random() < 0.5 + else "first line\n" + secret) + quote = rng.choice(QUOTES) + terminated = rng.random() < 0.8 + return rng.choice(PREFIXES) + quote + body + (quote if terminated else "") + + +def _statement(rng, secret): + operator = rng.choice(OPERATORS) + comparison = "==" in operator or "!=" in operator + return rng.choice(TARGETS) + operator + _value(rng, secret, allow_bare=not comparison) + + +def _snippet(rng, secret): + parts = [_statement(rng, secret)] + if rng.random() < 0.25: + decoy = rng.choice(["b = 1", "other = x", f"a = {secret}"]) + parts = [parts[0], decoy] if rng.random() < 0.5 else [decoy, parts[0]] + line = "; ".join(parts) + if rng.random() < 0.4: + line += f" {rng.choice(COMMENT_MARKERS)} {rng.choice(COMMENT_BODIES)}" + if rng.random() < 0.15: + line = f"def f():\n {line}\n return None" + if rng.random() < 0.1: + line = " " + line + " " + return line + + +def test_a_generated_credential_never_survives_masking(): + rng = random.Random(SEED) + for _ in range(ITERATIONS): + secret = rng.choice([SHORT_SECRET, LONG_SECRET]) + snippet = _snippet(rng, secret) + masked = redact_snippet(snippet, credential_finding=True) + assert secret not in masked, f"leaked\n in : {snippet!r}\n out: {masked!r}" + + +def test_a_generated_credential_is_never_partly_revealed(): + # The head-and-tail reveal is deliberate for a plain value, but it must not + # expose a recognizable run of a credential that was masked another way. + rng = random.Random(SEED + 1) + for _ in range(ITERATIONS): + snippet = _snippet(rng, LONG_SECRET) + masked = redact_snippet(snippet, credential_finding=True) + for fragment in (LONG_SECRET[:8], LONG_SECRET[-8:]): + assert fragment not in masked, ( + f"partial reveal\n in : {snippet!r}\n out: {masked!r}" + ) + + +def test_generated_non_credential_snippets_are_untouched(): + # Masking must not reach a finding whose match is ordinary code. + rng = random.Random(SEED + 2) + for _ in range(ITERATIONS // 4): + snippet = _snippet(rng, "ordinary_identifier") + assert redact_snippet(snippet, credential_finding=False) == snippet diff --git a/uv.lock b/uv.lock index 21d9722..7ddf171 100644 --- a/uv.lock +++ b/uv.lock @@ -244,7 +244,7 @@ name = "exceptiongroup" version = "1.3.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "typing-extensions" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/0b/9f/a65090624ecf468cdca03533906e7c69ed7588582240cfe7cc9e770b50eb/exceptiongroup-1.3.0.tar.gz", hash = "sha256:b241f5885f560bc56a59ee63ca4c6a8bfa46ae4ad651af316d4e81817bb9fd88", size = 29749, upload-time = "2025-05-10T17:42:51.123Z" } wheels = [ @@ -692,7 +692,7 @@ wheels = [ [[package]] name = "socket-basics" -version = "3.3.0" +version = "3.4.0" source = { editable = "." } dependencies = [ { name = "jsonschema" },