feat(content-drive): bulk folder duplicate backend (#37062)

[zJaaal]

Backend half of folder duplication in Content Drive (#37062). Bottom of the stack: this PR, then the frontend in #37761 on top.

Status: draft, in progress. Built test first, one story at a time.

• Done: the shared types. BatchFailureReason.PARENT_PERMISSION_DENIED, a broader COVERED_BY_PARENT description that covers both delete and duplicate, the BULK_FOLDER_DUPLICATE_COMPLETED event type, and the request form, with unit tests.
• All five stories built, in CI now. The endpoint and its refusals; each folder duplicated beside its original, holding everything including generic content and archived items, in one transaction per folder; the _copy naming; a named reason for every refused folder, and a folder inside another selected folder skipped; cancellation between folders, recording where the run stopped; the completion push and durable notification to the submitter; a heartbeat for long folders, and no retry of an abandoned run.
• Spec convergence: four gaps closed (where a cancelled run stopped, a malformed path's reason, progress only when it changes, and how long an outcome is kept).
• Review fixes: the AI review's findings are answered on each thread. The latest (5bd342d): when the permission check itself fails, each folder it covered is now reported UNCLASSIFIED with the error, not as permission denied, since a failed check proves nothing about the author's rights. 4501422 fixes two limited-user tests that granted READ only on a parent folder, which does not reach the folder under it.
• Quickstart walk (on a shared test instance): found one bug, fixed in 1c00f7f. A path with no site in front of it, such as /blogs/, came back PROTECTED_FOLDER; only //hostname/ is a site root now, and anything else is reported PATH_NOT_FOUND.
• Not-a-back-end-user refusal: 445b1f2 documents the 401 the endpoint really returns, the same generic role check bulk delete uses, instead of a 403 NOT_ENTITLED nothing emits. openapi.yaml regenerated; Postman checks it.
• Review by @ihoffmann-dot: a request with no body now gets 400 EMPTY_SELECTION instead of a 500 (4a49608), and each submitted folder is resolved without listing its contents (4da0ebe). Bulk delete shares both and is left for a follow-up.
• Review by @dario-daza (4d52a52):
  • A folder inside another selected folder is COVERED_BY_PARENT only when that folder was actually duplicated. If the ancestor is refused or its duplicate rolls back, the child is duplicated on its own. If a cancellation stops the run before the ancestor, both are skipped with no reason.
  • Two overlapping duplicates of the same folder both succeed now. Before, the second one failed on identifier_pkey. A duplicate now waits for any other duplicate landing in the same place, and nothing is ever refused.
  • An abandoned run is notified as interrupted, and no longer as "0 duplicated".
  • An archived menu link's copy stays archived.
  • The second pass compares against what actually landed in the duplicate, instead of re-running the walk's search-index queries for every folder.
  • FR-016, FR-033, FR-037 and the contract are amended to match.
• Follow-up: the limit on very large folders, which the shipped copy already had, is tracked in Bound the recursive folder copy: page a folder's contents, then decide on the transaction #37769.

What the contract pins (specs/37062-folder-copy-backend/contracts/folder-bulk-duplicate-api.md)

• POST /api/v1/assets/folders/_bulkduplicate with site-qualified assetPaths such as //demo.dotcms.com/blogs/alpha/. Each folder is duplicated in place, beside its original, under a name the server derives.
• 202 with {jobId, statusUrl, submitted}. The run is a background job.
• Refusals before any job exists: 400 EMPTY_SELECTION, 400 OVER_MAX_PATHS, and 401 for a caller who is not a back-end user. Unlike bulk delete there is no overlap refusal: a duplicate landing where another is being made waits for it rather than being refused.
• The per-folder outcome and its reasons: PERMISSION_DENIED, PARENT_PERMISSION_DENIED, PATH_NOT_FOUND, PROTECTED_FOLDER, UNCLASSIFIED, and the skip COVERED_BY_PARENT. A skip with no reason means a cancelled run never reached the folder.
• The pushed completion event BULK_FOLDER_DUPLICATE_COMPLETED.
• The ceiling FOLDER_BULK_DUPLICATE_MAX_PATHS (default 50), advertised as folderBulkDuplicate.maxPaths on /api/v1/appconfiguration.

The spec itself is in #37648, merged into main; the amendments from review are in this PR.

🤖 Generated with Claude Code

This PR fixes: #37062

[claude]

Claude finished @zJaaal's task in 6m 4s —— View job

---

Backend code review — bulk folder duplicate (#37062)

Reviewed the full diff (origin/main...HEAD): the two processor classes, the duplicator, the REST/helper/exception layer, the completion listener, and the modified enums, config, i18n, and test wiring.

New issues introduced by this PR

None found. I traced the paths that usually break in a batch/async job and each holds up:

• SQL is parameterized — both pg_advisory_xact_lock(hashtext(?)) (FolderDuplicator:127) and the identifier-table read (FolderDuplicator:259-263) bind every runtime value via addParam; the nosemgrep on the lock is a correct false-positive suppression.
• Every submitted path ends with exactly one outcome — decideUpFront routes each path through a single record(...) branch, and the run loop records one result per remaining folder, so results.size(), and successCount + failedCount + skippedCount, both equal total on every exit (clean, partial, failed, cancelled).
• Coverage is decided after the ancestor actually runs (process:175, recordUnreached:273), so a refused or failed ancestor leaves its descendant to be duplicated on its own rather than silently dropped — matches FR spec intent.
• The duplicate is transactional — duplicate() wraps the walk + complement pass in one LocalTransaction.wrapReturn, so a complement-pass failure rolls the whole duplicate back and leaves nothing partial (FolderDuplicator:85-107).
• @NoRetryPolicy is correct and necessary — duplication is not idempotent; a re-run would create a second set of copies, and the interrupted-notification wording (FolderBulkDuplicateCompletionListener:172-174) correctly avoids telling the author "0 duplicated" when per-folder commits may already have landed.
• Thread-safety holds — counters are Atomic*, results is CopyOnWriteArrayList, total/stoppedAt are volatile; the only cross-thread field the heartbeat touches is the tracker, and lastReportedPercent is written only on the process() thread.
• Test wiring is complete — all five ITs are registered in Junit5Suite1 and the Postman collection is added to config.json, so nothing is silently skipped in CI. Notification keys added to Language.properties cover every branch.

Existing issues (pre-existing, not introduced here)

• The advisory lock and the direct identifier-table read are Postgres-only. This is consistent with the shipped FolderBulkDelete* code and dotCMS's Postgres-only stance, so it's not a regression — noted only for awareness.

Resolved since the prior review round (commit 4d52a52)

• Concurrent-duplicate identifier collision → fixed by the transaction-scoped advisory lock keyed on the landing parent/site (FolderDuplicator:123-130).
• Coverage ordering → ancestorsFirst + coverage-after-ancestor-runs now duplicate a folder on its own when its ancestor was refused or rolled back.
• Archived links reappearing active in the duplicate → archiveTheCopiesOfArchivedLinks re-archives each copy, recursively at every level (FolderDuplicator:199-217).
• Interrupted-run wording → distinct interrupted message + !outcomeRecorded branch (FolderBulkDuplicateCompletionListener:172).
• Second-pass query → reads names/identifiers from the identifier table, off the search index, loading each contentlet only as it is copied (FolderDuplicator:257-267).

Minor observation (non-blocking, likely intentional)

On cancellation, recordUnreached records unreached folders as SKIPPED/COVERED_BY_PARENT and bumps skippedCount, but does not bump processedCount (FolderBulkDuplicateProcessor:270-284), whereas a COVERED_BY_PARENT skip during the normal loop does count as processed (line 191). So after a cancel, getResultMetadata's processed is less than total while the three counts still sum to total. This reads as a deliberate meaning of "processed = folders actually reached," and every number a client needs is derivable from the three counts, so it's not a defect — flagging only so it's a conscious choice.

Verdict: no blocking issues. The prior-round findings are all resolved and I found no new bugs in this diff.

---

🤖 · reviewed origin/main...HEAD

[dotCMS-Machine-User]

✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.

_{approved automatically by dotbot}

[dario-daza]

A few comments on the bulk duplicate backend: two correctness points (a descendant skipped as covered when its ancestor is never duplicated, and concurrent duplicates of the same folder), the notification after an abandoned run, a test suggestion for archived links and non-default-language items, and two non-blocking follow-ups (shared code with bulk delete, repeated queries in the second pass).

[dotCMS-Machine-User]

✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.

_{approved automatically by dotbot}

[github-actions]

🟡 [P2] setenv.sh:326 guard OTel/Pyroscope agent attach

Posted as a general PR comment because the referenced file is not part of this PR's diff.
Original target: dotCMS/src/main/resources/container/tomcat9/bin/setenv.sh:323-326.

Current code:

[ "$OTEL_JAVAAGENT_ENABLED" = "true" ] || return 0
    echo "Adding OpenTelemetry agent to CATALINA_OPTS"
    export CATALINA_OPTS="$CATALINA_OPTS -javaagent:$CATALINA_HOME/otel/opentelemetry-javaagent.jar"

Problem: Missing jar aborts boot; re-sourcing duplicates agent flag.

Fix:

[ "$OTEL_JAVAAGENT_ENABLED" = "true" ] || return 0
    [ -r "$CATALINA_HOME/otel/opentelemetry-javaagent.jar" ] || { echo "WARNING: OTel jar missing"; return 0; }
    case "$CATALINA_OPTS" in *opentelemetry-javaagent.jar*) return 0;; esac
    export CATALINA_OPTS="$CATALINA_OPTS -javaagent:$CATALINA_HOME/otel/opentelemetry-javaagent.jar"

[github-actions]

dotbot code review:

• Reviewer: meta/muse-spark-1.3 (medium)
• Overall: patch is incorrect
• New findings this run: 1
• Prior unresolved dotbot findings still relevant: 0
• Active findings total: 1
• Findings remapped to general PR comments: 1 (missing file map=1)

Opt-in agent wiring works when jars exist; remaining gap is non-blocking robustness for missing jars and duplicate attach.

Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads.

_{reviewed by dotbot · meta/muse-spark-1.3 · medium}

[github-actions]

⚪ [P3] setenv.sh: OTEL/Pyroscope toggles lack the missing-jar guard just added for Glowroot

Posted as a general PR comment because the referenced file is not part of this PR's diff.
Original target: dotCMS/src/main/resources/container/tomcat9/bin/setenv.sh:324-326.

Current code:

add_otel_agent() {
    [ "$OTEL_JAVAAGENT_ENABLED" = "true" ] || return 0
    echo "Adding OpenTelemetry agent to CATALINA_OPTS"
    export CATALINA_OPTS="$CATALINA_OPTS -javaagent:$CATALINA_HOME/otel/opentelemetry-javaagent.jar"

Problem: A -javaagent pointing at a missing jar aborts JVM startup. add_glowroot_agent now guards against that, but add_otel_agent/add_pyroscope_agent don't — a leftover OTEL_JAVAAGENT_ENABLED=true against a dist built before the copy-observability-agents execution kills startup.

Fix:

add_otel_agent() {
    [ "$OTEL_JAVAAGENT_ENABLED" = "true" ] || return 0
    if [ ! -r "$CATALINA_HOME/otel/opentelemetry-javaagent.jar" ]; then
        echo "WARNING: OTEL_JAVAAGENT_ENABLED=true but $CATALINA_HOME/otel/opentelemetry-javaagent.jar is not in this image; starting without OpenTelemetry"
        return 0
    fi
    echo "Adding OpenTelemetry agent to CATALINA_OPTS"
    export CATALINA_OPTS="$CATALINA_OPTS -javaagent:$CATALINA_HOME/otel/opentelemetry-javaagent.jar"

Apply the same check to add_pyroscope_agent.

Assumption: existing images/dist builds predate this patch, so a stale toggle can meet a missing jar.
What to verify: run an image built before this change with OTEL_JAVAAGENT_ENABLED=true — the JVM fails to start.

[github-actions]

dotbot code review:

• Reviewer: ~z-ai/glm-latest (medium)
• Overall: patch is incorrect
• New findings this run: 1
• Prior unresolved dotbot findings still relevant: 0
• Active findings total: 1
• Findings remapped to general PR comments: 1 (missing file map=1)

The agent wiring is opt-in, operator variables correctly override defaults, the removed init-container approach is replaced by an in-image copy guaranteed by the pom, and the compose file's OTEL_/PYROSCOPE_ settings remain valid for the bundled agents. Only a low-severity robustness gap (no missing-jar guard for the new agents, unlike the one added for Glowroot) was identified.

Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads.

_{reviewed by dotbot · ~z-ai/glm-latest · medium}