Support channels in v2 sync by midigofrank · Pull Request #1566 · OpenFn/kit

midigofrank · 2026-09-29T17:57:30Z

Short Description

Adds v2 sync support for project channels. Channels are managed locally in a new resources.yaml file at the workspace root, and pulled, merged and deployed along with the rest of the project. resources.yaml is meant to hold other server-side resources too; collections will move there later.

Fixes OFN-4553

Implementation Details

resources.yaml format. Channels live under a channels key. Each channel is keyed by an id (slugified from its name on pull) and has a required name. There are no uuids, and credentials are referenced by name (owner|name), the same way steps reference them:

channels:
  my-channel:
    name: My Channel
    destination_url: https://example.com/hook
    enabled: true
    credential: jane@example.com|My Credential

If resources.yaml is missing, or has no channels key, channels aren't managed locally, and merge and deploy leave remote channels alone. This protects projects synced before resources.yaml existed. channels: {} means "no channels", so any remote channels get deleted.

@openfn/project

from-fs reads channels from resources.yaml if they're there. to-fs writes them only if the project has channels.
New util/resources.ts converts between the file format and ChannelState.
to-app-state resolves credential names to uuids, mints an id for any channel created locally, and never sends the local key.
Merge: source channels win, and channels are matched by their resources.yaml id, so a rename keeps the remote channel. Remote channels are matched by their slugified name. Sandbox merges used to drop channels completely. That's fixed.
Deletes: merge records channels dropped from resources.yaml on project.removedChannels, and to-app-state sends them as delete: true, the same way removed steps and workflows are handled.

@openfn/cli

deploy treats channel changes as deployable. Previously a channels-only change reported "Nothing to deploy".
checkout removes a leftover resources.yaml when the target project has no channels, so one project's channels don't get deployed to another.

@openfn/lexicon

Adds ChannelState: a Channel with an optional id, a local key, and a destination_credential_id that can hold a credential name.

Known limitation: Lightning currently rejects channel deletes through the provisioning API (422). That's being fixed on the Lightning side; until then, a deploy that removes a channel fails.

QA Notes

Pull a project that has channels and check that resources.yaml is written with names and credential names, not uuids.
Add and edit a channel in resources.yaml, then deploy. Check that each change shows up in Lightning.
Rename a channel (change name, keep its key) and deploy. Check that the channel is updated in place, not recreated.
Remove a channel and deploy. Check that a delete is sent (Lightning will reject it until the server-side fix lands).
Deploy with no resources.yaml and check that the remote channels are unchanged.
Check out a project without channels from one that has them, and check that resources.yaml is removed.
Merge a sandbox that has channels into its parent, and check that the channels survive and keep their ids.

AI Usage

I have used Claude Code
I have used another model
I have not used AI

You can read more details in our
Responsible AI Policy

josephjclark

Just left some musings Frank - it may not all be valid

josephjclark · 2026-10-01T15:56:35Z

      logger?.warn('WARNING! No content for file', f);
    }
  }
+  // Remove any channels.yaml left over from the previous project, so its


Joe to check: this is a little awkward isn't it. Not sure I see a way around it though...

Agreed. It's needed because checkout rewrites openfn.yaml but doesn't touch other files, so a channels.yaml from the previous project would be left behind and its channels deployed to the new one

josephjclark · 2026-10-01T15:58:04Z

+
+// Channels dropped from the merged project (ie, removed from
+// channels.yaml) need an explicit delete entry in the deploy payload
+export const deletedChannels = (merged: Project, remote: Project) => {


I think this should be happening in the Project code - the same way as we track deleted steps and workflows.

CLI should have as little logic as possible

Done. Merge now flags removed channels with Project.removeChannel(), and to-app-state sends them as delete: true. fyi, I borrowed this logic from Collections, there is a deletedCollections function in deploy.ts

josephjclark · 2026-10-01T16:00:18Z

@@ -0,0 +1,42 @@
+import type l from '@openfn/lexicon';


Are we sure this warrants its own file?

I'm open to folding this to another file, just not sure where

josephjclark · 2026-10-01T16:01:07Z

+import type l from '@openfn/lexicon';
+import getCredentialName from './get-credential-name';
+
+export const CHANNELS_FILE = 'channels.yaml';


I am tempted to suggest that we convert this to resources.yaml and use it to track extra server-side resources. Collections, in particular. But maybe later environments, global functions, all sorts

Aaah, good idea. Can I proceed with it on this branch?

josephjclark · 2026-10-01T16:04:13Z

 }

+// Source channels win, but keep the target's id on a name match.
+// If the source has no channels at all (no channels.yaml), keep the target's


Why this rule? Maybe the user just deleted channels.yaml to remove all those pesk channels?

is this tracking a specific use case?

It protects existing workspaces. Anything pulled before this change has no channels.yaml, so if a missing file meant "no channels", the next deploy would delete every channel on the project

josephjclark · 2026-10-01T16:09:15Z

  }));
 }

+// Source channels win, but keep the target's id on a name match.


Possibly this should recognise deletes too?

The way we do it in merge-workflow is:

identifiy removed stuff druing the merge

call Workflow.remove(id)

This tells the workflow class that something was removed

That step is generally ignored from workflow.steps

But during state serialization we get the list of removed items and set delete: true

This makes me want to add Project.removeCollection() and have it work in just the same way. Then all the delete logic is only handled by the provosioner serialisation - no-one else needs to worry or care about it.

We'd have to do the same for collections too, but happy to spin out an issue for that.

Good point. I've done this for channels. If we're to do resources.yaml then maybe we should do it for Collections too

josephjclark · 2026-10-01T16:10:03Z

  };

+  // channels.yaml is optional: if it's missing, channels stay undefined and
+  // are left untouched on merge/deploy


Also slightly questioning this...

…ids on merge

…aml on checkout

Move channel delete detection out of the CLI deploy handler and into the project's merge step, so any caller of toAppState sends removed channels to Lightning as deletes.

…d of channels.yaml

…e channel

josephjclark

This is great Frank, thank you

I'm just going to look at adding slightly better output in the CLI. I'll either add it and merge it, or spin out an issue if it takes too long. But Ithink it's small...

* restructure project-diff * diff channels in project * log resource diff in CLI * changeset * one more changeset for the road * format * simplify

* Remove trigger.enabled (#1564) * lexicon: remove trigger.enabled from spec * update handling of trigger.enabled * update version hash * changeset * remove log * update version util and fix tests * add notes to docs * remove .only * fix test * update test * fix integration test * Support channels in v2 sync (#1566) * project: read and write channels via channels.yaml * project: fix sandbox merge dropping channels and keep remote channel ids on merge * cli: deploy channel changes and deletions, and clear stale channels.yaml on checkout * project: track removed channels on merge and send them as deletes Move channel delete detection out of the CLI deploy handler and into the project's merge step, so any caller of toAppState sends removed channels to Lightning as deletes. * project: store channels under a channels key in resources.yaml instead of channels.yaml * project: key resources.yaml channels by id with a required name * project: match channels by their resources.yaml id so renames keep the channel * Resource Diffs (#1572) * restructure project-diff * diff channels in project * log resource diff in CLI * changeset * one more changeset for the road * format * simplify --------- Co-authored-by: Joe Clark <joe@openfn.org> * No checkout after deploy (#1573) * don't checkout after deploy * test * changeset * fix test * version --------- Co-authored-by: Midigo Frank <39288959+midigofrank@users.noreply.github.com>

taylordowns2000 added this to Core Sep 29, 2026

github-project-automation Bot moved this to New Issues in Core Sep 29, 2026

midigofrank marked this pull request as ready for review September 30, 2026 11:00

midigofrank requested a review from josephjclark September 30, 2026 11:00

josephjclark changed the base branch from main to release/next October 1, 2026 15:53

josephjclark reviewed Oct 1, 2026

View reviewed changes

midigofrank added 4 commits October 2, 2026 11:11

project: read and write channels via channels.yaml

9a26105

project: fix sandbox merge dropping channels and keep remote channel …

3f5af6e

…ids on merge

cli: deploy channel changes and deletions, and clear stale channels.y…

e96f13b

…aml on checkout

project: track removed channels on merge and send them as deletes

9c88fa8

Move channel delete detection out of the CLI deploy handler and into the project's merge step, so any caller of toAppState sends removed channels to Lightning as deletes.

midigofrank force-pushed the frank/ofn-4553 branch from b2fc66c to 9c88fa8 Compare October 2, 2026 08:20

midigofrank added 3 commits October 2, 2026 13:02

project: store channels under a channels key in resources.yaml instea…

a22e71a

…d of channels.yaml

project: key resources.yaml channels by id with a required name

e78ea36

project: match channels by their resources.yaml id so renames keep th…

49e2e37

…e channel

midigofrank requested a review from josephjclark October 2, 2026 13:11

josephjclark approved these changes Oct 2, 2026

View reviewed changes

Resource Diffs (#1572)

a76544c

* restructure project-diff * diff channels in project * log resource diff in CLI * changeset * one more changeset for the road * format * simplify

josephjclark merged commit dd7832e into release/next Oct 2, 2026
7 of 11 checks passed

josephjclark deleted the frank/ofn-4553 branch October 2, 2026 17:16

Conversation

midigofrank commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Short Description

Implementation Details

QA Notes

AI Usage

Uh oh!

josephjclark left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephjclark left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

midigofrank commented Sep 29, 2026 •

edited

Loading