Repository navigation
Sync: split up openfn.yaml - #1567
Merged
Merged
Conversation
Collaborator
Author
|
Are we absolutely sure the There's no other binding between the checkout an the project file. So you have to run It also means if you look at the branch on github, yes you can see its contents and any tracked state files. But you can't know which project/sandbox you're looking at. If the branch and and sandbox name agree you can infer it I suppose. But you sill want to run a checkout locally. I don't think we're there yet. |
Project metadata now lives in .openfn/checkout.yaml, openfn.yaml holds workspace config (plus collections) as flat, sorted keys. Integration tests pin OPENFN_BRANCH=false so they don't pick up the kit repo's branch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Idea: maybe when you pull the branch from GH, and you get no state, and so the CLI doesn;t know what's checked out... but the CLI knows this! It could prompt the user "which project is tracked" and pick one from the list. That's all we need. It could even auto-infer from the branch name. We just need that one hook to get started. The only possible concern then is that the user doens't have forked_from references for other versions. Does that matter? Does that block merging or deploying? If forked from is articifically created fresh does that user lose anything? |
This was referenced Oct 7, 2026
* add util for checking if there's a valid checkout * add helpers for writing checkout metadata * refactoring * use new helpers in deploy * prompt user for checkout * add user prompt * simplify * fix tests * better fix
Collaborator
Author
|
Ok, this is all good to go. I'm happy with it! |
josephjclark
added a commit
that referenced
this pull request
Oct 12, 2026
* Sync: split up openfn.yaml (#1567) * lexicon: remove trigger.enabled from spec * update handling of trigger.enabled * update version hash * changeset * remove log * update version util and fix tests * add notes to docs * remove .only * fix test * update test * fix integration test * first swing at refactor to split up openfn.yaml * order keys * fix ropey merge issues * add logic to checkout on git without tracking * Update tests for checkout file and flat openfn.yaml Project metadata now lives in .openfn/checkout.yaml, openfn.yaml holds workspace config (plus collections) as flat, sorted keys. Integration tests pin OPENFN_BRANCH=false so they don't pick up the kit repo's branch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * format * move collections to resources.yaml * Add prompt to reworked openfn yaml (#1577) * add util for checking if there's a valid checkout * add helpers for writing checkout metadata * refactoring * use new helpers in deploy * prompt user for checkout * add user prompt * simplify * fix tests * better fix * ignore checkout meta (#1578) * changeset * changesets and docs --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> * CLI: handle multiple local copies of the same project (#1575) * project: tell local copies of the same project apart by alias A workspace can hold several local copies of one project, each from its own state file (main@app.openfn.org.yaml, backup@app.openfn.org.yaml). They share a UUID and id, so anything keyed on those mixed them up. Copies overwrote each other's entries in the path map, the checkout had no way to say which copy was checked out, and get() threw even when given an exact alias. matchProject now returns the exact alias match when all the other matches share its UUID. Real ambiguity still throws, and the error lists each clash as alias@host and suggests using one. Workspace keeps project paths by Project rather than by id, so each copy keeps its own file. getProjectPath() now takes a Project. extractCheckout writes the active project's alias to the checkout file, and getTrackedProject() looks that alias and host up first. The alias is left out of project.openfn so the metadata doesn't store it twice. merge used getProjectPath(id) to find where to write. With --base it only got the right file because the id-keyed map happened to point there. It now writes back to the --base file directly. * cli: don't fail fetch on duplicate local projects when the target is explicit `openfn project fetch <uuid> --endpoint X -o file.yaml` threw MultipleMatchingProjectsError if two local project files shared the UUID. With a full UUID, an endpoint and an output path, the command has everything it needs, so the local copies don't matter. Skip the local workspace lookup in that case. Also stop searching the workspace for an output target once -o is given, because the path is the target. Duplicates still throw when the UUID, endpoint or output path is missing. * cli: use the checked-out copy for pull and deploy without a target When no project was given, pull and deploy fell back to the active project's UUID. That throws if another local copy shares the UUID, even though the checkout file says which one is checked out. If the checkout file records an alias, pull now defaults to the tracked project's alias@host and deploy uses the tracked project. Without an alias, both fall back to the UUID as before. * add changeset * address review feedback getTrackedProject now tries alias, uuid then id in plain steps, and matches the alias on the full endpoint origin, so copies on different ports of the same host (localhost:4000, localhost:5000) are told apart. deploy picks its sync target in one place: an explicit target, the local copy of a deployed file, or the tracked project. The ambiguity error now suggests alias@domain or a path. Adds a test for fetching by alias when local copies share a UUID. * little tidy --------- Co-authored-by: Joe Clark <jclark@openfn.org> * versions * versions * better guard and message on ad-hoc checkout * revert ensureCheckoutIgnored We need to track the checkout files in github for github sync deploy actions to work properly (without the checkout files, the sync aciton cannot connect workflows/ to a deploy target) * unit test --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Midigo Frank <39288959+midigofrank@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Short Description
This PR splits openfn.yaml into two parts: workspace settings (including colllections and channels) still sit in openfn.yaml. But all local state stuff in the
projectkeys is moved to a.openfnfolder, which should be gitignored.This should really help clean up github diffs. The stuff you don;t want to merge - like the checked out project uuid - can be untracked.
To make this work, the checkout code will recognise if you're using git, and will track state per branch. So you can switch branches or pull the project and it'll configure itself. This should be mostly invisible and seamless.
Some caveats:
Still to do:
AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy
Release branch checklist
Delete this section if this is not a release PR.
If this IS a release branch:
pnpm changeset versionfrom root to bump versionspnpm installpnpm changeset tagto generate tagsgit push --tagsTags may need updating if commits come in after the tags are first generated.