Conversation
Split benchmark_models_petab access into v1/v2 submodules mirroring petab.v1/petab.v2, so a later piece of work can add problems/<id>/v2/ directories without further library or docs groundwork. Top-level get_problem/get_problem_yaml_path/get_simulation_df keep their current (v1) behavior unchanged. Also documents the v1/v2 equivalence policy in CONTRIBUTING.md and adds a corresponding optional PR-template checklist item. References Benchmarking-Initiative#327. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dweindl
commented
Sep 29, 2026
Comment on lines
+16
to
+17
| A problem's `v1/` and `v2/` encodings must represent *exactly* the same mathematical problem: | ||
| same model, same data, same objective. |
Member
Author
There was a problem hiding this comment.
This will probably need some refinement. How do we want to deal with the whole parameter scaling topic? Encode it in the model or condition table, ignore it, ...?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #327.
Prepares
benchmark_models_petaband the repo docs so thatproblems/<id>/v2/directories can be added later without further groundwork. Nov2/problem directory is created here — that's separate, later work.benchmark_models_petab.v1/.v2submodules, mirroringpetab.v1/petab.v2. Each exposesget_problem,get_problem_yaml_path,get_simulation_df. Top-levelget_problemetc. keep their current (v1) behavior unchanged (base.py/test_base.pyrenamed tov1.py/test_v1.py).v2.get_problem_yaml_path/get_problemraise a clear error when a problem has nov2/yet (the normal case for essentially every problem right now).v1/andv2/encodings must represent exactly the same mathematical problem (same model, data, objective), thoughv2/files may look structurally different to make idiomatic use of v2 features.v1/andv2/are documented as optional (at least one must be present), rather than implyingv1/is mandatory.Out of scope, left for follow-up work: generating/hand-curating any actual
problems/<id>/v2/directory, an automated v1-vs-v2 equivalence checker, and updating the README overview table / INSTALL.md usage docs for v2 (postponed until real v2 problems exist).🤖 Generated with Claude Code