Fix trampoline notebook filename collision#5508
Open
simonfaltum wants to merge 2 commits into
Open
Conversation
Wrapper notebooks were named notebook_<job_key>_<task_key>. Since both keys can contain underscores, distinct pairs like (a_b, c) and (a, b_c) mapped to the same filename and one task silently ran the other task's wrapper. Prefix the job key length to make the name unique per pair. Co-authored-by: Isaac
Contributor
Waiting for approvalBased on git history, these people are best suited to review:
Eligible reviewers: Suggestions based on git history. See OWNERS for ownership rules. |
Collaborator
|
Commit: 953c623
22 interesting tests: 15 SKIP, 7 KNOWN
Top 29 slowest tests (at least 2 minutes):
|
Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Found during a full-repo review of the CLI. The python wheel wrapper (trampoline) names its generated notebook
notebook_<job_key>_<task_key>. Both keys can contain underscores, so distinct pairs can produce the same name: joba_bwith taskcand jobawith taskb_cboth map tonotebook_a_b_c. The second wrapper overwrites the first, and both tasks point at the same notebook, so one task silently runs the other task's wrapper.Changes
Before, two different (job key, task key) pairs could collide on one wrapper filename; now every pair gets a unique name. The name is now
notebook_<job_key_length>_<job_key>_<task_key>(inbundle/trampoline/trampoline.go). The length prefix makes the encoding unambiguous for any keys, with no new dependencies. The example above becomesnotebook_3_a_b_cvsnotebook_1_a_b_c.Existing wrapper notebooks get a new path on the next deploy. This only affects bundles with
experimental.python_wheel_wrapper: true, and the wrapper is regenerated and the job spec updated on every deploy, so no action is needed.Test plan
TestGenerateTrampolineWithCollidingKeyscovering the colliding pair, asserting two distinct wrapper files and notebook paths (fails without the fix)bundle/trampolinetests for the new naminggo test ./bundle/trampoline/ -count=1passes./task fmt-q,./task lint-q, and./task checkspassThis pull request and its description were written by Isaac.