Conversation
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <deepsainivarun@gmail.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
fix Open() calls; replace Finalize() with Close(); close state file in plan
Move state open/close management to process.go so the lifecycle is transparent. process.go opens state for read (with WAL recovery) after PullResourcesState and defers close. Deploy/destroy upgrade to write mode via the new UpgradeToWrite() method which initializes the WAL without re-reading state JSON. Internal functions (CalculatePlan, ExportState, InitForApply, ValidatePlanAgainstState) no longer manage their own open/close — they expect state to already be open. Self-managed callers (bind, migrate, yaml_sync, diff) handle their own state lifecycle. Plan command uses ProcessBundleRetWithPlan to compute the plan while state is still open for read inside processBundleRetInternal. Co-authored-by: Isaac
WAL is recovered on next run via WithRecovery open in process.go; deployCore already calls Finalize+Open explicitly before PushResourcesState. Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
Flush WAL to local state while the state DB is still open, before remote files are deleted. Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
3 jobs exercise the same DAG + partial-WAL recovery path with 3x fewer output lines. Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
- drop corrupted-wal-middle (same code path as corrupted-wal-entry) - drop multiple-crashes (covered by crash-after-create) - drop summary-after-crash (incomplete output; crash coverage in crash-after-create) - fix empty-wal echo: (unexpected) -> (expected) - fix parent test.toml: exit code 137 -> [KILLED] only; errors show Exit code: 1 Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
On Linux, KillCaller (SIGKILL) may produce exit code 1 instead of 137. Add a context-sensitive replacement to normalise exit code 1 only when it directly follows [PROCESS_KILLED], so genuine error exits (exit code 1 from cat/jq) remain visible as Exit code: 1 in the output. Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
- chain-3-jobs: fix stale echo "job_10" -> "job_03"
- corrupted-wal-entry, future-serial-wal, lineage-mismatch, stale-wal,
wal-with-delete: commit static fixture files (resources.json,
resources.json.wal) instead of creating them inline in script;
wal-with-delete: commit databricks.yml as resources: {} instead of
overwriting it at runtime
Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Why
Tests