-
Notifications
You must be signed in to change notification settings - Fork 133
Config remote sync command #4289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Commit: 52c71ae
24 interesting tests: 22 KNOWN, 1 SKIP, 1 RECOVERED
Top 49 slowest tests (at least 2 minutes):
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add some acceptance tests?
You can see examples here that modify remote resource:
% git ls-files | find . -type d | grep remote
./bundle/resources/permissions/jobs/deleted_remotely
./bundle/resources/permissions/jobs/added_remotely
./bundle/resources/permissions/jobs/reorder_remotely
./bundle/resources/volumes/remote-delete
./bundle/resources/volumes/remote-change-name
./bundle/resources/jobs/remote_add_tag
./bundle/resources/jobs/remote_matches_config
./bundle/resources/jobs/remote_delete
./bundle/resources/jobs/remote_delete/destroy
./bundle/resources/jobs/remote_delete/deploy
./bundle/resource_deps/remote_field_storage_location
./bundle/resource_deps/remote_app_url
./bundle/resource_deps/remote_app_url/app
./bundle/resource_deps/jobs_update_remote
./bundle/resource_deps/remote_pipeline
| schema Generate JSON Schema for bundle configuration | ||
| summary Summarize resources deployed by this bundle | ||
| sync Synchronize bundle tree to the workspace | ||
| validate Validate configuration |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The output seems to be the same except for whitespace?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, probably a bug in help/Cobra? New command is hidden, but help reserves space for it.
| isDefault: isEmptyStruct, | ||
| }, | ||
| { | ||
| pattern: regexp.MustCompile(`^webhook_notifications$`), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can probably drop these, they now using different reason: #4338
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep that works good now, thanks!
| currentValue := b.Config.Value() | ||
|
|
||
| for i, n := range nodes { | ||
| if key, ok := n.StringKey(); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StringKey might be rendered with a dot or with [""] or with [''] depending on string contents.
I suggest to output PathNode and then convert it to string rather than recreate the string directly.
Changes
New experimental command
databricks bundle config-remote-sync. The command fetches the latest remote changes of resources and compares them to the deployed state. With the--saveflag, it also writes changes back to YAML files.The command leverages the diffing algorithm of the
bundle plancommand. Also note that new dependency is added to do yaml patching that preserves commentsThis is a first PR, I will continue fixing all limitations in the next PRs
Current limitations:
bundle plantasks[task_key='my_task']need to be tested for cases when the object or parent doesn't existNo changelog entries are needed as the command is intended to be private for now, and we don't want to encourage users to use it due to the unstable API
Why
This command serves as the backend for the new visual authoring feature in DABs in the Workspace.
User edits job or pipeline in the Workspace UI, then clicks the "Sync" button, and then the new command applies the diff to the source configuration files. Then the user may accept or reject these changes in the editor
Tests
Currently, only unit tests to speed up devloop, but once I have full functionality, I'll add proper acceptance test coverage
Acceptance tests are failing because of the change in formatting (even though the command is hidden)