Add duckdb build/concepts and use SQLGlot to convert BigQuery SQL into other dialects#1689
Merged
alistairewj merged 47 commits intomainfrom Feb 20, 2024
Merged
Add duckdb build/concepts and use SQLGlot to convert BigQuery SQL into other dialects#1689alistairewj merged 47 commits intomainfrom
alistairewj merged 47 commits intomainfrom
Conversation
…)...committing before gutting dead code.
…es without a large amount of RAM, and this way it can be skipped with `--skip-indexes`
tompollard
approved these changes
Jan 9, 2024
Member
tompollard
left a comment
There was a problem hiding this comment.
Thanks, looks good to me, with the caveat that I haven't run the code! (and that breaking this into smaller PRs would be nicer). I added a couple of notes about typos but all very minor, so feel free to go ahead and merge. The sqlglot package looks very cool.
How much manual tweaking of the outputs is needed?
Ideally I think we would maintain a single source format (psql/bigquery) and then create other dialects in a GitHub workflow. My preference would be to have dialects on separate branches, rather than trying to maintain them all on main.
Member
Author
|
Thanks for the review! Fixed the typos; regarding some unaddressed points:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A few things in this PR, mainly to get the concepts to work with duckdb. This builds off #1529 / @SphtKr. However #1529 has the conversion for MIMIC-III, whereas here I've converted MIMIC-IV. It should be straightforward to adapt for MIMIC-III, but I can't explain the parsing errors I'm getting when running it over that folder, so I've left it for a future PR.
I also overhauled the READMEs a bit for clarity.