Skip to content

refactor: parser #322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Apr 11, 2025
Merged

refactor: parser #322

merged 21 commits into from
Apr 11, 2025

Conversation

psteinroe
Copy link
Collaborator

@psteinroe psteinroe commented Apr 9, 2025

this rewires the workspace server for better DX and adds support for sql function bodies.

closes #178

hear me out:

  • the Document is responsible for extracting statements from a source file. right now, this is always SQL, but in the future we could make this generic.
  • the Parser is the new layer between the workspace server and the document. there is one parser per document, and the parser owns the document.
  • the parser is responsible for parsing the raw sql coming from the document, and is the main API layer for the workspace.
  • the consumer of the parser apis simply gets the statements and the related parse results. the consumer does not need to know anything about sub statements
  • I added a nice pattern where the consumer of the Parser can define a mapper and / or a filter and then simply calls parser.iter(ExecuteStatementMapper, CursorPositionFilter::new(pos)) and gets exactly the data it needs.
  • I also optimised the parsing in general: the only thing that actually cares about modified statements is tree sitter. for pg_query, we always just remove and add when we receive a modify statement. hence, we can simply lazily parse all but tree sitter and just use modified / remove notifications to clear the cache. this also opened up the way for a nicer api because if you want the pg_query results, you always get Result<Node, SyntaxError> and we do not need to wrap it in an Option<> anymore.
  • the statement id is now not just an usize anymore but an enum that can reflect sub statements. if we remove parse results for a parent, we also remove it for its sub statement.

@psteinroe psteinroe marked this pull request as ready for review April 11, 2025 10:53
Copy link
Collaborator

@juleswritescode juleswritescode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR! :)
I was scared that accomodating nested statements would make things more complex, but this even makes it simpler!!

@psteinroe
Copy link
Collaborator Author

@juleswritescode for me ParsedDocument or DocumentParse makes more sense than ParsedStatements

@psteinroe psteinroe merged commit 4867189 into main Apr 11, 2025
7 checks passed
@psteinroe psteinroe mentioned this pull request Apr 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support sql functions bodies
2 participants