I think its worth considering but I don’t agree it’s always best to use the language parser.
replyA language parser should be correct. A LSP parser should be fault tolerant. My understanding is you cant have both.
It absolutely can be done [1] but it is a lot of effort to do high quality error recovery. It's easier in languages with natural "synchronization points" [2], harder in languages which don't have them.
replyHaving built a lot of protobuf tooling, I'd estimate that protobuf largely falls into the former camp; most of your time working with .proto files is operating on fields which terminate using `;` at the end of the line (though multi-line is also possible).
[1] source: I've done it for SQLite SQL at https://github.com/LalitMaganti/syntaqlite/
[2] e.g. SQL naturally has this at statement and expression boundaries which covers almost all of the cases people care about.
Adding to this, rustc's parser is both correct and fault-tolerant, yet rust-analyzer ended up building its own for different reasons (needing a CST and not an AST).
replyI settled on starting with a CST using rowan, doing semantic analysis, having an agnostic editor services, exposing both an LSP interface with tower, and a WASM interface (for Monaco in the browser [non-LSP!!]) and having the runtime binary doing the interpretation/compilation with the same pathway, with a slight jog, as the editor services. Phew!
replyIt is my understanding that basically all LSP implementations use tree-sitter because of its incremental rebuild capability, and this requires re-implementing the parser.
replyThat's just not true. rust-analyzer does not use tree-sitter. Neither does gopls. Nor Pylance. Or tsc. In fact I don't know if there is a single popular LSP that uses it (but there probably is).
replyMany editors use tree-sitter, but that is separate from the LSP.
buf is a parser / compiler as an alternative to protoc https://buf.build/docs/migration-guides/migrate-from-protoc/
replyAh, so does the LSP just call out to the Buf binary?
replyThe LSP is one of the subcommands provided by the buf binary. My LSP configuration for protobuf is `buf lsp serve`.
replyMakes sense, thanks for the clarification.
replyThe lsp is a subcommand of buf, from the page: `buf lsp serve`
reply