In order to have direct function calls, you need to load the plugin's code in your own memory. Which means you expose your own memory to the plugin. Any malicious/buggy plugin could wreak havoc on your program - even managed code doesn't solve that problem in general.

IPC is the natural solution to that - run a process in its own address space and communicate with it via I/O. TCP is not most optimal, but it is uniquous. Same thing with JSON.

In LSP, most of the processing happens in the LSP server anyway - communication overhead between client and server is negligible in comparison to it.

TCP isn't really necessary. Usually a language server just uses stdin/stdout, which are just pipes.

And overall overhead for running a language server in a separate process isn't that big. LSP is designed in such a way that only minimal amount of information is needed to be passed, like edits or short responses. Packing/unpacking JSONs isn't a bottleneck, the heaviest job like program analysis is done in the language server itself without interprocess communication overhead involved.

I agree, the only reason LSP exists as it does is for a world running on electron based applications. Plug-ins and application extensions are not new technology and they are nearly universally more efficient in the forms designed and used prior to 2005-ish. I understand why VSCode exists and why it is used so often by developers, but it is certainly a downgrade from more language specific options that could exist.

There are some arguments that do carry water in favor of using a client/server protocol transmitting JSON, in particular, the ability for a nearly complete decoupling of analysis of code and the displaying and editing of that code. Also, LSP was (to my knowledge) the first language/platform/usecase agnostic protocol intended for use in code editors.

I get why this is where a big chunk of developers have ended up, but I do bemoan the lost potential for a language to grab mindshare and popularity on the usability and performance of its tooling and developer experience via-a-vis a custom designed and hyper specific code editor. I mean, Rust and Elm received endless praise for their error messages as a massive boon to developer experience, so it is a facet of language design and implementation that can act as great advertisement. I just hate that the prevalence of LSP at this point precludes custom editors as a first choice in the current zeitgeist.

It only makes sense in the context of host security and stability, as proven by plugin issues in those IDEs.

However, to come back to your point, there are much better high performance OS IPC mechanisms for inter process communication than sending JSON down through a TCP wire.

As others point out, Electron.