upvote

points

by jrmg4 hours ago |
comments
upvoteby randusername15 minutes ago|
next
[-]
Personally I'd start with llamafile [0] then move to compiling your own llama.cpp.

It's not as bad as you might think to compile llama.cpp for your target architecture and spin up an OpenAI compatible API endpoint. It even downloads the models for you.

[0]: https://github.com/mozilla-ai/llamafile

reply
upvoteby AstroBen4 hours ago|
prev|
next
[-]
Ollama or LM Studio are very simple to setup.

You're probably not going to get anything working well as an agent on an M2 MacBook, but smaller models do surprisingly well for focused autocomplete. Maybe the Qwen3.5 9B model would run decently on your system?

reply
upvoteby jrmg3 hours ago|
parent|
[-]
Right - setting up LM studio is not hard. But how do I connect LM Studio to Copilot, or set up an agent?
reply
upvoteby NortySpock3 hours ago|
parent|
next
[-]
I tried the Zed editor and it picked up Ollama with almost no fiddling, so that has allowed me to run Qwen3.5:9B just by tweaking the ollama settings (which had a few dumb defaults, I thought, like assuming I wanted to run 3 LLMs in parallel, initially disabling Flash Attention, and having a very short context window...).

Having a second pair of "eyes" to read a log error and dig into relevant code is super handy for getting ideas flowing.

reply
upvoteby AstroBen3 hours ago|
parent|
prev|
next
[-]
It looks like Copilot has direct support for Ollama if you're willing to set that up: https://docs.ollama.com/integrations/vscode

For LM Studio under server settings you can start a local server that has an OpenAI-compatible API. You'd need to point Copilot to that. I don't use Copilot so not sure of the exact steps there

reply
upvoteby brcmthrowaway3 hours ago|
parent|
prev|
[-]
Basically LM Studio has a server that serves models over HTTP (localhost). Configure/enable the server and connect OpenCode to it.

Try this article https://advanced-stack.com/fields-notes/qwen35-opencode-lm-s...

I'm looking for an alternative to OpenCode though, I can barely see the UI.

reply
upvoteby AstroBen3 hours ago|
parent|
[-]
Codex also supports configuring an alternative API for the model, you could try that: https://unsloth.ai/docs/basics/codex#openai-codex-cli-tutori...
reply
upvoteby chatmasta2 hours ago|
prev|
[-]
Any time I google something on this topic, the results are useful but also out of date, because this space is moving so absurdly fast.
reply