These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.
Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.
Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.
Now its collecting data to make a domain specific BERT and do what Jev does.
Also curious about if you plan on doing some sort of routing for the requests. Like detecting the type of task to decide which model to route the request to
This whole thing started because I wanted something to help me play Dune Imperium. Even relatively large models with vision encoders couldn't reliably extract the full state of the board. Now that I have ~2k labeled screenshots, I want to train heads on top of SigLIP2 to extract all of that data in one go.
That's how it started. Now the thing supports multiple kinds of datasets:
Images - currently the Dune Imperium and Bolatro screenshots, with SigLIP2 heads being the next step.
STT - my self-hosted Linux dictation tool feeds this dataset. I run Nemotron ASR tuned for my voice.
TTS - for Piper TTS, trained to speak like SHODAN. Trained from data generated by Qwen3-tts + original video games files.
Text pairs - for a 1.2B model that converts normal text into "what would SHODAN say?"
FastApply - a Qwen3.5-4B LoRA adapter for doing fast edits.
Chat threads - all agent/chat threads get saved too, so eventually I can turn the useful ones into a dataset and train a LoRA for a really good Rust-specialized version of Qwen3.8-27B.
Tool calls (extracted from chat threads) - this is where I want something Jev-like, mainly to add an auto-approval mode to my agent harness.
The FastApply dataset is already ~20k entries, with the majority of outputs being 8k–16k tokens. The STT dataset is roughly 30 hours and growing.
Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need.
Amazing, thank you for sharing your setup. Very cool applications
this misses the point of jev somewhat - the point is that this is a foundational, general purpose classifier model - see some good sources https://x.com/mparakhin/status/2101683565520199887?s=12
30KB model, 40-50ms inference. Pretty happy with the results so far!
I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation/training data creation, data cleaning, etc). Or just use a general classifier like Jev/Kev ha
What’s the model architecture?
I'm assuming you only need to consider a single language for your emails?
Haven’t tried with more
Do you have a specific use case?
In my case Norwegian, English and Japanese.
The type of task in which it does really well, especially against Laya, is classification with >50 classes
But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one)
For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP
This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.