So, its not a general language model, focused on tool call strictly for tiny edge-devices. There are solutions everywhere for high-capacity devices, Needle is for sub-$200 devices.
replyIt seems to me that the model struggles to have enough general intelligence, knowledge, or reasoning capacity for arbitrary prompted tool calling. At this size, not surprising.
replyI am VERY interested in seeing how it could perform with some fine-tuning for a specific family of tools/tasks. That would be a great addition to the demo.
i would assume a model this size would require finetuning tbh. even functiongemma recommends that.
reply14mb? More like sub-$20 devices.
replyMost pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
replyWell running from QSPI flash (even the internal memory versions use SPI internally) so any inference would be very slow streaming from that compared to RAM. The featured article says: “With a peak session RAM around 28MB, Needle runs on newer microcontrollers like ESP32-S3.” So I don’t see this doing anything useful on a Pico. The Pico 2 (RP2350) for example has 520k of RAM.
replyAn ESP32 has the same amount of SRAM as the Pi Pico. You can hook up PSRAM to the Pi Pico just like ESP32 to get 16MB more RAM.
replyI'm quite impressed by the results of the web demo, especially given its size and the precision with which it uses the three available tools (tested with German commands). I could imagine that this LLM would fit well into a setup with multiple micro-sized LLMs for different purposes; so 14 MB for precise tool invocation is a reasonable memory footprint when a number of other local models are running (e.g. STT, TTS + language models).
replyyes, that's what we had in mind while building
replyIt's called a LM :). LLM stands for Large Language Model.
replyLLM as in Little Language Model
replyOk we'd be adopting Little Language Model officially haha
replyTLM
replydeleted
reply