I asked it to design a submarine for my cat and literally the instant my finger touched return the answer was there. And that is factoring in the round-trip time for the data too. Crazy.
replyThe answer wasn't dumb like others are getting. It was pretty comprehensive and useful.
While the idea of a feline submarine is adorable, please be aware that building a real submarine requires significant expertise, specialized equipment, and resources.it's incredible how many people are commenting here without having read the article. they completely lost the point.
replyWith this speed, you can keep looping and generating code until it passes all tests. If you have tests.
replyGenerate lots of solutions and mix and match. This allows a new way to look at LLMs.
Not just looping, you could do a parallel graph search of the solution-space until you hit one that works.
replyInfinite Monkey Theory just reached its peak
replyYou could also parse prompts into an AST, run inference, run evals, then optimise the prompts with something like a genetic algorithm.
replyAnd then it's slow again to finally find a correct answer...
replyIt won't find the correct answer. Garbage in, garbage out.
replyThis doesn't work. The model outputs the most probable tokens. Running it again and asking for less probable tokens just results in the same but with more errors.
replyDo you not have experience with agents solving problems? They already successfully do this. They try different things until they get a solution.
replyAgreed, this is exciting, and has me thinking about completely different orchestrator patterns. You could begin to approach the solution space much more like a traditional optimization strategy such as CMA-ES. Rather than expect the first answer to be correct, you diverge wildly before converging.
replyThis is what people already do with “ralph” loops using the top coding models. It’s slow relative to this, but still very fast compared to hand-coding.
replyOK investors, time to pull out of OpenAI and move all your money to ChatJimmy.
replyA related argument I raised a few days back on HN:
replyWhat's the moat with with these giant data-centers that are being built with 100's of billions of dollars on nvidia chips?
If such chips can be built so easily, and offer this insane level of performance at 10x efficiency, then one thing is 100% sure: more such startups are coming... and with that, an entire new ecosystem.
RAM hoarding is, AFAICT, the moat.
replylol... true that for now though
replyYeah, just cause Cisco had a huge market lead on telecom in the late '90s, it doesn't mean they kept it.
reply(And people nowadays: "Who's Cisco?")
You'd still need those giant data centers for training new frontier models. These Taalas chips, if they work, seem to do the job of inference well, but training will still require general purpose GPU compute
replyNext up: wire up a specialized chip to run the training loop of a specific architecture.
replyI think their hope is that they’ll have the “brand name” and expertise to have a good head start when real inference hardware comes out. It does seem very strange, though, to have all these massive infrastructure investment on what is ultimately going to be useless prototyping hardware.
replyTools like openclaw start making the models a commodity.
replyI need some smarts to route my question to the correct model. I wont care which that is. Selling commodities is notorious for slow and steady growth.
Nvidia bought all the capacity so their competitors can't be manufactured at scale.
replyIf I am not mistaken this chip was build specifically for the llama 8b model. Nvidia chips are general purpose.
replyYou mean Nvidia?
replyI dunno, it pretty quickly got stuck; the "attach file" didn't seem to work, and when I asked "can you see the attachment" it replied to my first message rather than my question.
replyIt’s llama 3.1 8B. No vision, not smart. It’s just a technical demo.
replywhy is everyone seemingly incapable of understanding this? waht is going on here? Its like ai doomers consistently have the foresight of a rat. yeah no shit it sucks its running llama 3 8b, but theyre completely incapable of extrapolation.
replyHmm.. I had tried simple chat converation without file attachments.
replyI got 16.000 tokens per second ahaha
replyI get nothing, no replies to anything.
replyMaybe hn and reddit crowd have overloaded them lol
replyThat… what…
replyWell it got all 10 incorrect when I asked for top 10 catchphrases from a character in Plato's books. It confused the baddie for Socrates.
replyFast, but stupid.
reply Me: "How many r's in strawberry?"
Jimmy: There are 2 r's in "strawberry".
Generated in 0.001s • 17,825 tok/s
1. How is this worth it over diffusion LLMs (No mention of diffusion LLMs at all in this thread)
2. Will Talaas also work with reasoning models, especially those that are beyond 100B parameters and with the output being correct?
3. How long will it take to create newer models to be turned into silicon? (This industry moves faster than Talaas.)
4. How does this work when one needs to fine-tune the model, but still benefit from the speed advantages?The blog answers all those questions. It says they're working on fabbing a reasoning model this summer. It also says how long they think they need to fab new models, and that the chips support LoRAs and tweaking context window size.
replyI don't get these posts about ChatJimmy's intelligence. It's a heavily quantized Llama 3, using a custom quantization scheme because that was state of the art when they started. They claim they can update quickly (so I wonder why they didn't wait a few more months tbh and fab a newer model). Llama 3 wasn't very smart but so what, a lot of LLM use cases don't need smart, they need fast and cheap.
Also apparently they can run DeepSeek R1 also, and they have benchmarks for that. New models only require a couple of new masks so they're flexible.
LLMs can't count. They need tool use to answer these questions accurately.
reply[dead]
replyI asked, “What are the newest restaurants in New York City?”
replyJimmy replied with, “2022 and 2023 openings:”
0_0
Well, technically it's answer is correct when you consider it's knowledge cutoff date... it just gave you a generic always right answer :)
replychatjimmy's trained on LLama 3.1
replyIs super fast but also super inaccurate, I would say not even gpt-3 levels.
replyThat's because it's llama3 8b.
replyThere are a lot of people here that are completely missing the point. What is it called where you look at a point of time and judge an idea without seemingly being able to imagine 5 seconds into the future.
reply“static evaluation”
replyIt is incredibly fast, on that I agree, but even simple queries I tried got very inaccurate answers. Which makes sense, it's essentially a trade off of how much time you give it to "think", but if it's fast to the point where it has no accuracy, I'm not sure I see the appeal.
replythe hardwired model is Llama 3.1 8B, which is a lightweight model from two years ago. Unlike other models, it doesn't use "reasoning:" the time between question and answer is spent predicting the next tokens. It doesn't run faster because it uses less time to "think," It runs faster because its weights are hardwired into the chip rather than loaded from memory. A larger model running on a larger hardwired chip would run about as fast and get far more accurate results.
That's what this proof of concept shows
replyI see, that's very cool, that's the context I was missing, thanks a lot for explaining.
replyIf it's incredibly fast at a 2022 state of the art level of accuracy, then surely it's only a matter of time until it's incredibly fast at a 2026 level of accuracy.
replyyeah this is mindblowing speed. imagine this with opus 4.6 or gpt 5.2. probably coming soon
replyI'd be happy if they can run GLM 5 like that. It's amazing at coding.
replyWhy do you assume this?
replyI can produce total jibberish even faster, doesn’t mean I produce Einstein level thought if I slow down
Better models already exist, this is just proving you can dramatically increase inference speeds / reduce inference costs.
replyIt isn't about model capability - it's about inference hardware. Same smarts, faster.
Not what he said.
replyI think it might be pretty good for translation. Especially when fed with small chunks of the content at a time so it doesn't lose track on longer texts.
reply