This model was likely trained months before deepseek released their paper.
replyModels are generally posttrained to a window shorter than you think.
replyDoesn't mean they didn't apply something similar. They could have also come up independently with their own version, the speculation is not they copied it, rather that they have performance breakthroughs which perhaps is a result of work in same domain
replyUnless they already have something similar of their own, which is always possible, they'd be stupid not to. I don't suppose we'll ever know, though. It would not be a good look if after the trillions of dollars that have been thrown at US labs, investors found out that they're down to copying Chinese tech.
replyI don't think it's particularly relevant?
replyThey might be using something like this, or they might be using some other "increased sparsity" techniques, of which there are a great many. They also might be optimizing for something else - like less RAM use for KV cache.
Alternatively, they might be cutting into their margins and dropping the price because of stiffer competition from Astra. I do think that's unlikely though.
Getting a 403 on that link. Mind checking it once?
replyhttps://web.archive.org/web/20260922172456/https://miraflow....
replytired: AI startup attempting to build their own website
wired: a nonprofit founded in 1996
I'm able to access it on my laptop at home. Maybe a misconfigured bot protection rule, try a different user-agent or IP?
replyIt's working for me (based out of California).
reply