> I feel like it can be good within a specific context like Sql as you mention.
replyYes I think very constrained task: known data universe, well known language etc should be the best possible place for small language models to play
Yes, I think though that, maybe 1) this shows 1-bit llm models working so more companies can do that so that we can get more competition within this space (+ ngram-embedding idea)
replyAnother point, but I feel like, we can see some really good fine tuned models out of this model, the community feels excited about 1-bit LLM architecture. We are gonna see some good innovation within this space in the upcoming future.