upvote
Hi! I’m the author of this blog.

I’m evaluating these VLMs to figure out which ones are good enough to auto-annotate my data, so I can fine-tune my detector.

I wrote a bit more about this here: https://x.com/skalskip92/status/2080334344061694429?s=20

reply
Did you evaluate any that could be self-hosted (or at least ow models), if so which one is the best you seen?
reply
Take a look here: https://playground.roboflow.com/evals. We have few ~30B.
reply
Thank you!

It seems Qwen is kicking ass, and Fable made me laugh when I saw it all alone on the far right of the graph :))

reply
Agreed, this like asking a chainsaw to carve a wooden spoon. Impressive it can, but definitely not the right tech to scale.

LLM needs to setup an image classifier to use as a tool call.

reply
Building a dataset is expensive, manual annotation is expensive. Datasets don't exist in every niche.

I remember around 2013-15 people were scoffing at uses of deep learning CNNs for various things, because why don't you just use an SVM on HOG features? Or face detection is solved, just use Viola-Jones.

What if you give the benefit of doubt and assume the author knows about alternatives and uses VLMs for their strengths? They use it to auto-annotate training data for regular deep learning models.

reply
Now maybe, but the gap is closing.
reply
How are we supposed to pay off all these data centers and chips if you’re not willing to burn a microwave burrito worth of electricity for each prescription? Think of the benchmarks
reply