I can highly recommend gliner2.5, been using it for tagging news articles, fine tuned on my home 5090 in a few minutes on 500 articles tagged by gpt 6.1, works fantastically. Deployed on hetzner running on a cpu .
replygliner-style models are absolutely crazy, I learned about them a couple weeks ago when a co-worker fine-tuned an LFM encoder for promptable classification. as far as I know, you don't need to fine-tune; you can just prompt for classification. My co-worker's demo is here: https://huggingface.co/spaces/LiquidAI/prompt-routing
replyYou're doing something similar to what I've been prototyping for the automatic metadata enrichment part. So far I've been using deterministic older techniques + manual for my personal data stream, but I'm trying to learn what the state of the art is for efficiently annotating data with LLMs and newer ML.
replyThis seems interesting, but if anyone else is also doing a similar thing, I'd appreciate tips on how to approach it. So far, I think I have some small RAG model in a demo or the very small qwen models. I think fine tuning seems interesting too. Some of the decision models applied in layers (like a binary search on a pre-generated corpus) could be fast enough to help too?
idk too many options, i'm sure this is a solved problem lol
It's definitely not a solved problem, and it has been kicking my ass. I don't just want metadata tagging/categorization, I want something that identifies the networks between the information, graduates topics to actual objects, and routinely consolidates/synthesizes narratives that can be easily looked up.
replyThat's very cool. I'm curious if you need a GPU for your use case (email and text triaging). There are some open source alternatives[0] that run/train on CPU, so you can self-host or run locally (and train on your own data for better accuracy), would something like that work for you?
replyHave you considered fine-tuning on your dataset?
replySeeing the fine-tuning guides that are coming out with LFM3, I am now considering it :)
replyWould be a fun experiment.
You might also consider the decision models, d1-2B is very strong and good for calibrated classification. Would be happy to share links/resources if you're interested.
replyYes, once it's in LocalAI, I intend to try it out.
replyHow do you determine confidence levels? logprobs?
replyI initially tried annotating everything three times, then judging the annotation with the same model. That failed predictably, so yeah just logprobs right now, judged on a hand-annotated dataset. Mostly correlated - ~80%.
reply[flagged]
reply