Constant hallucinations. OpenAI:s latest on max settings. If you are to naively feed say, a short corpus of text to turn it into a parallel corpus in a few different languages, the original text gets subtly mangled and no longer matches the original. Say you have several hundred annotated sentences. Without hand-coding some regex to make sure that each sentence in the source column occurs in the original corpus you’re bound to get hallucinated sentences with an error rate that exceeds 1:100. Whatever you use as the output, JSON or XML, you will end up with columns that just repeat the original instead of translating it, especially for languages that are very close to each other or represent the same language.

Yes, LLMs can be SOTA for NLP, but you’re going to have to use them to write software or workflows that are more deterministic.

If simply running things thrice-over was enough to stop "hallucinations" (and not incur other problems) we wouldn't be here talking about it today, it'd have been "solved" months or years ago.