( I was trying to quote a single sentence and then realised it ran on for the whole paragraph. )
Given how difficult I found that to follow, are you sure your prompt is actually "Clear, not too long not too short"? We now only have your word for it. I too had assumed that was a prompt given to an LLM to further prompt agents.
It sounds like they're hitting a data source quality issue, which is hardly uncommon in scraping.
It's common for job boards to obscure who the real clients are, and if the scraping engine is LLM powered ( rather than LLM written ), then I would expect it to accidentally present agencies as the contracting organisation sometimes.
Breaking down the process so you can inspect the messy middle of a data pipeline is an important part of software engineering, but it sounds like they've tossed a messy task at an LLM and expected it to be proficient end-to-end.
"General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks" https://arxiv.org/abs/2604.11778
"...General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks..."
Question: Strangers A, B, C, D, and E line up from youngest on the left to oldest on the right. Their clothing
colors and shoe colors all differ, and they come from five different regions.
Known facts:
1. A is from Morocco.
2. D is five years older than B.
3. E is older than A.
4. C stands next to D.
5. A stands next to B.
6. The person in teal shoes is not adjacent to the person from Vanuatu.
7. One twelve-year-old wears yellow shoes.
8. The person in orange shoes wears white clothing.
9. The person in blue clothing is from Chile.
10. The youngest person wears red shoes.
11. Counting from the right, the fourth person comes from South Africa.
12. E wears yellow clothing.
13. The person in green shoes does not wear multicolored clothing.
14. Two people are twelve years old, ordered by birth month.
15. One adult is thirty-five years old, and that age is sixteen less than the combined ages of the other four.
If you multiply every possible age C might have, what number do you obtain?
Nowhere is "multicoloured" defined, are we to assume it should be treated as a colour and implied that someone else must be wearing "multicoloured clothing"? Because strictly that doesn't logically follow, and it ought to be phrased as "The person in green shoes is not the person wearing multicoloured clothing" if that is the case.
This is an extremely hard logic puzzle, especially since it's revealed at the end that there are multiple solutions.
I'd expect anyone to struggle unless armed with prolog.
It means that it is possible for someone to be wearing a white shirt and yellow pants (say), but the person in green shoes came from the set of a Wes Anderson film.
white, blue, yellow, and is otherwise undefined.
But we know from [1], [2], [4], [5] and [11], that the order must be:
A, B, C, D, E or A, B, E, D, C.
Which makes C either the older 12 year old or the 35 year old.
The key this is that D can't be a 12 year old without A being in slot 2, but A can't be in slot 2 because slot 2 is South Africa and A is Morocco.
Trying to reach the shoes + Vanuatu clue is a complete waste of time, paying any attention to shoes or clothing is a waste of time, it feels like there ought to be a way to narrow it down to one of those two configurations, but the clothing is too ambiguous, the shoes end up irrelevant.
What a frustrating puzzle, where half the clues are seemingly redundant.
I guess they were not optimising for a satisfying solving experience! (I'm not sure what 'expansion' means in this context, but it sounds like maybe the 'expansion strategy' referred to in the paper involved padding out problems with red herring premises?)
I particularly found the note about “local collapse” helpful (near the end of section 3). The idea is that even though benchmarks contain a wide variety of different reasoning tasks, each individual problem requires only a few skills - unlike this benchmark where they deliberately construct tasks that span many categories.
If a human misunderstood this, they'd be a dumb human.
"Frontier LLMs Still Struggle with Simple Reasoning Tasks" https://arxiv.org/abs/2507.07313
"General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks" https://arxiv.org/abs/2604.11778
"...General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks..."
You're clearly operating in bad faith, but just for the record: the General365 problems are very difficult as you can see from the examples at https://arxiv.org/html/2604.11778v1#A1. It's actually impressive that Gemini 3 Pro got 62%, and the strongest OpenAI and Anthropic models they tried were GPT-5.1 and Sonnet 4.5.
It seems to me that such simplified tasks tend work better. The rest of the loop is just scraping websites, which doesn’t really have a reason to rely on ai agents.