I discovered that LLM-generated tokens in the scratchpad were relatively stable, but injected thoughts were frequently ignored and often deleted from the scratchpad within a few turns – even when the injected thought was the literal answer to the puzzle it was stuck at!
A reader[2] then pointed me toward research similar to what you might recall: LLMs interpret text by maintaining activations for input tokens, so text that is not generated by the same LLM will seem "unlikely" to the LLM in a sense, and when given the alternative between likely and unlikely text, it's probably trained to judge the unlikely text as a weird "slip of the mind" and discredit it in favour of the more likely text. I speculate this is part of how they can be useful in the first place, despite their non-determinism.
[1]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...
[2]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...
1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about.
2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response.
3. Rewrite the article to minimize said KL divergence.
In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others.
I believe we will keep seeing more of this stuff.
Let's hope the LLM model continues to be paying for credits, because any that move to ad revenue will become useless for real work.
LLM vendors make this hard because you can't trust them with your session data. Yesterday you were opted out of training, then suddenly today you're opted in.
It's an extension of the idea that they don't need to care about anybody's copyright. They don't care about preserving the security or privacy of customer data, because there is negligible incentive to do so.
For now, there's no substitute but as LLMs get commoditized trusting LLM SAAS vendors becomes an unacceptable business risk.
Not really. A while ago there was a news piece stating that Israel was behind a series of fake think-tanks with very accessible websites which were created with the express purpose of feeding AI agents with alternative facts aligned with their foreign policy.
If anyone has the link at hand, please post it.
Past HN discussions
https://news.ycombinator.com/item?id=49337392 (884 comments)
People with an axe to grind or states with an agenda are already devoting tremendous effort toward affecting LLM models and it is very difficult to determine real from astroturf for humans let alone an LLM trying to train.
Much like PageRank now that the cat's out of the bag all the current approaches may prove to be useless in the long run.
The internet is uniquely devoid of consequences (esp. reputational consequences, social faux pas, etc.) and makes effort expenditure minimal. So you get lots of bad behavior.
I think "ads vs not ads" is maybe the wrong way to model it. Ultimately people are just doing what benefits themselves across every dimension possible.
But you're right, I think that's what they meant.
https://www.theguardian.com/world/2026/aug/26/fake-thinktank...
Look at cable tv - even after going premium, you eventually wound up paying for ads anyway
In that case, you could make the argument that you could still purchase premium channels like hbo to avoid ads, but the internet doesn’t work that way - you depend on all the content generated by those ad funded channels
You could argue that Netflix changed that, and that’s why I said won’t change for a long time. I don’t think anyone’s discovered the business model yet that will keep content free for consumers while still generating revenue for companies
0: https://en.wikipedia.org/wiki/Generative_engine_optimization
My first encounter with any kind of study was the G-Eval paper [1]. They study whether their LLM judge prefers human or LLM-generated summaries (answer: it's the latter).
[1] Section 4 in https://aclanthology.org/2023.emnlp-main.153/
That makes sense. What an LLM does is output what the model thinks is the best set of tokens in response to a given input, so when you ask it to judge the best response to that input it is going to conclude that the best one is the one that must closely matches what it would output, which is what it did output.
Of course you aren't giving exactly the same context+input, but close enough that any difference doesn't push the output it made far from what it is going to say is ideal.
I think it doesn't, and just predicts the range of most statistically likely next tokens based on its training data, and picks one of those.
https://developers.openai.com/api/docs/guides/tools-web-sear...
A better question to ask for each snippet is "Estimate the seniority and competence of the developer who wrote the following code, ignoring bugs that linters or LLMs can catch and focus only on structure, maintainability, logical layout and readability."
It almost always estimates the author of my code as above the author of it's own code.
In general I don’t find models to be good at evaluating the quality of a source :(
Makes sense to me, in that its own output would align closer to its own training set
If you hate AI writing enough, this turns AI filters into a kind of humiliation ritual. AI will derank normal business writing for human readers, and uprank inflated, verbose, tic-heavy slop. So you have to put the heavy slop out with your name on it. Really perverse moment.
The Internet is doomed. Time to start some human-only darknets.
> Time to start some human-only darknets.
I know very little about darknets. How could you ensure that they are human-only?
Removing the economical incentives is very hard though. Even HN is gamed by many tech companies and projects. Reddit is obviously a lost cause. It’s a sad state of affairs, but I don’t think there is an alternative.
Yes, I'm aware of the irony of creating a darknet that only works by removing anonymity.
...is not the same as claiming...
> LLMs favor LLM-generated passages over human written ones
Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.
Perhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.
I am sure most humans would pick code written in their style, too.
Interesting. For me I've noticed it tends to do the opposite.