And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]
[0] https://en.wikipedia.org/wiki/Operation_Earnest_Voice
[1] https://www.latimes.com/entertainment-arts/story/2022-07-19/...
[2] https://www.tortoisemedia.com/2024/02/26/depp-v-heard-who-tr...
They were instrumental in the whole "Uyghur genocide" thing where the only proof was Adrien Zenz suggesting there would be a possible future decline in population growth and called it a "cultural genocide".
The US State department has been running things like "Radio Free Asia" as state propaganda mills for decades. This has broken out into an industry of insiders getting board seats on various think tanks and churning out essentially policy desires as "studies". These studies are then used as citations for actual news articles to launder the policy desires into something that seems like a reliable source. Then people say "Look! this is really happening!" while 100s of articles are written citing the same untrustworthy sources, laundered through dozens of thinktanks funded by the DoD and state dept.
Neither of these sources support this claim as none of them include any discussion on Amber Heard('s legal team) using bots?
I think your bias is showing. From what I recall, both sides employed the exact same tactics, and we often saw blatantly inorganic discussions popping up in random subreddits on how $(SIDE_A) was the real villain and $(SIDE_B) only had supports because they were paid bots driven by their PR firm.
Things reached a point where I'm sure the same template was used by each counterpart.
On a bigger picture, I'm not sure why anyone still goes to places like reddit and believes anything showing up in the mainpage is still organic.
brand defence, nation-state influence operations.
There are many different flavors of the same thing, trying to mislead competitors or enemies. Or to dupe consumers ofc... Since the tech is more and more available, now also 12 year olds can run insanely large influence campaigns, to boost their streaming profiles or other somewhat benign things, but obviously modelled after what larger players have been doing forever.
The first transatlantic commercial transatlantic phoneline (TAT-1) was immediately tapped on both sides by UK/US... before that it went over radio, which was simply intercepted.
Because of that nature of the coms system being easily listened in on, just like they do/did with the lil mirror-beam-splitters at GCHQ etc.., its been natural effect to then provide disinformation over those channels. perhaps even use those channels exclusively for disinformation, and have a secure channel to do _actual_ coms.
So as you can imagine there has been many decades now of practice in why and how to share disinformation effectively to undermine other peoples work or processes.
Anyways, I got really interested in this topic around mid 2010s because of some really interesting reporting by Codastory[0] covering Russian "troll factories" especially used to influence opinion in eastern Ukraine, but also in many other circumstances. In Houston in 2016, Russian troll accounts planned both an ANTI Muslim rally AND a counterprotest to that rally all on Facebook.[1] Hundreds of real human beings showed up!
Russia got a ton of attention for their use of troll farms but the truth is that the US and certain allies have actually been using stuff like this for far longer and, imo, with much more success. In fact, Coda Story itself is funded by the NED which is a massive pillar of global US, uh, "soft power".
[0] https://www.codastory.com/
[1] https://www.npr.org/2017/11/01/561427876/how-russia-used-fac...
There have been so many bot armies exposed over the years that one would think every narrative being pushed should be considered to be backed by bots. Some state-level actors even operate their narrative-pushing campaigns openly, yet people still happily deny their entire existence.
HN is very much not an exception to this rule. It's very hard to accept that maybe some thoughts you have were actually installed.
Maybe a few are just “people” (i.e. the same people who normally post in the community.)
But in most cases I’d suspect any such botting campaign brings along with it a few human agitprop agents embedded onto each target platform, well-positioned to deny and discredit anyone pushing back on the narrative the bot farm is spreading.
(Sometimes this is pretty obvious — a lot of accounts coming out of the woodwork to deny Chinese bot farms have a conspicuous “English as learned as a second language in Beijing, to a competent but not fluent level” writing style that does not often otherwise crop up in other discussions in those same communities. But other times it can be far more subtle, making use of operatives hired from within the community’s own majority-culture rather than relying on foreign nationals playing the part of same.)
There are many Ivorians and Senegalese promoting the CCP’s ‘Community of common destiny’ on Reddit, for example.
Edit: Community of common destiny for mankind/‘Community of Nations’
Talking about a country that sets up fake think tanks is a good way of triggering them.
The sheer volume of comments make it super obvious when it happens compared to normal conversation
https://en.wikipedia.org/wiki/Pamphlet_war#History_of_the_pa...
> Throughout Europe in the 16th century, printed tracts were used to argue religious doctrine and foment support for religious causes. In England, Henry VIII used print literature to justify his break from the Catholic Church. During the subsequent reigns of Edward and Mary, print polemics escalated into propaganda warfare, as print media gained enormous potential to sway common opinion.
> Over the next century, the pamphlet became the principal means of garnering support for a cause or an idea, and was particularly influential during the English Civil Wars (1642-1651) and the Glorious Revolution of 1688.
https://en.wikipedia.org/wiki/English_Civil_War
> Casualties: Parliamentarians 34,130 killed; Royalists 50,700 killed; 127,000 non-combat deaths (including some 40,000 civilians)
https://en.wikipedia.org/wiki/Williamite_War_in_Ireland
> Casualties: Williamites: 10,000 killed or died of disease; Jacobites: 15,293 killed or died of disease
(never mind the fact that he didn't even have the full truth available to him but you get the point)
Of course, a great deal of panic over fake news wasn’t necessarily over fake news per se, but whose fake news and the loss of control over the “official narrative” that legacy media used to have a monopoly over. This isn’t a vindication of the heaps of slop published on the internet, just a more synoptic take IMO.
People miss Cronkite, because that style of media made them feel certain in their opinions and in “official consensus reality”, regardless of whether it was true or not and whether it was a trustworthy authority or not.
You might have to shadow library this one, but it's a good read. Interestingly it seemed to be something entirely separate from Bernays whose observations were based on a derivation of what began as a homebrewed effort in a corner of the market. Valerie Steele wrote extensively about the trends and the after life but the source seems to originate in corsetry and ended up being copied by the garmet industry generally in the 20s.
Thankfully the resources at our local libraries are a) selected by a combination of librarian curation and public demand, b) often published before 2022, c) heavy things printed on paper or stored in a database and unlikely to suddenly change their content or disappear overnight when a new news cycle breaks. Written works aren’t necessarily credible even if they aren’t slop, but thanks to curators who care, our libraries are significantly less susceptible (although definitely not immune [2]) to ai slop, and propaganda and manipulation efforts than a search engine. The internet continues to enshittify, and truth becomes further eroded by this coming wave of ai enabled propaganda, we are going to have to turn somewhere if we want to find trustworthy resources. Personally, if it gets to the point that verifying sources from public search results becomes too hard with all the new misinformation and slop, I’ll start propritizing human curation over SEO and start visiting my local library’s website a lot more.
[1] https://thenewthings.com/p/apple-big-ai-book-slop-problem
[2] https://www.404media.co/ai-generated-slop-is-already-in-your...
Books where the author has successfully hoodwinked the publishing process into believing that AI generated text is their own, those will turn up, just as plagiarised books will.
Probably a handful of books in such libraries already have some text in them where the author has tried to pass off fragments of LLM-generated prose as their own and have got that past editors, reviewers and publishers, and that will sadly get worse.
I think using LLMs to research is completely fine, as long as authors would take the information and synthesize their own opinions based on the findings. I read books/articles/posts/comments to hear other PEOPLE's opinions, not LLM generated opinions.
There are many studies, but each can be wrong and they can collectively show a bias. Even things of which are the most non-political of facts can have very strong biases. Look at the Millikan measurement of the electron and how in created confirmation bias and an anchoring effect on some property that has absolutely no real world significance to the things people are tribalistic about (aka, no political relevance). Now imagine the same applied to fields like economics or psychology which do have massive legal/political implications.
For a different example, ask the question if X committed crime Y. There are cases where they weren't found guilty but it is reasonable to assume they did. But being found guilty doesn't make it a fact either, as some people are wrongly convicted. Some eventually are overturned, but even if it isn't, it still isn't a fact they committed a crime.
Then there is the simple ambiguity of statements. Language generally can't support facts. It is why legalize, and programming code, and math's are effectively their own languages. For a simple example, consider the Betrand paradox(1).
>Consider an equilateral triangle that is inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?
Is the answer 1/2, 1/3, or 1/4? Well, it is all three at once, depending upon what you meant by random. Now, imagine how this impacts things like research studies, where the randomness is much harder to quantify and there is constant pressure to p hack a result.
1. https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)
The word you're looking for is "reputation". It's been a critical part of society since forever.
One consequence is that one should be extra-wary when asking AI about current, controversial topics.
But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?
Whining about AI is a red herring. If anything, LLMs lowered the cost of producing this type of propaganda.
What's novel about this approach is that the emphasis on building up a facade of authority and credibility is not a central part of the propaganda effort. They can get away with outputting the content in a way thats ingested by other LLMs among the reputable sources, and then they lean upon people's gullibility towards LLM output as some sort of credibility laundering.
There is a reason why one of Wikipedia's central policy is requiring reputable primary sources to support content.
"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."
I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.
I guess the new angle might be that AI scrapers have even less discernment than humans, so you don’t have to put much effort into creating a fake front online to sell it. Humans have at least a little bit of radar for this.
… which means the AI companies now have to run input through existing models to audit it before training on it? That’s interesting.
Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.
I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.
Mostly because they already are responsible for systematizing documents that are needed to recognize us as citizens, land or car owners, etc.
Some laid back countries might allow companies to take and manage that function, but I don't think most of them would. Especially not in current landscape where people heard "you are the product" from all tech companies for the last 10 or so years.