(europeanbusinessmagazine.com)
Except when you look into who counts as a wine expert, it ends up being a bunch of college students and people that took a weekend course.
Financial accounting is a very real thing, it's complicated, there are standards, vagaries, companies do odd things sometimes out of laziness, malice, or necessity.
It's a very real discipline.
>"Wine experts can't tell" is a myth
It's really not a myth. This has been tested several times (at least once a year) in several different ways. Even the most expert of experts don't really have a clue.
[1]https://www.theguardian.com/lifeandstyle/2013/jun/23/wine-ta...
Does such an expert exist? (I have no idea.)
They literally have accreditation + levels, professional bodies, very well defined standards, 'apprenticeships', and usually institutional professionalism aka firms with practice, client bases, standards, internal know-how.
It's totally the opposite of 'wine tasting'.
I'm not saying they're wrong not to want to waste their good folks on boring low-stakes/low-revenue clients but it does feel a bit dishonest that they try to get those signed in the first place when they don't really care about them in the first place.
This might of course be regionally different - it's a worldwide operation so conduct can vary of course.
Part of the test is a blind taste test. Most people fail because it because you need to be able identify not just the type of wine but it's vintage and even year.
Perhaps I'm explaining it badly, but my belief is that most people doing any given thing are quite bad at it, and a small minority of skilled individuals raise the average (mean).
Also a footnote regarding wine tasting in particular, even judges in top wine competitions aren't statistically much better than randomness, see for example: https://www.theguardian.com/lifeandstyle/2013/jun/23/wine-ta...
> a bunch of college students and people that took a weekend course
Do you by this include people who graduated with a two-year degree in the study of winemaking? Perhaps it's not exactly being an "expert", but it's quite far from a weekend course, I'd say. A better term might be "wine professional". In any case, such a sample were unable to tell red wine apart from white wine with red dye in it...
> The wine comparison test was carried out by 54 undergraduates from the Faculty of Oenology of the University of Bordeaux
https://www.sciencedirect.com/science/article/abs/pii/S00939...
You could state this much more strongly even. It's the entire industry, not just KMPG.
The point of AI is to compete with the human workforce by replacing labor contracts and working hours for human workers with service contracts and usage quotas for AI. If a company can reduce its human workforce capacity use by 20% through the use of AI, it can lay off up to 20% of its workforce and spend up to 100% of the savings on AI - even spending only 50% of it on AI would be a major win for AI companies once they operate at profitable margins.
I'm saying that as someone who uses AI every day and has a Claude Ultra subscription. The point of AI is to move the money paid in wages to money paid to the AI companies (or if the AI is being developed and operated in-house, keep it inside the company and thus generate more profit to be extracted by shareholders/investors).
Much as with real people, the context window is addressed through collaboration, and it turns out agents are extremely good at that too.
how well would a human fare, trying to recall the details from ten of these?
The entire consulting industry works on this.
Average consultants are more profitable than skilled ones.
Plus companies get to overpay for the education of juniors.
After all, they have all those fresh ideas the big companies are looking for.
Oops, did I say consultant instead of accountant? AI definitely will run most of an accounting department on its own.
And every year a new human comes along and has to learn all of it again.
Of course ONE AI agent and teaching ONE AI is massivly in an advantage as soon as its as good as the avg person in the job and it blows everyone else out of the water as soon as it gets a tag better.
[1]: https://www.who.int/teams/integrated-health-services/patient... [2]: https://en.wikipedia.org/wiki/Battleshort
But even if all of those other fields did derive the standardization of procedures and checklists from aviation checklists, so what? My point isn't that others invented checklists, too. It's that pilots aren't the only organization that attempts to standardize a process to the point where operators are interchangeable without changing the output.
[1] https://www.solist.blog/pilots-did-not-invent-checklists-in-...
It’s literally a monoculture.
For example, if an iteration of the singular AI ends up being crashable with a magic visual pattern or QR code (highly likely at some point), that is a major national security risk if that AI is piloting aircraft, as the entire nations aircraft could be taken down by anyone projecting said image in it’s visual plane until the fix gets pushed.
And these systems are far too complicated to exhaustively test all possible inputs before release.
There is a reason why nature rarely has such monocultures - that live for very long, anyway.
Its a possible as an attack vector for sure.
But in comparision to nature, compute can speedrun all of this with RL.
This has different failure modes than monocultures, which would also be bad for us: a collection like that behaves as a species, and can evolve faster than one single thing, and evolution is a blind watchmaker that cares only for success rather than details.
Even a lot of what software professionals do isn't verifiable.
For a human you need to train everyone every year and if you have more and more material to teach, harder it will be for the humans.
Not so for AI.
Then we will also see more simulations, an AI will be able to play through scenarios and we will have models being able to invent new scenarios.
While this is happening, hardware will be faster, more and more improvements on every front. Humans have not progressed really
More generally, if you assume that magic can replaces the laws of statistical learning, then yes, AI can do everything and we're all doomed.
I reject this premise on the basis of knowing how RL and LLM training works.
How many people learn one particular job? If its a job which educates 10.000 people per year, when is the financial motivation big enough to create enough training data to make them obsolete and transfer the learning into an AI model?
Also we don't know yet were the journey goes, 'dreaming' models finetuning over night might help.
Or its already just enough to have the biggest RL Loop in history of humanity: everyone of us using the chatbots and giving them signals like thumbs up and thumbs down and stuff like "DUDE WTF I TOLD YOU..." you know :)
Then we also know that the frontierlabs are paying experts and people with PhDs to generate challanges and answers.
Educating just a 1000 people every year over and over is very very expensive even if LLMs need more training data. In worst case frontierlabs will just keep investing in generating traingindata manually with their earnings and it takes. longer.
Or we find a breakthrough like how LeCun is complaining all the time that LLMs are not learning properly. We are still early and progress is still happening fast.
Or maybe cut back on the acid.
Thinking it can detect the unknowable at the time is a big ‘ML magic’ thinking error.
It’s the same thing that fed the grey goo panic around the 2000’s.
Turns out, entropy and the law of thermodynamics applies to nano machines too.
With AI you have to teach one model once. It makes a mistake? Teach it and it will not make it again.
This is a lot cheaper than training every pilot on the planet and its a lot more persistent then training every new pilot
Has this been your experience? Models still invent answers for me. Directives go unrespected, etc.
I expect it to get significantly better in the next few years but I also expect either to see continues improvement or new optimized models for specific tasks.
If you have access to the auditing AI in advance then you would pre-check your bad behaviour to see if it flags. And iterate until it doesn’t.
Bad actors already do this with humans, they know where auditors are likely to look and act accordingly. Even with my limited financial training I learnt where some big murky bits are and how to spot them abuse.
With a human there’s always a risk that they’ll see it, spot the swap in the shell game etc. An AI isn’t exactly deterministic but it’s a lot closer so the audit risk goes down.
One of the most misinformed things I’ve read on Hacker News recently. That’s not how any of this works.
Has the average quality of discussion plummeted since the advent of LLMs and every tech bro starting to comment wildly out of their depth?
How about you are less snarky and try to be clear what your problem with my statement is?
Should I say one model/one model family? etc.?
The task of auditing a business is extremely complicated by virtue of data/information being all over the place. AI had not gotten to a point of where it could outperform humans but in this experiment they find out a group of agents are good enough for this task. I still think there are so many poorly defined tasks that auditors just understand implicitly, eg data discovery; before the audit even starts, that humans will always be needed to initiate and conclude audits just like software projects.
I expect for 80% of the audits AI will be 'good enough.' And those are the bread and butter for the public accounting firms.
But in terms of executing on the check list I think it would consistently do much better.
If you have every asked AI to investigate some sort of mystery - like something no one truly knows the answer to - it is very good at gathering information and even forming good hypothesis. I've done this and when it usually gets to a dead end or gives up, you can find facts that it shares and ask more questions: "You said X and Y. That makes me think of Z, which doesnt directly have to do with anything we talked about, but could it shed any new light?" Etc. Just making connections and sparking new avenues to explore.
I have so say, it's really, really fun to use AI for this sort of thing. To investigate novel problems. Doesn't have to be full on scientific research. You will see that you can really spark progress by just making observations.
The difference is important because I don’t want idiots out there reading this comment and drinking the KoolAid, believing that an entire client’s audit activities can be blindly automated.
Just like a self-driving car’s entire drive can’t be blindly automated, even though X% of the maneuvers that the car has to perform can be automated.
I’m curious if you think this isn’t true.
Pretty much all code is AI generated at any serious company now.
Does not mean 5000 line vibecoded slop, just normal code done faster.
And it's true.
We're getting to a point where you don't even have to look at the code anymore, and I'm serious.
> AI had not gotten to a point of where it could outperform humans but in this experiment they find out a group of agents are good enough for this task
The more this is done, the better it gets. Especially as more resources are poured into making it better.
It's only a matter of time before AI auditing is the common practice and humans are the backup for the AI.
Because FAANG still employ a lot of them, do they not?
Also most companies don't want to pay for people to go around checking every comma since it brings very little value, so there is not time for it.
With that said I think AI will change auditing, as most everything else, eventually and allow different ways of working, it's just a weird flex.
On the other hand, with AI we'll use reinforcement learning to create agents without such evolved-in tendencies. Where (so long as we can solve alignment and cheating) they are trained to do the job, no matter how much effort it takes, or what's involved.
But when it comes to audits/compliance, it's mostly a cost center meant to mitigate risk. Going deeper may be valuable, but it can also add costs in terms of time and attention that just aren't worth it—this is the balance that human auditors walk every day, and it's in a squishy human domain where AI really struggles to demonstrate any common sense let alone street smarts, instead acting like an extremely precocious 13-year-old with no sense of human incentives or tacit understandings.
They are less lazy than humans though
It has a max capacity but you can just loop it forever with that max capacity.
You cant loop a human forever without dropping mental capacity.
And this does not just manifest as the model being lazy under resource constraints, it also manifests as the model taking shortcuts on easy tasks in general, or making assumptions where it could go out and find the facts. Higher effort levels help ("effort" is an inverse laziness slider after all) but it does not completely eliminate it
Or just being tired. Auditors are famously run into the ground.
I think you are right. Auditing is a pretty standard procedure. There is real tricky investigation sometimes but for the most part the formula is to throw data into a rules engine. And there are robust processes in place for handling mistake. There are consequences for errors. If they incur too high of a cost for the firms they will reform the can reform their strategies.
I'm all for expert AIs in the future, but at present, AI systems have a very serious sycophancy problem; in other words, they aren't opinionated at all, which is necessary. It's extremely easy to steer the opionions of an AI.
I'm curious how this will develop, as training AIs for non-binary outcomes is difficult, but probably, for success in certain fields, it will be a requirement.
AI labs were working with companies like Mercor for more than a year paying the consultants and accountants themselves from these very firms a good amount per typical task they work on everyday, some even providing actual documents.
The expectations were more rigorous, and the rubrics pretty expansive. so yeah, the agent trained from tens of thousands of consultants would eventually ask better questions than an individual consultant.
Opus, with access to all of our policies, repo, and the SOC2 control standards is more thorough and exacting about evidence than any auditor I've ever dealt with for any kind of security/compliance review.
It's almost to a fault, it sets unrealistic standards for certain things that need to be nudged back and it's also aggressive about inventing work to do that will keep you busy forever.
I think this is the source FT article
2/5
and
> "one AI agent failed to correctly match regulatory requirements with the company’s documentation"
This is a depressing pattern. The AI can bury you with the volume it produces, forcing you into cognitive surrender. Then it makes the dumbest of mistakes that you're too overwhelmed to spot. Luckily for them, that one didn't make it to the client.
Solutions, implementation, perfect detail - that's another question.
Quotidian FP&A, controlling and treasury functions are done faster and more reliably with AI. I feel like people are purposefully ignoring the evidence here. Most of corporate work IS slop/busywork and that’s what AI excels at. Without AI, I’d prob have 4 more headcount on my team.
Most famously - https://en.wikipedia.org/wiki/Enron_scandal
Is anyone in this space, to speak to whether AI agents asking "better" questions is a benefit, vs. a bad product-market fit?
I know some people in the big four world. At least here (NL), it’s prohibited to be the auditor and provide consulting services at the same time.
Like, there are a lot of problems with auditing as currently practiced, but to imply widespread corruption is entirely wide of the mark.
Once upon there used to be a Big 5, they are now four because of one case like you're describing
That's exactly what it is. And yes, the incentives at every level are laughable.
An AI model won't care about becoming partner. It won't be overwhelmed when it's deliberately snowed in with a blizzard of uncorrelated PDFs. It won't be intimidated by the stature of the target. It won't be exhausted by the conditions to which it is deliberately subjected. It won't care about making flights.
There are huge opportunities here. Much auditing is mandated by statute or contract, and then farmed out to easily and willingly bamboozled firms and their junior conference room occupiers. What might happen if the auditing party chose to leverage the labor capacity of AI and conduct audits themselves? What happens when under resourced adversaries leverage the same power?
A reckoning is coming. I can't predict the outcome, but I can say that the clownshow we have now will not be operative. A new and very different clownshow will emerge to replace it, but along the way there will be great trauma.
The reason these things happen is compliance is paid for by the company. The Enron account was massive, so auditors incentives are misaligned.
There are many downsides to this in that business cannot ever be perfect and a sample-based regime is useful because it allows for a few mistakes here and there it's not waste everyone's time at audit time
So similar to cybersecurity where you're going to need to apply AI offensively and defensively I suspect we'll see the same in accounting
But even moving from sampling to an exhaustive analysis, the time-wasting mistakes could still be rejected as non-material (i.e. too small to care about). If it wasn't possible to automate this, I doubt an exhaustive approach would take off.
The very first time I ever worked with a McKinsey consultant, he turned out to be a jr. consultant half year into his gig at McKinsey. In all likelihood a really sharp and ambitious person, but extremely junior when compared to the domain experts (with decades of experience) in the same room.
3/5
4/5
Alas, would that we had independent audits, ratings agencies, etc. But I'm just a beautiful dreamer that way.
To be clear, this describes the vast majority of coding.
Rules, rulings and following them. I have an extremely hard time coming up with something that requires intelligence but lacks rules and rulings. The rules and ruling might be complex but they are always there even if in the background (like in, say, visual arts).
No. Not currently.
> Rules, rulings and following them
Does not describe an auditor's role in the context of litigation.
> rules and ruling might be complex but they are always there even if in the background (like in, say, visual arts)
Uh, sure. Q.E.D.
I'm not trying to denigrate accountants, I'm just amazed that this is a headline finding in 2026.
I guess for industries heavily tied up by regulatory norms and regimes it doesn't matter how much better AI is at the job. Hence why Anthropic's claims of 15% GDP growth are unserious childlike predictions from nerds who have never interfaced with the world outside silicon valley.
The ThinkPad army will be manually typing business buzzwords into their powerpoint presentations for another 10 years still, regardless of how fast the actual tech is moving.
The accountants will not have this ability to use AI because their fantasy of managing and overseeing IT wasn't able to be handled.
Very much not my experience (which is also why I don't believe the claims in this article). My experience is that when trying to troubleshoot issues with LLMs they are prone to make wild leaps and insistently chase down esoteric edge cases, in situations where a human would start with common solutions first.
Average to below average consultants are way more profitable.
But they can also fail to ask "obvious" questions and get stuck at points that an average person wouldn't be stuck in.
Which makes them for adversarial processes - like auditing or hacking. But no so good at producing defect-free software or planning without guidance. With guidance, they're great aids in this too. But when people AI processes to "turn off their brains" or act autonomously, then you get problems. And you get a lot of pressure to very lightly if at all scan AI output when AI investment is supposed to show productivity gains.