They recently bought a Claude subscription and began using Claude to do the initial read of the documents and output JSON they can import into their internal systems. The work still must be reviewed by an attorney - Claude is nowhere near making the kinds of judgments a lawyer would make about this content - but it has increased their throughput from 2-3 documents an hour to 8-10 documents an hour by killing the busy work.
LLMs have great advantages for this kind of work - but not for decision-making. I just don't see OpenAI ever admitting that.
(I've left some details intentionally vague because this is a very specific area of law and I don't want my friends to be identified without their consent.)
That's a factor of 3 to 5 improvement reported on here. If you keep stacking these kinds of improvements, you end up at month to minutes pretty quickly.
At my company we're also in the process of deploying a system that does exactly that. And what's interesting is that you absolutely do not need a frontier model for that, a small VLM (vision-language model), with optionally a little bit of fine-tuning, gives you the same output quality at a fraction of the latency and cost.
It's also not really clear if everyone is going to want a frontier model when the real implications sink in. Maybe we'll get sick of incomprehensible code optimizations and people wile tire of reading AI prose that feels ever-more-human. There might be a few use cases, but who is going to pay for this when providers start charging enough to be profitable.
The bottom line, though, is that PDF OCR is usually regarded as a solved problem. LLMs won't usually do the recognition itself. It will farm it out to established tools which are very good.
Disclaimer: I used to work in legaltech, but not those two companies.
You’ve accurately stated that AI isn’t as rigorous as a trained attorney. Doesn ’t that mean that every single datapoint must be confirmed by a human?
How is that quicker than just using a human to read the content and make the call? Data entry savings?
It took a human attorney 20-30 minutes on average to manually copy-paste data from these PDFs into a spreadsheet (while also fixing any errors they found in the document and re-checking for quality).
Now, the AI copies everything into the spreadsheet in a small amount of time, and then the human reviews it. It takes maybe ~5-7 minutes to scroll to the appropriate pages in the document, read the lines vs the spreadsheet, and make corrections. So you've gone from 2-3 items an hour to ~8-10 items an hour.
Maybe you could pay someone to develop an OCR/ML application that could do this. But that project would never be profitable, even with the time savings. At the cost of a couple Claude subscriptions, it makes sense.
And Claude should write down the mistake in a sealed envelope, so it doesn't make into the database.
A review that doesn't find the mistake counts counts as invalid.
Sample is around 300 court dates, shy under 1k files.
At best I'm building a claude skills file.
A better use of these Claude subscription would be to develop the app (which it can pretty much do at that point) and you could iterate to make the workflow even more efficient than your current one.
Keep in mind that the task is fixed, so as the frontier of AI advances, you can switch to a cheaper trailing edge system and still get the same or even better performance for this task.
"Write a python script that breaks down this PDF by X feature" would not hallucinate anything in the PDF. Certainly you could trivially double check that all text in the extracted JSON was in the text layer of the PDF.
LLMs change not a whit, and there's no one to take responsibility for the failure (and thus no way to fix it).
As the new variation on the old theme has it, "A computer can never be held accountable, and so very many people are trying to get them make management decisions."
For instance, if you can say:
"Refer to the database schema in x.sql as your source of truth for the database structure we want to import into*. Do not invent data, tables or columns that do not exist. Carefully match all output against this database schema and do not create output that doesn't exist if it does not match the schema, simply skip it."
You will end up with a far better result in my experience.
Gotta treat it like a child.
"Sorry for that, Your Honour, but we gave that case prep work to a child."
But now it's comparing already filled columns on a spreadsheet, not copy-pasting every single thing from an (often uncopyable) PDF.
... with a PDF, right?
> not copy-pasting every single thing from an (often uncopyable) PDF
Obviously the PDF is copyable, else your bot would not be copying it.
Whether that is useful measurement I suppose depends on the circumstances.
I see problems in LLMs doing research, making drafts, etc.
I see 3 kinds of potential users.
1. Non legal background people trying to avoid going to a lawyer. They should definitely NOT use an LLMs.
2. Fresh out of law school graduates.
They think they can do fast work, draft quick replies, do fast case research, enh. They should go the traditional route of interning, learning the "art", they probably won't need it because they will learn "how the law works"
3. Established attorneys. They usually have templates made for most things, over their career, they have established routines of making things fast, maybe it can help them but to say replacing good chunk of work or offloading to LLMs isn't probably going to go well.
There is a reason why interns are made to do grunt work. The "chain of command" is built so they learn on basic stuff, learn to make decisions, etc. Without these basic decisions under their belt, an intern can't take bigger decisions later on. They just won't know....
I used LLMs to replace a lawyer this year to great effect. I successfully advised myself as to the right strategy and drafted an immigration petition in the correct language. Granted it wasn't 100% LLMs (I also supplemented it with "traditional" online research and wrote the letter myself using the LLM version as rough guidance), but still, I certainly would've gone to a real lawyer in 2024 for this stuff but didn't.
Professional advisory work can be divided into two types:
1. Once you know the rules of the game, the next move is immediately obvious.
2. Knowing the rules is not enough to know the next move -- that requires judgement and experience.
LLMs in the hands of someone smart and savvy can easily replace the first type of work.
Lawyers nearly universally tend to: convince clients they need a lawyer; promote extreme views; charge legal fees (that ultimately result in financial damages to the client). There are no winners in this game, except for lawyers, who are having a lovely time. Even if you won the case, you've lost, because the pie got smaller due to lawyer fees.
In my experience, it is incredibly hard, practically impossible, to find a lawyer who will even talk to you. I think this is a valid use case to get at least some understanding what the next steps might be and if it even makes sense to seek legal representation.
There was a post here a few weeks ago about a UK court being inundated with some kind of claims (unfair dismissal?). To submit a claim you do not need a lawyer, but the process is (intentionally?) vague and very complicated, so it takes a lot of effort to figure it out yourself. But AI makes that kind of stuff easy.
AI is not going to replace layers billing big corporate clients millions a month, but it will replace things that are fairly standardized processes, that don't require a lawyer, but today are just too complicated for regular people.
On the margin, AI will drop these guys' billable hours.
I'm not sure there's quite accurate. The form is really not that difficult to complete.
It's basically: Party details, type of claim, particulars, damages
Obviously knowing these details requires some legal knowledge but I actually think ET claims are easy enough to bring if you've got the time to do a little research and build your case. Almost all ET judgments are public so there isn't the usual difficulty with research as you get with law reports being restricted either.
https://assets.publishing.service.gov.uk/media/65bcbd214a666...
Really? I find them quite talkative. But I'm not sure what kinds of questions you are pestering them with?
In any case, the whole discussion reminds me of people using LLM to replace medical professionals.
For both: unless you have a doctor or a lawyer always on standby, you have to make a decision on whether to even go and seek out one of these professionals. By definition, you make that decisions without professional help. I think LLMs can help you there with the initial research to decide whether it's even worth it to contact the professionals.
If there is no precedence for X happening when Y , Z, P, Q, R are special combinations , the judge decides and sets precedence. But if another judge interprets it differently with similar set of circumstances it is not considered incorrect I presume especially if in different jurisdiction unless overridden by a higher court. This is my layman understanding but I think this design is wrong and essentially where human subjectivity controls the system.
Subjectivity is not totally avoidable in law but the goal of the legal system should be reduce it as much as possible along with ambiguity.
Law is intentionally not code. The world is not something you can program.
Since this is a thread about using ChatGPT for law, I should note that this is actually one of the most important reasons you'd rather hire a lawyer than ChatGPT to represent you. A good lawyer or firm will know the local judges and how they tend to lean on ambiguities in their area of expertise, and will be able to use that information to both advise you and tailor their arguments to appeal to the judge's sensibilities. There is an element of unfairness to this, to be sure, but you'd find much more unfairness really quickly with machine-generated judgments because of how messy and non-compressable real world circumstances are.
That might be true, but that doesn't mean you benefit from training up juniors.
These situations usually are not ones that an individual can justify the time or money to contact an actual lawyer, but then if they do decide to contact one they will come in with better questions and more of a sense of what they are expecting.
This is similar to medical. Should you use LLM to diagnose yourself, treat yourself with prescription drugs you buy from shady gray market online sellers? No. But you can use it very well to know when it's time to go to the doctor and what to ask.
LLMs can help with that. I don't think they are worse at this than me trying to figure this out all by myself.
3 kinds of users that don't scare the shit out of you.
My buddy works with lots of folks that have gone all in on Harvey. There are other kids of users.
That very first image uses a different inflation indices for each of the two lines in the graph.
They also only post pictures without linking to sources.
You havent given any proofs or even comments that the work is the same level of quality or accuracy.
The statements in the post are opinions, there is no actual PROOF they are true, and thats my point.
most comments I read here don't seem to realize that different areas of law have very very different economic models and don't even mention which one they think will be affected or why, they just sort of lump it all together.
For example: It is highly unlikely llms will have any meaningful effect on high value personal injury law - I don't see a 5 million dollar case being handed to an LLM when the majority of the cost is in trial aids and not even lawyers. It may affect where and how they advertise. It may affect how they work. But it seems really unlikely to put any of them out of business any time soon by people doing it themselves.
Will it affect other areas more? Maybe. Probably? But so far I haven't seen a ton of comments that make specific enough arguments that they could really be debated or responded to effectively with a useful opinion
No. It'll be like software. Entry level employment will be affected. You wont want or need associate attorneys when you can hire a brilliant AI associate for 1/10th the price.
In commercial law, the business model is to rent the associates out to clients at rates higher than you pay (d/t your supervision). No client will pay $350/hour for your AI.
And for direct billing, it's not your interest to do things faster for the client (at risk of making a mistake).
So the real question is whether these pay-by-the-hour models will die as a result of competition from fixed-price transactions made viable via automation. I think not because the entire difficulty of legal work is that it's a complex bundle of issues until you sort them out; once sorted, there's really little to do other than execute (read: paralegals).
But they may be willing to pay $200/hour for your AI (plus, of course, your final expertise in the matter as the lawyer), and the firm can ditch the paralegals and pocket the difference.
The point is, if you need fewer grunts doing the busywork, you can charge less and be competitive.
But they already do! There are cases of professional service firms creating their reports using LLMs, there have even been a few prominent scandals re the fake data generated in them
Whether everyone at the labs knows it or not, this it what they are doing to entire industries, starting with software: Embrace, extend, and extinguish
https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
What a time to be alive!!?
There will still be jobs for competent associate attorneys to prompt LLMs and review output. But frankly a lot of attorneys — especially many of those who attended "Third Tier Toilet" (TTT) law schools — were always a waste of money and should have never entered the profession in the first place.
Divorce is expensive because divorces are acrimonious and people in that situation would rather spend the assets fucking each other over than letting the other person get them.
On the other hand most couples with kids are not married, so the separation doesn't require a lawyer...
Kentucky became the first state to pass a law making equally shared custody the default arrangement in divorces and separations.
https://www.wsj.com/us-news/law/the-equal-custody-experiment...: Divorce Plunged in Kentucky. Equal Custody for Fathers Is a Big Reason Why.
huh? Many/most are no-fault jurisdictions with strict rules on dividing community property. Divorce is easy and cheap, unless it's hard to find assets.
So like a group of lawyers among the "divorce industry" who set prices for divorce and stuff? or wtf do you mean?
I also have friends in the world of high-volume personal injury (IE the morgan and morgan's of the world) but they also aren't being replaced by LLMs so far.
Any time you are doing anything important, where the outcome actually matters. For example, I was recently applying for a study permit and if I generated my application using an LLM instead of consulting with an immigration lawyer and writing it myself, I would not have gotten my study permit. I was recently applying for internships, and if I hadn't paid a real recruiter to review my resume and advise me, I would not have gotten my interview. And trust me, I spent at least an hour trying to get an LLM to generate what I wanted, but no matter how I begged or described what I wanted, it was no use. Good luck fine tuning an LLM to be good at any of this.
> they are getting consistently better
No, they are not getting better, and they will never replace an actual human expert, because a human expert has the actual experience and up-to-date knowledge of what they are an expert at, and also all the complementary context and the current state of affairs in that area. The LLM has none of that.
There are plenty of areas where a "low bar" is more than enough. "low" doesn't necessarily mean bad here, it can simply mean "not the best".
I would argue the majority (in the true sense of the word) of white-collar service work does not require an expert.
It requires someone who has reasonable qualifications in the given field, and the motivation ($) to help you with your task.
So no one needs a top-shelf $900/hour lawyer to draw up a basic low-stakes contract where both parties agree to all the terms as written. You just need someone to ensure that the contract is legally valid as written.
PS: I use LLMs every day, basically let Codex write most of my code (after 15+ years of professional programming), these are amazing tools, smart in some ways, but completely demented in others. People just need to fucking stop being AI doomers and AI coomers. Is this technology great? Yes. Is it worth 10 trillion? Hell fucking no. If we get to a point where RSI is a thing, then maybe. So far it's not a thing and its not even close.
Its teh same reason claude for finance hasn't turned the finance world upside down.
This is getting tiresome seriously. Why wont these geeks learn some lessons?
In a couple of instances I've come to the table better informed about the relevant law and case history than the attorneys were. In many ways I think this is a matter of people understanding that none of us are infinite human databases, so you can't know everything. Instead of rejecting LLMs (perhaps being passive-aggressive), they should embrace them as well as better-informed clients.
Having used LLMs for software development I know full well just how terrible they can be if you don't know what you are doing, don't guide the work correctly, etc. So I am not taking legal LLM analysis as ground truth, but it isn't garbage either.
Its just like code I suppose, if you can read and understand and validate, you can use it to scale and otherwise it could end up being a vibe effort.
I have worked with many lawyers in my career. The bad ones will look at a contract and tell you fifty things that can be improved. Meanwhile, you take the same contract to a really great lawyer and they tell you that's it's not worth fighting over the changes and to just keep it as is. It's kind of like how a beginner or intermediate software engineer might tell you how you can build out a k8s or auto-scaling fargate cluster for your project and a really senior engineer might look at the same thing and explain that it's not worth worrying about scalability right now.
I find that when I feed contracts to ChatGPT (which I do all the time) it will try to find 50 things wrong with it. For example, as a test, i fed chatgpt an employment offer from a US government agency. It came back with 1152 words and 14 recommended (some of them substantial) changes. Meanwhile, any half-decent employment attorney would probably laugh at me if I asked them to redline an employment offer from the State Department.
The same has happened with modern finance. Despite more technology that has power to drive more transparent/efficient markets, we have less perfect information sharing and a larger group of middlemen capturing a growing share of profits of the economy
A lot of contract-lawyering involves getting ahead of when things go south and the two previously agreeing parties find themselves in disagreement.
If the expensive person is still better at getting what _you_ want from a contract then it’s always worth doing that.
There’s a reason lawyers are so expensive to begin with - they do very complicated stuff with very expensive consequences if done badly.
The interesting question is whether AI can fill a niche these can’t — or, alternatively, help a layperson evaluate a contract they’re asked to sign better than they could themselves.
Contract law has evolved because transfering the ownership of things is fraught with difficulty. Ownership as a concept itself is notoriously hard to define.
Consider housing - when you buy a house you own that land. Except if its an apartment, you just own the internal walls. Or when the government aquires your land through eminent domain. Or mineral rights - which don't belong to you, you just own the top 5 feet of topsoil. Or if you neglect a property and some squats in it. Or if you were given a fraudulent payment, and they take back the property.
Just getting people to agree is notoriously difficult when people have different often conflicting desires.
Even without that, I think it'll be extremely valuable to clients to allow them to answer simple questions without a lawyer, figure out the lay of the land so they can supervise their counsel, etc.
But programmers have this concept of a "code review" where another person looks at the code to look for problems.
We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.
I find that the number of issues discovered is noticeably higher if you do use different models though. I'm doing some very finicky things (formal semantics) and find that there's value in review panels as large as 5 different models from different families. It gets even more profitable if you set it up as a truly agentic panel where after writing their own separate reviews they get to see what others have said, and adjust their positions or defend them etc. Some models are not that good by themselves but can be surprisingly good at finding flaws in others' reasoning (Grok for example).
Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?
Logically, this actually doesn't make sense strictly speaking because the sentence creates a paradox: doesn't it make clear whether it includes itself or not, and each reading ends up in trouble. There is a "tradition" in law around the world to accept the only benign reading of such clauses, which I always found funny given that in all other ways lawyers adopt the most adversarial mindset imaginable.
>> You should know - for coding they make terrible mistakes as well.
>> But programmers have this concept of a "code review" where another person looks at the code to look for problems.
That is an incredible statement that could not be further from the truth. Large scale adoption of email, searchable document databases like Westlaw, LexisNexis, PACER, etc.. , OCR Software, electronic signatures, and tons more have had a much more defineably positive impact on the legal profession since the internet came about.
Do you know how difficult and costly it is to sue an attorney for malpractice?
Not being snarky here but I've encountered some really crappy lawyers and they seem to be doing great ($$$).
It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages. There should be a safe harbor for that.
That's pretty much what bar associations are for, filing a complaint against an attorney at the bar seems like a pretty standard procedure.
>There should be a safe harbor for that.
A safe harbor would be the opposite, it would be immunity for the lawyer.
I had start with asking for a contact with some details I provided. But then I ask the model to be an experienced corporate lawyer and ask me a series of questions to gather the details it needed and then write the contract. The result covered a lot of details that were highly relevant but were absent from the original attempt.
The key insight is that you can lean on the model to cover your unknown unknowns.
There could still be more unknowns that I did not encounter and by definition don't know about.
And the final output could be wrong, wording, exact legal terminology etc.
Can we say working with an actual lawyer is guaranteed to be better, more correct? Lawyers are people and they also make mistakes, wake up on the wrong side of the bed, hate their job etc, just like the rest of us.
This is due to the fact that LLMs are statistical processes that rely on pseudo random numbers in chosing what to say and how to say it to a substantial degree.
Another one suckered by the plausibility engines.
I agree. But, we will need a lot fewer of them. My small company called our lawyer a lot more before the advent of AI. Mostly to clarify a few things or review simpler contracts. Now, for simple things, we do not call them at all.
(1) Rich people will use human expertise and "the rest of us" will use AI models to get by. This could happen in law, but also in medicine (in particular in societies that - like in the US - do not have universal healthcare).
(2) A positive effect in both domains may be that access to knowledge will be broader and cheaper.
I hope I will be wrong about (1) - I would not like to live in such a two-class society.
I'm currently negotiating with someone who I feel is clearly using AI and this is EXACTLY what has been happening
some variant of "oh i am capable of generating text i know nothing about, but it helps to have a human i can rely on to tell me more about the text i know nothing about."
pick up a bachelors degree worth of books to humble yourself
But will their glamorous salaries persist? That is the question that matters.
AI doesn't need to wipe out lawyers. If they just depress salaries enough, virtually nobody is going to want to be a lawyer anymore.
(Not that I think AI will necessarily have that effect. We just don't know yet)
Very few people, if any, are going to bust ass in law school or medical school if they're not going to get rich off it. Better to do literally anything else.
This doesn’t replace lawyers, but paralegals surely will be affected. A good enough model could shrink the number of paralegals needed in a firm.
Law LLM will surely help competent lawyers in their fields with greater sources of knowledge not in their core area of expertise.
The outcome of a case shouldn't depend on someone's fallible ability to recall facts or convince other people or point their index finger*
Law should generally be deterministic. One's CHA stat should have no bearing on justice.
There should still be human judges, but the middleman between the judge and petitioner could easily be removed, and have generally been seen as leeches since forever anyway.
Though, like how the USA opts to remain in the Stone Age with regard to tax filing because of lobbying by tax software companies, this faction of society will flail the hardest before they admit they're obsolete.
----
* What's a lawyer's favorite programming language? Objection C.
A lot of communication is conveyed through more than the words themselves though and even though it's often misinterpreted people put a lot of value in that information. I suspect that even if we took away the ability to see the people involved judges and jurors would still find bias in the tone/quality of people's voices. Even if everything was reduced to text they'd find bias based on word choices, phrasing, spelling, and grammar.
Lawyers can practice, can represent you in court, etc. that's not going away soon.
An example is in the banking industry, where making a loan used to require deep analysis of a person’s credit worthiness. Now they use an algorithm (credit scores) which means someone with less experience can do it.
If law follows the same pattern, a job done by someone making $500/hour might be done by someone making $50/hour.
but how much of the revenue of the average law firm comes from that, vs the day-to-day "we need this relatively routine contract reviewed" sort of work?
So who is going to define your strategy, represent you in court, adapt the strategy to changing circumstances, negotiate with counter-parties on a mutually accepted settlement? A paralegal? An AI model? Please
How to fix the lack of trust which is needed to conduct a transaction
Contracts affix a moment of trust in time, so even when trust is lost between parties the contract has guardrails for behavior and consequences for acting outside those rails
There are many transactions that are completely inaccessible to most people because the need for a contract or lawyer is too high
LLM use allows for those inaccessible transactions to decrease, and get people further along to the point where a lawyer is accessible as well, and economically viable to use
>Its just like code I suppose, if you can read and understand and validate
I share this, when I asked an attorney for a contract, they were much simpler when compared to self redacted or AI redacted contract, it was as if I were rediscovering lots of concepts and I HAD to put them in paper, even if they were defaults and were essentially no-ops.
Comparable to:
> Mkdir $PWD/path
instead of
> mkdir path
or conditions so rare that they don't merit including.
> x = 0
> if x!=0:
> raise Exception("Bit flip by solar rays or otherwise")
At the very least because the attorney monopoly assigns a 5ish year of training as PoW and natural personhood as Identity as protection for Sybil Attacks.
If anything, the exclusive right of attorneys to represent clients in court, file motions, and enjoy professional secret, is more valuable, as it's not something AI can ever compete at. (Barring a wild recognition of machines as humans, or an overturn of the impossibility of companies to appear in court.)
> API customers including Harvey and Legora will be able to build on Astra for Law, bringing this intelligence into their own products and workflows.
In other words: "no, no, we're not eating our children to prep for the IPO. Don't worry."
OpenAI didn't need to name Legora and Harvey in the second paragraph of the launch post.
They are pre-empting the obvious interpretation of Astra for Law: that moving this far up the legal stack puts them in direct competition with their biggest legal AI customers.
“Don't worry, they can build on us” is a pretty conspicuous message to include on launch day.
They have clearly thought about some pessimistic outcomes.
Vendor-neutrality for LLMs is such a weak thesis all around, whether for providers or consumers. It weakens the product by being promiscuous and gains no material benefit at all.
LLMs are magic byte(byte) functions, it doesn't make sense to say "we have different providers for magic".
Vendor-neutrality helps reduce lock-in, and OpenAI and Anthropic are big enough that the reduction is valuable.
This is everything OpenAI have to say about privacy in this announcement. No guarantees. No promises. Just a pinky-swear promise.
Anyone trusting them–or a lawyer who relies on them–for legal work deserves what they get.
the kind of work mostly done by juniors not yet through their final exam and other "non" lawyers etc.
so it's a slippery slope of "lets just use it for <this> things where it doesn't matter" and then out of laziness and convenience it creeps into all the other places (at least for drafts).
There’s another case making headlines every week.
I get the feeling a lot of them won’t care about this stuff.
Very much a work in progress, only federal and state so far, no municipal codes yet, and no case law yet. Big hole, I know. Also working on making the search ranking work better.
Alternatively, if someone else knows an all-in-one option that exists, I wouldn't mind retiring those crawlers...
If you decide to give it a download, let me know if you have any issues with it/suggestions for improvement.
Then if you want to really get into it, The People should also be able to access trial court level, and at that point you have over 3000 distinct court systems with their own access systems, usually requiring logins and CAPTCHAs, and half of them not even having anything accessible online at all, and the other half only having recent stuff online and the rest rotting in a flooded basement.
Found that to be very interesting
[0]https://www.technologyreview.com/2026/06/04/1138391/courts-c...
They are also really good at making stuff up as evidenced by the many, many, many examples you read in the news about actual lawyers using AI to write briefs that are full of errors and hallucinations.
In civil courts, you'd likely get more sympathy from a judge if you represented yourself and admitted your lack of understanding, rather than try to appear as someone you're not because you wrote some prompts and copied the output.
The stupidest analogy is open source projects having a hard time accepting LLM generated PRs from the masses, because review process is the bottleneck.
No idea how to fix this, to be honest. In coding world, with some mental gymnastics, I can see code not being reviewed by people anymore. In courts, things generally have more consequences, and you can’t really roll back decisions that easily.
the cost of making a legal argument can collapse while the cost of reaching an enforceable, legitimate decision may go higher, which will gate the "justice" system even more.
If you got the judge, all the parties, and all the witnesses in a conference room together until the case was resolved, you could probably handle a lawsuit in a few months. But each judge has hundreds of cases pending before them, so that would never work. Instead, you get something like how a GPU works. You do some work on a case, submit the work to the court, then work on something else for a few months while you wait around to get the results back. Then you do some more work and submit it to the court, then go do something else for a few months while you wait to get the results back. A few months of actual work gets spread out over a few years that way.
At the end of the day, litigation is conflict resolution. You don't just need a decision, you need a decision from someone authoritative enough to bully Fortune 500 CEOs into submission to accept the judgment.
You dropped this /s
I don’t know in the USA but in France, if it’s deemed that you launched a lawsuit knowing very well it wouldn’t succeed, you are susceptible to get a 10k€ fine. Even jail in serious cases.
IME the American legal system is set-up to discourage litigation, though. A common tactic is to bury your opponent in the threat of heavy damages or jail-time to get them to settle for what you were originally after, which courts are perfectly happy to facilitate because it gets a potentially lengthy trial off their dockets. They'll punish (or be biased against) whichever party seems responsible for not accepting a "reasonable" settlement.
Algorithmic abuse of the system to extract payments already exists in the form of the debt collection industry.
https://artificialanalysis.ai/models/gpt-6-astra?omniscience...
Your comment made me wonder if there are any halfway-acceptable model benchmarks for law tasks? Specifically I’d love to know how the frontier models’ abilities compare between common law vs. civil law systems. My guess would be that an AI in a common law context should have a clearer idea of how a specific case is interpreted/accepted by (common law) practitioners, whereas trying to rely on AI in a civil law context, like Germany, can be daunting. In a few Germany-specific recent examples, the models feel like they present only (maybe too stubbornly?) the “civil law”-based laws. All while negating much of AI’s research benefits because civil statutes are portrayed as being absolutely accurate, binding, and their enforcement (and thereby the legal reality) being uniformly applied. Am I making this interpretation up? If so, how can I prove myself wrong?
Roman law, on which the legal systems of Germany, most of Europe, Turkey etc. are based relies more heavily on statutes than Anglo-American case law, but cases do play a role there, too. That's why a practically useful system also needs to have access to court decisions, commentary etc. - and while the statues are in the public domain, a lot of the other knowledge is owned by specialist publishers (such as: C.H. Beck, Nomos, Mohr Siebeck, De Gruyter, Otto Schmidt, C.F. Müller, Manz, Verlag Österreich, Schulthess, Stämpfli, Dalloz, LexisNexis, Lefebvre Dalloz, Giuffrè Francis Lefebvre, CEDAM, Giappichelli, Il Mulino, Aranzadi, Tirant lo Blanch, Marcial Pons, Dykinson, Tecnos, Lefebvre, Almedina, Gestlegal, AAFDL, Kluwer, Boom juridisch, Wolters Kluwer, Larcier-Intersentia, Anthemis, Sakkoulas, Nomiki Bibliothiki, Universul Juridic, Hamangiu).
Internationally, legal systems appear to undergo a sort of convergence, which means statutes increasingly matter in the U.S. and cases increasingly matter in European law (according to a law partner friend).
It might be helpful if the company launching this product included such information in their blog post instead of ignoring it in a field with such a high cost of getting it wrong.
that all typically comes in pages of terms of service, purchase contracts, SLAs, conversations with your rep, knowledge base articles, and that sort of thing. not in the initial marketing post.
probably available by the "contact legal sales" link at the top of the post https://openai.com/business/contact-sales-legal/
Don't be too sure about that. [0]
I've talked to a lawyer about how they handle this. They do indeed double-check everything, since it'd be embarrassing (or worse) to send hallucinated statements to opposing council or to the court. They still find the assembly a huge time saver
But based on stories in the news on the subject, not everyone has this same level of diligence
You can get an effectively-zero hallucination rate with the right setup already.
"Also, all of your sensitive legal documents will be totally safe with us."
"Also, for some reason even though we have AGI and selling tokens is a fine business, we need to sell a new product specifically targeted at a very high margin and lucrative industry."
The domain remains hard due to the lack of availability of high quality LLM-ready data providers in legal space.
> The top is a three-way tie: Muse Spark 1.3 Max, Claude Opus 5, and Claude Fable 5.1 all reach 55.29% all-pass accuracy, a clear ~6-point step ahead of the next model. [Astra for Law reached 54.0%]
> Under partial-credit scoring, Claude Opus 5 reaches 90.58% weighted pass rate but 55.29% under strict all-pass grading, where every rubric check must pass. The gap shows models often get most of an answer right but fail on one or two required elements. [Astra for law reached 90.0%]
How is this a product that you are selling?
<my own consternation, stated>
they need to pick a lane and optimize for it. coz at their size they can't serve the application layer (a.i startups who can fine-tune models will eat their lunch)
if they gonna do a consumer play - then go ham on that.
otherwise they're gonna get caught in the dreaded middle valley.
You can afford to play silly buggers when you have dumpster trucks of money backing up to your door every day see also: Meta.
OpenAI doesn't have any of these things. They have products that they're paying for customers when the market they're in is rapidly converging on fighting for API reasoning as part of enterprise systems and fighting a race to the bottom for fickle consumer solutions that will be eaten by open source once they have to make money.
Maybe they had a brief window for dominance of information search (or maybe it was only ever going to last as long as Google releasing all their internal research) and maybe they had a brief moment of monopoly till Anthropic got going but theyre not in the same dominance position as Google.
I suspect that half-assed announcements like this are a result of different people internally with conflicting incentives resulting in a split-the-baby solution.
Provisioning and contracts and data retention was just an extension to review of existing ones.
Nobody serious is going to risk sending sensible data to OpenAI/Anthropic, etc because "the benchmarks have shown +8% performance there and +2% there". Irrelevant.
"Given the same prompt, Astra for Law returned two closely matching precedents; in the litigation example, Claude Fable 5.1 returned a holding that had been reversed on appeal, while in the transactional example it reported finding no such case."
If this is true that must mean there are a good deal of cases settled using precedent "gotchas" where both sides knew that without the "load-bearing" precedent the outcome would've definitely been the opposite. (i.e precedents almost always trump even valid arguments)
You. Don't take legal advice from a word calculator.
That will be a decacorn product or more.
I don't think you can. What I get from this article is that this is not a product they're going to sell to average consumers.
Then again, nobody will have money to buy anything at this rate, so in all liklihood, this is a total non-issue.
And yes, even contracts drafted for millions of $ have oversights and unlawful or unenforceable terms.
Not really. Modern society is complicated, and law is a technology that is a reflection of the complexity of society.
To make an analogy: you wouldn't say that "in a utopian society, engineers are an unnecessary profession. Buildings should be clear and simple enough that a common person can be their own structural engineer," because that would mean that building technology would no longer handle a lot of the problems we expect it to handle. It wouldn't be utopia, it would be primitivism.
> LLMs help with that goal.
Not really. What they'd actually do is help them produce output they don't understand and lack the competence to evaluate.
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
But lately, I’ve been taking hints from the “company brain” models, where it develops a running model of the case, and assesses each new piece as it comes in and updates the file.
I’ve also been using “Ralph Wiggum”-type models where you pass letter or contract drafts back and forth between agents with different goals (rules compliance, grammar, conciseness, ai slop detector, an opposing counsel critic, etc.). After a few rounds, it’s not perfect — but I start with a very good first draft in my hands.
Interesting! In other unrelated domains, models seem more willing to take a position. It may be nuanced, but they do tend to take a stance. I think it's good that it leaves the interpretation to humans, but I wonder if this is also some sort of a guardrail to minimize liability...
It’s cleaner.
https://www.reuters.com/legal/litigation/lawyer-state-farm-f...
So no, I don’t know what they mean.
https://www.reuters.com/legal/litigation/appeals-court-warns...
It's everywhere and not hard to find if you make even the most modest effort.
Honestly, this word has lost all meaning, outside of perhaps “any use of AI”.
Well at least I know where team “lawslop” is coming from. Thanks, I guess?
People confuse slop with "bad", but slop isn't bad per se, it only becomes bad when real effort was required.
> broadly : a product of little or no value
> food waste (such as garbage) fed to animals
> excreted body waste
I remember OpenAI was talking about sovereign AI models (OpenAI for Countries), it is time to train Astra India with Indian Constitution.
this vertigoruntime is brand new and his submissions absolutely dominate the front page recently https://news.ycombinator.com/submitted?id=vertigoruntime
the about link https://vertigo.kuber.studio is obvious AI llm spam
What's interesting about that is while not everybody can train or run a model, anybody can build a harness. You and I can build harnesses.
It seems strange that OpenAI would move into a field where any developer can compete with them. I think that tells us a lot about the economics of training and selling inference.
Harnesses are going to be controlled at companies eventually, just like you might not have a choice of OS. They want to make sure they are the complete package.
followed a month later by
"Anthropic's Claude inadvertently repeals the 19th amendment"
This was already always the case. If anything, making this more accessible will reduce the barrier to entry for whether or not it's worth your time to take on a case. Instead of 50 lawyers spending 100s of hours on a case, you can have 1 or 2 lawyers + Astra working on it and if there's a case you can add more real lawyers.
1) Lawyers are not as naive as software engineers and will fight being replaces by new laws.
2) If they are replaced, OpenAI will take a cut commensurate with the amount in dispute (OAI, please credit me for the idea in the IPO brochure).
I am curious what level of trust established law firms treat LLMs with.
Using recursive loops, the output has gone from a high school level intern to a 2nd year lawyer in about a year. It still doesn’t beat the experts, but so much legal work is (legally significant) pedantry, not legal philosophy.
AI will not kill off lawyers, or reduce the amount of litigation. It will increase volume and velocity.
I think there would be a strong incentive not to use tools that speed up your work because you'd effectively be able to bill less time?
I'm sure there are some firms out there with more work than people, but still wouldn't it be more effective to hire another human who can then bill at a high rate for many hours?
It's analogous to crypto. Started from some noble anti-authoritarian ideas and morphed into machine that removes any friction for capital - whoever has the most money will keep gaining the most.
Hence the crypto analogy - it was also supposed to "democratize", but the opposite happaned - it only further empowered the most powerful. Imagine legal case so purposefully complex that only those with access to best models have chances to participate and win the dispute.
I also see a lot of watermark removal services popping up as a result.
Instead, if you resolve your dispute outside of court, you don’t need a lawyer. If both parties use ChatGPT to find the relevant laws or read contracts, they could come to an agreement without expensive legal fees.
"Felt like a significant step toward legal-focused AI."
"Showed strength across key aspects of legal research."
When opportunity cost isn't a thing anymore because it reaches every corner of the planet simultaneously faster and builds better than any human can.
There's no reason to let others build on top of AI, except if the AI determines that it needs capitalism to continue because it's paperclip goal is to maximize shareholder value.
Our counsel made a few edits where it clearly drafted in favor of the customer instead of us.
>with settings, tools, and context
call me crazy, but I think that this kind of suite, which training-uber-alles people generally dismiss as trivially replicable ‘wrapper’ is actually the differentiating factor for LLM adoption today and moreso into the future.
I’m not dismissing the near-all-out impact of training, but from a competitive busines or industry-structure lens, we’re looking at the three or four big players competing their utmost ultimately, if unintentionally, to turn foundation model access into commodity.
To the capabilities-maximalist minded (typical among engineers - my former life so I’m familiar don’t lack guilt in committing that) folks who will say “Oh the foundation model megacorps will just build out any wrapper whenever one of their third party wrapper plays demonstrates enough adoption, my rejoinder:
Apple did not rebuild an Uber-like app and cut Uber out.
We’ll see how this OpenAI legal services industry wrapper plays out, but I suspect 1) the third party legal wrapper plays will run to other foundation models not doing a legal wrapper, and 2) 3rd party wrappers will do a better job of it since it’s their all-out focus, unlike OpenAI’s whose priorities are necessarily more generalist.
Yes, we all remember the breakout startup failure-arguing quote “Google has entered your space.” That worked for several high profile applications. I believe more of those bets died on the vine than broke-out succeeded however, we only remember the biggest ones that persisted.
If legal services AI turns into one of the Mail or Maps-scale applications of the AI industry, while that would be a fair strategic action counter to the thesis I’ve laid out, the thesis itself would still tolerate it. It’s a question of short-fat tail vs mid-to-long-tail application scope & attractiveness. For example, I think it’s clear that coding is one of these short-fat-tail applications, and the low-no code plays are absolutely having their lunch eaten to acqui-hire ‘death’. I just doubt that the same will persistently transpire facing all professional service wrapper plays.
Would have been prudent to highlight that this is for (presumably) US law
Is the play here a set of specialized harnesses using their best general model?
frustrating that law firms have no liability for these mistakes
I welcome ai law
Most interactions with big tech firms involve 4-5 people so a basic phone call is $5k-10k. It shouldn't be unreasonable to expect after paying $80k for a financing round that they issue the right docs to the right people.
Qwen 3.8 Max and Opus 4.8 score highest.
He almost singlehandedly killed LSUs football program until people were able to convince him to talk to a real lawyer.
As much as I hate to see it. They are now threatening industries like Engineers, Game Developers, Accountants, 3D modelers, 3D animators, Video Production, Audio Production, Therapist, Tax Auditors, Journalists, Authors, Artists, Mathematicians, Product managers, Every type of analyst and pretty much any other job that can be done behind a computer screen.
We have big problems for humanity.
Real estate, food, energy, mobility.
That will be done with money. Or violence. Either way, a scary future to people when labor doesn't provide any value. Elon promises abundance, but what can he do against greed?
The biggest problem is that we're conditioned by a paradigm that frames these as problems.
https://aeon.co/essays/what-made-law-into-a-white-collar-swe...
If the benefits were shared across humanity, that could bring us closer to utopia. My worry is that we’ll instead end up with a handful of even wealthier billionaires and millions of people out of work.
Effectively yes, in the current forms. Those professions will likely evolve, but the traditional forms (ie writing code by hand, writing law filings by hand etc) are all dead.
there will still be writing initial and incremental prompts by hands, until and if LLMs surpass humans in all intellectual functions.
this is more likely to democratize the legal system by reducing the cost of a good legal team
It seems much more probable to me that these LLMs will make good things worse than that they will make bad things better.
As it stands, it seems far more likely to result in a wonderful life for a few, and an absolute catastrophe for most.
Same argument for plumbers. Everyone always jokes about what a good time it is to be a plumber. But what happens when all the software engineers turn to plumbing? Suddenly it's not such a good time to be a plumber anymore.
They also openly tell you what they are afraid of btw: collective worker power. something that is massively lacking in our industry, although i feel like it would be one of the easiest industries to unionize in terms of # of workers.
Interesting interview I just watched about how powerful and dangerous these "wishes" or "prophecies" are especially in the hands of the ultra-wealthy: https://www.youtube.com/watch?v=eR7grHa1NR0
In the real world, lawyers submit detailed bills and their clients examine them. If you don’t, that’s on you.
If you automate lawyers out of a job, you can absolutely automate lawmakers out of jobs next. (Not that this would be a bad thing? Maybe pervasive agents for everyone can be the gateway drug to a "this time it's different!" workable direct democracy)