Just like breaking crypto in the age of cloud is more about cost than time, this will lead to legal attacks based on the same principle. The biggest wallet wins.
This was already always the case. If anything, making this more accessible will reduce the barrier to entry for whether or not it's worth your time to take on a case. Instead of 50 lawyers spending 100s of hours on a case, you can have 1 or 2 lawyers + Astra working on it and if there's a case you can add more real lawyers.
Many lawyers are owners/partners compared with software engineers who are more like cogs in the machine. They also bill hourly/contingency per case compared to engineers who are salaried. If a partner in a firm thinks they can take on more cases because of AI assistance then they will because that's just more money in their pockets.
With regard to certain legal questions this has always been the case. AT&T Fought the US Government for 20 years and eventually won because the government gave up. Without some kind of national anti-SLAPP law we're all one irritated oligarch away from having our lives financially ruined.
I am curious what level of trust established law firms treat LLMs with.
Will these models eventually replace all knowledge work, leaving lawyers, doctors, product managers, software developers, and others out of a job?
If the benefits were shared across humanity, that could bring us closer to utopia. My worry is that we’ll instead end up with a handful of even wealthier billionaires and millions of people out of work.
I mean that's what all of these execs are openly telling everyone: they want you out of work, they want their ai to be the one to bring the world to it's knees, they want to surveille every second of your day, they want killer drones to use, they want to lay all of your cities to rubble and build "paradises" on top of them like in gaza.
They also openly tell you what they are afraid of btw: collective worker power. something that is massively lacking in our industry, although i feel like it would be one of the easiest industries to unionize in terms of # of workers.
Interesting interview I just watched about how powerful and dangerous these "wishes" or "prophecies" are especially in the hands of the ultra-wealthy: https://www.youtube.com/watch?v=eR7grHa1NR0
Will these models eventually replace all knowledge work, leaving lawyers, doctors, product managers, software developers, and others out of a job?
Effectively yes, in the current forms. Those professions will likely evolve, but the traditional forms (ie writing code by hand, writing law filings by hand etc) are all dead.
in that case, legal cases would just come down to who has more compute lol. Many times cases win on their merits, but we've also seen evidence where overwhelming legal pressure can influence cases.
May I just note that there are other jurisdictions on this planet that are less money-biased than than the US one but will be disrupted by law-LLMs as well?
It seems much more probable to me that these LLMs will make good things worse than that they will make bad things better.
If we have enough energy and raw materials to keep building, yes, it will be an utopia made real. But if there is energy scarcity, then other two outcomes can arise.
Doubt it. When someone can go to Astra MD for 75% of what they used to go to the doctor for, then the remaining doctors only have 25% as many visits. When doctors only have 25% as many visits, they have to compete on price and they make less per visit.
Same argument for plumbers. Everyone always jokes about what a good time it is to be a plumber. But what happens when all the software engineers turn to plumbing? Suddenly it's not such a good time to be a plumber anymore.
Genuinely wondering (aka not snarky): Has anyone found frontier models to provide useful research in the context of European civil law systems?
Your comment made me wonder if there are any halfway-acceptable model benchmarks for law tasks? Specifically I’d love to know how the frontier models’ abilities compare between common law vs. civil law systems. My guess would be that an AI in a common law context should have a clearer idea of how a specific case is interpreted/accepted by (common law) practitioners, whereas trying to rely on AI in a civil law context, like Germany, can be daunting. In a few Germany-specific recent examples, the models feel like they present only (maybe too stubbornly?) the “civil law”-based laws. All while negating much of AI’s research benefits because civil statutes are portrayed as being absolutely accurate, binding, and their enforcement (and thereby the legal reality) being uniformly applied. Am I making this interpretation up? If so, how can I prove myself wrong?
So those lawyers will likely want to know about the likeliness of it, strategies to work around it, and overall mitigations.
It might be helpful if the company launching this product included such information in their blog post instead of ignoring it in a field with such a high cost of getting it wrong.
agreed, but i dont think there are many (any?) product launch blogs that cover all the fine details and legal minutia of the product.
that all typically comes in pages of terms of service, purchase contracts, SLAs, conversations with your rep, knowledge base articles, and that sort of thing. not in the initial marketing post.
Does hallucination matter for this application? We've moved beyond raw recall being that important, it seems like for law specifically all relevant facts will be cited and checked easily by humans.
all relevant facts will be cited and checked easily by humans
I've talked to a lawyer about how they handle this. They do indeed double-check everything, since it'd be embarrassing (or worse) to send hallucinated statements to opposing council or to the court. They still find the assembly a huge time saver
But based on stories in the news on the subject, not everyone has this same level of diligence
Sure but just like generating 100x more code, someone has to review it. So you are wasting everyone in court's time (defendants, prosecutors, judges, staff) by making them parse through what is quite often a bunch of hallucinated slop. Time that could be much better spent on parties who prepared and reviews their own arguments.
Interesting to see the callout to companies like harvey in the post itself as consumers rather than competitors? I guess openai isn't quite willing to step into those customer relations themselves?
I already use them for that, they are pretty excellent at it. Much better than the terms of use generator products that used to exist. That said, nobody cares to sue your business for the most part until you're big enough to be worth it. By that time, you'll have a team of legal analyst to assist you... or agents should I say.
Then again, nobody will have money to buy anything at this rate, so in all liklihood, this is a total non-issue.
I'd expect they could indemnify you against hallucinations or similar if this gets good enough for that to be a very rare occurrence? Or you could buy insurance on it that's cheaper than hiring a lawyer (not a high bar to clear). I wouldn't rely on it currently, though.
The product isn’t meant for you or me it is for lawyers. If you can’t take on personal liability for a badly written contract then you shouldn’t be using it.
People that are saying OpenAI is screwed because a lack of profit, I'm not of that opinion. They are encroaching on every industry they can. They have name brand recognition, a huge user base and are showing they can be a valuable tool to all types of businesses.
As much as I hate to see it. They are now threatening industries like Engineers, Game Developers, Accountants, 3D modelers, 3D animators, Video Production, Audio Production, Therapist, Tax Auditors, Journalists, Authors, Artists, Mathematicians, Product managers, Every type of analyst and pretty much any other job that can be done behind a computer screen.
I used ChatGPT the other day to resink a CPU with termal paste, replace a PSU in my PC and snake my kitchen sink from the wall. I didn't exactly need it's help but wanted someone looking over my shoulder so to speak. It can help you repair all types of stuff, but as far as county / building codes and such, that probably isn't far off. It seems to understand things quite well.
By that point money ceases to have value, because the value of money comes from the motivation it gives people to work. If AI does everything, then money is useless (unless AI like money for some reason).
There will still be a need to distribute and exchange ressources, goods and services.
Real estate, food, energy, mobility.
That will be done with money. Or violence. Either way, a scary future to people when labor doesn't provide any value. Elon promises abundance, but what can he do against greed?
there are always be next frontier of problems which require creativity, unless AI become supersmart completely make humans redundant in all cognitive functions.
Most HNers are clueless that if you have Top Talent + Capital you already have an insurmountable moat. OpenAI, SpaceX, Anthropic all have that and none of the regular guys can compete against them (if they choose to attack that industry)
Why the hell are they not fighting fire with fire? This is not sustainable. But it is annoying that the AI labs get to play arms dealer, selling to both sides.
you could probably write a cool little gotcha of an SF short story about a barren wasteland of a planet that keeps broadcasting out legalese that's revealed to just be LLM chatbot lawyers pedantically arguing with one another about xeno legal doctrine
The legal system, which is a machine/technology by itself, will be eaten out. I wonder what will replace it. Botnet law arbitrage? – Personal assistants constantly negotiating with each other to avoid permanent civil lawsuits?
the cost of making a legal argument can collapse while the cost of reaching an enforceable, legitimate decision may go higher, which will gate the "justice" system even more.
You presented the concern from my adjacent comment perfectly (“LLM performance: common law vs civil law” essentially). So is AI possibly just growing the “Reverence for Professional Experience” factor that plays such a big role for legal compensation here in the US?
It's really not. The legal system is slow and inefficient because it's a deeply pipelined system built to maximize the throughput of the bottleneck resource: judges. Judges are constitutional officers who exercise independent authority in meat space and thus are necessarily limited in number. The rest of the design flows from that.
If you got the judge, all the parties, and all the witnesses in a conference room together until the case was resolved, you could probably handle a lawsuit in a few months. But each judge has hundreds of cases pending before them, so that would never work. Instead, you get something like how a GPU works. You do some work on a case, submit the work to the court, then work on something else for a few months while you wait around to get the results back. Then you do some more work and submit it to the court, then go do something else for a few months while you wait to get the results back. A few months of actual work gets spread out over a few years that way.
I find the idea that people can use LLMs to exercise their rights as citizens appealing however. Many people aren't aware of the rights they have, and LLMs are pretty good at surfacing some stuff without having to pay lawyers. Having to hire a lawyer is imo actually a huge way of gatekeeping people from exercising their rights. I heard a lot of local German public institutions are currently being flooded with people arguing their case with the help of LLM that they previously weren't really realistically able to do. So I don't see it all as bad.
are pretty good at surfacing some stuff without having to pay lawyers
They are also really good at making stuff up as evidenced by the many, many, many examples you read in the news about actual lawyers using AI to write briefs that are full of errors and hallucinations.
In civil courts, you'd likely get more sympathy from a judge if you represented yourself and admitted your lack of understanding, rather than try to appear as someone you're not because you wrote some prompts and copied the output.
Current court systems around the world are just not built to handle the flooding of cases from the citizens.
The stupidest analogy is open source projects having a hard time accepting LLM generated PRs from the masses, because review process is the bottleneck.
No idea how to fix this, to be honest. In coding world, with some mental gymnastics, I can see code not being reviewed by people anymore. In courts, things generally have more consequences, and you can’t really roll back decisions that easily.
I don’t know in the USA but in France, if it’s deemed that you launched a lawsuit knowing very well it wouldn’t succeed, you are susceptible to get a 10k€ fine. Even jail in serious cases.
Yes. And you(r lawyer) can collect lawyer's fees and you can be made to pay the court fees, if you lose.
IME the American legal system is set-up to discourage litigation, though. A common tactic is to bury your opponent in the threat of heavy damages or jail-time to get them to settle for what you were originally after, which courts are perfectly happy to facilitate because it gets a potentially lengthy trial off their dockets. They'll punish (or be biased against) whichever party seems responsible for not accepting a "reasonable" settlement.
Algorithmic abuse of the system to extract payments already exists in the form of the debt collection industry.
By using the legal search index, Astra for Law can search U.S. case law, statutes, regulations, court rules, and administrative decisions across a corpus of more than 230 million URLs, with sources added daily. Our work with Free Law Project, the nonprofit behind CourtListener, brings its case-law collection covering more than 99.9% of published U.S. precedential case law (opens in a new window) into this research experience.
CourtListener already has an MCP interface and Grok is quite good at pulling from it. In my experience, Grok 4.6 is quite good at analyzing legal cases and human-written documents. Better than Opus 5. I'm not sure if it's better than Fable 5.1 on that task, b/c I'm not willing to spend my precious Fable tokens on case law searches lol.
Yeah, it should be freely available, you have to be able to know the rules you're supposed to obey in order to obey them well. I've been making a free API for US law search, you can point whatever model you want at it: https://law.agentlookups.ai/
Very much a work in progress, only federal and state so far, no municipal codes yet, and no case law yet. Big hole, I know. Also working on making the search ranking work better.
Right now it's just a bunch of crawlers for the individual states. If there's interest, I could periodically stand up snapshot torrents or something. That something you'd be interested in?
Alternatively, if someone else knows an all-in-one option that exists, I wouldn't mind retiring those crawlers...
Not op, but that's a very interesting proposition. While the law and legal code are technically property of the people, I'm not aware of any single point of download for it all.
There’s no single point of download for it all because there’s thousands of autonomous entities that issue law and adjudicate cases, at least 51 of them distinct sovereign entities.
We could enforce (suggest?) a common format / api at the federal level. Especially if it’s incentivized with funding that more than justifies the cost of maintenance. Similar to how federal interstate funding is only available to states with a 21+ drinking age.
Yeah, going to the courts and municipal code seems like it's going to be a heavy lift. Many of them seem to hang off of municode, though, so maybe it's not a huge number of unique crawlers.
There is none, not for statutes and definitely not for case law; even at the appellate level where you have multiple federal circuits, then 50 states, then territories, military, tribal and a whole host of other niche courts. And the appellate court systems can be split into districts, and by lower and higher levels.
Then if you want to really get into it, The People should also be able to access trial court level, and at that point you have over 3000 distinct court systems with their own access systems, usually requiring logins and CAPTCHAs, and half of them not even having anything accessible online at all, and the other half only having recent stuff online and the rest rotting in a flooded basement.
Max Junestrand has consistently said Legora treats the model layer as swappable, selecting across frontier providers rather than building the product around one.
OpenAI didn't need to name Legora and Harvey in the second paragraph of the launch post.
They are pre-empting the obvious interpretation of Astra for Law: that moving this far up the legal stack puts them in direct competition with their biggest legal AI customers.
“Don't worry, they can build on us” is a pretty conspicuous message to include on launch day.
They have clearly thought about some pessimistic outcomes.
the thing is a lot of the legal work which will go through this is drafting 100 and 1 variations of draft versions of standard contracts not containing any trade secrets where the contract can be drafted with "replacement/place holder names"
the kind of work mostly done by juniors not yet through their final exam and other "non" lawyers etc.
so it's a slippery slope of "lets just use it for <this> things where it doesn't matter" and then out of laziness and convenience it creeps into all the other places (at least for drafts).
open-ai have proven they can make good / decent models but business strategy is just spray and pray.
they need to pick a lane and optimize for it. coz at their size they can't serve the application layer (a.i startups who can fine-tune models will eat their lunch)
if they gonna do a consumer play - then go ham on that.
otherwise they're gonna get caught in the dreaded middle valley.
I think the comparison is unfair. Google had/have a product that was 10X better than anything else at the point of release (search then chrome). And they have several money making products like YouTube and android that are either singular or in a duopoly.
OpenAI doesn't have any of these things. They have products that they're paying for customers when the market they're in is rapidly converging on fighting for API reasoning as part of enterprise systems and fighting a race to the bottom for fickle consumer solutions that will be eaten by open source once they have to make money.
Maybe they had a brief window for dominance of information search (or maybe it was only ever going to last as long as Google releasing all their internal research) and maybe they had a brief moment of monopoly till Anthropic got going but theyre not in the same dominance position as Google.
They have a lot of compute, so I think it makes sense to spray and see what works. Anthropic is limited in that regard, and focused in coding/tech, but OpenAI don't need to do the same. They even got the lead without having to focus in coding only, which is remarkable.
Enterprise is eventually get caught (if it isn't already) by the Microsoft/Google. Because with office/teams/suite they were already in every enterprise, and that just added a new tool to existing offerings.
Provisioning and contracts and data retention was just an extension to review of existing ones.
Nobody serious is going to risk sending sensible data to OpenAI/Anthropic, etc because "the benchmarks have shown +8% performance there and +2% there". Irrelevant.
They can't be seen to commit to strongly to a specific product experience, because if they are understood as a regular tech product business that has way different financial scaling considerations than "superintelligent everything-factory"
I suspect that half-assed announcements like this are a result of different people internally with conflicting incentives resulting in a split-the-baby solution.
Perhaps, but then someone would do the same but bill you for 7 hours, someone else would undercut them again, until the price reaches a lower equilibrium.
Wouldn’t this and similar efforts to centralize bureaucracy make AI the new gatekeeper? Without reliable transparent models we’re just trusting OpenAI instead of a hundred top legal firms.
So much for caring about the spirit of the law. Now we'll start an arms race for abusing every possible letter of the law.
It's analogous to crypto. Started from some noble anti-authoritarian ideas and morphed into machine that removes any friction for capital - whoever has the most money will keep gaining the most.
Now there will be even less friction to horribly abuse the law. To democratize the law we would need to move in the opposite direction - to always keep it simple and aligned with our intuitions.
The more intricate and complex legal arguments become, the more abstracted they are from their original purpose and spirit.
Hence the crypto analogy - it was also supposed to "democratize", but the opposite happaned - it only further empowered the most powerful. Imagine legal case so purposefully complex that only those with access to best models have chances to participate and win the dispute.
Any lawyers here who have used AI agents heavily for their work? From what I've heard, they're currently very good at searching, analyzing and drafting documents like contracts and patents, but some say they suck at interpreting the law.
They are excellent, especially the latest models. That said, (a) I wouldn't feel safe filing something without a real lawyer looking at it; (b) it can't (easily? legally?) do oral arguments for you; and (c) a lot can happen in the hallways outside the courtroom to move a case forward that the AI can't easily do.
I’m sure there are a wide variety of experiences out there, but here’s my perspective as a former biglaw associate and current solo litigator:
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
In my experience it basically doesn't even try to interpret the law. It just summarises the publicly available law/guidance out there and, if there is a question about how to interpret some provision, might set out the arguments for each interpretation. It doesn't really take a position. It is pretty good at drafting though. (Legora)
I've always said this will be when we get the real Butlerian Jihad, when the AI firms start trying to liquidate the legal profession.
If you automate lawyers out of a job, you can absolutely automate lawmakers out of jobs next. (Not that this would be a bad thing? Maybe pervasive agents for everyone can be the gateway drug to a "this time it's different!" workable direct democracy)
I really hope the bar associations continues to hold lawyers to high standards but I have feeling they may not be ready to handle fallout of AI slop-law.
The need for actual lawyers will persist I think from my own experience. I attempted drafting a contract with some points myself using AI, but after several edits I wasn't sure if it was correct. Sending it to an actual lawyer ended up in so many corrections I couldn't imagine the first time. One big thing was the overly excessive protective clauses which didn't make sense for reality or conflicted with another.
Its just like code I suppose, if you can read and understand and validate, you can use it to scale and otherwise it could end up being a vibe effort.
LLMs are the first genuinely useful legal tech since the Internet. I'm pretty shocked, though, at the delta between how competent Claude is on code versus legal work. It's good for research and data organization, but terrible for drafting. I wonder if this is a structural problem with the lack of feedback loops. In law, there's no compiler to check for logical or continuity errors in your brief, and there's no unit tests to check for correctness or performance.
Even without that, I think it'll be extremely valuable to clients to allow them to answer simple questions without a lawyer, figure out the lay of the land so they can supervise their counsel, etc.
I have found that it’s useful generally speaking to get the intent of contracts and red lines, but actual drafting I agree is where I lose all confidence. My guess is that the significance of the difference between using a word like “and“ or “or“ can be so meaningful that that level of nuance can often be lost. But I know nothing I’m not in the space, I just pay too much money for lawyers.
LLMs are the first genuinely useful legal tech since the Internet
That is an incredible statement that could not be further from the truth. Large scale adoption of email, searchable document databases like Westlaw, LexisNexis, PACER, etc.. , OCR Software, electronic signatures, and tons more have had a much more defineably positive impact on the legal profession since the internet came about.
The internet came well before both of those. And none of the example I gave have lead to lawyers regularly being fined in court because of "hallucinations"
You should know - for coding they make terrible mistakes as well.
But programmers have this concept of a "code review" where another person looks at the code to look for problems.
We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.
Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.
Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?
The need for actual lawyers will persist I think from my own experience.
The outcome of a case shouldn't depend on someone's fallible ability to recall facts or convince other people or point their index finger*
Law should generally be deterministic. One's CHA stat should have no bearing on justice.
There should still be human judges, but the middleman between the judge and petitioner could easily be removed, and have generally been seen as leeches since forever anyway.
Though, like how the USA opts to remain in the Stone Age with regard to tax filing because of lobbying by tax software companies, this faction of society will flail the hardest before they admit they're obsolete.
----
* What's a lawyer's favorite programming language? Objection C.
Yep, just yesterday, Nike removed one leach middlemen called retailers and sell directly through their app. Turns out working out very well for them too.
The typical pattern is called “deskilling”. It doesn’t usually mean a skilled profession will disappear overnight. Instead, the job might be done by less expensive folks like paralegals.
An example is in the banking industry, where making a loan used to require deep analysis of a person’s credit worthiness. Now they use an algorithm (credit scores) which means someone with less experience can do it.
If law follows the same pattern, a job done by someone making $500/hour might be done by someone making $50/hour.
i don't think that's the point anybody is making. "good lawyers" is not the profession at risk here. a top tier courtroom lawyer won't be replaced by AI.
but how much of the revenue of the average law firm comes from that, vs the day-to-day "we need this relatively routine contract reviewed" sort of work?
Yes. And it would be the continuation of a trend that’s already been going on for two decades. When I started out as an associate at a fancy Wall Street firm, we had 30 people working on reviewing documents for this matter even that level of firm today would not be able to justify doing that anymore. We’re going to move from a pyramid model to more of a diamond shaped model with many fewer associated attorneys doing the legal research and information marshaling.
Instead, the job might be done by less expensive folks like paralegals.
So who is going to define your strategy, represent you in court, adapt the strategy to changing circumstances, negotiate with counter-parties on a mutually accepted settlement? A paralegal? An AI model? Please
Not being snarky here but I've encountered some really crappy lawyers and they seem to be doing great ($$$).
It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages. There should be a safe harbor for that.
Assuming you are in the US, there is a safe harbour called "opinion based on disclosed facts" for that. You could also fairly argue that the conduct of a lawyer is a matter of public concern
Not trying to be dismissive here but a lawyer will always edit your proposal. I do think lawyers will persist but not as many. And they will work very differently, much like we already use claude / codex to code - a big part of the contract probably won't be read by the lawyer.
I would rather argue they SPECIFICALLY will read everything. However, they'll likely often just be like "I'd phrase this differently but that works, too".
The best writer I know of was an associate attorney. He didn't have the technical background nor the in depth computer related knowledge, and relying on the information I fed him. But man the briefs he filed to the court were amazingly good. Reading them I would have been convinced his side was right if I were in the jury.
Law LLM will surely help competent lawyers in their fields with greater sources of knowledge not in their core area of expertise.
As costs for legal work drop we might expect demand for that work to increase. For example, it may be the case that legal help becomes accessible to entire segments of the population that had no access previously.
I was in a similar situation and tried a different approach.
I had start with asking for a contact with some details I provided. But then I ask the model to be an experienced corporate lawyer and ask me a series of questions to gather the details it needed and then write the contract. The result covered a lot of details that were highly relevant but were absent from the original attempt.
The key insight is that you can lean on the model to cover your unknown unknowns.
Your comment is a great example of the phenomenon where people think AI is an expert in areas that are not because they have no way to determine if the seemingly good looking result is genius, gibberish, or somewhere in between.
The point was that you can go further than "write me a contact that says x,y and z". I actually believe the process I went through with the AI was a more custom, tailored experience than any interaction I have had with actual lawyers, who'd usually apply a template. The AI asked me a bunch of questions, many of which were relevant, and led to follow up questions. I could ask my own questions, make corrections, etc. and all along, they made sense to me. This part of the experience was great.
There could still be more unknowns that I did not encounter and by definition don't know about.
And the final output could be wrong, wording, exact legal terminology etc.
Can we say working with an actual lawyer is guaranteed to be better, more correct? Lawyers are people and they also make mistakes, wake up on the wrong side of the bed, hate their job etc, just like the rest of us.
I want to take a step back and observe that you're saying expensive specialists will continue to be required if two parties want to make an agreement. That seems like something worth fixing rather than just accepting.
Same for SWE. AIs will churn out happy path vibe slop and fail when something unexpected happens by swallowing errors or having a bunch of terrible "fallbacks".
In my experience, those expensive specialists help me protect against things me and the counterparty cannot even imagine when making an agreement. Do you mean that this would be fixed by AI or by making simpler agreements?
Millions if not billions of agreements are made every day without the involvement of lawyers. Lawyers are only needed for particularly complex situations and even there they are generally only "needed" in the sense that parties feel they would add value.
And to add to your last point yes it should also be like code: where the costs of development are coming down. Legal costs should trend down if law firms are effectively leveraging LLMs to accellerate their procedures (though I won't hold my breath)
In my recent experience, it seems like LLMs may even increase need/demand for actual lawyers (more ideas + more words = more lawyer time). I expect lawyers will generally adopt the technologies that will benefit them, rather than the technologies that would increase standardization and trust
The same has happened with modern finance. Despite more technology that has power to drive more transparent/efficient markets, we have less perfect information sharing and a larger group of middlemen capturing a growing share of profits of the economy
We need a term for the dark pattern of zooming into just that part of the y-axis where the two closely competing benchmarks sit, to make the top one appear maximally better.
this essay details how law firms became sweatshops from 80s. They charge hundreds of dollars to do make busy work by the junior most staff. LLM will kill the goldengoose of the law industry.
I'm not familiar with the Vals AI Legal Research Benchmark. But their website has other frontier models' scores, and the scores OpenAI is now revealing for "Astra for Law" are slightly less than Claude and Muse:
The top is a three-way tie: Muse Spark 1.3 Max, Claude Opus 5, and Claude Fable 5.1 all reach 55.29% all-pass accuracy, a clear ~6-point step ahead of the next model. [Astra for Law reached 54.0%]
Under partial-credit scoring, Claude Opus 5 reaches 90.58% weighted pass rate but 55.29% under strict all-pass grading, where every rubric check must pass. The gap shows models often get most of an answer right but fail on one or two required elements. [Astra for law reached 90.0%]
I've had top SV lawfirms whos partner charged our company $2000/hr and still couldn't get the right docs in the signature packet. and another getting share counts wrong during raise.
frustrating that law firms have no liability for these mistakes
Since the cost of building software is now cheap, there is nothing stopping them from building everything imaginable. They'll soon have an app store with every app built by them and they'll say its for security reasons. Nothing is stopping this coming monopoly
At a certain point why would they sell anything other than services and products their eventual (actual) AGI/ASI builds in literally every market.
When opportunity cost isn't a thing anymore because it reaches every corner of the planet simultaneously faster and builds better than any human can.
There's no reason to let others build on top of AI, except if the AI determines that it needs capitalism to continue because it's paperclip goal is to maximize shareholder value.
The most interesting use case in my mind is skipping law suits. Obviously you need lawyers in court. But lawyers are people you are basically paying to fight for you.
Instead, if you resolve your dispute outside of court, you don’t need a lawyer. If both parties use ChatGPT to find the relevant laws or read contracts, they could come to an agreement without expensive legal fees.
I find the watermarking dynamic to be really interesting in the legal space, as more large model providers provide increasingly powerful legal capabilities, and adoption (presumably) also increases. Attorneys aren’t the same as developers as their work can be traced back to them, and there are personal bar licenses and reputations at stake. I wonder if knowing the likelihood that AI generated something helps or hurts in that respect.
I also see a lot of watermark removal services popping up as a result.
Just like breaking crypto in the age of cloud is more about cost than time, this will lead to legal attacks based on the same principle. The biggest wallet wins.
That’s how the legal system in the US has always worked though
This was already always the case. If anything, making this more accessible will reduce the barrier to entry for whether or not it's worth your time to take on a case. Instead of 50 lawyers spending 100s of hours on a case, you can have 1 or 2 lawyers + Astra working on it and if there's a case you can add more real lawyers.
There are other possibilities:
1) Lawyers are not as naive as software engineers and will fight being replaces by new laws.
2) If they are replaced, OpenAI will take a cut commensurate with the amount in dispute (OAI, please credit me for the idea in the IPO brochure).
Many lawyers are owners/partners compared with software engineers who are more like cogs in the machine. They also bill hourly/contingency per case compared to engineers who are salaried. If a partner in a firm thinks they can take on more cases because of AI assistance then they will because that's just more money in their pockets.
That's already the case in law because you could just hire the best or most lawyers.
With regard to certain legal questions this has always been the case. AT&T Fought the US Government for 20 years and eventually won because the government gave up. Without some kind of national anti-SLAPP law we're all one irritated oligarch away from having our lives financially ruined.
I am curious what level of trust established law firms treat LLMs with.
Will it be able to sue itself?
That’s a great question for Astra for Law.
Will these models eventually replace all knowledge work, leaving lawyers, doctors, product managers, software developers, and others out of a job?
If the benefits were shared across humanity, that could bring us closer to utopia. My worry is that we’ll instead end up with a handful of even wealthier billionaires and millions of people out of work.
Just a few years ago people were paying 400k for pictures of apes. We always make up new stuff to spend money on.
You know most of that was wash trading, right?
True, unless you're a surgeon.
I mean that's what all of these execs are openly telling everyone: they want you out of work, they want their ai to be the one to bring the world to it's knees, they want to surveille every second of your day, they want killer drones to use, they want to lay all of your cities to rubble and build "paradises" on top of them like in gaza.
They also openly tell you what they are afraid of btw: collective worker power. something that is massively lacking in our industry, although i feel like it would be one of the easiest industries to unionize in terms of # of workers.
Interesting interview I just watched about how powerful and dangerous these "wishes" or "prophecies" are especially in the hands of the ultra-wealthy: https://www.youtube.com/watch?v=eR7grHa1NR0
Effectively yes, in the current forms. Those professions will likely evolve, but the traditional forms (ie writing code by hand, writing law filings by hand etc) are all dead.
in that case, legal cases would just come down to who has more compute lol. Many times cases win on their merits, but we've also seen evidence where overwhelming legal pressure can influence cases.
bigger spender already has a huge advantage, up to millions of dollars
this is more likely to democratize the legal system by reducing the cost of a good legal team
May I just note that there are other jurisdictions on this planet that are less money-biased than than the US one but will be disrupted by law-LLMs as well?
It seems much more probable to me that these LLMs will make good things worse than that they will make bad things better.
Everywhere a good lawyer costs a lot of money.
there will still be writing initial and incremental prompts by hands, until and if LLMs surpass humans in all intellectual functions.
If we have enough energy and raw materials to keep building, yes, it will be an utopia made real. But if there is energy scarcity, then other two outcomes can arise.
This also assumes that the AI owners _want_ a global utopia, or that governments will enforce that outcome.
As it stands, it seems far more likely to result in a wonderful life for a few, and an absolute catastrophe for most.
XIX century dickensian society and Gilded Age barons were replaced by the time of the New Deal.
No, it will only leave those without a professional license jobless. Like software developers.
Doubt it. When someone can go to Astra MD for 75% of what they used to go to the doctor for, then the remaining doctors only have 25% as many visits. When doctors only have 25% as many visits, they have to compete on price and they make less per visit.
Same argument for plumbers. Everyone always jokes about what a good time it is to be a plumber. But what happens when all the software engineers turn to plumbing? Suddenly it's not such a good time to be a plumber anymore.
Eagerly awaiting your robot plumbing startup. I highly doubt anything in plumbing is compute-limited.
No word on model hallucinations in the blog post.
https://artificialanalysis.ai/models/gpt-6-astra?omniscience...
Not like common law doesnt already have a lot of hallucination going on.
can “penumbras and emanations” compete with hallucinations?
Common law is all about rummaging around in dead mens’ letters, LLMs are a natural fit.
Genuinely wondering (aka not snarky): Has anyone found frontier models to provide useful research in the context of European civil law systems?
Your comment made me wonder if there are any halfway-acceptable model benchmarks for law tasks? Specifically I’d love to know how the frontier models’ abilities compare between common law vs. civil law systems. My guess would be that an AI in a common law context should have a clearer idea of how a specific case is interpreted/accepted by (common law) practitioners, whereas trying to rely on AI in a civil law context, like Germany, can be daunting. In a few Germany-specific recent examples, the models feel like they present only (maybe too stubbornly?) the “civil law”-based laws. All while negating much of AI’s research benefits because civil statutes are portrayed as being absolutely accurate, binding, and their enforcement (and thereby the legal reality) being uniformly applied. Am I making this interpretation up? If so, how can I prove myself wrong?
The benchmark you linked to shows GPT-6 Astra having the lowest hallucination rate of all tested models.
The blog post makes no mention. If the rate isn't 0% it should be mentioned for a field like this.
Agreed. Hallucinations and reliability are the main hurdles to anything being 'agi' in my book
The definition of AGI is whatever a frontier lab wrote a blog about doing in the last week.
why this maximalism? There's nothing that has 0% hallucination including humans. lets use reasonable baselines.
Humans can be sanctioned and fined, and eventually disbarred if they continue to lie in court filings.
presumably that will remain true for the lawyers using the product
So those lawyers will likely want to know about the likeliness of it, strategies to work around it, and overall mitigations.
It might be helpful if the company launching this product included such information in their blog post instead of ignoring it in a field with such a high cost of getting it wrong.
agreed, but i dont think there are many (any?) product launch blogs that cover all the fine details and legal minutia of the product.
that all typically comes in pages of terms of service, purchase contracts, SLAs, conversations with your rep, knowledge base articles, and that sort of thing. not in the initial marketing post.
probably available by the "contact legal sales" link at the top of the post https://openai.com/business/contact-sales-legal/
Does hallucination matter for this application? We've moved beyond raw recall being that important, it seems like for law specifically all relevant facts will be cited and checked easily by humans.
Don't be too sure about that. [0]
0: https://www.damiencharlotin.com/hallucinations/
I've talked to a lawyer about how they handle this. They do indeed double-check everything, since it'd be embarrassing (or worse) to send hallucinated statements to opposing council or to the court. They still find the assembly a huge time saver
But based on stories in the news on the subject, not everyone has this same level of diligence
Sure but just like generating 100x more code, someone has to review it. So you are wasting everyone in court's time (defendants, prosecutors, judges, staff) by making them parse through what is quite often a bunch of hallucinated slop. Time that could be much better spent on parties who prepared and reviews their own arguments.
Interesting to see the callout to companies like harvey in the post itself as consumers rather than competitors? I guess openai isn't quite willing to step into those customer relations themselves?
memes and snark aside, can you use this to create legitimate terms and contracts for my products and if I do who is getting sued when it is wrong?
I already use them for that, they are pretty excellent at it. Much better than the terms of use generator products that used to exist. That said, nobody cares to sue your business for the most part until you're big enough to be worth it. By that time, you'll have a team of legal analyst to assist you... or agents should I say.
Then again, nobody will have money to buy anything at this rate, so in all liklihood, this is a total non-issue.
You. Don't take legal advice from a word calculator.
The business value will be when platforms can underwrite their LLMs' legal products.
That will be a decacorn product or more.
That or a hydrogen balloon of financial engineering
As opposed to taking advice from the glucose wetware?
The glucose wetware will actually fight your case...
Wetware is a new phrase that both horrifies me and makes me crack up in laughter :)
Yes, because glucose wetware can be held liable for malpractice if it gives you nonsensical advise.
You will be the one getting sued. It's just a tool.
I'd expect they could indemnify you against hallucinations or similar if this gets good enough for that to be a very rare occurrence? Or you could buy insurance on it that's cheaper than hiring a lawyer (not a high bar to clear). I wouldn't rely on it currently, though.
I don't think you can. What I get from this article is that this is not a product they're going to sell to average consumers.
Same for SWE when pushing code they don't understand.
Solved without AI https://commonpaper.com
https://commonpaper.com/standards
You are.
when you shoot someone with a gun, who gets sued? you? or the gun manufacturer?
The product isn’t meant for you or me it is for lawyers. If you can’t take on personal liability for a badly written contract then you shouldn’t be using it.
You can create terms and contracts even if you're a 7 year old with no AI.
And yes, even contracts drafted for millions of $ have oversights and unlawful or unenforceable terms.
You thought you could sue your lawyer when a contract they drafted turns out to not mean what you thought it meant?
People that are saying OpenAI is screwed because a lack of profit, I'm not of that opinion. They are encroaching on every industry they can. They have name brand recognition, a huge user base and are showing they can be a valuable tool to all types of businesses.
As much as I hate to see it. They are now threatening industries like Engineers, Game Developers, Accountants, 3D modelers, 3D animators, Video Production, Audio Production, Therapist, Tax Auditors, Journalists, Authors, Artists, Mathematicians, Product managers, Every type of analyst and pretty much any other job that can be done behind a computer screen.
We have big problems for humanity.
I'd kill for an AI that can tell me how to fix shit on my own, with proper permitting and building codes factored in.
I used ChatGPT the other day to resink a CPU with termal paste, replace a PSU in my PC and snake my kitchen sink from the wall. I didn't exactly need it's help but wanted someone looking over my shoulder so to speak. It can help you repair all types of stuff, but as far as county / building codes and such, that probably isn't far off. It seems to understand things quite well.
The biggest problem is that we're conditioned by a paradigm that frames these as problems.
I think the future is going to look pretty boring, we are all going to review and put our signature below LLM generated output...
By that point money ceases to have value, because the value of money comes from the motivation it gives people to work. If AI does everything, then money is useless (unless AI like money for some reason).
There will still be a need to distribute and exchange ressources, goods and services.
Real estate, food, energy, mobility.
That will be done with money. Or violence. Either way, a scary future to people when labor doesn't provide any value. Elon promises abundance, but what can he do against greed?
People are still going to be motivated to eat and have shelter.
Of course, but who (what?) is going to make the food and the shelter?
there are always be next frontier of problems which require creativity, unless AI become supersmart completely make humans redundant in all cognitive functions.
As one wise man said - the oldest profession will also be the last to go.
Most HNers are clueless that if you have Top Talent + Capital you already have an insurmountable moat. OpenAI, SpaceX, Anthropic all have that and none of the regular guys can compete against them (if they choose to attack that industry)
The courts are about to overrun with AI-generated lawsuits (even more so than they have been[0]).
[0]https://www.technologyreview.com/2026/06/04/1138391/courts-c...
They seem unaccountable. Every other profession has increased their productivity.
You dropped this /s
justice doesn't have to be productive it has to be fair
how so? delayed justice is denied justice
It's completely failing at that too. Public trust in the courts is lower than it's ever been, and it was never all that high to begin with
https://www.damiencharlotin.com/hallucinations/
Why the hell are they not fighting fire with fire? This is not sustainable. But it is annoying that the AI labs get to play arms dealer, selling to both sides.
How are we fighting it online? You cannot
It'll be like radar detector detector detectors, which were a thing for a while. Each side (cops, speeder) buying detectors to detect the detectors.
There's a Judas Priest joke in here somewhere.
you could probably write a cool little gotcha of an SF short story about a barren wasteland of a planet that keeps broadcasting out legalese that's revealed to just be LLM chatbot lawyers pedantically arguing with one another about xeno legal doctrine
JG Ballard could probably turn that into something darkly atmospheric, and even more dystopic than his usual depopulated worlds...
Relevant:
https://www.ftrain.com/nanolaw
The legal system, which is a machine/technology by itself, will be eaten out. I wonder what will replace it. Botnet law arbitrage? – Personal assistants constantly negotiating with each other to avoid permanent civil lawsuits?
the cost of making a legal argument can collapse while the cost of reaching an enforceable, legitimate decision may go higher, which will gate the "justice" system even more.
Call it a Prediction Market
You presented the concern from my adjacent comment perfectly (“LLM performance: common law vs civil law” essentially). So is AI possibly just growing the “Reverence for Professional Experience” factor that plays such a big role for legal compensation here in the US?
I am not sure "eaten out" was the correct phrase to use here.
The legal system is slow and inefficient by design.
The legal system is robust and deliberate by design. That winds up being necessarily slow. But it doesn't need to be inefficient.
It's really not. The legal system is slow and inefficient because it's a deeply pipelined system built to maximize the throughput of the bottleneck resource: judges. Judges are constitutional officers who exercise independent authority in meat space and thus are necessarily limited in number. The rest of the design flows from that.
If you got the judge, all the parties, and all the witnesses in a conference room together until the case was resolved, you could probably handle a lawsuit in a few months. But each judge has hundreds of cases pending before them, so that would never work. Instead, you get something like how a GPU works. You do some work on a case, submit the work to the court, then work on something else for a few months while you wait around to get the results back. Then you do some more work and submit it to the court, then go do something else for a few months while you wait to get the results back. A few months of actual work gets spread out over a few years that way.
I find the idea that people can use LLMs to exercise their rights as citizens appealing however. Many people aren't aware of the rights they have, and LLMs are pretty good at surfacing some stuff without having to pay lawyers. Having to hire a lawyer is imo actually a huge way of gatekeeping people from exercising their rights. I heard a lot of local German public institutions are currently being flooded with people arguing their case with the help of LLM that they previously weren't really realistically able to do. So I don't see it all as bad.
They are also really good at making stuff up as evidenced by the many, many, many examples you read in the news about actual lawyers using AI to write briefs that are full of errors and hallucinations.
In civil courts, you'd likely get more sympathy from a judge if you represented yourself and admitted your lack of understanding, rather than try to appear as someone you're not because you wrote some prompts and copied the output.
Current court systems around the world are just not built to handle the flooding of cases from the citizens.
The stupidest analogy is open source projects having a hard time accepting LLM generated PRs from the masses, because review process is the bottleneck.
No idea how to fix this, to be honest. In coding world, with some mental gymnastics, I can see code not being reviewed by people anymore. In courts, things generally have more consequences, and you can’t really roll back decisions that easily.
Don’t you have fee when initiating a lawsuit ?
I don’t know in the USA but in France, if it’s deemed that you launched a lawsuit knowing very well it wouldn’t succeed, you are susceptible to get a 10k€ fine. Even jail in serious cases.
Yes. And you(r lawyer) can collect lawyer's fees and you can be made to pay the court fees, if you lose.
IME the American legal system is set-up to discourage litigation, though. A common tactic is to bury your opponent in the threat of heavy damages or jail-time to get them to settle for what you were originally after, which courts are perfectly happy to facilitate because it gets a potentially lengthy trial off their dockets. They'll punish (or be biased against) whichever party seems responsible for not accepting a "reasonable" settlement.
Algorithmic abuse of the system to extract payments already exists in the form of the debt collection industry.
Second paragraph:
In other words: "no, no, we're not eating our children to prep for the IPO. Don't worry."
If there is any cartel that deserves to be broken its legal search. May they die a miserable death.
Found that to be very interesting
CourtListener already has an MCP interface and Grok is quite good at pulling from it. In my experience, Grok 4.6 is quite good at analyzing legal cases and human-written documents. Better than Opus 5. I'm not sure if it's better than Fable 5.1 on that task, b/c I'm not willing to spend my precious Fable tokens on case law searches lol.
Yeah, it should be freely available, you have to be able to know the rules you're supposed to obey in order to obey them well. I've been making a free API for US law search, you can point whatever model you want at it: https://law.agentlookups.ai/
Very much a work in progress, only federal and state so far, no municipal codes yet, and no case law yet. Big hole, I know. Also working on making the search ranking work better.
is there a dataset/torrent with all the underlying laws?
Right now it's just a bunch of crawlers for the individual states. If there's interest, I could periodically stand up snapshot torrents or something. That something you'd be interested in?
Alternatively, if someone else knows an all-in-one option that exists, I wouldn't mind retiring those crawlers...
Not op, but that's a very interesting proposition. While the law and legal code are technically property of the people, I'm not aware of any single point of download for it all.
There’s no single point of download for it all because there’s thousands of autonomous entities that issue law and adjudicate cases, at least 51 of them distinct sovereign entities.
We could enforce (suggest?) a common format / api at the federal level. Especially if it’s incentivized with funding that more than justifies the cost of maintenance. Similar to how federal interstate funding is only available to states with a 21+ drinking age.
Yeah, going to the courts and municipal code seems like it's going to be a heavy lift. Many of them seem to hang off of municode, though, so maybe it's not a huge number of unique crawlers.
There is none, not for statutes and definitely not for case law; even at the appellate level where you have multiple federal circuits, then 50 states, then territories, military, tribal and a whole host of other niche courts. And the appellate court systems can be split into districts, and by lower and higher levels.
Then if you want to really get into it, The People should also be able to access trial court level, and at that point you have over 3000 distinct court systems with their own access systems, usually requiring logins and CAPTCHAs, and half of them not even having anything accessible online at all, and the other half only having recent stuff online and the rest rotting in a flooded basement.
A third-party project converted an official XML source of German laws to Markdown documents, but the repo is a bit dormant:
- https://okfn.de/en/projekte/bundesgit
- https://github.com/bundestag/gesetze
https://axiom.org is working on this but very early.
Cool, thanks for the pointer!
Let's swap it out for the AI cartel!
Max Junestrand has consistently said Legora treats the model layer as swappable, selecting across frontier providers rather than building the product around one.
OpenAI didn't need to name Legora and Harvey in the second paragraph of the launch post.
They are pre-empting the obvious interpretation of Astra for Law: that moving this far up the legal stack puts them in direct competition with their biggest legal AI customers.
“Don't worry, they can build on us” is a pretty conspicuous message to include on launch day.
They have clearly thought about some pessimistic outcomes.
"We are also expanding our work on privacy and governance to give law firms specific controls for confidential client work"
This is everything OpenAI have to say about privacy in this announcement. No guarantees. No promises. Just a pinky-swear promise.
Anyone trusting them–or a lawyer who relies on them–for legal work deserves what they get.
the thing is a lot of the legal work which will go through this is drafting 100 and 1 variations of draft versions of standard contracts not containing any trade secrets where the contract can be drafted with "replacement/place holder names"
the kind of work mostly done by juniors not yet through their final exam and other "non" lawyers etc.
so it's a slippery slope of "lets just use it for <this> things where it doesn't matter" and then out of laziness and convenience it creeps into all the other places (at least for drafts).
Lawyers are already llm’ing their work and getting slapped around in court for making shit up.
There’s another case making headlines every week.
I get the feeling a lot of them won’t care about this stuff.
open-ai have proven they can make good / decent models but business strategy is just spray and pray.
they need to pick a lane and optimize for it. coz at their size they can't serve the application layer (a.i startups who can fine-tune models will eat their lunch)
if they gonna do a consumer play - then go ham on that.
otherwise they're gonna get caught in the dreaded middle valley.
Do they still do Atlas, the web browser? They are indeed throwing shit at the wall, seeing what stick.
a la Google.
Google went to that model after they had a massive cash cow though.
You can afford to play silly buggers when you have dumpster trucks of money backing up to your door every day see also: Meta.
I think the comparison is unfair. Google had/have a product that was 10X better than anything else at the point of release (search then chrome). And they have several money making products like YouTube and android that are either singular or in a duopoly.
OpenAI doesn't have any of these things. They have products that they're paying for customers when the market they're in is rapidly converging on fighting for API reasoning as part of enterprise systems and fighting a race to the bottom for fickle consumer solutions that will be eaten by open source once they have to make money.
Maybe they had a brief window for dominance of information search (or maybe it was only ever going to last as long as Google releasing all their internal research) and maybe they had a brief moment of monopoly till Anthropic got going but theyre not in the same dominance position as Google.
They have a lot of compute, so I think it makes sense to spray and see what works. Anthropic is limited in that regard, and focused in coding/tech, but OpenAI don't need to do the same. They even got the lead without having to focus in coding only, which is remarkable.
Enterprise is eventually get caught (if it isn't already) by the Microsoft/Google. Because with office/teams/suite they were already in every enterprise, and that just added a new tool to existing offerings.
Provisioning and contracts and data retention was just an extension to review of existing ones.
Nobody serious is going to risk sending sensible data to OpenAI/Anthropic, etc because "the benchmarks have shown +8% performance there and +2% there". Irrelevant.
They can't be seen to commit to strongly to a specific product experience, because if they are understood as a regular tech product business that has way different financial scaling considerations than "superintelligent everything-factory"
I suspect that half-assed announcements like this are a result of different people internally with conflicting incentives resulting in a split-the-baby solution.
Cool, now you can pay an attorney $500/hr for them to prompt Astra for 30 minutes, and bill you like they spent the normal 8 hours on the case.
Perhaps, but then someone would do the same but bill you for 7 hours, someone else would undercut them again, until the price reaches a lower equilibrium.
That is an ethics violation. Even in the pre-AI era, getting caught billing for hours not worked was seriously punished.
"7.5 hours - reviewing output"
Sure, if they actually spent 7.5 hours, and it was a reasonable amount of time. If not, perhaps it would be seen as evidence of incompetence.
In the real world, lawyers submit detailed bills and their clients examine them. If you don’t, that’s on you.
How could that be enforced/checked?
Law is a solved problem.
where are the benchmarks
Wouldn’t this and similar efforts to centralize bureaucracy make AI the new gatekeeper? Without reliable transparent models we’re just trusting OpenAI instead of a hundred top legal firms.
Seems like another “product” that will be killed in 6 months, but is good for the IPO so they can say they solved law.
I wonder if Lawslop is gonna become a mainstream expression. Anyone got a better term?
Slawp?
Haha clever
Drop the s and p. Just law.
It’s cleaner.
i hope that someone can come up with something catchier than "thing-slop" or "slop-thing". the word has become basically meaningless from overuse.
I love the slop word, to me it means effortless, average.
People confuse slop with "bad", but slop isn't bad per se, it only becomes bad when real effort was required.
Right. Pigs enjoy eating their slop. It’s not bad from their point of view.
Slop?
What do you mean by “lawslop”?
You know exactly what they mean. But if somehow you miraculously don't there are hundreds of stories like this:
https://www.reuters.com/legal/litigation/lawyer-state-farm-f...
Aren’t hallucinations different than slop?
So no, I don’t know what they mean.
hallucinations are a subset of slop
So is there other “lawslop” or is it just hallucinations in this context?
You must have a greater critical thinking ability than what you are showing now.
law-bearing
I imagine the point here is to separate out the APIs by different professions and charge accordingly.
So there's a strategy shift here. They launched financial services specific tools and now law?
Is the play here a set of specialized harnesses using their best general model?
So much for caring about the spirit of the law. Now we'll start an arms race for abusing every possible letter of the law.
It's analogous to crypto. Started from some noble anti-authoritarian ideas and morphed into machine that removes any friction for capital - whoever has the most money will keep gaining the most.
Do you really think the law isn't already horribly abused? Democratization of law has been needed for a thousand years.
Maybe I’m misunderstanding, but isn’t that what legislators do?
Now there will be even less friction to horribly abuse the law. To democratize the law we would need to move in the opposite direction - to always keep it simple and aligned with our intuitions. The more intricate and complex legal arguments become, the more abstracted they are from their original purpose and spirit.
Hence the crypto analogy - it was also supposed to "democratize", but the opposite happaned - it only further empowered the most powerful. Imagine legal case so purposefully complex that only those with access to best models have chances to participate and win the dispute.
Why do you think LLM's language abilities are unable to understand the spirit of the law?
Went from AI replacing my job as an attorney, to them begging me to use their tools
Any lawyers here who have used AI agents heavily for their work? From what I've heard, they're currently very good at searching, analyzing and drafting documents like contracts and patents, but some say they suck at interpreting the law.
Probably because the AI was also trained on HN comments.
worse, it was trained on reddit law comments
They are excellent, especially the latest models. That said, (a) I wouldn't feel safe filing something without a real lawyer looking at it; (b) it can't (easily? legally?) do oral arguments for you; and (c) a lot can happen in the hallways outside the courtroom to move a case forward that the AI can't easily do.
We do and it saves so much time. Of course human judgment is needed but it’s like cooking with someone else doing the mise en place.
I’m sure there are a wide variety of experiences out there, but here’s my perspective as a former biglaw associate and current solo litigator:
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
In my experience it basically doesn't even try to interpret the law. It just summarises the publicly available law/guidance out there and, if there is a question about how to interpret some provision, might set out the arguments for each interpretation. It doesn't really take a position. It is pretty good at drafting though. (Legora)
I've always said this will be when we get the real Butlerian Jihad, when the AI firms start trying to liquidate the legal profession.
If you automate lawyers out of a job, you can absolutely automate lawmakers out of jobs next. (Not that this would be a bad thing? Maybe pervasive agents for everyone can be the gateway drug to a "this time it's different!" workable direct democracy)
I really hope the bar associations continues to hold lawyers to high standards but I have feeling they may not be ready to handle fallout of AI slop-law.
The need for actual lawyers will persist I think from my own experience. I attempted drafting a contract with some points myself using AI, but after several edits I wasn't sure if it was correct. Sending it to an actual lawyer ended up in so many corrections I couldn't imagine the first time. One big thing was the overly excessive protective clauses which didn't make sense for reality or conflicted with another.
Its just like code I suppose, if you can read and understand and validate, you can use it to scale and otherwise it could end up being a vibe effort.
LLMs are the first genuinely useful legal tech since the Internet. I'm pretty shocked, though, at the delta between how competent Claude is on code versus legal work. It's good for research and data organization, but terrible for drafting. I wonder if this is a structural problem with the lack of feedback loops. In law, there's no compiler to check for logical or continuity errors in your brief, and there's no unit tests to check for correctness or performance.
Even without that, I think it'll be extremely valuable to clients to allow them to answer simple questions without a lawyer, figure out the lay of the land so they can supervise their counsel, etc.
I have found that it’s useful generally speaking to get the intent of contracts and red lines, but actual drafting I agree is where I lose all confidence. My guess is that the significance of the difference between using a word like “and“ or “or“ can be so meaningful that that level of nuance can often be lost. But I know nothing I’m not in the space, I just pay too much money for lawyers.
That is an incredible statement that could not be further from the truth. Large scale adoption of email, searchable document databases like Westlaw, LexisNexis, PACER, etc.. , OCR Software, electronic signatures, and tons more have had a much more defineably positive impact on the legal profession since the internet came about.
I think "email" and "westlaw" fairly count as "the Internet." LLMs might be bigger than either of those.
The internet came well before both of those. And none of the example I gave have lead to lawyers regularly being fined in court because of "hallucinations"
You should know - for coding they make terrible mistakes as well.
But programmers have this concept of a "code review" where another person looks at the code to look for problems.
We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.
Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.
Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?
I don't understand. I just provided advice about getting better output. Are you trying to reply to someone else?
You made some points worthy of expansion:
What model did you use?
The outcome of a case shouldn't depend on someone's fallible ability to recall facts or convince other people or point their index finger*
Law should generally be deterministic. One's CHA stat should have no bearing on justice.
There should still be human judges, but the middleman between the judge and petitioner could easily be removed, and have generally been seen as leeches since forever anyway.
Though, like how the USA opts to remain in the Stone Age with regard to tax filing because of lobbying by tax software companies, this faction of society will flail the hardest before they admit they're obsolete.
----
* What's a lawyer's favorite programming language? Objection C.
Yep, just yesterday, Nike removed one leach middlemen called retailers and sell directly through their app. Turns out working out very well for them too.
The law is much more interpretation and argument based than logic routing based than you seem to believe.
The typical pattern is called “deskilling”. It doesn’t usually mean a skilled profession will disappear overnight. Instead, the job might be done by less expensive folks like paralegals.
An example is in the banking industry, where making a loan used to require deep analysis of a person’s credit worthiness. Now they use an algorithm (credit scores) which means someone with less experience can do it.
If law follows the same pattern, a job done by someone making $500/hour might be done by someone making $50/hour.
If you think paralegals + AI can do the job of good lawyers you don't understand the profession at all.
i don't think that's the point anybody is making. "good lawyers" is not the profession at risk here. a top tier courtroom lawyer won't be replaced by AI.
but how much of the revenue of the average law firm comes from that, vs the day-to-day "we need this relatively routine contract reviewed" sort of work?
Yes. And it would be the continuation of a trend that’s already been going on for two decades. When I started out as an associate at a fancy Wall Street firm, we had 30 people working on reviewing documents for this matter even that level of firm today would not be able to justify doing that anymore. We’re going to move from a pyramid model to more of a diamond shaped model with many fewer associated attorneys doing the legal research and information marshaling.
So who is going to define your strategy, represent you in court, adapt the strategy to changing circumstances, negotiate with counter-parties on a mutually accepted settlement? A paralegal? An AI model? Please
Human professionals put their reputation and finances at risk when performing their work. This risk functions as a guarantee.
Do they?
Not being snarky here but I've encountered some really crappy lawyers and they seem to be doing great ($$$).
It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages. There should be a safe harbor for that.
Assuming you are in the US, there is a safe harbour called "opinion based on disclosed facts" for that. You could also fairly argue that the conduct of a lawyer is a matter of public concern
Even if there were it would just be another court case
Not trying to be dismissive here but a lawyer will always edit your proposal. I do think lawyers will persist but not as many. And they will work very differently, much like we already use claude / codex to code - a big part of the contract probably won't be read by the lawyer.
I would rather argue they SPECIFICALLY will read everything. However, they'll likely often just be like "I'd phrase this differently but that works, too".
The best writer I know of was an associate attorney. He didn't have the technical background nor the in depth computer related knowledge, and relying on the information I fed him. But man the briefs he filed to the court were amazingly good. Reading them I would have been convinced his side was right if I were in the jury.
Law LLM will surely help competent lawyers in their fields with greater sources of knowledge not in their core area of expertise.
Sure as long as they stop making things up.
But will their glamorous salaries persist? That is the question that matters.
AI doesn't need to wipe out lawyers. If they just depress salaries enough, virtually nobody is going to want to be a lawyer anymore.
This doesn’t replace lawyers, but paralegals surely will be affected. A good enough model could shrink the number of paralegals needed in a firm.
As costs for legal work drop we might expect demand for that work to increase. For example, it may be the case that legal help becomes accessible to entire segments of the population that had no access previously.
Lawyers sure. Paralegals and assistants are cooked in the way typist and secretaries were back in the day.
Nope. Paralegals and assistants do the type of thing GP hired a lawyer for (and lawyers do, as well).
Lawyers can practice, can represent you in court, etc. that's not going away soon.
Lawyers have that lovely regulatory wall, for better or worse.
I was in a similar situation and tried a different approach.
I had start with asking for a contact with some details I provided. But then I ask the model to be an experienced corporate lawyer and ask me a series of questions to gather the details it needed and then write the contract. The result covered a lot of details that were highly relevant but were absent from the original attempt.
The key insight is that you can lean on the model to cover your unknown unknowns.
Did you validate the output with a lawyer?
No admittedly not. I can try it and report back.
Your comment is a great example of the phenomenon where people think AI is an expert in areas that are not because they have no way to determine if the seemingly good looking result is genius, gibberish, or somewhere in between.
The point was that you can go further than "write me a contact that says x,y and z". I actually believe the process I went through with the AI was a more custom, tailored experience than any interaction I have had with actual lawyers, who'd usually apply a template. The AI asked me a bunch of questions, many of which were relevant, and led to follow up questions. I could ask my own questions, make corrections, etc. and all along, they made sense to me. This part of the experience was great.
There could still be more unknowns that I did not encounter and by definition don't know about.
And the final output could be wrong, wording, exact legal terminology etc.
Can we say working with an actual lawyer is guaranteed to be better, more correct? Lawyers are people and they also make mistakes, wake up on the wrong side of the bed, hate their job etc, just like the rest of us.
I want to take a step back and observe that you're saying expensive specialists will continue to be required if two parties want to make an agreement. That seems like something worth fixing rather than just accepting.
The problem is those two people are in conflict not collaboration.
If the expensive person is still better at getting what _you_ want from a contract then it’s always worth doing that.
There’s a reason lawyers are so expensive to begin with - they do very complicated stuff with very expensive consequences if done badly.
A lot of contract-lawyering involves getting ahead of when things go south and the two previously agreeing parties find themselves in disagreement.
Same for SWE. AIs will churn out happy path vibe slop and fail when something unexpected happens by swallowing errors or having a bunch of terrible "fallbacks".
In my experience, those expensive specialists help me protect against things me and the counterparty cannot even imagine when making an agreement. Do you mean that this would be fixed by AI or by making simpler agreements?
Millions if not billions of agreements are made every day without the involvement of lawyers. Lawyers are only needed for particularly complex situations and even there they are generally only "needed" in the sense that parties feel they would add value.
And to add to your last point yes it should also be like code: where the costs of development are coming down. Legal costs should trend down if law firms are effectively leveraging LLMs to accellerate their procedures (though I won't hold my breath)
He just gad better prompt
In my recent experience, it seems like LLMs may even increase need/demand for actual lawyers (more ideas + more words = more lawyer time). I expect lawyers will generally adopt the technologies that will benefit them, rather than the technologies that would increase standardization and trust
The same has happened with modern finance. Despite more technology that has power to drive more transparent/efficient markets, we have less perfect information sharing and a larger group of middlemen capturing a growing share of profits of the economy
I said this before but I wonder if Dan Kan will reboot Atrium. Rally up some old partners and hope Anthropic buys them out for a couple billion.
This is really cool.
We need a term for the dark pattern of zooming into just that part of the y-axis where the two closely competing benchmarks sit, to make the top one appear maximally better.
Truncating the Y axis or cropping the Y axis
crop-maxxing
crop-topping
this essay details how law firms became sweatshops from 80s. They charge hundreds of dollars to do make busy work by the junior most staff. LLM will kill the goldengoose of the law industry.
https://aeon.co/essays/what-made-law-into-a-white-collar-swe...
Astra isn't doing well on bullshit Benchmark https://petergpt.github.io/bullshit-benchmark/viewer/index.n... and is even worse in Legal department.
Qwen 3.8 Max and Opus 4.8 score highest.
This is the first blog post I saw OpenAI call out Claude directly like that.
I'm not familiar with the Vals AI Legal Research Benchmark. But their website has other frontier models' scores, and the scores OpenAI is now revealing for "Astra for Law" are slightly less than Claude and Muse:
https://www.vals.ai/benchmarks/legal_research
"Rogue OpenAI agent swarm accidentally overturns the Civil Rights Act"
followed a month later by
"Anthropic's Claude inadvertently repeals the 19th amendment"
I've had top SV lawfirms whos partner charged our company $2000/hr and still couldn't get the right docs in the signature packet. and another getting share counts wrong during raise.
frustrating that law firms have no liability for these mistakes
I welcome ai law
the partner shouldn't be doing sig packets
As a user of ai hooked up to my Gmail I assume I’ve lost all privilege
Since the cost of building software is now cheap, there is nothing stopping them from building everything imaginable. They'll soon have an app store with every app built by them and they'll say its for security reasons. Nothing is stopping this coming monopoly
At a certain point why would they sell anything other than services and products their eventual (actual) AGI/ASI builds in literally every market.
When opportunity cost isn't a thing anymore because it reaches every corner of the planet simultaneously faster and builds better than any human can.
There's no reason to let others build on top of AI, except if the AI determines that it needs capitalism to continue because it's paperclip goal is to maximize shareholder value.
most of the complexity in software isn't in the actual code
"Introducing Astra for Dog Walkers"
The end goal should be AI judges, I think China has implemented that to some degree.
The most interesting use case in my mind is skipping law suits. Obviously you need lawyers in court. But lawyers are people you are basically paying to fight for you.
Instead, if you resolve your dispute outside of court, you don’t need a lawyer. If both parties use ChatGPT to find the relevant laws or read contracts, they could come to an agreement without expensive legal fees.
except for those pesky things called rights
I find the watermarking dynamic to be really interesting in the legal space, as more large model providers provide increasingly powerful legal capabilities, and adoption (presumably) also increases. Attorneys aren’t the same as developers as their work can be traced back to them, and there are personal bar licenses and reputations at stake. I wonder if knowing the likelihood that AI generated something helps or hurts in that respect.
I also see a lot of watermark removal services popping up as a result.
From my experience LLMs seem to forget sometimes who they are representing when drafting clauses or editing / redlining.
Our counsel made a few edits where it clearly drafted in favor of the customer instead of us.