Im sorry given how bad of a situation that is, but it would be so ironic if they used Claude or codex for this

If the claim is true, I'd expect them to have used either those models or Grok or similar.

Of course if I was the FBI, I would make it so hackers trying to breach the system get a honeypot where all the data is fake, and with LLMs (even poor ones) it would be very easy to fake an entire alternative reality.

How about putting some cash bounty flags out there in dedicated mock systems.. and maybe allow the public to attempt cyber breaches with the same impunity that anonymous perpetrators enjoy.

I’m sure there’s some value in that.

Thats a major attack on the US.

If your systems are compromised and need to coordinate, what do you even do if you can't trust anything, assuming the attacker is still inside the network?

Is it, though?

If the goal is to exfiltrate data, I guess it is. If the goal is to make the people working in the FBI feel vulnerable - and pushing out this sample data would suggest that it is - I don't think it is. You could probably do the same with data from social media sites and data brokers.

Uh yeah, it dangerous. Public data brokerage doesn't identify FBI agents in a master roster?

Consider open investigations with covert agents. Leaking their identitys could compromise entire investigations.

Hopefully there was some foresight in washing undercover agents from these systems to other secure ones or something otherwise that's pretty bad.

Public data brokerage doesn't identify FBI agents in a master roster?

Are you saying that Ethan Hunt was involved?

In general, most governments have standard operational policies that mitigate such issues (ISO 15408.) =3

ISO and ICC start with the same letter, just saying...

Well that's a big one.

Perhaps firing expertise and hiring incompetents wasn't a good idea.

There's incompetence in every major company including Oracle. PeopleSoft isn't known for being the most modern or secure thing out there. Less reliance on 3rd party software like this will be a good thing going forward.

That's assuming that the result of this will be to switch away from PeopleSoft.

I think enterprise software in this vein used to have a quasi-monopoly due to the sheer work required to build software of its size (not enough engineers exist in the government to do so), and the difficulty for competitors to enter regulated markets and so pretty much 1-2 options to choose from.

AI now makes it possible to build this kind of software in-house, offering a 2nd choice, though it'll only be as good as the standards of the teams using it. Only time can tell.

Enterprise software is complicated mostly because of number of customers it can support.

Bespoke software can be orders of magnitude less complex. There are many reasons companies choose to use vendor solutions, but for large organizations it’s usually not “we literally can’t hire enough engineers to build it.”

There are so many counter examples.

Bespoke software can be orders of magnitude less complex.

Not in government. For payroll/HR, we're talking about hundreds of pages of legislative mandates, union bargaining rules, Title 5 statutory compliance, and FISMA/NIST regs. Just customizing something like PeopleSoft to fit is a large task in itself, let alone trying to implement bespoke software using an engineering pool that isn't even large enough to modernize the software it currently owns.

Bespoke sounds nice and works well in startups but that's not the context we're discussing. Check out Phoenix Pay: https://en.wikipedia.org/wiki/Phoenix_pay_system and understand the US is even more complex.

Not even the FBI has hostage negotiators good enough to get you out of an Oracle contract.

Do you think this is a consequence of the seemingly quarterly RIFs at Oracle? Is it that offshoring wasn’t such a good idea?

The irony, of course, is that this will likely have a negative impact upon Oracle's reputation, which will have a negative impact upon its value, which will then be resolved with more RIFs.

It's okay. Larry got another island.

I doubt it, look at all other breaches over the years.

If anything Oracle will "contribute" a lot to congress people's midterm reelection and in a few months all will be forgotten and Oracle will get more Gov. contracts.

Doesn’t seem new. Do you remember the 2015 OPM hack? https://en.wikipedia.org/wiki/2015_Office_of_Personnel_Manag.... My data got leaked in that.

Was that the one where due to political grand-standing, we had a government shutdown that led to furloughing the contractors who worked on security, so no one was watching the dashboard? Or was that a different one?

I’m not sure that breaches are prevented by “watching the dashboard”, but what do I know?

I always assumed DOGE + Ka$h would create an impenetrable fortress of strength; a beacon on the hill of brilliance and security.

If you use Trump Coins to reflect the Evil Eye away from you, nothing bad can happen to you.

It can only good happen.

I know a guy who named his kid Kash (before orange man) and he regrets it.

To be fair, it was a terrible name before Patel.

It's short for Kashyap

FBI in this past year has been desperate for hires. To the point they have ran campaigns to do 1 interview to hire. I've had a few recruitment calls to interview for positions but their pay is far below market rate for cybersecurity professionals. To the tune of like 30-50k/yr below what you could get elsewhere for analyst work. Add that in with the horrible reputation they have now, it's a grim situation.

pay is far below market rate for cybersecurity professionals. T

And there is no glamour on sense of duty when your main job is to make sure that the Epstein files are not revealed so USA citizens are in the dark about who were the rapists and criminals in the list. An awful job.

TLDR. A Peoplesoft (Oracle HR) instance was compromised which allowed movement into GovCloud (AWS)

PeopleSoft 0-day.

Just goes to show that the wall of IT bureaucracy does nothing. I'm sure they had an ATO, a several-hundred-page SBOM, compliance audits, etc.

The FBI had some very smart people. The current regime/administration's number 1 priority is to eliminate anyone competent in order to replace them with loyalists. Stuff is only going to get worse.

The FBI had some very smart people.

They trusted Oracle. I'm not sure I believe you.

I’m skeptical that multiple terabytes of data were exfiltrated quietly. I’m struggling to see this as anything other than a bluff.

If it was compressed prior to exfiltration it would be much smaller than the original data. I would expect this is mostly human readable text and a fairly high compression ratio. Agree it would still be large but 2 TB at 7:1 ratio drops to 285 GB. I don't know how much these servers are doing with data but I'd suspect sneaking out 10 GB/day over a month or 20GB/day over 2 weeks isn't going to trip up much. The CPU hog for the original compression might, but if you batch out the process in chunks (like a good data engineer), you probably wont trip many thresholds outside of expected use. Just a theory on how I might sneak that much data out of somewhere that has eyes on it, I don't know many details except what's in the article.

In the same week the head of the FBI went on national TV to claim credit for increasing the bureau's use of AI by 605%, whatever that's supposed to mean.

https://www.tomshardware.com/tech-industry/artificial-intell...

Is their name a reference to pokemon? Or to the meme that the FBI/CIA glows through the screen?

You're thinking of "glowie", "shiny" is def a Pokémon thing.

...and this is how the FBI makes you a higher priority target, and you wind up caught.

US keeps arresting and charging people from this group for well over 6 years now, and this happens in 2026. They don't sound very scared.

The group is not afraid, but the individuals who got charged probably stopped all black-hat activity: there was an individual in his early twenties who "was sentenced to three years in prison and ordered to return $5 million", which is life-changing to say the least.

There’s a scene in Battlestar Galactica (2004) where someone asks Captain Adama why the Galactica doesn’t have networked computers. So the cylons can’t hack the ship…

this is the way

It's not just a scene; it's the whole premise of the setting. It's why the Galactica survived and the newer ships did not. It's why the new Vipers got wiped out and they had to pull the old ones out of mothballs.

In the pilot, the Galactica was literally being turned into a museum, and that's why they lived.

"What do you hear, Starbuck?"

nothing but the rain

It was even worse: the Galatica was a museum ship, and it was being decommissioned!

No, GP is correct. In the pilot, the ship was in the process of being decommissioned from military use and converted to a museum. It wasn't already a museum prior to the events in the pilot (if that's what you meant -- maybe I'm not really sure what your comment was saying.)

I believe they were nearing completion and already had the ceremony when the Cylons attacked. It's been a while since I watched my favorite tv series.

I miss these kinds of shows. BSG, StarGate, Star Treks, FireFly. I'm sad that they went out of fashion and the genre has moved on to other concepts.

Only Expanse and The Orville compares in the past decade I guess.

Which was always pretty stupid. At best it means the Cylons can't hack the whole ship, just whichever part they exploited to gain access remotely in the first place. But really the Cylons would've needed exploits for each individual system anyway, since simply connecting them with wires shouldn't just do that. And unless those other systems are completely air gapped with no wireless or other access, it would be trivial for them to still gain remote access, or have one of their infiltrators provide a local connection of some kind. AFAIR that was never a plot point, which was odd.

And unless those other systems are completely air gapped with no wireless or other access

That is exactly the canon.

I think it's genius to use this element of computer security in narrative fiction. But I think they still had comms across the ship. The cylons could still clone the captain's voice and have random crew members perform arbitrary tasks by directing people to do so over the ship's radio.

whichever part they exploited to gain access remotely

I think you missed the point of "no networking", you have to actually physically sit in front of the computer. There is no remote access.

I didn’t follow the whole thread. Just want to point out that over the last 10 years hacking air-gapped systems has been a primary objective. Hard drive clicks, keyboard clicks, CPU squeaks, speaker-to-speaker. Exactly what advanced adversarial AI would be specifically good at.

There's also a scene where they do network the computers and are hacked nearly instantly.

but thanks to Baltar's firewalls and code obfuscation, just barely are able to escape after calculating the jump and having to literally wipe and reinstall all code on all systems.

I loved the writers giving every firewall its own LED indicator, so we knew exactly how far the toasters had got.

Sometimes I wonder if filmmakers have the right of it. They need to show what's going on in the computer to a distracted audience in a heartbeat. When they pull that off on screen, I wonder if us software devs should be taking notes.

Isn't that what dashboards are for? They may not look like simplistic LEDs, but being able to "see" what the automation is doing is something everyone that's ever made automation wants to do. It starts with making logs before migrating to live dashboards.

Its basically that staring at the log patterns in graph form or system stats you get a pretty good idea of what 'normal' looks like and anomalies stick out pretty quickly.

Though having a person interpretingthe degradation progress and relaying it to the rest of the team could be a usable scene.

A guy I went to high school with used to do that for a living. I have no idea what they call it, but it's basically theatrical UI, where the screen has to act and usually deliver a bunch of exposition.

In real life, malware turns the MacBook camera on and off so fast that the user doesn't even see the LED light up.

Could this be resolved with a capacitor? LED can stay on for at least a second or two once the camera is off? Not sure on exact components but this feels solvable and only problem may be slim design of their laptops.

Does this mean we'll see new startups in the sneaker-net space?

Hmm ShinyHunters seems to be in the news quite a bit recently. Most high profile was the Canvas LMS hack last spring right during college finals. Wonder if there will be a ransom for this data as well.

They got paid for the Canvas hack allegedly about 10M$ by Infrastructure, so it makes sense to continue. This one probably has a larger price tag

Looks like it was an Oracle PeopleSoft 0-day so I imagine there are a lot more systems vulnerable.

I wonder if that was really a 0-day or an intentional backdoor?

it's Oracle, 99% chance it's a 0-day lol

it's Oracle, 99% chance it's a back-door lol

So, what are the odds this was done with an open-weight LLM?

Uh 0%?

There are many people that run open weight LLMs. And unsurprisingly, they don't all have a copy of the FBI employee database.

If you mean "using" an open weight LLM in combination with other tools or even potentially frontier models, then that's a lot more likely.

That feels like publicly announcing that you want to be in a lot of trouble.

Imagine the FBI's relief when the hackers only got a list of their employees, not the UFO files.

At this point, no one seems capable of keeping a large database safe. I assume all medical and biographical information that exists is in the hands of the major state actors.

China hacked 22.1 million records of US government employees:

https://en.wikipedia.org/wiki/2015_Office_of_Personnel_Manag...

Google seems capable

And the city wonders why I don't want to put my credit card info in their crappy parking app and would instead prefer to put a quarter into the meter for 30 mins.

What info can be gleaned from that? Surely the mere fact that you have a credit card means your name and billing address are floating around.

I guess your parking history around town could be valuable if someone is targeting you.

What info can be gleaned from that?

The card number?

In USA, folks who check their statements monthly are at little risk of immediate financial pain there.

When your lifetime of credit card transactions leaks, that could be financially painful, embarrassing, etc. (can be discriminated against, including with pricing)

I do dislike creating a log of where I park on some random company’s server. Nice that ALPRs/govt.-funded corp spycams/Ring/etc. make sure the quarter method is minimally marginally effective at protecting privacy.

In USA, folks who check their statements monthly are at little risk of immediate financial pain there.

I had to fight a bank for months over a clearly fraudulent charge. Sometimes it's easy; other times it isn't.

In 2026, having my credit card number compromised is the least of my worries. At least here there is an established process for denying charges and ordering a new card. As long as you're not using a debit card, this is not a big deal.

My bank will reverse debit card charges. Based on that, I assumed that was a standard thing now.

The problem is debit cards leave a window where you don't have access to your own money until it gets reversed.

What happens with credit card? Is your line of credit reduced until it gets reversed?

Ok, your card is compromised. It's been cancelled.

One, how much money is in your pocket so you can eat?

ok, you'll use your second ca.... oh, it has to be cancelled now too.

Ok, lets wait a few days for another card, and lets go use it the first time, what hacked already, I guess I need to wait a few more days.

As long as you're not using a debit card, this is not a big deal.

So screw 60% of all transactions done on a card? This doesn't seem workable.

Meh, I’ve been giving out my credit card number willy nilly to merchants for 20+ years, and it hasn’t been used without my authorization over the course of what must be tens of thousands of transactions.

I have a text alert setup for transactions, so I presume I’d be able to successfully challenge any fraudulent ones pretty quickly.

who stores card numbers other than the processors? that should be a hangable offense. I've integrated card processing on multiple sites, and not once does the form come from me. I add the processor's JS, and it collects the data to move along. They then return to me a bit of information that includes success/fail so that I can decide what to do from there.

I add the processor's JS, and it collects the data to move along.

Consumers aren't gonna notice the difference if the site gets hacked and that JS is swapped out for a malicious set.

Yea, it's insane seeing this person arguing about the nature of credit card theft when we have a million different examples of how it happens and how rarely the end user knows until it's far too late. We almost always learn about itpost ad hoc.

You've moved the goal posts. A typical site isn't storing the numbers so when they get hacked, that data is not available. If you're suggesting hackers directly injecting malicious JS to hijack card data then that's totally different. I'm not insane about this particular subject. You're just standing on a soapbox

That what services that offer disposable and merchant-locked virtual cards are for. I have had good experiences with Privacy.com and Revolut.

And how do you pay them? And how secure are their systems.

It is unthinkable to me that anyone believes there is such a thing as computer security after so many years of nonstop hacks and leaks. If you have a computer and it is connected to a network with access to the Internet, assume that computer is semi-public. Meaning, if someone was interested enough in accessing your computer, they could do it. Do not hook any computer with access to anything that would be devastating if it was made public to the Internet. Do not put anything that would be devastating if it was made public onto someone else's Internet-connected computers.

For example, do not hook your goddamn water or traffic or electricity infrastructure up to the goddamn Internet, and then, do fire the guy who suggested it.

The correct analogy for computer security is not locks and keys and doors and gates. It is a house in a floodplain. Your house will not survive the flood of it hits you. Do not store anything critical or irreplaceable in that house.

A generation of coders who can't/are scared to write "Hello world" in C without Claude doing it for them has not helped.

I think you're exaggerating a bit.

Does this answer your question?

/s

I've seen it.

People flaunting their credentials in multiple languages, then sweating bullets and apologizing profusely when they see

  int t = 4;

You can either code or you can't; the language is merely a vehicle.

I agree, but then learning to code isn't much of a hurdle. It's a similar effort to learning vim. The difficult part is getting to know the language. I never coded in Haskell for example and learning to use that language would take effort. On the other hand, it would be pretty easy with an LLM at hand.

It might even help in figuring out whether Haskell would be a good fit. Something I couldn't do, as I do not know the language. Then again, it's not a question that really gets asked much in a corporate setting. Most things are just solved in a few popular languages, whether that makes the most sense or not.

learning to code isn't much of a hurdle... The difficult part is getting to know the language.

I agree. The people who depend on chatbots to write their code for them won't have either of those skills though. They don't know (or are in the process of forgetting) how to code, and they're missing out on the opportunity to really learn the language by turning off their brain and letting a bot spoon-feed them code.

An LLM would only get in your way if you actually wanted to learn Haskell.

We’re like 6-7 tiers deep on that aren’t we? Does every c developer understand the instruction set on the cpus their code is executing against? Does every c#/java/other managed memory language deeply understand their garbage collector?

It’s abstractions all the way down and most people aren’t going to have an intimate understanding of every layer, and it’s not economically worth it for the vast majority to even try

Two things:

Claude hasn’t been around for a generation yet.

It’s a good thing that people are scared to hand write memory-unsafe languages. 50 years of exploitation has finally sunk in…

It used to be that nothing was secure but that was OK because at least adversaries would have to expend effort. If you are one of a million companies why would anyone hack you. Maybe if you are a target you need a lot of investment, but most orgs only prevent the most egregious of vulnerabilities.

The calculus has certainly changed. Hacking is becoming even more frequent and… I’m not really sure what the equilibrium looks like.

It’s not really an option to stop using computers or networks. But it’s going to be way too expensive (or maybe even impossible) to secure even just critical systems.

Maybe banks and governments can secure themselves (and that’s a big IF) but it really feels like something fundamentally has to change.

It’s not really an option to stop using computers or networks. But it’s going to be way too expensive (or maybe even impossible) to secure even just critical systems.

Admiral Adama says otherwise.

The military has significantly different incentives.

Even just consider banks and e-commerce. They are hugely lucrative and making them even a tiny bit less accessible directly impacts their revenue. As an example, Amazon seeing that latency has a measurable effect on purchase behavior.

Maybe the military (fictional or otherwise) can go back to the ARPANET but most economic activity created by the internet cannot afford to disconnect

People buying less shit on Amazon (or e-commerce in general) would be side benefit. So much waste.

I am reminded of the scene of a guy walking through various layers of security to access a computer that isn't connected to any network and still wonder what the hell this guy's job was in Mission Impossible (1996). The data got stolen either way, because of course it did, but what highly sensitive work can you even do on a computer not connected to any network?

If there's too much security in the way, it seems to me that work becomes impossible.

We had water and traffic control and electricity for decades and centuries before the Internet. It is less convenient and more expensive, but it also means hostile countries can't literally poison your drinking water from across the planet. It's not a difficult trade to consider.

Is it really more expensive to not connect a water treatment plant to the internet? I can imagine the vendor selling that idea but I struggle to come up with how that could make a water treatment plant cheaper to operate.

Yes. Without a remote system you must have a real person check levels, pumps, pressures, and many other devices thus be present. This person must be trained and you will likely need a backup as well.

If not a person you need more redundancies built in. Bigger tanks, multiple backup systems. When items start failing you need them to be shutoff in a timely manner. Water pumps at these facilities are in the 50-100k range. When it starts failing you want to know.

Think of it like driving a car and it starts making funny noises. The longer you wait to fix it the more it costs.

This smells wrong to me. You already have people working at the water treatment plant. You don’t need it connected to the internet?

Why that drastic, checking system state (telemetry) can be done using 'data diode' style networking - one network just broadcasting sensor data, other network allows changing parameters and 3rd one allows allows software upgrades and so on.

You can be very defensive and design any remote sensing controller to act as two systems - one management cpu only does data routing (no other connection than administrative tasks), sensor cpu works only with sensors. As bonus you can have management cpu act as active firewall.

Main problem it is necessary to have in house expertise (hw, fw and process knowing) which in making company lean are optimized first and outsourcing custom solutions suddenly too expensive.

Maybe banks and governments can secure themselves (and that’s a big IF) but it really feels like something fundamentally has to change.

The problem is that most companies don't care if they get hacked so long as the hackers are just taking data and not interfering in their ability to bill customers and make money.

They face zero meaningful consequences if their data gets leaked. The money they save by not taking security and employee/customer privacy seriously will more than pay for the year of "identity protection" they'd have to pay for (assuming the hack gets found out) anyway.

They actually care about ransomware, but most of the time that's also something they can comfortably buy their way out of. We've seen a lot of companies pay off ransomware gangs rather than invest in the kinds of robust backups that would make recovery possible/less painful than rewarding the hackers.

What's needed for change is regulation with actual teeth that makes not protecting their data either meaningfully expensive or criminal resulting in executives spending time behind bars for their negligence. Without that, things are only going to get worse, especially as companies experiment with using AI and increase dependence on third parties and cloud providers who themselves become rich targets.

That probably still won't help the FBI though. Our government isn't exactly big on holding themselves accountable or even prioritizing competency right now.

Many years ago, I regularly played cyberpunk tabletop RPGs with a number of other computer-inclined friends. We all used to laugh at ridiculousness of a key assumption of the game - the idea that giant corporations would ever connect their internal networks, full of valuable data, to the larger global telecommunications network.

It is unthinkable to me that anyone believes there is such a thing as computer security after so many years of nonstop hacks and leaks.

Of course there is. For example, SeL4’s security and reliability proofs still hold in the world of LLMs. The problem is that most software isn’t written on that firm foundation. Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”. You don’t get secure software by working like that, because security vulnerabilities aren’t visible.

We - humans - know how to write secure software. Just like we know how to make safe aeroplanes. The problem isn’t that we lack the capability to make secure computers. The problem is we don’t have a culture of security. Secure software is - somehow - niche. And as such, it’s much more expensive. And nobody wants to pay.

The bank has the best doors, the best locks, and the best cameras, and it is patrolled by a guard who props the doors open to so he doesn't have to keep fooling with the locks and points the cameras the other way to extend his smoke break. SeL4 would be another system used by humans.

It's always possible to break a perfect system by moving an additional layer of abstraction outward, and attacking one of the assumptions upon which it's built. Some of our era's highest security systems - game consoles - have been broken by undervolting them until the logic failed.

It's always possible

It's often possible. But not all systems are vulnerable to undervoltage attacks. For example, I don't think the iphone secure enclave is vulnerable to this.

And good security uses "defence in depth". Multiple layers which each individually need to be compromised to break the whole thing. To hack chrome, you need a vulnerability in the renderer or VM. Then you also need a sandbox escape, and a way to use that to attack the browser's parent process. This is much harder to do.

not all systems are vulnerable to undervoltage attacks. For example, I don't think the iphone secure enclave is vulnerable to this.

Then there's decapping / depotting, a world of different types of microscopy - some destructive some not, directed EM attacks, etc.

And good security uses "defence in depth"

And automation has enabled "offense in depth"

To hack chrome, you need a vulnerability in the renderer or VM. Then you also need a sandbox escape, and a way to use that to attack the browser's parent process.

Or you just phish the user into installing your exploit. There's always another layer. Always a potential exploit. Because ultimately the same properties of the universe which permit computation within a closed system allow for predictably observing and influencing it. The expense and hassle of doing so are widely variable, of course.

There's always another layer. Always a potential exploit.

So what? Most attackers aren't nation state adversaries. They're some kid in Wyoming messing around with deepseek. We live in a world where most exploits happen because someone was running an unpatched, 8 year old copy of wordpress. Because they put their insecure mongodb instance on the open internet. Because they used admin / "12345" as the username and password. We don't need to make hacks physically impossible for a nation state adversary. Just really, really difficult and expensive to pull off.

Honestly. If people talked about physical security like they talk about computer security, you'd have people telling you that, because walls can be physically smashed through, they don't bother locking the front door to their house.

I'd stick with always. Defense vs offense in anything reasonably complex suffers from one issue that simply cannot be overcome. To defend, you need to defend against every single possible imaginable attack, from now until forever. To attack, you need to find a single attack that works. And on a practical level all systems need to be accessible by somebody, yet that somebody is himself also now a part of your security structure and is never going to be 100% reliable, both in terms of corruption and incompetence.

It's always possible to break a perfect system

A perfect system is either extremely limited in scope or flawed in it's assumptions.

I once worked on AUD 450M banking project, the root password was kept in a kickstart file and unchanged, root SSH was allowed. The bank didn't care until I told the external security auditor who included it as part of their report.

Snitch

I sure hope you're joking, warkdarrior!

There's absolutely no way to account for humans, who can be tricked, or pressured, or just make human sized mistakes.

Then there's no such thing as security.

By the way, there are countless ways to account for humans. There are entire branches of engineering devoted to this. If you don't want someone to leave the bank with a pen customers use for signing checks, you just chain it to the desk. If you don't want the installer to forget to put the pen-chain in, make a photo of the chain part of the checklist required to get paid. If you want to... etc.

The idea is that you determine an acceptable level of risk, then secure to that level. Maybe the acceptable level of risk chosen by companies is wrong. Maybe we need to increase that risk exposure via heavier fines and regulations. Maybe the cost of reducing that risk is too high already. Maybe we need to fund that. Maybe it's too confusing and we need to research better standard practices. I dunno. But this is not some unsolvable problem.

Then there's no such thing as security.

There really isn't

Ask anyone seriously involved in security - whether computer science related, or in general.

A thought experiment: Think about the most important secrets a country can have - now think how they are still discovered by competing countries, enemies, etc.

As long as there are humans in the loop there is a known weakness.

Again, of course there is.

Decades ago, I worked in a bank in an old building. The door had a card reader for access. You boop your card and the door opened. People would hold the door open for each other all the time out of politeness, even when they didn't know each other. Security told us not to do that, but it's hard to convince people to stop being polite.

I had a laptop stolen from my desk in a place like that once. (Not a bank - but similar door-card reader system). This guy came in in the middle of the day, wearing overalls. He confidently walked through the door after someone, like he belonged there. He walked up to my desk, swiped my laptop and just strolled out.

At the bank, they've replaced the door with mechanical gates and a security guard. The gates - physically - only let one person to walk through at a time. You can't hold a gate open any more. And the security guards stop anyone who tries.

Is it 100% foolproof? No. But it's way more secure. It would have stopped that laptop thief.

There's this pernicious, defeatist attitude that if you can't make a system 100% secure, so you shouldn't try. That's misguided. Most systems can be made orders of magnitude more secure than they are today. It just takes a bit of care and work.

As soon as you make something foolproof, the universe evolves a better fool

This is the key truth that we keep forgetting

Fine. Make the universe work for it. The whole system becomes more resilient as a result.

Look at our immune system. Incredibly complex and clever, and able to keep us alive in the face of all sorts of pathogens. It exists because of this cat and mouse game, played over millions of years.

There's people in the highlands of PNG who regularly eat each other. Of course, many are thought to have died due to prion diseases. But now these tribespeople seem to have become largely immune to prion disease. Incredible.

Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”.

I work in secure systems and it’s shocking how many people believe this - the incentives from management are all about it too.

I like to think behind every Dev anxious to ship half baked software sits an omniscient middle manager with a vague idea of what the product was supposed to do, maybe

This will all of course end when our AGI lords lovingly manage everything /s

No because it's a problem of human communication. Taking humans out of the loop creates other social problems that have been thoroughly documented in cyberpunk mythology, not actual a solution, just trading one big problem with multiple little ones.

Please note the sarcasm.-

It's worse: it seems like almost every developer on this very site is aspiring to be that middle manager, with LLMs as their underlings.

We are headed for scary waters.

Guess you don’t remember the days of ssl on login pages, ssl strip, exfiltering data via JavaScript prototype pollution, and a million other things like that.

Only just when we started to have a resemblance of security we got agile and startups breaking things (making rubbish software to capture a few bucks faster) and now vibe coding and llm assisted hacking.

The point of my, arguably rant, is that there is nothing new under the sun.

I'm not sure my worries are assuaged by thr fact that it's been bad before, too.

I believe the technical term is “Move fast, and break things.” MVP is a huge disaster. I can see it working for applications that don’t process PID, but only an idiot ships data handling software before it’s been dragged through a lot of testing. I tested my app for two years, before finalizing, and an LLM still found a couple of holes (minor ones, but ones I missed).

After the DOGE debacle, I suspect that all the previously really secure stuff, is now out there, too. In fact, I wouldn’t be surprised if some of these leaks, came from that.

FBI employee data is very bad.

The issue is that in consumer and enterprise software, move fast-and-break-things outcompetes secure-by-default every time. Critical infrastructure needs to have a different set of priorities, but it’s very hard because the expertise is so thin on the ground. Why would anyone with the expertise to make these calls bang their head against the wall trying to educate bureaucrats about these things for $150k a year when they can easily make multiples of that in big software companies that don’t own that level of risk.

The incentives have to change. Any breach regarding PID should have fines as a percentage of revenue of the company. Any breach intentionally covered up and found out later by a third party should mean jail time for the C level. Yes, I know it is hard to make such laws "foolproof". And yes, in the current political and economical climate it will not happen anyway.

The real problem is that even for companies that wish to pay more and wait more for secure-by-default can't easily tell the difference.

The only solution I can come up with is some form of certification or paid code review from a third party. I know that at least for Windows prior to 7 Microsoft actually allowed some parties to come in and check the code/checksum on an air-gaped computer. We somehow moved to "trust more" in the last decade, and now we can trust nobody

If DOGE is going to have an effect on network security, it's not going to be for many more years.

It's not as if best practices aren't well documented, or as if CVEs don't come out every day, or as if the information is somehow unavailable to even the most junior devs to take basic security measures.

Not all hacks are caused by pure negligence, laziness or stupidity, but most of them are. Even a little effort goes a long way.

My grandfather spent a couple decades as a builder, ran a construction crew. Whatever the project was, he wanted to know everyone he hired personally was going to reinforce and report to him anything they had the slightest doubt about. "Always hammer in an extra nail" was basically his motto.

What we do ain't that different. The difference is that when an apartment building collapses, it's bigger news than when a govenrment database does.

I'd argue the management has a point there; without a pressure to ship, nothing would ever get released, because computer security has not yet understood the basic concepts that every non-computer security work does:

- nothing is, can be, or even should be 100% secure; the optimal rate of security incidents in society is not 0 (with apologies to 'patio11)

- security is a simultaneous trade-off against costs and usability, and those two other factors are more important:

-- security is achieved primarily through raising costs for attackers to beyond profitability, and reducing impact of such attacks (due to "not in isolation from the world" below, this also mostly translates to costs)

-- if "properly secured" (in the current cybersecurity sense) product/service cannot fulfill its function anymore, then you may just as well not make it; either way, no point in paying you for security work

- security isn't done in isolation from other systems and the world at large; "if this happens we'll go straight to filing crime report with the police" is perfectly legitimate security measure (even if it works somewhat less well on the Internet); similarly, "this is secured by us having insured against it" is also a valid solution to some security problems

what's patio11?

The problem isn’t that we lack the capability to make secure computers.

Depends on the "we". "We" have the capability to make secure computers like how "we" have the capability to make EUV lithography machines. There exists a relatively small number of people and organizations in the world who can do so. Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc. All of those organizations have tried for literal decades, thumped their chests about how they have awesome security year after year, and yet have totally and utterly failed despite their best efforts.

Acquiring the capability to do so is difficult and challenging and requires years to invent if you start right this very second and know what you need to do, which these organizations emphatically do not. We need security at scale and fast. The only way forward is to scale up working solutions rather than letting the bozos who put us in this spot fail at scale with yet another promise that this time for sure they will solve the problem they have repeatedly failed at for decades.

It's nothing like EUV.

Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc

This is all by choice. They could easily have that capsbility, very unlike EUV.

And the reasoning behind that choice is looking at the tradeoffs and saying “shipping more features faster is more important”

Part of the argument (aiui) is that there is no way that culture could ever change to producing secure systems. There are too many weaknesses embedded in the organizational structure.

Oh, Microsoft can just figure out how to make unhackable systems, they just choose not to. They spend all of those billions of dollars per year on security and spent all of those decades on failed attempts as a prank.

You really think that if they could have they would not have, even just for bragging rights? Or are we going with that it is some kind of task demanding enormous expenditure even though the organizations that have made secure systems are infinitesimally small in comparison?

Microsoft has spent orders of magnitude more money and time than the organizations that have succeeded and the result of their efforts is Windows. That says everything you need to know about their capabilitys.

Multiple literal trillion dollars organizations have spent literal decades failing at it. You are really underselling the capability gap.

It's not rocket science.

If windows was reimplemented as a capability based microkernel like sel4, it would be far more secure. Run drivers in their own isolated processes. Do interprocess communication between them via capabilities and shared memory. Remove all ambient authority from programs. All the programs a user launches stop automatically inheriting all of that user's permissions.

There's no secret knowledge required to do this. The SeL4 team has written extensive documentation of how they did it. They also opensourced their kernel implementation, with correctness proofs for the whole thing.

The reason windows hasn't done it is the cost. You'd have to rewrite half of the NT kernel and refactor everything else. All existing windows drivers would need to be rewritten. If you forced windows userland use a capability based system, you'd essentially be inventing a new way to write windows programs. You'd need to document that, and write a compatibility layer for legacy programs. And solve some UX problems. It would be terribly inconvenient for everyone. Oh, and some programs would run slower as a result.

They could do it if they wanted to. But microsoft just doesn't care about security as much as they care about performance and compatibility. Linux is the same.

The big irony is that security would be a lot cheaper for microsoft if they designed the NT kernel to be more like sel4. Microsoft has to spend millions on security every year because any tiny bug in the kernel (including in drivers) might result in the whole OS being compromised. In a microkernel, a buggy driver is nowhere near as dangerous.

Your knowledge is out of date: Windows has supported capabilities from the start of NT and does run a lot drivers in isolated processes, along with most OS services. It has done for years. It has the most sophisticated IPC framework of any OS (DCOM) which is integrated with the operating system kernel's security frameworks, and used pervasively both internally and by apps. And Microsoft has created a new way to write Windows programs in which apps are expected to advertise the capabilities they need (see WinRT and MSIX). This is also over 15 years old.

macOS does the same but with more developer adoption. Apps don't have the ambient capabilities of the user and must advertise what they need via entitlements embedded in the binaries, or get permission just in time.

Which is all very good, and modern platforms are much more secure than they once were. Yet "capabilities" as a silver bullet are academic overpromises. This article I wrote is more about language/runtime level capabilities but OS capabilities are not much better.

https://blog.plan99.net/why-not-capability-languages-a8e6cbd...

SeL4 isn't secure because of One Weird Trick that others would adopt if only if they could be made to care enough, it's "secure" because it hardly does anything, which is why nobody uses it and why it has no impact on real world computer security.

The hard part of desktop security is not changing the operating system. The hard part is getting app developers to care. Most security features added to operating systems are ignored by developers, which is why Apple forces you to adopt some of them as the price of admission to the app store. If they didn't nobody would use them, as can be seen for apps distributed outside of the app store. The reason is security is a market for lemons. Nobody can see the result of security investments so it's irrational to invest. SeL4 has no solution.

Yeah it's about costs. I care about a lot of things, but the causes I actually give money to is a much shorter list. If personal data was radioactive, and leaking it cost companies real money, then more money would get spent on security. (and insurance, and lawyers.)

Yeah I've long said we should treat data leaks like food safety. The only way we'll see software security improve is if there were serious fines and/or jail time for leaking user data due to negligence.

PCI compliance works pretty well.

I would like to see restricted access to only us workers for us citizens data though.

Transferring us customer data outside the country should be illegal

PCI compliance is a joke. That isn't even close to good enough. I'll break all your PCI compliant stuff trivially.

sel4's guarantees break if you have DMA, e.g., from your NIC. It doesn't help with timing attacks. It doesn't cover your network stack. AFAIK, no one has taken up the mantle from Project Everest, so you'll need to write a verified TLS library. Once you've built all that, you can start thinking about your database/application/whatever. Then of course you'll have to verify all your dev's machines and scripts to ensure nobody is misusing a credential that can get stolen.

"So what?" you say. "Making a heavier-than-air metal tube take off and land millions of times per year without a catastrophe is also hard, and we no longer expect most or even many of those tubes to blow up or fall down."

Mother nature is not spending $$$ using AI and HI adversarially trying to find the exact combination of atoms that will cause your device to fail.

NASA was supposedly a CMM level 5 organization and they still managed to crash a mars probe because of bad unit conversions.

Did any martians hack the probe?

Jeff Goldblum with a thinkpad, maybe

Jev Goldblum? He’s all the hype today

A piece of software Lockheed made gave its outputs in US Customary units (not following their specification), and NASA expected it in SI units (as their specification expected)

That should have been checked multiple times by NASA before they launched it.

It seems like it should have been checked by Lockheed not NASA since supposedly NASA provided a specification, that specified the units, and paid for the software no?

SeL4 or proof assistant are not panacea. They do not help if assumptions about the task are wrong. And correctly formulating the task in real world is very messy.

So physical security is just as important. I really like how ARINC serial bus on planes work. One can have a reader that is physically incapable of sending anything to the writer. This allows to connect entertainment systems to flight data sensors safely.

In Airbus this system is replaced with Ethernet switches that in software ensures separation of traffic. The software was proven mathematically. But I am skeptical that it is absolutely bulletproof as a client under malicious control can influence Ethernet signaling and may exploit hardware bugs.

SeL4 or proof assistant are not panacea. They do not help if assumptions about the task are wrong. And correctly formulating the task in real world is very messy.

Nobody said sel4 was a panacea.

My claim is that doing this kind of computer security is possible. It's just expensive and inconvenient. We know how to make computers a lot more secure than they are today. The limiting factor isn't humanity's knowledge. The limit is that barely anyone wants to pay the bill.

The limit is that barely anyone wants to pay the bill

Do you even have any experience with how most companies work? SMEs barely have the cashflow to cover their daily expenses, let alone suddenly pay thousands for regular professional security audits and overhauls of their code. This is why security is an afterthought.

This is why security is an afterthought.

Security is always a cost center and rarely a profit center. That's the only thing that needs to be said.

The main issue is that market evolution will always surfaces the most cost-efficient entities within the ecosystem pressures.

That means designing the ecosystem pressures is crucial: things more meaningful than just pure capitalist private-profit logic must by enforced by thoughtful regulation or otherwise the ecosystem converges for private-profit of a small sliver of individuals (billionaires) to the detriment of all other ecosystem members (99% of the world population).

This holds for anything broader than pure private gain, may it be security, social fairness or ecological topics. Sole monetary-value optimization for private gain must be properly constrained or else it results in pure predatory capitalism that implodes society from within, may it be through leaky security, poisoned environments or social unrest.

SeL4’s security and reliability proofs still hold in the world of LLMs

Oftentimes the weakest link is the human operator who has direct access to those systems, not the computer system itself. Good old social engineering, in other words.

Even though one could use LLMs for social engineering, come to think of it, like re-enacting the movie "Her" involving a modern AI as Scarlett Johansson and an engineer working for the water utility as the romantic target.

...there is such a thing as computer security...

Of course there is. (...continues to talk about software security)

A bit over-optimistic when looking at hardware vulneraribilties like Spectre/Meltdown.

Secure software isn't worth much when it runs on vulnerable hardware (at least Spectre/Meltdown could be worked around in software though, but at a cost.

There's "no such thing as computer security", because this one time computer security researchers found vulnerabilities?

You know that's their actual job right? That and fixing the problem. Which they did in both software and hardware.

One time? lscpu on my system lists two dozen CPU vulnerabilities known to the kernel. I stopped tracking them a long time ago when it felt like every week brought another information leak or bypass.

When I run that on my computer I see a big list of CPU vulnerabilities. All of them are either mitigated in software or fixed at the hardware level.

Looks to me like the security researchers have been doing their jobs.

Yes, except that they haven’t been able to fix this completely in sw and the community knows this.

Secure software is - somehow - niche. And as such, it’s much more expensive.

Its not expensive because its niche, its expensive because its hard. Its a lot easier to learn a bit of html and javascript and knock up some web projects then it is to become proficient at all the things necessary to be good at security. Generally it also takes consulting with maths experts who have spent their life studying cryptography and as such command a decent wage.

Generally it also takes consulting with maths experts who have spent their life studying cryptography and as such command a decent wage.

It doesn't take a maths degree to look out for SQL injection attacks or to audit software & write up a risk assessment.

This is the major problem I see with flock cameras. They say it’s okay because they’re only using it for good. But can they actually protect the honeypots they create? No. Is it their fault if it gets stolen? Yeah but at that point cats out of the bag.