Hacker News

Top stories

Live mirror
30 storiesupdated just nowView source snapshot
  1. Laya the open source version of Jev(convaiinnovations.com ↗)
    143comments
  2. A graphical desktop for the ZX Spectrum(github.com/mindbox77 ↗)
    53comments
  3. AI-generated posters don’t have to be horrible(john.hartnup.uk ↗)
    466comments
  4. Human brain is two separate organs, Stanford Medicine-led research finds(stanford.edu ↗)
    183comments
  5. Tin: full-text search for Postgres(planetscale.com ↗)
    43comments
  6. Supabase (YC S20) Is Hiring for OrioleDB(supabase.link ↗)
    discuss
  7. “The Secret Life of Circuits” is here(coredump.cx ↗)
    52comments
  8. Asking Authors About Their Own Papers(medium.com/tmlrorg ↗)
    40comments
  9. Black Holes or Black Hole Stars? Astronomers Spar over 'Little Red Dots'(quantamagazine.org ↗)
    15comments
  10. I built the fastest PHP webserver in the world(qbixserver.com ↗)
    33comments
  11. Android 17 is the first since 3.x to add new APIs without releasing to the AOSP(grapheneos.social ↗)
    554comments
  12. San Francisco Onion Futures Company(onionfutures.com ↗)
    117comments
  13. GPT-6 Astra Solves a WWI German Radio Cipher(prinzai.com ↗)
    134comments
  14. New evidence for hidden chambers beyond Tutankhamun's tomb(nature.com ↗)
    7comments
  15. Agreement between the USA and Denmark (1951,2004) [pdf](state.gov ↗)
    21comments
  16. If math is more than proof, we need to better celebrate the rest of it(terrytao.wordpress.com ↗)
    193comments
  17. What Zig felt like, coming from Rust(besok.github.io ↗)
    110comments
  18. Cloudflare Quick Tunnels(cloudflare.com ↗)
    301comments
  19. How to Write with an LLM(sockpuppet.org ↗)
    360comments
  20. You can run Git on object storage if you re-make packfiles(tigrisdata.com ↗)
    24comments
  21. Saving another 100TB of RAM(cloudflare.com ↗)
    91comments
  22. Communication by means of modulated Johnson noise(pnas.org ↗)
    19comments
  23. SDCC – Small Device C Compiler(sourceforge.net ↗)
    23comments
  24. Almost Never Use AI to Write Anything Substantive(erichgrunewald.substack.com ↗)
    discuss
  25. Ray Ozzie and the Optimism of Being Early(reproof.app ↗)
    11comments
  26. Science Is Open Software(jepedersen.dk ↗)
    51comments
  27. The first new cat species discovered in 100 years(nationalgeographic.com ↗)
    124comments
  28. Why building a Rust LSP is hard(rust-glancer.github.io ↗)
    46comments
  29. OpenJev(openjev.com ↗)
    280comments
  30. Ctenophores: Wonders of Biology(quantamagazine.org ↗)
    8comments

Google Chrome Courgette differential compression algorithm

71 pointsby 15y agodev.chromium.org
21 comments
Courgette uses a primitive object disassembler adjuster to compute "assembly language" differences that are about 10% size of bsdiff.
15y agoHN ↗

This is exactly how updates should be done. Usually, I'm not that big of a fan of things that Google does, but this is a great achievement. Hats off to those who contributed to this. I hope all update systems eventually work this way.

15y agoHN ↗

Well, most binary updates do go through bsdiff, which was designed for exactly this. I'm surprised they were able to beat it by such a margin. Shit, the guy who made bsdiff did his doctoral thesis on the subject (actually, the algorithm used in bsdiff generates patches about 25% larger than the one in the thesis, but at huge performance increase (or, the thesis generates patches 20% smaller)).

15y agoHN ↗

Really, I wasn't aware that that many projects used bsdiff.

15y agoHN ↗

Yeah, me neither. What projects are we talking about here? I'm in Linux so I'm getting the full deb each time. I'm okay with that considering it's managed for me. I'll take that any day.

Adobe? Apple? Those are the two vendors that I can imagine people having installed on most, if not all Windows computers. And I know for a fact they both distribute updates as full copies of their software waying in at a couple hundred to several hundred megabytes, requiring manual updates, pop up windows and restarts.

I've yet to see anyone do updates as fast or seamlessly as Google.

15y agoHN ↗

yum-presto on Fedora uses deltarpm, which uses bsdiff.

15y agoHN ↗

Firefox, FreeBSD, OS X, Courgette (it preprocesses binaries and then feeds them to bsdiff), Sophos, Mochi Media, the Amazon Kindle...

15y agoHN ↗

I think Google could handle spending a bit of extra time on a single diff, in order to save 25% of their update bandwidth. Maybe they should just implement the thesis.

15y agoHN ↗

I'm surprised they were able to beat it by such a margin.

Technology marches on. I have ideas for improving bsdiff, too.

15y agoHN ↗

Call me stupid, but why do you have to disassemble the program to get the symbol pointers when you just finished compiling the thing in the first place?

Couldn't this intermediate step of pointer collection be part of the prelink process and skip all this guesswork?

15y agoHN ↗

Sadly, most compilers are not modular so it's generally easier to add a post-compilation step than to add an intermediate compilation step.

15y agoHN ↗

If the source is in a single language, compiled by a single tool, with one single version, and entirely linked by a single linker strictly at the end of the process, then that might be feasible. But that's probably not the case. If there is much heterogeneity at all, getting proper symbol information may be a lot more work than analysing the binary for control and data flow. For example, intermediate linking steps may resolve symbol fixups internally and just leave a list of relocations behind, or perhaps use strictly relative addresses within what looks to the final linker like a monolithic blob.

15y agoHN ↗

If we're talking about a way to generate an upgrade image for a single architecture, taking a known version # of an executable to another known version #, built on a production build machine, I would think that could cover the case easily.

That kind of led to my second question: this makes really small images, but only from one known version to another, right? What happens if the target to be upgraded is 80 revisions behind?

15y agoHN ↗

Presumably it's still a known version. Just an older one.

You only need to keep one diff for each version you've previously published. Shouldn't be hard.

15y agoHN ↗

I can't speak to what chrome does, but usually there would be rollup patches to take advantage of, cutting down the number of patches to apply dramatically. E.g. To get from 23 to 76 you might only need to apply 23->25->75->76. This would be slightly less efficient than going directly both in terms of time patching and patch size, but not terribly so (one can assume most patches are decently disjoint). Most importantly, this keeps the total number of supported patches manageable.

15y agoHN ↗

We want smaller updates because it narrows the window of vulnerability. If the update is a tenth of the size, we can push ten times as many per unit of bandwidth. We have enough users that this means more users will be protected earlier.

They're trying to get the vulnerability window down to minutes! If they succeed, this is going to have an impact on the economic viability of running malware.

15y agoHN ↗

We're also giving google implicit permission to push code to our machines anytime they want.

We should have conflicting feelings about this.

15y agoHN ↗

We're also giving google implicit permission to push code to our machines anytime they want.

We should have conflicting feelings about this

You have a point. But I think the answer to that is in the market.

Through Chrome, the browser is evolving into a new kind of platform. It will be a platform that encompasses locally execution and memory and all of the flexibility and availability of the cloud, and it will all just work.

There are sure to be competitors. If one competitor fails, we can always take our business elsewhere. We only have to worry if the government legislates us out of a means for oversight.

15y agoHN ↗

Red Bend's attempt at a preliminary injunction was quashed recently, with the court stating that "with insufficient evidence that Courgette infringes on at least one patent claim, Red Bend cannot prove a likelyhood of success",

I'm no lawyer, but this sounds pretty bad for Red Bend.