Am I missing something? Max boost went up by only 15% but the base frequency went up 38%

Better thermals maybe? Less throttling?

2 years sounds very fast. It seems to be because the 3d cache SKUs of each generation were released at different stages. The Zen 3 one was a later variant.

First zen 3 November 5, 2020 with desktop processors. First desktop Ryzen 9000 processors on August 8, 2024. So the generations were about 4 years apart.

I suspect the improvements are even more dramatic going from Zen 1 through to Zen 5. AMD has really hit the jackpot with how scalable the Ryzen CPU is considering how they're able to improve the performance from year to year. This is a stark difference to the FX series during the 2010s, which saw very small YoY performance increases by comparison. Ryzen really is AMD's equivalent to what Nehalem/Core was for Intel back in the mid 2000s.

That is all true but I will defend the FX series a little. Mostly now that there is a lot of software that scales across cores better now, they haven't aged as terribly as others have. They aren't great but not terrible considering.

Mostly now that there is a lot of software that scales across cores better now

That's pretty much irrelevant since the AMD's FX arch's issues weren't that SW at the time wasn't using all the 8 cores. Intel dropped the Core 2 Duo and Quad into the era where most SW was still stuck in single threaded for a long time and those CPUs still ripped single-threaded SW tasks regardless.

Here's the big reasons why the FX sucked back then and why they still suck today in the multi-thread SW era:

  Instead of discrete, fully independent cores, AMD grouped processing units into "Modules" where each module contained two integer execution units, but they had to share critical resources like one FPU, the instruction fetch/decode pipeline, and the L2 cache so when both "cores" inside a module were heavily taxed especially with math or physics-heavy calculations (like in videogames), they choked fighting over shared hardware.

  AMD designed Bulldozer with a very long pipeline, betting they could sacrifice efficiency per clock cycle in exchange for extraordinarily high clock speeds(a-la Intel Pentium 4) but the IPC was so bad that an FX core was often slower clock-for-clock than AMD’s previous-generation Phenom II chips and also their power consumption exploded. 

  FX processors were plagued by high cache latencies and an inefficient memory subsystem as another bottleneck.

So unless you're into collecting vintage CPUs as display pieces, this one definitely belongs in the e-waste pile instead of burning electricity, because it did not age like wine with the adoption of SW multi threading like people were hoping.

AMD designed Bulldozer with a very long pipeline, betting they could sacrifice efficiency per clock cycle in exchange for extraordinarily high clock speeds

It blows my mind that AMD watched Intel try to do basically the same thing only a few years prior with NetBurst, and fail so badly that they had to scrap that entire evolutionary branch and start over – and AMD still went and did it again themselves anyway.

Hyperthreading/SMT is a significant boon for heavily threaded workloads. What makes that such a win while Bulldozer's implementation of "two integer units sharing a front-end, cache and FPU" is supposedly so bad? Because that description makes the 8 core Bulldozers sound exactly like a 4 core with SMT.

Hyperthreading/SMT is a significant boon for heavily threaded workloads.

That's hugely debatable and depends on SW workloads and the SMT implementation + CPU pipeline design.

In SMT the execution engines, ALUs, FPUs, and caches are completely shared. When one thread stalls waiting for RAM, the second thread sneaks into the idle execution units. At best, SMT yields a ~10% to 20% throughput boost over a single thread.

Because that description makes the 8 core Bulldozers sound exactly like a 4 core with SMT.

It's not the same thing. Bulldozer arch sits between a true 8-core and 4-core + SMT implementation.

At best, SMT yields a ~10% to 20% throughput boost over a single thread.

Exactly, which is a significant benefit for how marginal the costs are.

It's not the same thing. Bulldozer arch sits between a true 8-core and 4-core + SMT implementation.

Then surely it should be even better than 4 cores with SMT?

If you're going to argue that the problem with Bulldozer was it's weird semi-SMT solution, you need to explain how it would've been better without it (aka as a regular quad core). Because even if it just gets the 10-20% performance improvements from being a form of SMT it would be better to have it than to not. And if you have lots of integer unit-bound threads, it should be even better than that.

Then surely it should be even better than 4 cores with SMT? If you're going to argue that the problem with Bulldozer was it's weird semi-SMT solution, you need to explain how it would've been better without it

I explained all the bottlenecks of the architecture in a comment above, that the issue was more than 4-core +SMT instead of true 8 cores. Please read it.

I did read it. It's not clear from it why you think having 2 integer units per core in a SMT-like configuration makes it worse. If you have <=4 threads it doesn't matter, just schedule the threads on different proper cores. If you have >4 memory/FPU/front-end heavy threads you should see the same benefit as SMT. If you have >4 integer arithmetic-bound cores, you should see a significant benefit beyond what SMT would give.

Now the very long pipeline and high memory latency are obviously significant issues with the architecture but those seem disconnected from the 4-core+SMT issue? I'm not questioning those issues at all, it's just not the part of your comment which interested me

EDIT: okay so in this comment: https://news.ycombinator.com/item?id=49809017, you explain that there's actually a fairly large part of a core that's duplicated, not just two integer units. If each "core" gets its own integer unit, register file and L1 cache, you're actually paying a ton of die space for it, unlike SMT which is "free". I can totally get how that can be a terrible trade-off for most workloads if it all ends up mostly starved due to front-end/FPU/memory throughput.

Instead of discrete, fully independent cores, AMD grouped processing units into "Modules" where each...

Yeah that's hyper-threading intel was doing it as well and all modern CPUs do it as well. Where AMD dropped the ball, was they did not disclose that in their marketing as clearly as they should.

All CPUs today are marketed as x cores 2x threads, back then some AMD marketing genius in their infinite wisdom put 8 cores on the box, instead of the honest 4 cores with hyperthreading.

No, AMD's FX "fake" 8-core was more than just 4-cores + hyperthreading. In SMT(hyperthreading) the execution engines, ALUs, FPUs, and caches are completely shared, whereas on FX design, they built two completely separate integer pipelines (schedulers, register files, ALUs, and L1 data caches) inside one module. Only the instruction fetch/decode front-end, the FPU, and the L2 cache were shared. So the FX design would be an in-between a 4-core + SMT and a true 8-core.

Fair, I had not delved into the details, but still they were not full cores and the marketing did not make a real distinction.

I had a pilledriver one, it was a perfectly good cpu, I would buy it again. If I remember back then it was the best overall performance per dollar, the alternatives if I remember correctly were i7-39.. and i7-38.. and were at best 50% more expensive for 10-15% more performance.

Depends what you were doing with it. The Piledriver only beat the Intels in heavily multi threaded (preferably integer) workloads like media encoding, which is why it was popular with media creator workstations on a budget, but for most consumer real world tasks at the time, like video games, Intel was way ahead in performance even though it was more expensive.

The Piledriver would win the consumer bang/buck mindset back then because of the 6-core part was reasonably priced and unlocked for overclocking, so people would overclock them to beat the more expensive (locked?) 4-c/8-t Intels at a lower price, but that ignored the costs of massive extra power draw(100+ W) over the Intel, the need for beefier more expensive coolers and power supplies, more expensive AMD motherboards with beefier MOSFET power delivery stages built to withstand the higher power draws of the Piledriver, so in the end the actual bang/buck gain of the AMD system wasn't remotely as big as people were making it out to be, they were just happy to get a "6-core" AMD cheaper than a 4-core Intel thinking more cores = more "better", same how having more mega-herz was also more "better" a decade before that.

The Team AMD VS Team Intel wars on forums on these topics were wild back then.

Instead of discrete, fully independent cores, AMD grouped processing units into "Modules" where each module contained two integer execution units, but they had to share critical resources like one FPU, the instruction fetch/decode pipeline, and the L2 cache so when both "cores" inside a module were heavily taxed especially with math or physics-heavy calculations (like in videogames), they choked fighting over shared hardware.

They did just fine in parallel workloads, so I think this is not accurate. The design scaled just fine. The problem was that each core was weak.

They did just fine in parallel workloads, so I think this is not accurate

Depends how you define "doing just fine in parallel workloads". The contemporary competition from Intel that was 4-core + SMT was beating AMD's 8-core FX CPUs in most real-world tasks and benchmarks at the time. The 8-core AMD broke even and rarely won only in >4-thread strictly integer benchmarks and some >4-thread media encoding tasks/benchmarks. So if you wanted a prosumer media encoding workstation a budget then yeah, the AMD was better, but for most real world task, it really wasn't.

The design scaled just fine. The problem was that each core was weak.

Can you elaborate and be more exact? What you wrote is technically vague and doesn't mean anything in technical dissection/terms.

When we say a design "scales", that means that increasing the size of the workload does not incur a lot of overhead. If contention between shared resources meant that the design was not able to achieve an ~8x speedup when run with eight parallel threads, that would mean the design was not scalable. But we did in fact see a roughly 8x speedup with eight threads, so the design scaled just fine. The problem with the design was that each core was individually crummy, so even eight cores running in parallel had lackluster performance.

The myth that each two-core module functioned more like one core with hyperthreading would suggest that these CPUs would have much higher per-core performance when lightly loaded than when fully loaded. That is not what happened. Each core was crummy even when lightly loaded, but under full load you would have eight crummy cores, which would beat four Intel cores on a lot of workloads.

The only time contention was a serious problem was with workloads that were dominated by floating point, which were relatively rare.

They never shipped the zen 5 epyc big cache chip (e.g. 9685X). :( I'd hoped they pulled it to produce a HBM integrated chip instead, but they didn't do that either.

The AMD Ryzen 9 9950X3D / EPYC 4585PX is a beast for 99% of compute workloads.

It has the highest base clock of any modern AMD chip (4.3 GHz), plus 16-cores, which is already more than most workloads need.

Can’t wait to see what Zen 6 brings early next year. Rumors say 24-cores and another big jump in single-core performance.

I looked at game benchmarks, and sometimes 5600 or 5600X were fastest. With all X3D processors so fast that I would not decide to upgrade above 5700X3D or anything cheaper. Now, with current RAM pricing, getting a X3D CPU with 90+ MB L3 Cache seams the way to go instead of doing a DDR4 -> DDR5 leap.

All stated for gaming, that is.

I still love my 5800X3D. Drop-in upgrade from a 3700X so I kept my board, RAM, cooler, everything.

I feel like most benchmarks don’t really capture CPU benefits to gaming.

It’s even difficult to find CPU benchmarks that don’t overemphasize 1080p and eSports scenarios.

My upgrade from 5600x3D to 9850x3D felt kind of dumb at the time, but I decided to do it because Micro Center’s bundle deals are so far below market pricing.

I was shocked at how much better it is. Benchmarks and FPS don’t really show things like micro stutters and little performance wrinkles like that. I’m not even sure 1% low FPS counts capture it.

A great example game for this is Oblivion Remastered. Upgrading my CPU alone with the same GPU took away the environment loading slowdown almost entirely.

The benchmark will tell you that I didn’t gain any FPS during gameplay but every time I open a door into the new environment my CPU is positively impacting the experience.

I would have said the exact same thing you are saying until I actually experienced upgrading to the best on the market. For the record, this is the first time in my life I’ve actually owned the best CPU on the market for gaming.

My old advice would have been to buy one of those sweet spot cheaper mid-range gaming CPUs, but my newer advice is really if you’ve already spent all that you’re willing to spend on a GPU (I have a 9070XT, my only upgrade paths are insanely expensive), buy the highest gaming CPU on the list for gaming benchmarks (e.g., I wouldn’t go crazy with a 9950X3D2 since it doesn’t have any gaming improvements above the 9850X3D).

And the thing about CPUs is they’re not insanely expensive like GPUs. We are talking a price delta of $200 between this beast of a CPU and something way more middling.

Oblivion remastered is an absolute dumpster fire for performance in general.

Given the massive increase in hardware prices lately, we are going to have to get by with mid range or even low end hardware for a lot longer. Studios will be forced to actually optimise games to not run like shit on a sub $8000 PC.

Every game should be targeting the switch 2 and steam deck in terms of power.

Every game should be targeting the switch 2 and steam deck in terms of power.

I think PS5 is a perfectly reasonable spec to be targeting, which was the higher end of mid range about 5 years ago, and still significantly out performs Switch 2 (which significantly out performs steam deck). I’m sure the GTA sales will demonstrate that the install base at this spec is wide enough for broad commercial success.

I don’t really want game experiences to be locked at a PS5 forever. That console is actually somewhat old now. There are phones that have more graphical power than a PS5. A MacBook Air has a more powerful GPU than a PS5.

I grew up with gaming systems that got rapidly better and while it can be argued that we are at a diminishing returns plateau of gaming horsepower, I don’t think the solution is to never get hardware upgrades ever again, even in the current RAM-constrained environment.

GTA VI is coming out with FPS maximum of 30FPS on the PS5 including PS5 Pro. That’s a subpar experience that is limited by hardware.

I didn’t mean to imply that we should target ps5 as an exclusive platform, but ps5 is a good hardware spec to target, as there are several platforms that can match it and it has a wide install base.

Yeah, and I see what you mean there. I think even when the PS6 comes out, game developers largely won’t be taking advantage of its capabilities just like how today they don’t really leverage the capabilities of an RTX 5090.

We saw with much of the PS5’s lifespan that many of its cross-platform games came out on PS4 and original Switch with minimal compromise.

Not enough of the install base owns those higher end solutions, so for the most part someone with a better performing rig is getting better FPS, higher resolution, and not much else. Even gaining additional draw distance is rare these days, and higher resolution textures are basically impossible to see.

If the console is going to survive rather than decline as a whole I think that Microsoft and Sony will need to get more serious about innovation rather than just making a samey spec bump at a high price that doesn’t motivate their users to upgrade. Nintendo already proved that basic and rather mild level of innovation can excite the market (original Switch).

It has the highest base clock of any modern AMD chip (4.3 GHz), plus 16-cores, which is already more than most workloads need.

Does the base clock even mean anything beyond something that may or may not vaguely indicate the performance? Its lower than what these chips can sustain with sufficient cooling from my experience. It's higher than what the chips will clock down to when not sufficiently cooled.

Is this what's helping drive their stock price to the moon?

They're starting to catch up with NVIDIA. Because AI.

Serious question with 1GB L3 isn't it theoretically possible to boot a full Linux without ram?

Edit: dug in it, no, because cache isn't addressable and it's directly managed by the cpu.

Intel has a cache-as-ram (CAR) mode / Non-Eviction Mode, which is what you wanted but it is only available before DRAM is brought up via core initialization. Not sure about AMD.

I'm not sure focusing on just 3 processors is enough to claim that Ryzen got 50% faster. There were already other, faster Ryzen processors available without the additional cache. The extra cache was a very new thing (and very temperature sensitive), so it makes sense that they start carefully and then get a lot of improvement quickly). And the 5800x3d was for an older chipset, with all the limitations that come with that.

It's not that temperature sensitive, 90C is plenty. If you can't keep the non-x3d processors underneath that you're just throwing away performance anyways

Are you? I'm pretty sure when I bought my 7800x3d, everybody was saying you had to be very careful about heat, because these damage more easily from high temperatures than regular CPUs. I just checked, and they do throttle at a lower temperature than others: 89 vs 95 degrees C.

The x86 architecture still going strong. I've always though Ryzen was a fitting name, AMD had Ryzen from the ashes and struck a blow to the long dominant Intel.

When I built my Ryzen 1800X desktop, I named it "AMD Ryzing". My optimism turned out well placed.

I recently upgraded from that to a Core Ultra 7 that I got from work (e-waste recycling). It was not my intended upgrade path, and I hope Lisa isn't too mad lol (I kept my Radeon though).

a Core Ultra 7 that I got from work (e-waste recycling)

How the hell does a high end CPU that's barely 2 years old end up in e-waste recycling? Was Richie Rich using them or something?

Was Richie Rich using them or something?

Sort of: there were about half a dozen in micro PCs in a load picked up from a hospital. I was kinda bummed that they were BIOS password locked (as were the other PCs from there). The model didn't have a password reset jumper (or any reset mechanism outside of contact Dell support with proof of purchase), so I couldn't sell them as whole systems, and stripped them for parts.

We get DDR5-based systems from time to time, but they're understandably rare. Most of the time, they still work. My rule: any day I come across DDR5 is a good day.

But why was the hospital throwing away what was essentially new high end systems that can run all the latest SW and operating systems? For what reason? It just makes no sense to me. Or am I too poor of a peasant to understand?

Are you by any chance in some super-rich part of the US? Because then no wonder your healthcare is so expensive when hospitals treat high end PCs as single use disposables. Here in Europe I sometimes see hospitals and doctors practices using 10+ year old PCs and still rocking the old 17-19 inch 1280x1024 CCFL LCD monitors from like the mid-2000s.

Regardless, I'd love to just run into high end PCs being thrown away here and pick them up for free, but where I live I see people barely throwing away their value-line Dell/HP Core 2 Duo towers with audacity to ask for 25+ Euros for their e-waste with the classic "no lowballs, I know what I got" attitude.

For what reason? It just makes no sense to me. Or am I too poor of a peasant to understand?

I don't understand it either. Two scenarios make sense to me:

Somewhat probable: everyone in a department got upgraded, no exceptions. 'This is a really nice PC, but the boss says it has to go.' (There were about 100 i5 9th gen mini PCs (still OK), and 500 t640 thin clients in that same load.)

Less probable: some department got closed and cleaned out.

Are you in some rich part of the US?

Pittsburgh, Pennsylvania. Not particularly rich (rust belt), but healthcare is a significant part of the local economy. About 90% of healthcare here seems to run on HP, but Dells can appear.

Here in Europe I sometimes see hospitals and doctors practices using 10+ year old PCs and still rocking the old 17-19 inch 1280x1024 CCFL LCD monitors from like the mid 2000s.

My stomach turns seeing the dozens of 24" probably 1080p monitors we scrap every day. Pretty much all are significantly scratched when they come off the truck, otherwise reselling them might be good business.

Somewhat probable: everyone in a department got upgraded, no exceptions. 'This is a really nice PC, but the boss says it has to go.'

Here in Europe all that is 100% resold either locally, or further to balkans/eastern-europe due to much lower purchasing power making the resale efforts of ~5 year old HW economically viable. I check the local "craigslist" often and I notice when a business is clearing house due to a flood of used HPs/Dells/Lenovos from a single user. However the list prices aren't remotely palpable to be good deals. I assume the person/business hired for clearing the house is marking them up significantly hoping to squeeze a big win from a mark.

Some of the tech companies I worked for here would first auction their older HW internally at every upgrade cycle to the employees before selling what was left to a clearing company, which makes me feel we're being robbed to be offered by employers to bid for their e-waste when in the US people find better stuff in the trash for free. Europoor indeed.

otherwise reselling them might be good business

I assume those businesses don't bother reselling 1080p monitors and 2 year old PCs, and instead just throw them away, because the price of labor is so high in those parts of the US, that paying a full-time employee to take care of such resale tasks will cost them more money than they expect to make from the sale, so it's assumed to be cheaper and less hassle to just throw everything in the trash instead of wasting time on the used market dealing with tire-kickers just to gain what is essentially peanuts money for those businesses' bottom line.

Am I close with my assessment?

Oh come on, stop felling sorry for yourself. I'm from one of those countries compared to which even Eastern Europe you offload hardware to is a paradise.

None of what you're describing is available here at all. I sometimes see Europeans willing to ship here (already very few) offer heavily used and scratched ThinkPads and Dells for money they weren't worth when they were new. Add at least $80 on top for shipping, and no warranty, you get what you get.

On the local market, you will find those Core 2 Duo with precisely the attitude you're describing, but at 2-4 times the cost.

So I buy everything new, which is expensive with our salaries, but we have no choice. At least new hardware is available now thanks to selling over the internet and improved logistics.

10 years ago you could only buy what was offered by local shops, which was always at least two generations behind, and at least twice as expensive as it was in the EU (forget the US).

For example, if things continued like this to this day, I'd estimate the newest CPU you could buy right now would be something like Ryzen 3600, at twice the cost it was in your country when it just came out. If we're lucky, maybe 7600 would already appear at a similar overprice.

I'm sure many parts of the planet are even in a worse position than we are.

Here in Europe all that is 100% resold either locally, or further to balkans/eastern-europe

My company's policy is that retired laptops are destroyed. (I suppose properly wiping them or removing storage devices should be enough but for some reason it's not considered sufficient.)

There are indeed lots of second-hand former company laptops being sold, and some companies even specialize in refurbishing and selling them. I don't know what percentage ends up being recycled but it's definitely not 100 %. I'd assume the more security-oriented the organization gets, the more likely it might be to avoid recycling devices intact.

With that said, I haven't seen two-year-old high-end devices getting retired. But if there are only a few of them, and a large fleet is being renewed, I suppose it can make sense from a fleet management point of view to renew the few perfectly current ones as well.

Single core performance still worse than an iPhone.

The x86 folks lost the ball completely. For workstation/ mobile needs you go to arm. For matmul at scale you go to gpus.

I guess windows gaming with separate gpu is the only remaining market for them.

Unless Nvidia releases a motherboard with an arm cpu on top of their RTX cards, and then that market is gone too.

The IPhone runs on a battery, with no active cooling.

There is no chance the sustained single core perf is better than a 150=200+w ryzen with a big noctua cooler on it. The power draw alone would kill the battery in a few minutes(?), and the heat would make it catch fire. lol

If you compare like for like, ie. take the process node out of the picture, then you need to compare the Ryzen 9000 series to the A16, as both use TSMC's 4NP process.

It's impressive and yet it surely depends what you're doing with it? 50% better performance isn't going to make a local LLM feel quick and yet so many other aspects of computing are quite fast anyhow. I can browse the web fairly comfortably on a Raspberry Pi and it's a bit slow but manageable.

If your goal is web browsing you will want faster single core performance. Unfortunately that means Apple M series. If your goal is local LLM, I’m afraid a several-year-old GPU will smoke the fastest CPU available today.

IIRC, compared to GPUs, M-series is currently still stuck in memory bandwidths from around 2016. M7 might catch up to 2019 or so. So GPUs will be better for LLMs for a pretty decent while.

How so? I don't know of any other mainstream platform with >1TB/s memory bandwidth. Personally I don't want to deal with macOS but between the memory bandwidth and out of box Thunderbolt networking it's hard to argue that Apple doesn't have a couple significant advantages over the current alternatives.

I don't know of any other mainstream platform with >1TB/s memory bandwidth.

I mean, RTX 5080 has nearly 1TB/s, 5090 has nearly 2TB/s. Maybe you are talking about CPUs / unified memory platforms? I agree nobody else does it better. But for LLMs, GPUs can still be significantly faster than even the most advanced Apple silicon on the planet. TTFT in particular is super inferior with Apple, for now.

That's probably also the reason Apple had to reluctantly give into Nvidia servers for the initial rollout of Siri AI, though they claim to use trusted computing extensions to reach an acceptable level of privacy. (I do not trust that nearly as much as the Apple Silicon nodes)

It'll be amazing five years or whatever down the line to see Apple reaching those figures. They seem to be heading in that direction lately.

I Picked a threadripper for a box that needed a lot of IO and figured that with that many PCI lanes I couldn't go wrong. But I have to admit I've been more than pleasantly surprised by the performance of the CPU as well, it - easily - outperforms all of the XEON and I7 based boxes that I have. The only thing I wished I would have done different is to max it out with 256G DIMMs when they weren't the price of a car.

Zen 5 really is nice. I swapped my RAM from a Xeon W Sapphire Rapids machine into a Threadripper Pro 9000 series machine and I get almost double the memory read performance, plus it's a heck of a lot faster in single and multi core performance, and it runs cooler and quieter. Huge win all around. Aside from the price... (I went from Xeon W5-3435X to TR Pro 9985WX, eep.)

I have the complete opposite experience. Got a 9550x with 64gb ddr5 and a fairly high end mobo about two year ago. Just running the memory at stock speed. About 50% of the time I’d reboot and one or both of the sticks would only be detected as 2gb. Would need to do a hard shutdown to get it back.

I eventually gave up and turned off memory context restore and now I just deal with the minute plus (!) time to Post.

Not sure if amd memory controllers are just garbo or what but I’m going back to intel next chance I get.

What brand of ram, and was the ram listed on the supported spec sheet for the motherboard?

Up to date on bios updates?

Yes and yes, it’s crucial ram. Nothing exotic.

Weird, just had to check.

DDR5 has been kind of a mess imo, just in general.

On the desktop Intel has better memory controllers last time I looked, but it’s not that big a difference. Something in your setup is just broken. Failures will happen with any brand, so I wouldn’t chalk it up to AMD vs Intel… figure out where your problem is and get the faulty part replaced if you can.

I feel you though, regardless of the cause, that is an extremely frustrating place to be.

Yea I’m just bummed because it’s a real expensive time to be swapping memory parts around. Like I said, things seem to just work without memory context restore so I’m fine with just paying the cost when I reboot once a week or whatever. I feel like the fact that disabling MCR fixes it should narrow down the issue to some part, I’m just not sure which.

it’s a real expensive time to be swapping memory parts around.

It is, but isn’t it all warranty in your scenario?

It would be painful to claim though, as what part is at fault?

I went the opposite way and went back to Intel after many many years on AMD. I snagged a Xeon 654 Granite Rapids ($850) for a new workstation geared towards local inference. I don’t need a ton of CPU cores and all Granite Rapids have 8 memory channels where as you need very high end Threadrippers (9975wx for $4000) for an equivalent due to their CCD design.

As a bonus Granite Rapids is super power efficient and runs much cooler than my previous TR and Ryzens. Intel seems to be doing good things again! My only minor complaint is that P2P doesn’t work on my multiple GPU setup because every PCIe5x16 lane has its own dedicated root to the CPU, but all that bandwidth is useful for MoE models that are offloaded to RAM.

You could get a PCIe switch backplane, stick your GPUs on there and then have them communicate locally through the switch using p2p. For some GPUs you may have to use a hacked driver (google p2p 3090 for instance).

I wanted to do that, but didn't like any of the PCIe switches / backplanes I found (mostly due to price). Now, it might be worth buying one of these instead of trying to build a fancy PC. https://c-payne.com/products/pcie-gen4-switch-5x-x16-microch...

Threadripper is underrated for IO-heavy boxes. I made the same mistake with RAM prices, though; should have maxed it out when it was cheap

Ye, that's like saying one should've bought Amazon, Apple and Google stock when it was low.

That is only true if you only look at the market value of an asset.

One is a tool, the other is a speculative financial asset.

That's why I'm all in on hammers. Garage full of them.

In the computer world, wait a few years and things get cheaper. That was broken by an exceptional event. Who would have thought you can't afford DDR4 in 2026?

The upcoming shortage was public knowledge before the prices rose.

We seem to be going back to the days when desktop PCs had a little lock hole.

They (in theory) could padlock them so the drives and ram couldn't be stolen?

https://francisuniverse.wordpress.com/2017/10/07/the-turbo-b...

Weird they didn't talk about the DDR4->DDR5 transition. The 5800X3d->7800X3D went from DDR4->DDR5.

DDR5 is literally twice the price per GB. It's better for sure, higher bandwidth but waaaay more expensive. The 9800X3D supports much faster and even more expensive DDR5 than the 7800X3D too (DDR5-2667 to DDR5-5600).

There was a brief and glorious time before September 2025 where DDR5 was actually pretty comparable.

Even really good DDR5 could be had for cheap; I remember paying like 300$ for 96GB of high-quality M-die hynix DDR5.

Ram is getting better, faster etc. except for the latency. I guess we still cannot bend the rules of physics.

Latency has roughly remained the same since DDR2, no? So that has just about nothing to do with the improved performance of every generation.

Zen3 is 7nm, Zen4 is 5nm and Zen5 is n4p. That's also why Apple is ahead; TSMC is a big factor.

Perhaps better looking at it without the 3D Cache on GB6 scores.

2020 Zen 3 AMD Ryzen 7 5800 - 1852

2024 Zen 5 AMD Ryzen 9600X. - 2914

2027? Zen 6 - Est 3200?

And compare this to Apple.

M6 4000+

M5 3600+

M4 3300+

Even the Snapdragon X2 Elite X2E is better at 3300.

The gap isn't exactly shrinking. And I would love x86 to prove me wrong. But even a hypothetical Zen 7 in 2028 ( 2029 for consumer ) may only catch up to Apple's M5 released in 2025.

So far, AARCH64's advantages mostly seem to affect single core performance. AMD is also behind in the nanometer race, which is one of the ways Apple managed to improve their peformance over the years as well. If they manage to catch up in terms of die shrink, I think their multi-core perf will once again put AMD in the lead for data processing at the very least.

Although, if you count EPYC, which you might as well with the prices Apple charges for their top-end models, AMD still beats Apple with ease at multi-processing.

It's a real shame that these processors are made by Apple (and Qualcomm, I suppose). They'd be my go-to chip if they weren't.

With CPUs becoming wider and wider, it's almost like they run VLIW instructions, only more flexible.

Huh no, wide vector units or lots of execution units per core is nothing like wide instruction words? Superscalar out-of-order execution is specifically a way to get instruction level parallelism without the wide instruction words

I was thinking that for this new generation, "Superscalar" doesn't cover it anymore, except as a generic term for the design technique. "Ultrascalar"? "Superscalar Pro"?

Sure, but in my mind it feels like the CPU is slurping up individual instructions until it has enough to execute a whole bunch of them at once in one step (which would be equivalent to a big VLIW).

I was recently able to get a workload of mine optimized enough on Zen 5 to hit a sustained 6.0 IPC/core (3.0 / thread) at 5.1 GHz. Seeing the a > 99.8% branch prediction rate and a > 99.99% L2 cache hit rate retiring > 1T instructions every 3 seconds feels amazing.

Zen5 is incredible when you're able to make the most of it. I’m super excited about Zen6.

zen5 is really the first CPU of the avx512 era (since it's the first time normal compiler devs have them to play with)

We had Ice Lake and Rocket Lake too, they seem to have disappeared from the collective memory

Can you share any more details about the workload? Always interesting to hear of something like that which isn't a useless microbenchmark.

Does it use vector? What can you hit with SMT disabled?

It’s a backtracking search program looking for integer solutions to a specific problem. I’ve tuned a series of bloom filters to fill my L2 cache so that I rarely have to touch main memory (this alone took my IPC from 0.1-0.3 to 3.0 per thread). Without SMT it’s 4.6 IPC/core.

I think it’s only able to exceed 4.0/thread with SMT off because of a uops cache? From what I’ve read the Zen5 front end only had a 4-wide instruction decode per thread.

coded entirely or partially in assembly, i presume?

I work on game servers. I can only dream of such an IPC, but it doesn’t stop me trying, heh.

For someone who has never touched workload optimization, can you share a little about how you measure this? Does IPC mean inter-process communication in this context?

I know nothing about workload optimization, and I'd like to know more - right now I feel like the Good Burger gif. "Yeah, I know some of these words".

I've never done workload optimisation myself, but have looked into it just-in-case. It's a really interesting field and unfortunately, you need to know about both your compiler and target CPU/GPU to get the big gains. I believe most modern CPUs have registers for cache use. You have to do some guesswork to get IPC numbers, since core frequency can vary.

Increasing instructions-per-clock is all about minimising program branching (essentially 'if' statements). Because a CPU core can execute instructions faster than main memory can fetch em. It's a fun game to look at an 'if' statement and figure out how you could instead make it an arithmetic operation :).

Maximising cache hits is all about how you structure and access data. For example, if a cache entry is n bytes long, you want to ensure your struct is smaller than n bytes. Having very consistent access patterns can also help (e.g. arrays-of-structs vs structs-of-arrays).

This field is super deep, it's very fun to learn about!

I'm not too familiar with the CPU world of performance measurement, but in GPU land, and I suspect for CPUs too, there are a slew of hardware "performance counters" which are registers that increment every time the event they measure happens.

So there could be a "instructions completed" counter and a "cycle" counter, and before starting the benchmark, you record the current value of both counters, then after the benchmark, record the final values, and compute the Instructions Per Cycle (IPC) as the difference in instructions completed over the difference in cycle count.

As for optimization, at a high level it's about minimizing the amount of time any part of the CPU is waiting for other parts of the CPU. The specifics require a lot of background knowledge about how modern CPUs work, more than can fit in a post, but if you're interested, topics to read about include:

### CPU cache hierarchy

CPUs store copies of data from RAM in smaller, faster memory physically closer to where the computation happens, so it's available more quickly. The CPU decides what values to store in the cache, and gives only limited control to the program, so an optimal program needs to be careful to not make the CPU make bad caching decisions (including for synchronizing the cache between multiple threads of execution).

### Instruction pipelining and out-of-order execution (a.k.a. "superscalar" execution)

Modern CPUs operate like an assembly line. A new instruction can start executing before the previous instruction(s) finishes. CPUs also have redundant hardware, so multiple instructions can be in progress at the same step of the pipeline. But there are limitations; sometimes the input to one instruction depends on the output of the previous instruction, so the whole pipeline stalls until the result is ready. An optimal program orders its operations to avoid these stalls as much as possible.

### Branch predicition

When the code to execute depends on the result of a computation, like in an `if` statement, we call it a branch. Branches can stall the pipeline, because the CPU doesn't know what instructions to execute next until the current result is ready. However, to mitigate this, modern CPUs predict which code-path will be taken when a branch is reached and begin executing the associated instructions immediately. If the prediction is right, the pipeline stall is avoided, but if it's wrong the pipeline state has to be restored to what it was before the wrong branch started executing, which is even more expensive than a stall. Usually, the CPU predicts branches correctly, so branch prediction is a net gain. Optimal programs need to understand how the CPU makes these predictions and make their branches as predictable as possible.

### Single Instruction, Multiple Data (SIMD)

Some programs do the same operations to each item of a set of data. CPUs have so-called SIMD instructions to accelerate this by, unsurprisingly, performing the same operation to multiple items at once. For example, they could add 4, 8, or even up to 64 pairs of numbers at once, depending on the instruction set and the range of the inputs. Suitable programs are optimized by arranging their inputs and operations so that SIMD instructions can be used — either directly or by being written in a way that a compiler can translate individual operations to SIMD operations.

Thanks. Is there any convenient tool you'd recommend for watching those registers?

I've said in another comment that any optimisation I'd be doing would be looking at the entire stack, and CPU/GPU optimisation would only be as a learning exercise. I'm a tiny bit familiar with caching and instruction pipelines thanks to college, and branch prediction thanks to Spectre-class bugs. I want to find some time to dive deeper, now.

Because nobody cares about CPUs anymore and they thought now was the time to flex those numbers that were always possible? Because all eyes are on NVIDIA GPUs?

Mainly iterative IPC improvements combined with maturing process nodes. That initial TSMC 7nm was solid but had room to grow.

No mention of Moore's Law...

The number of transistors is way up, by about 50%, from roughly 11 billion to 16 billion. Most of the extra transistors went into the core, not the cache.

From https://en.wikipedia.org/wiki/Moore%27s_law

Moore's law is the observation that the number of transistors in an integrated circuit (IC) doubles about every two years, with minimal increase in cost.

Moore's law is mostly dead these days. Process scaling still happens, but nowhere near at the rate predicted, and there are tradeoffs everywhere now. The Zen 5 core is in fact bigger in terms of die size too.