Embarrassing, but probably little practical impact, since these hardware random numbers are typically not used directly and instead seed a CSPRNG.

It is just possible they decided crypto code that uses it was safer to skip zeros. (Whist mathematically it should be no more likely; it is vastly more likely someone will actually try that key).

It is also possible that their code was generating too many zeros and the easiest fix was to discard them all.

Can you clarify what you mean by "it is vastly more likely someone will actually try that key"?

I'm guessing you don't think there are people calling rdrand in a loop and throwing away the output with high probability except when it is 0, but I can't see how else you imagine people would be vastly more likely to use the output when it is 0?

In lots of scenarios I know the software used to generate the key; the only unknown is the random numbers used. If I am searching for weaknesses it is highly likely I would try keys with different seeds; zero, one, are going to me much more likely choices here then hoping I can guess the right values.

The probability of generating a zero is incredibly low if you use the normal distribution curve.

So it is not necessarily that it doesn't generate zero, they did not run enough times to increase the probability of actually generating a zero.

Why would it be a normal distribution?

From what I can see they were trying to generate 16bit integers, so the probability is 1 in 65536 and they were running the test for 11 hours.

You definitely would expect a roughly equal number of 0s as any other of those numbers since it's uniformly distributed. And definitely not 0

You definitely would expect a roughly equal number of 0s as any other of those numbers since it's uniformly distributed.

How would random numbers be uniformly distributed?

I'm getting 16-bit zeros on my Zen 3 chip (+1:3821, 0:3893, -1:3895), I will wait to get some statistically significant samples for the 32-bit values and update the forum thread. Maybe it was fixed after Zen 2?

This is not the first RNG bug on Zen 2, I recall after I first got mine that some application or other would quit immediately at startup because rdrand always returned -1, i.e. all 1s. It was fixed with a microcode update.

Do we now learn that they fixed "always generate all 1s" with "never generate all 0s"??

EDIT: I've been unable to reproduce the problem on my CPU, FWIW. It's a Ryzen 5 3600.

EDIT2: OK, update, I can reproduce it with rdrand16, rdrand32 is fine but rdrand16 can never generate all 0s. So my CPU does have this problem!

    return 4 # Determined by fair dice roll.

https://xkcd.com/221/ for those not in the know

And for the full fail story behind it, fail0verflow hacking the PS3 presentation is great and covers the bug: https://youtu.be/DUGGJpn2_zY

Most of the console hacking talks are great, both informative and entertaining.

Does rdrand32 and then taking the lowest 16 bits of its result yield any zeroes?

Basically I'm wondering if it's a bug in the version of the instruction that writes to a 16-bit reg, or a bug in the underlying RNG

Yes it does. rdrand32()%65535 was my first attempt, and generated zeroes at about the expected rate, that's why I initially erroneously thought my CPU did not have this problem.

You should be using &0xFFFF for masking. Your mod is off by 1 too.

I can reproduce it too with rdrand16 on Zen2.

But it looks like the rdrand16 instruction can produce zeros just fine, it just sets CF=0 erroneously (indicating an error and that the user program should retry).

So keep that in mind when you try to reproduce it too and use some abstraction that could implement retries internally.

Good observation, that seems like the most likely explanation. Do you ever see "true" CF=0 (with nonzero arg) or did they just take the lazy approach?

If I remember correctly, we had a setting in every Linux server we owned to remove CPU as a RNG seeder for the kernel because of those bugs with AMD CPUs.

I.e., we had `random.trust_cpu=off nordrand` in `GRUB_CMDLINE_LINUX`.

Adding bad randomness can't degrade good randomness, can it?

I thought the kernel would not replace anything just because it adds a potentially bad source.

E.g. if you have rand source A, and xor it with rand source B, then you get, at worst, the best of A and B,

Usually you do "rdrand % <some-number>" anyways, and in that case you will still get zeroes. True, your result might be skewed by 1/(maxint/some-number) but I guess that's not a big problem in practice

I always wonder how hardware bugs like this happen with the sheer amount of hardware validation that's done. It'd be fascinating to know how it slipped through the cracks, though I know almost nothing about this side of the industry sadly

So what? The point is to be non predictable not to pick all the numbers in the range with exactly the same probability. Would it be a problem if it never generated 16542?

Consider an 8-bit RNG.

By your argument, it would not be a problem if the RNG never generated 0. So, it must follow that it would also not be a problem if it never generated {1, 2, 3, ..., 253}.

That means that our RNG now only generates the values 254 and 255. Which of the values is generated is unpredictable on any given call. However, 7 of the 8 output bits are now always fixed and so completely predictable. Can you imagine how an attacker could exploit that?

Failing to generate only the number 0 is a weaker version of the same class of flaw.

This is the “what’s the big deal if I lost $100k in a casino, it’s really the same thing as if I had lost $5” argument.

I don’t think you can rebut “you only lose one of many values” with “it’s the same as only having one left”.

We're talking about whether a modification of the expected probabilities changes the dynamics of the game. The example I gave was deliberately extreme, because that makes it easier to reason about.

If you want a casino example, then consider a roulette wheel that always lands on 36 but still pays out as usual. I think you'd want to play on it. Now consider one that always lands somewhere between 30 and 36. Still worth it, right? With careful bets and a good starting float you're still coming away from the table up (with a very high probability).

In fact for a roulette wheel you only need two dead pockets for the player to get an edge. Bias is exploitable.

The value space goes from 2^16, 2^32, 2^64 to 2^16 - 1, 2^32 - 1, and 2^64 - 1 respectively.

The bug has zero practical impact.

Betty from accounting will have words.

What does Betty from accounting care about RNGs?

"Random" is used by most people to mean "random with an even distribution".

A weighted die is still random, but with an uneven distribution. This is effectively a 2^16-sided, weighted die.

https://www.amd.com/en/resources/product-security/bulletin/a...

The OP says they discovered this on a Zen 2, which is not covered by that bulletin (?)

[edit to add]: Also, the bulletin is solely about RDSEED zeros, whereas the OP is also reporting RDRAND zeroes.

Older AMD processors had issues as well:

https://github.com/systemd/systemd/pull/12536/commits/1c53d4...

Why does systemd use architecture-specific instructions rather than using the kernel-provided random interface in the first place?

0 is not a number, it's undefined

the what?

This is why I use, in security critical contents of my software (where the numbers have to be computationally infeasible to produce), a type of random number generator called an XOF (extendable-output function).

It takes entropy from multiple different sources, makes it all input to the XOF, then the XOF uses cryptography to output a stream that has as much entropy as the combined entropy of all of its sources of randomness. So if an XOF, for example, takes 100 runs of rdrand16, along with the system time in microseconds and the number of milliseconds between receiving 100 packets over the network, the XOF will output a completely random stream without artifacts like never returning 0x0000, even if rdrand16 never outputs 0x0000.

Isn’t this effectively what systems like /dev/(u)rand do? Pool multiple random sources together to hedge against these things?

I fail to see why one should either rely on a single random source nor roll their own.

Yes, /dev/(u)random is supposed to do that, but what if there’s a bug in a kernel (e.g. some embedded system which may not even be running Linux) which causes /dev/(u)ramdom to be less than secure? There’s also issues where, for example, it may no longer be possible to read /dev/(u)random after putting the process in a chroot() sandbox (chroot() isn’t defined in POSIX so its behavior is not guaranteed to be consistent across multiple operating systems).

getrandom() is often times suggested, but alas isn’t a standardized function, i.e. it’s not part of the POSIX specification. Considering how the C23 changes to the C specification caused a lot of perfectly good C code to no longer compile, I’m very anal about sticking to specs; I use '-std=C99' for my code these days (even though it can compile as C23 code) and stick to POSIX functions (except chroot() and setgroups(), but both of those predate POSIX, and even here I have a compile-time option to compile my code without those non-POSIX syscalls).

The code using a secure XOF (the algorithm was developed by the same team which later on made SHA-3, and includes people who helped make AES) has been around for nearly two decades (the code where I roll my own RNG to make secure random numbers has been around for over 25 years, but used AES before XOFs existed) and not one security problem has found with the RNG code has ever been found. [1] “Don’t roll your own RNG” is a suggestion, but it is possible to do so securely if one knows what they are doing (i.e. they have read Applied Cryptography and keep current with cryptographic developments).

For anything vibe coded (my code is 100% human written, for the record), rolling one’s own RNG is a really bad idea.

[1] There was a theoretical issue with cache timing attacks over two decades ago, so I put mitigations in place, and then chose to use an XOF for newer code.

[2] There was an issue where a separate implementation I made of this XOF would generate incorrect test vectors in clang, but only at some optimization levels. I now test the XOF in both GCC and clang at multiple optimization levels to make sure it acts correctly.

but what if there’s a bug in the kernel which causes /dev/(u)ramdom to be less than secure?

so instead you suggest trusting your own untested unlooked at implementation more?

Black-and-white thinking like this is always inaccurate.

untested

The automated tests includes tests that make sure the XOF is correctly implemented. [1]

unlooked at

People have been looking at my code for security holes for well over 20 years, and I have been getting multiple AI assisted security reports over the last year, things like “there’s a buffer overflow in this code which is nay to impossible to exploit, using code which hasn’t even been able to compile since 2022”.

[1] https://github.com/samboy/MaraDNS/tree/master/deadwood-githu... and https://github.com/samboy/MaraDNS/tree/master/deadwood-githu...

XOF correctly implemented doesn’t ensure you haven’t made other mistakes, such as using entropy sources correctly, doing needed math correctly to avoid any entropy bias, etc. etc….

You’re correct about black and white thinking. Then you invoke multiple straw men in this thread to defend that you’ll roll your own.

Disclaimer: I’ve been hired for multiple DoD projects to break hardware and software security systems, and I nearly always succeed, because so many people (and companies) roll their own.

i remember some linux kernel dev got ousted by the community because he/she wanted to not implement a backdoor that would compromise the results of /dev/urandom.

Who?

getrandom() is often times suggested, but alas isn’t a standardized function

The POSIX standard function is getentropy(), which internally calls getrandom() on Linux.

what if there’s a bug in the kernel which causes /dev/(u)ramdom to be less than secure?

It's often the other way around: the Linux kernel contains thousands of workarounds for buggy hardware, while the buggy hardware itself doesn't always get patched. Linux developers take this stuff very seriously. As a result it's often safer to rely on kernel APIs than to access the hardware directly.

The kernel code involving random number generation receives an exceptionally high amount of scrutiny because of its security implications, so I'd trust it to do the right thing over a naked call to RDRAND which nobody knows how exactly it's implemented in proprietary hardware or a handrolled solution to mix the RDRAND output with other entropy sources.

Remember the Debian openssl disaster from 2008? That happened exactly because someone had handrolled their entropy mixing solution, then someone else broke it.

Yes, on any modern system you should use the kernel provided random number sources.

The only legitimate reason to roll your own is when you're developing for an embedded system or a bootloader or something like that where there is no kernel API available.

You can effectively achieve the same result with this simple operation:

  hash = sha256(current_time());
  for i := 0; i < n; i++ {
      hash = sha256(hash.append(current_time()))
  }

This is because the number of nanoseconds between hashes is actually itself variable, and this is true for physics reasons that are basically beyond the control of any attacker trying to manipulate your entropy. If your time() function has a resolution of nanoseconds, you only need your loop to iterate about 50 times to get a cryptographically secure amount of entropy. If your time() function has a resolution of milliseconds, you need to let this run for more like 20 milliseconds, and if your time() function has a resolution of seconds you need to let it run for more like 5 seconds.

The reason I like doing it this way is that it happens entirely in userspace, it's genuinely a secure method of generating entropy, and it has no dependencies on potentially buggy firmware or microcode outside of the time() call, which is both fairly narrow, fairly heavily used (meaning a bug is likely to be discovered during testing, as the implementation is likely heavily scrutinized), and also fairly easy to test independently - just look at the number of nanoseconds that elapse at each consecutive call to sha256(current_time()) and verify that there's some statistical variance. The above suggestions are assuming about 2.5 bits of variance between calls, meaning there should be a range of at least 20 nanoseconds between your slowest and fastest hash call. This has been true on every CPU I've ever measured, including microcontrollers.

This comment demonstrates everything that's wrong with people trying to be clever and rolling their own crypto.

The security of your system depends on time() providing enough entropy, even though that's not what it's designed to do. It's built on top of the wrong primitive from the start.

The reason I like doing it this way is that it happens entirely in userspace

On Linux this is often true, but there is no portable way to get the current time that is _guaranteed_ not to do any system calls.

If your time() function has a resolution of nanoseconds, you only need your loop to iterate about 50 times to get a cryptographically secure amount of entropy.

You haven't proven that at all. It's easy to imagine that on a CPU running at a fixed frequency the interval between reads is constant, so if anyone knows (or can guess) the start time the resulting seed is entirely predictable.

This is completely independent of timer resolution. You seem to realize that as you were writing that:

just look at the number of nanoseconds that elapse at each consecutive call to sha256(current_time()) and verify that there's some statistical variance

Oh yes, because evaluating the quality of a random number generator is such a trivial thing to do, it's not like there is decades of research behind it or anything.

And assuming you are able to verify the statistical variance: are you going to put that logic in the loop, making it significantly more complex?

Or are you going to do this test on your machine and then ship your code on the assumption that if it works on your machine, it will work everywhere else, too?

if your time() function has a resolution of seconds you need to let it run for more like 5 seconds.

So not only is it insecure, it's agonizingly slow by design. Why do a system call that takes milliseconds at best, when we can run a loop in userspace for 5 seconds?

All this just so you can avoid writing the obviously correct oneliner:

    if (getentropy(&seed, sizeof(seed)) != 0) abort();

And to show my objections are not just theoretical I wrote a little program to check:

    #include <time.h>
    #include <stdio.h>
    
    static int estimate_entropy(long l) {
        int bits = 1; /* for the sign bit */
        if (l < 0) l = -l;
        while (l > 0) {
            ++bits;
            l >>= 1;
        }
        return bits;
    }
    
    int main() {
        struct timespec ts;
        if (clock_getres(CLOCK_REALTIME, &ts) != 0) {
            perror("clock_getres");
            return 1;
        }
        printf("Clock resolution: %ld.%09ld\n", (long) ts.tv_sec, (long) ts.tv_nsec);
        
        #define N 50  /* number of samples */
        struct timespec samples[N];
        for (int i = 0; i < N; ++i) {
            clock_gettime(CLOCK_REALTIME, &samples[i]);
        }
    
        printf("Deltas (ns):");
        long deltas[N - 1];
        for (int i = 0; i < N - 1; ++i) {
            deltas[i] = 
                (samples[i + 1].tv_sec - samples[i].tv_sec)*1000000000L
                + (samples[i + 1].tv_nsec - samples[i].tv_nsec);
            printf(" %4ld", deltas[i]);
        }
        printf("\n");
        long entropy = 0;
        printf("Deltas of deltas: ");
        for (int i = 0; i < N - 2; ++i) {
            long dd = deltas[i + 1] - deltas[i];
            printf(" %4ld", dd);
            entropy += estimate_entropy(dd);
        }
        printf("\n");
        printf("Maximum entropy: %lld\n", entropy);
    }

On my system this prints:

    Clock resolution: 0.000000001
    Deltas (ns):   55   51   23   23   25   24   24   24   24   24   25   25   24   24   24   24   24   25   24   24   24   25   25   24   24   23   25   24   24   25   24   23   25   25   26   23   25   24   24   25   26   24   23   25   25   26   24   25   24
    Deltas of deltas:    -4  -28    0    2   -1    0    0    0    0    1    0   -1    0    0    0    0    1   -1    0    0    1    0   -1    0   -1    2   -1    0    1   -1   -1    2    0    1   -3    2   -1    0    1    1   -2   -1    2    0    1   -2    1   -1
    Maximum entropy: 92

So no, 50 iterations of that loop does not provide 256 bits of entropy due to random fluctuations in nanontime between calls.

I would be very concerned if an RNG simply produced a natural 0.

I would be very concerned if an RNG simply produced a natural 1.