Oh, this could be really bad. I wonder if any private keys are compromised this way - would certainly be nice to know what rdrand is returning if it isn’t random data.

Catchy title. Nowhere it says it returns non-random data. It just fails.

Sure it does... read all the posts. Also read all the history.

Also read this from 2013 " I am so glad I resisted pressure from Intel engineers to let /dev/random rely only on the RDRAND instruction... Relying solely on the hardware random number generator which is using an implementation sealed inside a chip which is impossible to audit is a BAD idea. "

And this https://www.theregister.co.uk/2013/09/10/torvalds_on_rrrand_...

So, uh, this isn't news, and isn't limited to AMD.

Sure it'd be nice to fix.

RDRAND is not guaranteed to always succeed (and never was). You’re supposed to retry on failure.

(Although linked in that thread systemd code has fallback anyway, so I’m not sure how it fails at all).

Edit: not to mention that it’s better just to not use it, ever. Quite sane and sensible thing.

i don't see any reason to not xor rdrand output with a prng if rdrand is available. please enlighten us why you think otherwise as i'm really interested in assumptions that lead to this conclusion.

Do you mean if a PRNG is available? Because the answer is obvious if RDRAND is available: you don't do that because sometimes it fails, failure is almost always catastrophic but sublte, and CSRPNGs are not a bottleneck in almost all cases.

RDRAND has major benefits over any CSPRNG: there is no software-visible state that could possibly leak. With attacks like Spectre, there’s always a concern that your CSPRNG secrets could be leaked.

Sounds like a bit of code which was hard to trigger the negative test and therefore the fallback failed to work properly.

Not sure how the kernel devs generally go about testing the “this virtually never happens” code paths without adding debug switches to every unhappy path.

Certainly I doubt they are using DI/IoC to wrap an interface to RDRAND which allows unit testing the failure modes.

At least the result is a failure to generate a key, not a compromised key.

So - are they not checking for error in this case, and using a value anyway?

The instruction is signalling success, but returning FFFFFFFFFFFFFFFF as the random value.

* https://github.com/systemd/systemd/issues/11810#issuecomment...

H. Peter Anvin's educated guess was that some MSR flag, that has the effect of controlling this instruction, is not being saved and restored in the processor across a suspend, and the result is a processor state where it signals that the instruction is succeeding but it is not actually returning random values.

* https://bugzilla.kernel.org/show_bug.cgi?id=85911#c4

The instruction is signalling success

Let’s wait for clarifications how that person has done the tests.

However: the bug is about systemd failing to get entropy, not getting nonsense entropy.

How xe was doing this was in fact explained in the comment hyperlinked right at the start of this page, and H. Peter Anvin was thinking about this over 4 years ago. But for the doubters Vladislav has already reiterated the point.

* https://github.com/systemd/systemd/issues/11810#issuecomment...

There is no "however". This bug is about code that is, according to the AMD doco, using the instruction correctly; but that is, because the AMD processor has this possible state after a suspend+resume, getting all-ones as its random data, thereby causing ID collisions in a fairly wide range of possible things from freshly re-generated machine IDs to journal file header block IDs, and including unit invocation IDs.

* https://github.com/systemd/systemd/blob/717e8eda77b93ac396dc...

* https://github.com/systemd/systemd/blob/717e8eda77b93ac396dc...

* https://github.com/systemd/systemd/blob/717e8eda77b93ac396dc...

This indicates that a "should be fine" in another comment is not in fact true. (-:

* https://github.com/systemd/systemd/blob/717e8eda77b93ac396dc...

The Bugzilla report seemed to indicate that AMD was returning an error code, because OpenSSL was failing to generate a key, not generating a bad key?

Why would OpenSSL fail visibly if the API was returning success but with non-random data?

https://bugzilla.redhat.com/show_bug.cgi?id=1150286

If you look closely at one of the error messages, it says "too many iterations", in what looks like a random generator. I wonder if at some point it has to iterate to get a number different from another one, and it just gets an endless stream of identical values?

I can’t imagine the code is checking the returned value for uniqueness after seeing a SUCCESS code, and iterating in the hopes that becomes less true.

That would be quite something to read the comment on;

// Here we check to be sure the Earth is not flat

...

// Seeing that the Earth is indeed flat, we will iterate in this absurdity some more in the hopes it rounds out eventually

https://news.ycombinator.com/item?id=6336505

For the lazy, this is a link to a comment by Theodore Ts'o, kernel dev, who says:

I am so glad I resisted pressure from engineers working at Intel to let /dev/random in Linux rely blindly on the output of the RDRAND instructure. Relying solely on an implementation sealed inside a chip and which is impossible to audit is a BAD idea. Quoting from the article...

Theodore Ts'o is maintainer of the ext filesystems (particularly ext4), as well as, IIRC, /dev/random and other CSRNG related components of the kernel.

Thank you, Theodore Ts'o.

Theodore Ts'o is responsible for perpetuating the myth that entropy/randomness can run out, leading systemd (and other software) to do crazy things, such as trying to use RDRAND, to avoid "drain[ing] randomness from the kernel pool". The bugs and security vulnerabilities resulting from this myth probably neutralize the benefit that came from from his resistance to RDRAND in the kernel.

the myth that entropy/randomness can run out

Can you expand on this, or link to some sources that expand on this idea that the assumption above is wrong? As a person who has not dealt with crypto really at all I had heard this explained several times before and assumed it was generally accepted.

128-bits of random data is sufficient to securely generate a stream of 100s of terabytes of random data. It's not /that/ hard to find 128-bits of true entropy, even during boot phase. Here's one example:

    1. Seed with any fixed hardware IDs

    2. Mix-in the wall clock time

    3. Spin up a kernel thread and flip a bit on/off in a tight loop. Interrupt it every 100 nanoseconds and take the value of the bit at that time. Do this 256 times. Mix that in too.

    4. Mix-in 256-bits from RDRAND 

    5. Mix-in timings from other interrupts as and when they happen. 
 
    6. Repeat steps 4. and 5. ad infinitum. 

By step 4 we have taken 26 microseconds and we have the kind of entropy I would be comfortable generating an RSA private key with.

Note that step 3 is effectively a measure of how precise the system clock and CPU are. Attacks have been demonstrated against step 3, but they require co-resident processes and don't apply during the boot-phase, if you've got a dedicated core at least. In theory if system clocks and CPU got super precise it could become too deterministic, but the point is the likelihood of /both/ that happening /and/ RDRAND being broken.

128-bits of random data is sufficient to securely generate a stream of 100s of terabytes of random data.

What you are describing is /dev/urandom. Your argument is basically "urandom is good enough for anybody". If you want to use that, use it.

Wow, so when would be the worst time to suspend your computer?

The only time I imagine this, is when generating a private key for a production environment.

My reading is that systemd uses the cryptographically secure rng to generate a unique id for a filename, and doesn't handle collisions properly.

sigh

There shouldn't be collisions. I mean that really: if you see a collision it's so much more likely that your computer / program / source of randomness is faulty [as in this case] than that the two random numbers collided that it's not worth considering the collision case.

But there can be collisions, at least in theory. And as we can see, in practice too.

Not checking for collisions because you trust that an RNG returns unique values is a fallacy and should be avoided. Especially since it's so easy to deal with the problem.

It would be better to report the collision because most likely it means some error.

Is there a good reason this doesn't just use urandom/getrandom? This doesn't look like it's in the "we're so early in boot I haven't restored the random seed yet" case, for example.

Or better yet, the getrandom syscall, which has better semantics and avoids the filesystem.

No. There's a bad reason though. Want to guess what? https://github.com/systemd/systemd/blob/master/src/basic/ran...

Yes, the old "draining entropy" fallacy.

Why do you say draining entropy is a fallacy? It is certainly true that entropy recorded from I/O sources accumulates at a very limited rate.

Because after you gather ~256 bits of entropy from I/O sources (or rdrand^H^H^H rdseed [1]), using it to generate infinite output of /dev/urandom does not drain it. Adding more entropy only helps if the entropy pool inner state leaked, it is not needed to add more entropy because it drained through using it.

Proof:

1. Take 256 bits of entropy.

2. Use HKDF to generate 128 bit key and 64 bit nonce.

3. Use key and nonce for AES-CTR with all possible 2^64 counter values to produce 2^68 bytes of output.

4. goto 2.

If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool.

And here is a CCC talk about it: https://www.youtube.com/watch?v=OSfmtRc4VsE

1 - https://software.intel.com/en-us/blogs/2012/11/17/the-differ...

EDIT: Thanks for 'dragontamer for pointing out difference between rdrand and rdseed.

If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool.

This seems wrong to me.

If you can predict #1 (the 256-bits of entropy in step 1), then you can break the system. This is highly likely, as Intel only assures 65-bits of entropy across two rdrand calls. (64-bits of entropy from the first call, +1 bit from the 2nd call, since RDRAND is generated from an internal random number generator. So there's going to be a correlation between the two values).

In effect, if #1 is created with four calls to RDRAND, you only have around 68-bits of entropy, which can be brute-forced faster than a true source of 256-bits of entropy. Therefore, your RNG is broken. https://software.intel.com/en-us/blogs/2012/11/17/the-differ...

RDSEED is the instruction that guarantees 64-bits of independent / multiplicative entropy, but executes slower as a result.

----------

In effect, you're "simplifying" the problem. #1, getting 256 bits of entropy, is the hard part of the problem. You cannot ignore this part of the problem.

In any case, it seems to me that a large number of people don't know how to properly use the RDRAND function, in this thread as well as in the Github thread. RDRAND has a chance of failure, AND it doesn't even hold multiplicative entropy guarantees.

So this seems like a case of EVERYONE hasn't read the docs yet. Please people, do NOT use RDRAND as a source of entropy. Use RDSEED as a source of entropy. RDRAND is only a random number generator with only 64-bits of entropy guaranteed at any given state, and probably additive entropy at that.

I can ignore "how to do #1" when discussing whether entropy is drained by using it or not.

You need to get 256 bits of high quality entropy. We assume there are ways to do that.

On a device that has no way to do that, you can't do crypto (unless you accept the entropy from outside, which is key escrow, but might be ok for an IoT device that communicates only to its mothership).

I can ignore "how to do #1" when discussing whether entropy is drained by using it or not.

Ehhh... fair point. Still, the overall discussion is about RDRAND, so I feel like its very important to point out how RDSEED must be used to properly generate the 256 bits of entropy you require in step #1.

Each of these steps are tricky, and require thorough analysis to understand.

------------

EDIT: Its not so much that entropy is "used up". Its that RNG-sources of randomness gives "additive" entropy, while true entropy is "multiplicative".

Generating random numbers from a 256-bit RNG will give 256-bits of entropy from the first step, but be 100% predictable (and therefore "only" 256-bits on the 2nd step).

If the "next programmer" wants 512-bits of entropy, they will NEVER get 512-bits of entropy from your methodology, because you only started with 256-bits of entropy. Only by gathering "more" entropy will you be able to reach 512-bits of entropy.

The argument would go "who needs more than 256-bits of entropy", which is a fine point. But... that's how the math checks out.

It all comes back to the RDRAND vs RDSEED question. If the user just wants "unpredictable random numbers", then RDRAND and /dev/urandom is sufficient. But if you're creating independent random number generators (ex: If you're creating a service like random.org, and are guaranteeing certain amount of entropy per call to everyone), then you need to be using RDSEED.

Does 10 calls of your RNG produce 256-bits of randomness, or 2560-bits of randomness? What are your requirements? What are the requirements of the end user? For most people, having a RNG that "only" provides 256-bits of randomness across 10,000 calls is perfectly fine and sufficient. But there are plenty of cryptographic cases where that's not enough (and "true" 256-bits of entropy are needed in every call).

True about no way to deterministically get more than 256 bits of entropy out of 256 bits of initial entropy, but the requirements are to run wireguard, ssh, tls, generate web app session cookies. All those, I believe, would be satisfied by using a urandom which is implemented the way I described. What real use case has a requirement of more than 256 bits of "real" entropy?

Note that the real linux urandom is not as simple as I described, and djb's proposed key erasure RNG [1] is not as simple as I described. My design was just to talk about "draining" of the entropy.

1 - https://blog.cr.yp.to/20170723-random.html

So after reading this thread my understanding of the "draining is a fallacy" view is

(1) yes genuine non-programmatic entropy bits accumulate at a slow rate and can be drained

(2) in practise no one should care about (1) because once you have ~256 bits to initialise a CSPRNG you can use the output of that CSPRNG until the Earth is swallowed up by the sun; thats what the S of Cryptographically Secure Programmatic Random Number Generator promises.

(3) the linked systemd code is silly to even provide a function genuine_random_bytes that tries to get additional genuine non-programmatic entropy as every possible use case is covered by (2)

(4) The real fallacy is when people think that it might be possible to discover the seed of a CSRNG or predict its next outputs by examining a long enough run of its previous output. In practise such an attack should not be possible.

If I've got the summary right, the only point I would disagree on principle is (3). I can imagine that someone might rationally want "genuine" entropy independent of a kernel CSPRNG, for example for seeding their own CSPRNG.

I'd love to read a "cryptographic doom principle"-esque latacora blog post on all the failures over the years of not using your OS provided CSRNG.

Quoting from https://github.com/systemd/systemd/issues/11810#issuecomment...

BTW, the reason we use RDRAND in some cases instead of getrandom() [which we use in many others] is that we need to generate uuids early on (since every service we starts gets one passed, the "invocation ID", and for other stuff too), but getrandom complains in dmesg or blocks if we call it before the pool is initialized. Since systemd is one of the earliest programs that runs and thus very likely comes into contact with an uninitialized pool we attempt to avoid that by using RDRAND when generating uuids, since it should be good enough for that, as the usecase needs a "mid-quality" rng source: not crypt quality and not totally guessable either.

Why not use getrandom() with GRND_NONBLOCK?

Worst case, they could run RDRAND in a loop and write() it into /dev/random until getrandom() is unblocked. Then they're still using the ordinary kernel random device, more or less. They wouldn't have these catastrophic collisions due to near-zero entropy.

Because systemd is not x86-64-only. The actual worst case is where there is no RDRAND, because then the systemd people have the chicken-and-egg situation of not being able to seed urandom without running a system service unit, and not being able to run a systemd service unit without being able to assign a random GUID to it as its instance ID.

The design problem is that every service unit has an instance ID, whether meaningful and useful to it or not, the journal and other things have a machine ID, and journal files have header IDs. This requires that re-seeding urandom be pushed right to the start of the entire user-mode boot process (which begins with systemd in the initramfs on some operating systems), that something else (such as RDRAND) be used until urandom is re-seeded, or that the boot process simply stop and block running the very first service unit until urandom is re-seeded by the kernel.

Later in the thread:

Given that RDRAND is allowed to fail, it seems to me that you should either try it only once, or only a few times, before falling back to whatever code is used when RDRAND is not implemented.

... which is what the systemd code actually does. The problem is that there appears to be a possible AMD processor state, caused by suspend+resume, where the instruction succeeds but the data returned are not in fact random.

The other problem is that systemd's fallback is a non-random PRNG.

There is no mention anywhere, in this thread nor the one from 2014, of it returning non-random data from the issue reporters, just assumptions from onlookers.

Suspend/resume seems to be the cause of a whole host of bugs; I run into obnoxious suspend problems frequently on all platforms except maybe Windows. It's so common that I've pretty much stopped using suspend on most of my machines. It's a bit inconvenient to power down / power up things each day, but I'd rather deal with that than have intermittent wifi issues, display problems, etc.

I consulted once for a hospital that was experience seemingly random network outages. After a few questions with the staff there it seemed to happen whenever a person stepped away from their windows (Lenovo) workstations for too long. Say, after lunch or breaks. After a wireshark of the network I determined it was a network card driver that was causing a broadcast flood on the network from multiple points for stations with the same driver version. While not the fault of Windows (I dont think anyways, as an update of the driver fixed the issue) your comment did remind me of this experience.