Messages in this thread | | | Date | Fri, 2 Feb 2024 10:39:27 -0500 | From | "Theodore Ts'o" <> | Subject | Re: [PATCH 1/2] x86/random: Retry on RDSEED failure |
| |
On Fri, Feb 02, 2024 at 07:25:42AM +0000, Reshetova, Elena wrote: > This is a great summary of options, thank you Jason! > My proposal would be to wait on result of our internal investigation > before proceeding to choose the approach.
I'm happy for the option "Do nothing for now", but if we do want to do something in the absence of more detailed information, I'd suggest doing something simple for first, on the theory that it doesn't make things worse, and we can always do something more complicated if it turns out to be needed.
In that vein, my suggestion is:
> > Solution B) BUG_ON(is_early_boot && is_coco_system) in the RDRAND > > failure path (> 10 retries). > > > > This is slightly less simple than A, because we have to plumb > > CoCo-detection through to the RDRAND helper. [Side note: I feel > > ridiculous typing 'CoCo'.] Systems-wise, I don't see drawbacks. > > RNG-wise, the drawback is that this doesn't help deal with secure > > reseeding later in time, which is a RNG property that we otherwise > > enjoy.
If there isn't a global variable we can test to see if Confidential Compute is enabled, I suspect we should just add one. I would assume that /dev/random isn't the only place where we might need to do whether Confidential Compute is enabled.
So I don't think plumbing CC into the /dev/random code, and since we are only doing this in early boot, I wouldn't put it in the RDRAND helper, but rather in the caller of the RDRAND helper that gets used in the early boot path.
(Side note, internally, at least in my part of my company, we use CC as the acronym of convenience. And any comments that I make are my own opinion, and do not reflect the positions or desires of my employer...)
> > Question iii) How likely is Intel to actually fix this in a > > satisfactory way (see "specifying this is an interesting question" in > > [1])? And if they would, what would the timeline even be?
Here are at least two obvious ways that Intel could fix or mitigate this issue:
(1) Add more hardware RNG IP's for chips with a huge number of cores. This is the *obvious* way to address the problem with hundreds of CPU cores, although it's only something that can be done on newer chips.
(2) Have a per-core throttle where a core is not allowed to issue RDRAND or RDSEED instructions more than N times per millisecond (or some other unit of time). So long as N is larger than the maximum number of SSL connections that a front-end server can actually terminate, it's not going to impact legitimate workloads. This can be approximated by the number of public key operations per unit time that a single CPU core achieve. And if RDRAND isn't sufficient to support that today, then see solution (1), or CPU customers should switch to some other CPU vendor that can...
(3) Provide some kind of perf counter so the host can see which cores are issuing a huge number of RDRAND/RDSEED instructions, and which cores have been suffering from entropy exhaustion RDRAND/RDSEED failures. This would allow the operator of the host to detect which VM's might be carrying out DOS attacks, so that the operator can kill those VM's, and disable the customer account that was launching these abusive VM's.
Hopefully mitigation #2 in particular (and maybe mitigation #3) is something that Intel could implement as a firmware update; I'd love comments from Intel if that is the case.
I'll also note that the threat model where customer A is trying to start a CC VM, and customer B has purchased VM's that use all of the other cores on a server, is primarily the sort of thing that a public cloud vendor would need to worry about. And *if* this become a real issue, where some researcher demonstrates that there is a problem, the cloud provider will be hugely incentivized to apply host-side mitigations, and to lean on the guest OS providers to apply guest-side mitigations.
So if this is only a DOS which applies for CC VM's, and it turns out that solution (B) is not sufficient, we can do something more complicated, such as having the guest retry the RDRAND instruction for ten seconds. And if some hypothetical "RandExhaust" attack is being written about by the New York Times, I suspect it won't be that hard to get Red Hat to apply mitigations to the RHEL kernel. :-)
So I don't think it really is *that* big of a deal; if it turns out to be an issue, we will be able to deal with it.
- Ted
| |