Messages in this thread | | | Subject | Re: [PATCH v2 15/15] s390: doc: detailed specifications for AP virtualization | From | Tony Krowiak <> | Date | Tue, 27 Feb 2018 13:12:21 -0500 |
| |
On 02/27/2018 10:57 AM, Cornelia Huck wrote: > On Tue, 27 Feb 2018 09:28:13 -0500 > Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote: > >> This patch provides documentation describing the AP architecture and >> design concepts behind the virtualization of AP devices. It also >> includes an example of how to configure AP devices for exclusive >> use of KVM guests. >> >> Signed-off-by: Tony Krowiak <akrowiak@linux.vnet.ibm.com> >> --- >> Documentation/s390/vfio-ap.txt | 514 ++++++++++++++++++++++++++++++++++++++++ >> MAINTAINERS | 1 + >> 2 files changed, 515 insertions(+), 0 deletions(-) >> create mode 100644 Documentation/s390/vfio-ap.txt >> >> diff --git a/Documentation/s390/vfio-ap.txt b/Documentation/s390/vfio-ap.txt >> new file mode 100644 >> index 0000000..c599f30 >> --- /dev/null >> +++ b/Documentation/s390/vfio-ap.txt >> @@ -0,0 +1,514 @@ >> +Introduction: >> +============ >> +The Adjunct Processor (AP) facility is an IBM Z cryptographic facility comprised >> +of three AP instructions and from 1 up to 256 PCIe cryptographic adapter cards. >> +The AP devices provide cryptographic functions to all CPUs assigned to a >> +linux system running in an IBM Z system LPAR. >> + >> +The AP adapter cards are exposed via the AP bus. The motivation for vfio-ap >> +is to make AP cards available to KVM guests using the VFIO mediated device >> +framework. > Maybe drop a sentence in here that this makes heavy usage of the s390 > virtualization facilities, which do the heavy lifting? Will do. > >> + >> +AP Architectural Overview: >> +========================= >> +To facilitate the comprehension of the design, let's start with some >> +definitions: >> + >> +* AP adapter >> + >> + An AP adapter is an IBM Z adapter card that can perform cryptographic >> + functions. There can be from 0 to 256 adapters assigned to an LPAR. Adapters >> + assigned to the LPAR in which a linux host is running will be available to >> + the linux host. Each adapter is identified by a number from 0 to 255. When >> + installed, an AP adapter is accessed by AP instructions executed by any CPU. >> + >> +* AP domain >> + >> + An adapter is partitioned into domains. Each domain can be thought of as >> + a set of hardware registers for processing AP instructions. An adapter can >> + hold up to 256 domains. Each domain is identified by a number from 0 to 255. >> + Domains can be further classified into two types: >> + >> + * Usage domains are domains that can be accessed directly to process AP >> + commands >> + >> + * Control domains are domains that are accessed indirectly by AP >> + commands sent to a usage domain to control or change the domain, for >> + example; to set a secure private key for the domain. >> + >> +* AP Queue >> + >> + An AP queue is the means by which an AP command-request message is sent to an >> + AP usage domain inside a specific AP. An AP queue is identified by a tuple >> + comprised of an AP adapter ID (APID) and an AP queue index (APQI). The >> + APQI corresponds to a given usage domain number within the adapter. This tuple >> + forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP >> + instructions include a field containing the APQN to identify the AP queue to >> + which the AP command-request message is to be sent for processing. >> + >> +* AP Instructions: >> + >> + There are three AP instructions: >> + >> + * NQAP: to enqueue an AP command-request message to a queue >> + * DQAP: to dequeue an AP command-reply message from a queue >> + * PQAP: to administer the queues > Do you also want to explain how these entities show up on the ap bus in > Linux? It might make the explanations further down easier to > understand. (Is there any document for the ap bus you could point to?) I am not aware of any documentation for the AP bus, but I can provide a few sentences to explain how AP devices are managed by the AP bus. > >> + >> +Start Interpretive Execution (SIE) Instruction: >> +============================================== > Call this "AP and SIE" or so? You're not trying to explain the whole > SIE architecture :) Okay > > >> +Let's now see how AP instructions are interpreted by the hardware. > (...) > >> +Reserve APQNs for exclusive use of KVM guests >> +--------------------------------------------- >> +The following block diagram illustrates the mechanism by which APQNs are >> +reserved: >> + >> + +------------------+ >> + remove | | unbind >> + +------------------->+ cex4queue driver +<-----------+ >> + | | | | >> + | +------------------+ | >> + | | >> + | | >> + | | >> ++--------+---------+ register +------------------+ +-----+------+ >> +| +<---------+ | bind | | >> +| ap_bus | | vfio_ap driver +<-----+ admin | >> +| +--------->+ | | | >> ++------------------+ probe +---+--------+-----+ +------------+ >> + | | >> + create | | store APQN >> + | | >> + v v >> + +---+--------+-----+ >> + | | >> + | matrix device | >> + | | >> + +------------------+ >> + > Thank you for including diagrams, these are really helpful. You are welcomed > > (...) > >> +Initialize the CPU model feature for AP >> +--------------------------------------- >> +This design exploits a feature of the SIE architecture called interpretive >> +execution (IE). When IE is enabled for a KVM guest, the AP instructions >> +executed in the guest will be interpreted by the firmware and the commands >> +contained therein will be passed directly through to an AP device assigned to >> +the linux host. In order to enable interpretive execution for a KVM guest, SIE >> +must have access to the AP facilities installed on the linux host. A new CPU >> +model feature is introduced by this design to indicate that the guest will >> +directly access the host AP facilities. This feature will be enabled by the >> +kernel only if the AP facilities are installed on the linux host. This feature >> +must be turned turned on for the guest in order to ac: >> + >> + /usr/bin/qemu-system-s390x ... -cpu xxx,ap=on >> + >> + Where xxx is the CPU model being used. >> + >> +If the CPU model feature is not enabled by the kernel, QEMU >> +will fail and report that the feature is not supported. > The cpu model interface is supposed to be user space agnostic, although > it is only used by QEMU in practice. Mark this as an example more > clearly? Will do > > (...) > > I have not looked at this in detail (will probably come back to this > later), but this looks like a useful document. Excellent, I'm glad to hear that. It took some time to put together. >
| |