February 2019 - Devel - libvirt List Archives

[libvirt] Configuring pflash devices for OVMF firmware
by Markus Armbruster 22 Feb '19

22 Feb '19

We configure OVMF firmware for PC machine types with -drive if=pflash. This is pretty much the last remaining use of -drive in libvirt we can't yet replace by -blockdev. Such a replacement is desirable, because -blockdev + -device is more flexible than -drive if=pflash. Also, once we don't need -drive with new QEMU anymore, the path for deleting all -drive code in libvirt some day is open. As with all desirables, the benefit needs to exceed the cost. I'm going to describe the status quo, how we got there (briefly and much simplified), then sketch how to replace -drive if=pflash. I'm afraid this is fairly long; sorry. Please correct misunderstandings. Beware, my libvirt and OVMF fu is much weaker than my QEMU fu. In the beginning, board code read the BIOS from a fixed file and mapped it into the guest's address space. Life was simple. On physical hardware, the BIOS can persist a bit of state across (cold) reboots by storing it in (non-volatile) CMOS RAM. We didn't bother. Simple. Fast forward several years, and The Law of OS Envy (every program wants to grow into a full-blown operating system) has asserted itself: PC Firmware has grown from an 8KiB ROM using a few bytes of volatile and non-volatile RAM into a multi-megabyte beast with much more complex storage needs. On today's physical PC hardware, firmware is stored in flash memory. There's code, and there's persistent data. For obvious reasons, the code should be write-protected except when doing an upgrade. "Secure boot" additionally needs to restrict data writes to system management mode (SMM). Here's our first iteration of OVMF support, at QEMU level: -drive if=pflash,format=raw,file=/where/ever/OVMF.fd Generic code creates a block backend for it. Magic board code picks up the backend, creates a frontend (a cfi.pflash01 device), and maps it into the guest's address space. At libvirt level: <loader type="pflash">/where/ever/OVMF.fd</loader> Problem: while the flash device model provides read-only capability, it's all-or-nothing. You can't tell it to write-protect just the part holding code. The examples above don't write-protect anything. /where/ever/OVMF.fd better be writable exclusively. The flash device model could be enhanced, but we went down a different path: we split the single OVMF image OVMF.fd ("unified build") into a code image OVMF_CODE.fd and a data image OVMF_VARS.fd ("split build"). At QEMU level: -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=/where/ever/OVMF_VARS.fd OVMF_CODE.fd must be unit 0, and OVMF_VARS.fd must be unit 1. Generic code creates two block backends. Magic board code picks them up, creates a frontend (a cfi.pflash01 device) for each, and maps them into the guest's address space. Note there are *two* virtual flash devices now, whereas physical hardware commonly has just one. At libvirt level: <loader type="pflash" readonly="yes">/usr/share/OVMF/OVMF_CODE.fd</loader> <nvram template="/usr/share/OVMF/OVMF_VARS.fd">/var/libvirt/nvram/${guest}_VARS.fd</nvram> This treats OVMF_VARS.fd as a read-only template, and gives each guest its own writable copy, which is nice. The flash device model supports restricting writes to SMM (remember, that's required for secure boot). It's controlled by cfi.pflash01 property secure, off by default. If we created the device model with -device, we'd simply pass secure=on. But since we create it with -drive if=pflash, we can't. Instead we have to use -global driver=cfi.pflash01,property=secure,value=on This flips the global default value. Awkward, but works out okay, because (1) the flash device holding OVMF_VARS.fd wants this value, and (2) the flash device holding OVMF_CODE.fd doesn't care (it's read-only), and (3) there is no way to create additional flash devices. At the libvirt level, we add secure='yes' to the loader element. We also have to enable SMM emulation. At QEMU level: -machine smm=on At libvirt level: <features> <smm state='on'/> </features> Note that the above configuration examples involve selecting OVMF images. A bit of an inconvenience compared to BIOS, where the default "use the BIOS shipped with QEMU" pretty much just works. To add annoyance to inconvenience, different distributions have different ideas on where to install OVMF images. And because that's not complicated enough, we also have to pair code with data images. And because that's still not complicated enough, any specific machine type may work only with a subset of the available firmwares. The proposed way to deal with all that works as follows. Each set of firmware images comes with a descriptor file. These are JSON and conform to the QAPI schema docs/interop/firmware.json. Among the descriptors that declare support for the kind of machine we want, we pick (really: the management application picks) the one with the highest priority. The distribution provides default priorities, which system administrator and user can override. firmware.json documents this in much more detail. I wrote "proposed", because as far as I can tell, neither distributions nor libvirt are there, yet. After all this text, I'm finally ready to curve towards -blockdev. Going from -drive if=T, T!=none to -blockdev involves two steps. The first step replaces if=T with if=none and -device. The second step replaces -drive if=none with -blockdev. That step is "obvious" (it took us a few years to get to obvious, but I digress). The difficulty is in the first step. Two issues: (1) cfi.pflash01 isn't available with -device. (2) "Magic board code picks up the backend [created for -drive if=pflash], creates a frontend (a cfi.pflash01 device), and maps it into the guest's address space." When we replace if=pflash by if=none, we get to replicate that magic on top of -device. Issue (1) isn't too hard: we add the device to the dynamic sysbus device white-list, move a sysbus_mmio_map() from pflash_cfi01_realize() into pflash_cfi01_realize(). The latter requires a new device property to configure the base address. I got a working prototype. Since this makes the device model's name and properties ABI, review would be advisable. To solve (2), we first have to understand the magic. Device cfi.pflash01 has the following properties: num-blocks Size of the device in blocks sector-length Size of a block (admire the choice of names) width Bank width big-endian Endianess (d'oh) id0, id1, id2, id3 Some kind of device ID, guest-visible, default to zero, few boards change it name Memory region name (why is this even configurable?) phys-addr Physical base address (this is the new device property mentioned above) secure For restricting access to firmware, default off device-width you don't want to know, there is a default, but it's documented as "bad, do not use", yet pretty much all boards use it max-device-width defaults to device-width not actually set anywhere old-multiple-chip-handling back-compat gunk for machine types 2.8 and older The magic board code in hw/i386/pc_sysfw.c configures as follows: num-blocks computed from backend size sector-length 4096 width 1 big-endian 0 id0, id1, id2, id3 all 0 name system.pflash<U>, where U is -drive's unit number phys-addr computed so unit 0 ends right below 0x100000000, unit n+1 ends at right below unit n "secure", "device-width", "max-device-width", "old-multiple-chip-handling" are left at the default. One additional bit of magic is actually in libvirt: it configures "secure" by flipping its default with -global driver=cfi.pflash01,property=secure,value=on. Now let's consider how to replicate this magic on top of device. Perhaps machine-type specific defaults could take care of sector-length, width, big-endian, id0, id1, id2, id3. Leaves num-blocks, name, and phys-addr. Perhaps the realize() method could default num-blocks to size of backend. But that doesn't really help the management application, because it needs to mess with the size anyway to compute phys-addr. So scratch that idea. Moving the magic code to compute num-blocks, phys-addr and name to the management application is certainly possible, but ugly. Note that the values computed are fixed when the firmware gets deployed. If we record them in the firmware descriptor, the management application doesn't need magic, it can simply pass on the values obtained from the descriptor. We'd want to include sector-length in the descriptor then, to ensure num-block has a defined meaning. Same technique could take care of width, big-endian, ... in case machine-type specific defaults turn out to be inadequate for them. Opinions? One more problem: the magic board code does a bit more than just configure the cfi.pflash01 device. That additional magic needs to be generalized to work regardless of whether the device gets configured with -drive if=pflash or with -device. I got a working prototype.

7 44

[libvirt] Plans for next release
by Daniel Veillard 22 Feb '19

22 Feb '19

So it's getting time to think about 5.1.0, I'm thinking about entering freeze on Monday next week, push rc2 Wed and if everything goes well we can have the release out on Friday 1st, I hope that works for everyone, Daniel P.S. I will be on west coast so freeze and pushes likely to be later in the day than usual. -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH 0/1] update copyright notice
by David Kiarie 22 Feb '19

22 Feb '19

this updates the patch to use a full legal name on the copyright notice. David Kiarie (1): src/xenconfig: update copyright notice src/xenconfig/xen_xl.c | 1 + src/xenconfig/xen_xl.h | 1 + tests/xlconfigtest.c | 1 + 3 files changed, 3 insertions(+) -- 2.20.1

2 2

[libvirt] [PATCH 0/1] update copyright notice
by David Kiarie 22 Feb '19

22 Feb '19

these patche follows up a very co-operative review on a preview patch or lack of thereof. i have updated this to reflect the exact name i use in person. David Kiarie (1): src/xenconfig: update copyright notice src/xenconfig/xen_xl.c | 1 + src/xenconfig/xen_xl.h | 1 + tests/xlconfigtest.c | 1 + 3 files changed, 3 insertions(+) -- 2.20.1

2 3

[libvirt] [PATCH] qemu: fix memory leak in qemuBuildDiskDeviceStr
by Ján Tomko 22 Feb '19

22 Feb '19

Commit a1dce962 added the allocated scsiVPDDeviceId without freeing it. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- src/qemu/qemu_command.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f511193fca..74f34af292 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1890,7 +1890,7 @@ qemuBuildDiskDeviceStr(const virDomainDef *def, const char *bus = virDomainDiskQEMUBusTypeToString(disk->bus); const char *contAlias; char *backendAlias = NULL; - char *scsiVPDDeviceId = NULL; + VIR_AUTOFREE(char *) scsiVPDDeviceId = NULL; int controllerModel; if (qemuCheckDiskConfig(disk, qemuCaps) < 0) -- 2.19.2

2 1

[libvirt] [PATCH 0/7] Introduce automatic clearing for virBuffer
by Peter Krempa 22 Feb '19

22 Feb '19

First 4 patches clean up existing mess. Note that this series both fixes and then deletes virBufferEscapeN. The split is deliberate, so that if somebody needs virBufferEscapeN in the future they can revert the removal and get a working version. Peter Krempa (7): util: buffer: Remove misleading AUTOPTR func for 'virBuffer' util: buf: Fix memory leak in virBufferEscapeN tests: buf: Fix debug messages in 'testBufEscapeRegex' util: buf: Remove virBufferEscapeN util: alloc: Introduce 'VIR_AUTOCLEAN' macros for clearing stack'd structs util: buffer: Introduce VIR_AUTOCLEAN function for virBuffer qemu: domain: Use VIR_AUTOCLEAN for virBuffer src/libvirt_private.syms | 1 - src/qemu/qemu_domain.c | 27 +++------- src/util/viralloc.h | 32 +++++++++++- src/util/virbuffer.c | 107 --------------------------------------- src/util/virbuffer.h | 7 ++- tests/virbuftest.c | 57 +++++---------------- 6 files changed, 56 insertions(+), 175 deletions(-) -- 2.20.1

2 10

[libvirt] [PATCHv2 0/2] network: add netmask to dhcp range of dnsmasq conf file for IPv4
by Laine Stump 21 Feb '19

21 Feb '19

I sent V1 of this patch way back in October: https://www.redhat.com/archives/libvir-list/2018-October/msg00889.html but then self-NACKed it because I had written the patch to add the network prefix to dhcp-range, but IPv4 requires the netmask rather than range. Unfortunately, when I modified the patch accordingly the getnameinfo() function began failing, I couldn't immediately see why, and I had to pack everything up and go to KVM Forum. After that, I promptly forgot to look back at it. Today I finally revisited the problem (after the person who originally found it sent a reminder), and discovered the failure of getnameinfo() was due to a bug in a function that I added to the virSocketAddr library in 2010! Laine Stump (2): util: set missing data length in virSocketAddrPrefixToNetmask() network: add netmask to dhcp range of dnsmasq conf file for IPv4 src/network/bridge_driver.c | 27 +++++++++++++++---- src/util/virsocketaddr.c | 2 ++ .../dhcp6-nat-network.conf | 2 +- .../networkxml2confdata/isolated-network.conf | 2 +- .../nat-network-dns-srv-record-minimal.conf | 2 +- .../nat-network-dns-srv-record.conf | 2 +- .../nat-network-dns-txt-record.conf | 2 +- .../networkxml2confdata/nat-network-mtu.conf | 2 +- .../nat-network-name-with-quotes.conf | 2 +- tests/networkxml2confdata/nat-network.conf | 2 +- .../networkxml2confdata/netboot-network.conf | 2 +- .../netboot-proxy-network.conf | 2 +- .../networkxml2confdata/ptr-domains-auto.conf | 2 +- 13 files changed, 35 insertions(+), 16 deletions(-) -- 2.20.1

2 6

[libvirt] [PATCH 0/7] bulk snapshot list/redefine (incremental backup saga)
by Eric Blake 21 Feb '19

21 Feb '19

While looking at my work on incremental backups, Nir raised a good point: if we want to recreate a set of known checkpoints on one machine that will be taking over a domain from another machine, my initial proposal required making multiple API calls to list the XML for each checkpoint on the source, and then more API calls to redefine each checkpoint on the destination; it also had the drawback that the list has to be presented in topological order (the API won't let you define a child checkpoint if the parent is not defined first). He asked if I could instead add bulk APIs, for getting the XML for all checkpoints at once, and then for redefining all checkpoints at once. Since my checkpoint code borrows heavily from concepts in the snapshot code, I chose to tackle the problem by starting with this series, which does the same thing for snapshots as what I plan to do for checkpoints. That is, since this patch series adds virDomainGetXMLDesc(, VIR_DOMAIN_XML_SNAPSHOTS) and virDomainSnapshotCreateXML(, VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE_LIST), the checkpoint series will add virDOmainGetXMLDesc(, VIR_DOMAIN_XML_CHECKPOINTS) and virDomainCheckpointCreateXML(, VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE_LIST) with very similar code. Note that I'm posting the series now to start review; the bulk query is fully implemented (patch 6), while the bulk redefine still needs more work (patch 7 defines the API, but the actual implementation is trickier, so I'll be posting additional patches later once I get them working). Patch 1 is a revision of work posted previously in a different series where all other patches of that series are now on mainline. Also available at: https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/snapshot-bulk-v1 Eric Blake (7): domain: Document VIR_DOMAIN_XML_MIGRATABLE snapshot: Give virDomainSnapshotDefFormat its own flags snapshot: Refactor virDomainSnapshotDefFormat snapshot: Add virDomainSnapshotObjListFormat domain: Expand virDomainDefFormatInternal with snapshots domain: Add VIR_DOMAIN_XML_SNAPSHOTS flag snapshot: Add VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE_LIST flag include/libvirt/libvirt-domain-snapshot.h | 3 + include/libvirt/libvirt-domain.h | 1 + src/conf/domain_conf.h | 8 ++ src/conf/snapshot_conf.h | 18 ++- src/conf/domain_conf.c | 71 ++++++++-- src/conf/snapshot_conf.c | 162 ++++++++++++++++------ src/esx/esx_driver.c | 1 - src/libvirt-domain-snapshot.c | 21 ++- src/libvirt-domain.c | 14 ++ src/libvirt_private.syms | 2 + src/network/bridge_driver.c | 3 +- src/qemu/qemu_domain.c | 24 +++- src/qemu/qemu_driver.c | 7 +- src/test/test_driver.c | 10 +- src/vbox/vbox_common.c | 5 +- src/vz/vz_driver.c | 3 +- tests/domainsnapshotxml2xmltest.c | 16 ++- tools/virsh-domain.c | 7 + tools/virsh-snapshot.c | 13 ++ 19 files changed, 293 insertions(+), 96 deletions(-) -- 2.20.1

1 10

[libvirt] [PATCH] snapshot: Saner use of uuid
by Eric Blake 21 Feb '19

21 Feb '19

Most of the code base is fairly consistent about using the name 'uuidstr' when dealing with a formatted human-readable form, and 'uuid' when dealing with the smaller raw bytes form. Fix snapshot_conf to comply, as well as reducing the scope of a human string to only the error message that needs it. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- Pushing this one under the trivial rule. src/conf/snapshot_conf.h | 2 +- src/conf/snapshot_conf.c | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/conf/snapshot_conf.h b/src/conf/snapshot_conf.h index cc25164261..7a175dfc96 100644 --- a/src/conf/snapshot_conf.h +++ b/src/conf/snapshot_conf.h @@ -111,7 +111,7 @@ virDomainSnapshotDefPtr virDomainSnapshotDefParseNode(xmlDocPtr xml, virDomainXMLOptionPtr xmlopt, unsigned int flags); void virDomainSnapshotDefFree(virDomainSnapshotDefPtr def); -char *virDomainSnapshotDefFormat(const char *domain_uuid, +char *virDomainSnapshotDefFormat(const char *uuidstr, virDomainSnapshotDefPtr def, virCapsPtr caps, virDomainXMLOptionPtr xmlopt, diff --git a/src/conf/snapshot_conf.c b/src/conf/snapshot_conf.c index 1afc7de30c..489c25f511 100644 --- a/src/conf/snapshot_conf.c +++ b/src/conf/snapshot_conf.c @@ -688,7 +688,7 @@ virDomainSnapshotDiskDefFormat(virBufferPtr buf, char * -virDomainSnapshotDefFormat(const char *domain_uuid, +virDomainSnapshotDefFormat(const char *uuidstr, virDomainSnapshotDefPtr def, virCapsPtr caps, virDomainXMLOptionPtr xmlopt, @@ -744,10 +744,10 @@ virDomainSnapshotDefFormat(const char *domain_uuid, if (def->dom) { if (virDomainDefFormatInternal(def->dom, caps, flags, &buf, xmlopt) < 0) goto error; - } else if (domain_uuid) { + } else if (uuidstr) { virBufferAddLit(&buf, "<domain>\n"); virBufferAdjustIndent(&buf, 2); - virBufferAsprintf(&buf, "<uuid>%s</uuid>\n", domain_uuid); + virBufferAsprintf(&buf, "<uuid>%s</uuid>\n", uuidstr); virBufferAdjustIndent(&buf, -2); virBufferAddLit(&buf, "</domain>\n"); } @@ -1224,11 +1224,8 @@ virDomainSnapshotRedefinePrep(virDomainPtr domain, int ret = -1; int align_location = VIR_DOMAIN_SNAPSHOT_LOCATION_INTERNAL; bool align_match = true; - char uuidstr[VIR_UUID_STRING_BUFLEN]; virDomainSnapshotObjPtr other; - virUUIDFormat(domain->uuid, uuidstr); - /* Prevent circular chains */ if (def->parent) { if (STREQ(def->name, def->parent)) { @@ -1274,6 +1271,9 @@ virDomainSnapshotRedefinePrep(virDomainPtr domain, if (def->dom && memcmp(def->dom->uuid, domain->uuid, VIR_UUID_BUFLEN)) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + + virUUIDFormat(domain->uuid, uuidstr); virReportError(VIR_ERR_INVALID_ARG, _("definition for snapshot %s must use uuid %s"), def->name, uuidstr); -- 2.20.1

1 0

[libvirt] [jenkins-ci PATCH] lcitool: Don't import crypt
by Andrea Bolognani 20 Feb '19

20 Feb '19

It hasn't been needed since dbe55924f90d, and flake8 rightfully complains about that. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- guests/lcitool | 1 - 1 file changed, 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index 759eff6..e8263da 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -17,7 +17,6 @@ # with this program. If not, see <https://www.gnu.org/licenses/>. import argparse -import crypt import fnmatch import json import os -- 2.20.1

2 1