- Devel - libvirt List Archives

RFC: Qemu backup interface plans
by Vladimir Sementsov-Ogievskiy 25 May '21

25 May '21

Hi all! I'd like to share and discuss some plans on Qemu backup interface I have. (actually, most of this I've presented on KVM Forum long ago.. But now I'm a lot closer to realization:) I'd start with a reminder about image fleecing: We have image fleecing scheme to export point-in-time state of active disk (iotest 222): backup(sync=none) ┌───────────────────────────────────────┐ ▼ │ ┌────────────┐ ┌────────────────┐ backing ┌─────────────┐ │ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │ └────────────┘ └────────────────┘ └─────────────┘ ▲ ┌────────────┐ │ │ guest blk │ ──────────────────────────────────────────────┘ └────────────┘ Actually, backup job inserts a backup-top filter, so in detail it looks like: backup(sync=none) ┌───────────────────────────────────────┐ ▼ │ ┌────────────┐ ┌────────────────┐ backing ┌─────────────┐ │ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │ └────────────┘ └────────────────┘ └─────────────┘ ▲ ▲ │ target │ │ │ ┌────────────┐ ┌────────────────┐ backing │ │ guest blk │ ──▶ │ backup-top │ ───────────────────────┘ └────────────┘ └────────────────┘ This scheme is also called external backup or pull backup. It allows some external tool to write data to actual backup, and Qemu only provides this data. We support also incremental external backup: Qemu can manage dirty bitmaps in any way user wants, and we can export bitmaps through NBD protocol. So, client of NBD export can get the bitmap, and read only "dirty" regions of exported image. What we lack in this scheme: 1. handling dirty bitmap in backup-top filter: backup-top does copy-before-write operation on any guest write, when actually we are interested only in "dirty" regions for incremental backup Probable solution would allowing specifying bitmap for sync=none mode of backup, but I think what I propose below is better. 2. [actually it's a tricky part of 1]: possibility to not do copy-before-write operations for regions that was already copied to final backup. With normal Qemu backup job, this is achieved by the fact that block-copy state with its internal bitmap is shared between backup job and copy-before-write filter. 3. Not a real problem but fact: backup block-job does nothing in the scheme, the whole job is done by filter. So, it would be interesting to have a possibility to simply insert/remove the filter, and avoid block-job creation and managing at all for external backup. (and I'd like to send another RFC on how to insert/remove filters, let's not discuss it here). Next. Think about internal backup. It has one drawback too: 4. If target is remote with slow connection, copy-before-write operations will slow down guest writes appreciably. It may be solved with help of image fleecing: we create temporary qcow2 image, setup fleecing scheme, and instead of exporting temp image through NBD we start a second backup with source = temporary image and target would be real backup target (NBD for example). Still, with such solution there are same [1,2] problems, 3 becomes worse: 5. We'll have two jobs and two automatically inserted filters, when actually one filter and one job are enough (as first job is needed only to insert a filter, second job doesn't need a filter at all). Note also, that this (starting two backup jobs to make push backup with fleecing) doesn't work now, op-blockers will be against. It's simple to fix (and in Virtuozzo we live with downstream-only patch, which allows push backup with fleecing, based on starting two backup jobs).. But I never send a patch, as I have better plan, which will solve all listed problems. So, what I propose: 1. We make backup-top filter public, so that it could be inserted/removed where user wants through QMP (how to properly insert/remove filter I'll post another RFC, as backup-top is not the only filter that can be usefully inserted somewhere). For this first step I've sent a series today: subject: [PATCH 00/21] block: publish backup-top filter id: <20210517064428.16223-1-vsementsov(a)virtuozzo.com> patchew: https://patchew.org/QEMU/20210517064428.16223-1-vsementsov@virtuozzo.com/ (note, that one of things in this series is rename s/backup-top/copy-before-write/, still, I call it backup-top in this letter) This solves [3]. [4, 5] are solved partly: we still have one extra filter, created by backup block jobs, and also I didn't test does this work, probably some op-blockers or permissions should be tuned. So, let it be step 2: 2. Test, that we can start backup job with source = (target of backup-top filter), so that we have "push backup with fleecing". Make an option for backup to start without a filter, when we don't need copy-before-write operations, to not create extra superfluous filter. 3. Support bitmap in backup-top filter, to solve [1] 3.1 and make it possible to modify the bitmap externally, so that consumer of fleecing can say to backup-top filter: I've already copied these blocks, don't bother with copying them to temp image". This is to solve [2]. Still, how consumer of fleecing will reset shared bitmap after copying blocks? I have the following idea: we make a "discard-bitmap-filter" filter driver, that own some bitmap and on discard request unset corresponding bits. Also, on read, if read from the region with unset bits the EINVAL returned immediately. This way both consumers (backup job and NBD client) are able to use this interface: Backup job can simply call discard on source, we can add an option for this. External backup tool will send TRIM request after reading some region. This way disk space will be freed and no extra copy-before-write operations will be done. I also have a side idea that we can implement READ_ONCE flag, so that READ and TRIM can be done in one NBD command. But this works only for clients that don't want to implement any kind of retrying. So, finally, how will it look (here I call backup-top with a new name, and "file" child is used instead of "backing", as this change I propose in "[PATCH 00/21] block: publish backup-top filter"): Pull backup: ┌────────────────────────────────────┐ │ NBD export │ └────────────────────────────────────┘ │ │ ┌────────────────────────────────────┐ file ┌───────────────────────────────────────┐ backing ┌─────────────┐ │ discard-bitmap filter (bitmap=bm0) │ ──────▶ │ temp qcow2 img │ ─────────▶ │ active disk │ └────────────────────────────────────┘ └───────────────────────────────────────┘ └─────────────┘ ▲ ▲ │ target │ │ │ ┌────────────────────────────────────┐ ┌───────────────────────────────────────┐ file │ │ guest blk │ ──────▶ │ copy-before-write filter (bitmap=bm0) │ ─────────────┘ └────────────────────────────────────┘ └───────────────────────────────────────┘ Operations: - Create temp qcow2 image - blockdev-add to add the new image, setup its backing, and add two filters - some command to actually set backup-top filter as child of guest blk. That's a "point-in-time" of backup. Should be done during fs-freeze. - start NBD export on top of discard filter (and we can export bitmap bm0 as well, for the client) Now NBD client (our external backup tool) can: - import the bitmap - READ the data - send DISCARD requests on already handled areas to save disk space and avoid extra copy-before-write operations on host node Push backup with fleecing: ┌─────────────────────┐ │ final backup target │ └─────────────────────┘ ▲ │ backup job (bitmap=bm0, insert-filter=False, discard-source=True) │ ┌────────────────────────────────────┐ file ┌───────────────────────────────────────┐ backing ┌─────────────┐ │ discard-bitmap filter (bitmap=bm0) │ ──────▶ │ temp qcow2 img │ ─────────▶ │ active disk │ └────────────────────────────────────┘ └───────────────────────────────────────┘ └─────────────┘ ▲ ▲ │ target │ │ │ ┌────────────────────────────────────┐ ┌───────────────────────────────────────┐ file │ │ guest blk │ ──────▶ │ copy-before-write filter (bitmap=bm0) │ ─────────────┘ └────────────────────────────────────┘ └───────────────────────────────────────┘ Note, that the whole fleecing part is the same, we only need to run backup job instead of NBD export. Additional future idea. Why we need push backup with fleecing? To handle cases with slow connection to backup target. In any case when writing to remote target is slower than writing to local file, push-backup-with-fleecing will less disturb the running guest than simple backup job. But this is not free: 1. We need additional disk space on source. No way to fix that (that's a core idea of the scheme, store data locally), still discard-source option for backup job will help 2. If connection is not too slow than probably part of CBW (copy before write) operations could go to final target immediately. But with the scheme above all CBW operations go to qcow2 temporary image. This can be solved with help of ram-cache format driver (to be implemented, of course): ┌─────────────────────┐ │ final backup target │ └─────────────────────┘ ▲ │ backup job (bitmap=bm0, insert-filter=False, discard-source=True) │ ┌────────────────────────────────────┐ ┌───────────────────────────────────────┐ backing ┌─────────────┐ │ discard-bitmap filter (bitmap=bm0) │ │ temp qcow2 img │ ─────────▶ │ active disk │ └────────────────────────────────────┘ └───────────────────────────────────────┘ └─────────────┘ │ ▲ ▲ │ │ file │ │ │ │ │ file ┌───────────┐ │ └───────────▶ │ ram-cache │ │ └───────────┘ │ ▲ │ │ target │ │ │ ┌────────────────────────────────────┐ ┌───────────────────────────────────────┐ file │ │ guest blk │ ──────▶ │ copy-before-write filter (bitmap=bm0) │ ─────────────┘ └────────────────────────────────────┘ └───────────────────────────────────────┘ This way data from copy-before-write filter goes first to ram-cache, and backup job could read it from ram. ram-cache will automatically flush data to temp qcow2 image, when ram-usage limit is reached. We'll also need a way to say backup-job that it should first read clusters that are cached in ram, and only then other clusters. So, we'll have a priory for clusters to be copied by block-copy: 1. clusters in ram-cache 2. clusters not in temp img (to avoid copy-before-write operations in future) 3. clusters in temp img. This will be a kind of block_status() thing, that allows a block driver to give recommendations on sequence of reading to be effective. Not also, that there is another benefit of such thing: we'll implement this callback in qcow2 driver, so that backup will read clusters not in guest cluster order, but in host cluster order, to read more sequentially, which should bring better performance on rotating disks. -- Best regards, Vladimir

3 8

[libvirt PATCH 0/4] adjust max memlock limit when hotplugging a vDPA device
by Laine Stump 25 May '21

25 May '21

see patch 4 for the why. Found during testing of https://bugzilla.redhat.com/1939776 Laine Stump (4): qemu_hotplug.c: don't skip cleanup on failures of qemuDomainAttachNetDevice conf: new function virDomainNetRemoveByObj() qemu_hotplug.c: add net devices to the domain list earlier qemu: adjust the maxmemlock limit when hotplugging a vDPA device src/conf/domain_conf.c | 16 ++++++++++++++++ src/conf/domain_conf.h | 1 + src/libvirt_private.syms | 1 + src/qemu/qemu_hotplug.c | 41 +++++++++++++++++++++++++++++++--------- 4 files changed, 50 insertions(+), 9 deletions(-) -- 2.31.1

2 5

[PATCH 0/3] capabilities: Turn @cpus in _virCapsHostNUMACell into GArray
by Michal Privoznik 24 May '21

24 May '21

Honestly, I don't really like GArray-s. I find plain C arrays better (smaller memory footprint, direct access to items, no obscure g_array_*() bugs, fewer lines of code, etc.). But I'm posting these to stir discussion. However, 1/3 is still proper patch worth merging. Michal Prívozník (3): libxl: Break down an if() in libxlCapsInitNuma() capabilities: Turn @cpus in _virCapsHostNUMACell into GArray virCapabilitiesHostNUMAInitReal: Drop cleanup label src/conf/capabilities.c | 145 +++++++++++++++++---------------- src/conf/capabilities.h | 11 +-- src/libvirt_private.syms | 2 +- src/libxl/libxl_capabilities.c | 34 ++++---- src/test/test_driver.c | 12 +-- tests/testutils.c | 23 +++--- 6 files changed, 121 insertions(+), 106 deletions(-) -- 2.26.3

2 4

Ceph RBD Connection
by Mr. Gecko 24 May '21

24 May '21

Hello, I setup a Ceph Cluster on my system in hopes of using it with libvirtd, however I'm finding myself unable to have libvirtd make the connection. [root@server ~]# virsh pool-start "${CEPH_POOL}" error: Failed to start pool libvirt-pool error: failed to connect to the RADOS monitor on: localhost:6789,: No such file or directory I know that rbd is working as I am able to use it with the rbd utility. [root@server ~]# rbd list libvirt-pool Kolab [root@server ~]# rbd device map libvirt-pool/Kolab /dev/rbd0 [root@server ~]# lsblk /dev/rbd0 NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT rbd0 253:0 0 20G 0 disk ├─rbd0p1 253:1 0 1G 0 part └─rbd0p2 253:2 0 19G 0 part The libvirtd logs are not providing any useful information on the issue, I tried changing log-level to 1 and enabling the log_filters/adding "1:rbd" to the log filters. I have tried both the libvirt version 7.3.0 in my repository and the bleeding edge with a compile from git. Both returns the same error. Any help in troubleshooting this issue? I'm somewhat new when it comes to all of this. Thank you, James Coleman

2 3

[PATCH] hw/arm/raspi: Remove deprecated raspi2/raspi3 aliases
by Philippe Mathieu-Daudé 24 May '21

24 May '21

Remove the raspi2/raspi3 machine aliases, deprecated since commit 155e1c82ed0. Signed-off-by: Philippe Mathieu-Daudé <f4bug(a)amsat.org> --- docs/system/deprecated.rst | 7 ------- docs/system/removed-features.rst | 7 +++++++ hw/arm/raspi.c | 2 -- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index 80cae862528..7895bd4d849 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -238,13 +238,6 @@ this CPU is also deprecated. System emulator machines ------------------------ -Raspberry Pi ``raspi2`` and ``raspi3`` machines (since 5.2) -''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' - -The Raspberry Pi machines come in various models (A, A+, B, B+). To be able -to distinguish which model QEMU is implementing, the ``raspi2`` and ``raspi3`` -machines have been renamed ``raspi2b`` and ``raspi3b``. - Device options -------------- diff --git a/docs/system/removed-features.rst b/docs/system/removed-features.rst index 29e90601a51..8a8b8ca627b 100644 --- a/docs/system/removed-features.rst +++ b/docs/system/removed-features.rst @@ -312,6 +312,13 @@ This machine has been renamed ``fuloong2e``. These machine types were very old and likely could not be used for live migration from old QEMU versions anymore. Use a newer machine type instead. +Raspberry Pi ``raspi2`` and ``raspi3`` machines (removed in 6.1) +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The Raspberry Pi machines come in various models (A, A+, B, B+). To be able +to distinguish which model QEMU is implementing, the ``raspi2`` and ``raspi3`` +machines have been renamed ``raspi2b`` and ``raspi3b``. + linux-user mode CPUs -------------------- diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index 990509d3852..20bba0316f1 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -342,7 +342,6 @@ static void raspi2b_machine_class_init(ObjectClass *oc, void *data) MachineClass *mc = MACHINE_CLASS(oc); RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); - mc->alias = "raspi2"; rmc->board_rev = 0xa21041; raspi_machine_class_common_init(mc, rmc->board_rev); }; @@ -362,7 +361,6 @@ static void raspi3b_machine_class_init(ObjectClass *oc, void *data) MachineClass *mc = MACHINE_CLASS(oc); RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc); - mc->alias = "raspi3"; rmc->board_rev = 0xa02082; raspi_machine_class_common_init(mc, rmc->board_rev); }; -- 2.26.3

3 4

[PATCH 0/4] conf: Deduplicate NUMA distance code
by Michal Privoznik 24 May '21

24 May '21

While the number of saved lines is not that great, it prepares the code for what I'm working on - exposing HMAT in capabilities XML. My idea is to reuse XML formatters there too so that mgmt app can just take whatever we produce in capabilities XML and paste it into domain XML. Michal Prívozník (4): capabilities: Rename siblings to distances numa_conf: Rename virDomainNumaDistance to virNumaDistance numa_conf: Expose virNumaDistance formatter conf: Deduplicate NUMA distance code src/conf/capabilities.c | 60 ++++++++++++++------------------ src/conf/capabilities.h | 13 +++---- src/conf/numa_conf.c | 63 ++++++++++++++++++---------------- src/conf/numa_conf.h | 10 ++++++ src/conf/virconftypes.h | 2 -- src/libvirt_private.syms | 1 + src/libxl/libxl_capabilities.c | 20 +++++------ 7 files changed, 83 insertions(+), 86 deletions(-) -- 2.26.3

3 12

[PATCH 00/35] qemu: Implement support for transient disk hotplug and backing store sharing
by Peter Krempa 24 May '21

24 May '21

This series refactors the qemu disk hotplug code so that it can be easily extended for hotplug of transient disks and then also used at startup to plug in disks where the users want to share the backing image. Masayoshi Mizuma (1): qemu_snapshot: Add the guest name to the transient disk path Peter Krempa (34): qemu: snapshot: Extract setup of snapshot disk definition for transient disks qemu: process: Setup transient disks only when starting a fresh VM qemuSnapshotDiskPrepareOne: Pass in qemuSnapshotDiskContext qemuSnapshotDiskContext: Store virQEMUDriverConfig in the struct qemuSnapshotDiskPrepareOne: Use data from qemuSnapshotDiskContext qemuSnapshotDiskCreate: Use 'cfg' from the qemuSnapshotDiskContext qemu: snapshot: move transient snapshot code to qemu_process qemu: Move 'bootindex' handling for disks out of command line formatter qemu: Move bootindex usage logic into qemuBuildDiskDeviceStr qemuxml2argvtest: Remove pointless tests for keywrapping on s390 qemu: Move iothread and s390 address validation for disk devices into the validator Replace virDomainDiskInsertPreAlloced by virDomainDiskInsert conf: remove virDomainDiskInsertPreAlloced qemuBlockStorageSourceChainData: Add handling of 'copy-on-read' filter layer qemuDomainAttachDiskGeneric: Move 'copy-on-read' handling to qemuBlockStorageSourceChainData qemuDomainRemoveDiskDevice: Move 'copy-on-read' handling to qemuBlockStorageSourceChainData qemuDomainAttachDeviceDiskLiveInternal: Absorb qemuDomainAttachUSBMassStorageDevice qemuDomainAttachDeviceDiskLiveInternal: Absorb qemuDomainAttachVirtioDiskDevice qemuDomainAttachDeviceDiskLiveInternal: Absorb qemuDomainAttachSCSIDisk qemuDomainAttachDeviceDiskLiveInternal: Simplify call to qemuDomainAttachDiskGeneric qemuDomainAttachDiskGeneric: Move setup of disk into qemuDomainAttachDeviceDiskLiveInternal qemu: hotplug: Move post-insertion steps of disk hotplug to qemuDomainAttachDeviceDiskLiveInternal qemuDomainAttachDiskGeneric: Fix whitespace qemuDomainAttachDiskGeneric: Refactor cleanup qemuDomainAttachDiskGeneric: Move PR helper attach into qemuDomainAttachDeviceDiskLiveInternal qemuDomainAttachDiskGeneric: Refactor rollback handling qemuDomainAttachDiskGeneric: Split up frontend and backend attachment qemu: Track creation of <transient> disk overlay individually qemuDomainAttachDiskGeneric: Implement hotplug of <transient> disk qemuDomainAttachDiskGeneric: Pass the qemu async job type qemuDomainAttachDiskGeneric: Export conf: Introduce 'shareBacking' for <transient> disks qemu: Allow <transient> disks with images shared accross VMs tests: Add qemuxml2argv and qemuxml2xml test for <transient shareBacking='yes'> docs/formatdomain.rst | 6 + docs/schemas/domaincommon.rng | 5 + src/conf/domain_conf.c | 33 +- src/conf/domain_conf.h | 3 +- src/libvirt_private.syms | 1 - src/libxl/libxl_driver.c | 4 +- src/lxc/lxc_driver.c | 6 +- src/qemu/qemu_block.c | 12 + src/qemu/qemu_block.h | 4 + src/qemu/qemu_command.c | 134 ++----- src/qemu/qemu_command.h | 1 - src/qemu/qemu_domain.c | 2 - src/qemu/qemu_domain.h | 11 +- src/qemu/qemu_hotplug.c | 335 +++++++++--------- src/qemu/qemu_hotplug.h | 6 + src/qemu/qemu_process.c | 203 ++++++++++- src/qemu/qemu_snapshot.c | 160 +++------ src/qemu/qemu_snapshot.h | 27 +- src/qemu/qemu_validate.c | 73 ++++ .../disk-transient.x86_64-4.1.0.err | 2 +- .../disk-transient.x86_64-latest.args | 4 +- tests/qemuxml2argvdata/disk-transient.xml | 6 + tests/qemuxml2argvtest.c | 8 - .../disk-transient.x86_64-latest.xml | 48 +++ tests/qemuxml2xmltest.c | 3 +- 25 files changed, 644 insertions(+), 453 deletions(-) create mode 100644 tests/qemuxml2xmloutdata/disk-transient.x86_64-latest.xml -- 2.31.1

3 72

[PATCH] kbase: Fix broken link of migration doc
by Han Han 24 May '21

24 May '21

Signed-off-by: Han Han <hhan(a)redhat.com> --- docs/kbase/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index d483ca94de..91083ee49d 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -66,4 +66,4 @@ Internals / Debugging `VM migration internals <migrationinternals.html>`__ VM migration implementation details, complementing the info in - `migration <migration.html>`__ + `migration <../migration.html>`__ -- 2.31.1

2 1

[libvirt PATCH v2 0/2] storage_driver: add support for cluster_size QCOW2 option
by Pavel Hrdina 22 May '21

22 May '21

Pavel Hrdina (2): storage: add support for QCOW2 cluster_size option storage_file: add support to probe cluster_size from QCOW2 images docs/formatstorage.html.in | 6 ++ docs/schemas/storagecommon.rng | 7 ++ docs/schemas/storagevol.rng | 3 + src/conf/storage_conf.c | 12 ++++ src/storage/storage_util.c | 8 +++ src/storage_file/storage_file_probe.c | 70 ++++++++++++++----- .../qcow2-clusterSize.argv | 6 ++ tests/storagevolxml2argvtest.c | 4 ++ .../vol-qcow2-clusterSize.xml | 17 +++++ .../vol-qcow2-clusterSize.xml | 17 +++++ tests/storagevolxml2xmltest.c | 1 + 11 files changed, 134 insertions(+), 17 deletions(-) create mode 100644 tests/storagevolxml2argvdata/qcow2-clusterSize.argv create mode 100644 tests/storagevolxml2xmlin/vol-qcow2-clusterSize.xml create mode 100644 tests/storagevolxml2xmlout/vol-qcow2-clusterSize.xml -- 2.31.1

3 6

Qemu block filter insertion/removal API
by Vladimir Sementsov-Ogievskiy 21 May '21

21 May '21

Hi all! I'd like to be sure that we know where we are going to. In blockdev-era where qemu user is aware about block nodes, all nodes have good names and controlled by user we can efficiently use block filters. We already have some useful filters: copy-on-read, throttling, compress. In my parallel series I make backup-top filter public and useful without backup block jobs. But now filters could be inserted only together with opening their child. We can specify filters in qemu cmdline, or filter can take place in the block node chain created by blockdev-add. Still, it would be good to insert/remove filters on demand. Currently we are going to use x-blockdev-reopen for this. Still it can't be used to insert a filter above root node (as x-blockdev-reopen can change only block node options and their children). In my series "[PATCH 00/21] block: publish backup-top filter" I propose (as Kevin suggested) to modify qom-set, so that it can set drive option of running device. That's not difficult, but it means that we have different scenario of inserting/removing filters: 1. filter above root node X: inserting: - do blockdev-add to add a filter (and specify X as its child) - do qom-set to set new filter as a rood node instead of X removing - do qom-set to make X a root node again - do blockdev-del to drop a filter 2. filter between two block nodes P and X. (For example, X is a backing child of P) inserting - do blockdev-add to add a filter (and specify X as its child) - do blockdev-reopen to set P.backing = filter remvoing - do blockdev-reopen to set P.backing = X - do blockdev-del to drop a filter And, probably we'll want transaction support for all these things. Is it OK? Or do we need some kind of additional blockdev-replace command, that can replace one node by another, so in both cases we will do inserting: - blockdev-add filter - blockdev-replace (make all parents of X to point to the new filter instead (except for the filter itself of course) removing - blockdev-replace (make all parante of filter to be parents of X instead) - blockdev-del filter It's simple to implement, and it seems for me that it is simpler to use. Any thoughts? -- Best regards, Vladimir

3 8