Before posting another version of my patches [1], attempting to add support for the new qcow format to libvirt, I would like to know if this sounds reasonable: A new format named 'qcow3' would be added, along with a <features> sub-element for target. <volume> <name>qcow3test</name> <source> </source> <capacity unit='GiB'>8</capacity> <target> <path>/var/lib/libvirt/images/qcow3test</path> <format type='qcow3'/> <features> <lazy_refcounts/> </features> </target> </volume> I think that libvirt shouldn't care if the features are compatible or incompatible, as we don't know what features are supported by the hypervisor. Would the features be any good as tri-state (on, off, default?). While the qcow3 format is handled by the qcow2 driver in QEMU, <driver name='qemu' type='qcow2'/> should be enough for domains,

but in snapshot XML we treat the driver type as the format: <disk name='/path/to/old'> <driver type='qcow3'/> <source file='/path/to/new'/> <features> <lazy_refcounts/> </features> </disk> So I think we should allow the qcow3 driver type as well and translate it to qcow2 for QEMU. Jan [1] v2 here: https://www.redhat.com/archives/libvir-list/2013-February/msg00212.html

Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben: > On Mon, Mar 04, 2013 at 01:58:12PM +0100, Ján Tomko wrote: > > Before posting another version of my patches [1], attempting to add > > support for the new qcow format to libvirt, I would like to know if this > > sounds reasonable: > > > > A new format named 'qcow3' would be added, along with a <features> > > sub-element for target. > > > > <volume> > > <name>qcow3test</name> > > <source> > > </source> > > <capacity unit='GiB'>8</capacity> > > <target> > > <path>/var/lib/libvirt/images/qcow3test</path> > > <format type='qcow3'/> > > <features> > > <lazy_refcounts/> > > </features> > > </target> > > </volume> > > > > I think that libvirt shouldn't care if the features are compatible or > > incompatible, as we don't know what features are supported by the > > hypervisor. Would the features be any good as tri-state (on, off, default?). > > > > While the qcow3 format is handled by the qcow2 driver in QEMU, > > <driver name='qemu' type='qcow2'/> should be enough for domains, > > We should use qcow3 everywhere IMHO, regardless of whether qcow2 > technically works in this context. I think it makes much more sense to deal with it the way qemu does instead of inventing new names. This has much more of an (incompatible) feature flag than of a different image format. So to fit it in your proposed syntax:

But I guess you call all VMDKs just "vmdk", despite the fact that they are really just a collection of different subformats. Right?

Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben: > On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote: > > Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben: > > I think it makes much more sense to deal with it the way qemu does > > instead of inventing new names. This has much more of an (incompatible) > > feature flag than of a different image format. So to fit it in your > > proposed syntax: > > The issue is that QEMU is not the only thing that implements the qcow > format. There are a number of other impls out there, and we can't just > assume that they will all be providing a qcow2 driver that automagically > opens a qcow3 image format. Just in the same way we didn't assume that > a 'qcow' (version 1) driver would open a version 2 image. That's true. Other implementation actually tend to have a 'qcow' driver that deals with both qcow1 and qcow2. But these two are actually different enough that calling them two different formats might be acceptable. In contrast, version 3 images share _exactly_ the same structure with version 2 images, the just have additional header fields and support some new flags in some structures (that were previously reserved). If you call this a different image format, then scratch that whole <feature> idea, because then each newly added feature is a new image format by your standards.

> It so happens that with QEMU if you specify format=qcow2 and give it > a qcow3 image, QEMU will open it, but libvirt can't assume that, since > this is a mere implementation detail. Hence libvirt must explicitly > refer to 'qcow3' in the XML and map it to qcow2 if applicable when > talking to QEMU. If you need this information, sure, save it. I'm just requesting that you don't abuse the format name for it.

> > But I guess you call all VMDKs just "vmdk", despite the fact that they > > are really just a collection of different subformats. Right? > > Yes, but that is really a bug in our representation of vmdk. How are you going to fix it? Do you think having ten different format names all starting with "vmdk" will make tools user friendly?

Am 04.03.2013 um 15:46 hat Daniel P. Berrange geschrieben: > On Mon, Mar 04, 2013 at 03:38:54PM +0100, Kevin Wolf wrote: > > Am 04.03.2013 um 15:27 hat Daniel P. Berrange geschrieben: > > > On Mon, Mar 04, 2013 at 03:04:53PM +0100, Kevin Wolf wrote: > > > > Am 04.03.2013 um 14:09 hat Daniel P. Berrange geschrieben: > > > > I think it makes much more sense to deal with it the way qemu does > > > > instead of inventing new names. This has much more of an (incompatible) > > > > feature flag than of a different image format. So to fit it in your > > > > proposed syntax: > > > > > > The issue is that QEMU is not the only thing that implements the qcow > > > format. There are a number of other impls out there, and we can't just > > > assume that they will all be providing a qcow2 driver that automagically > > > opens a qcow3 image format. Just in the same way we didn't assume that > > > a 'qcow' (version 1) driver would open a version 2 image. > > > > That's true. Other implementation actually tend to have a 'qcow' driver > > that deals with both qcow1 and qcow2. But these two are actually > > different enough that calling them two different formats might be > > acceptable. > > > > In contrast, version 3 images share _exactly_ the same structure with > > version 2 images, the just have additional header fields and support > > some new flags in some structures (that were previously reserved). > > > > If you call this a different image format, then scratch that whole > > <feature> idea, because then each newly added feature is a new image > > format by your standards. > > No, that's not what I'm saying. The version 3 image format introduces > the ability to set a variety of features in an extensible way. Adding > new features to that list doesn't mean the version has changed. Why does libvirt care whether a new feature is indicated by incrementing one header field or by setting a bit in a different header field? These are image format internals, not external interfaces. > > > It so happens that with QEMU if you specify format=qcow2 and give it > > > a qcow3 image, QEMU will open it, but libvirt can't assume that, since > > > this is a mere implementation detail. Hence libvirt must explicitly > > > refer to 'qcow3' in the XML and map it to qcow2 if applicable when > > > talking to QEMU. > > > > If you need this information, sure, save it. I'm just requesting that > > you don't abuse the format name for it. > > The key distinction is that libvirt XML is recording an generic image > format, while the QEMU cli args are referring to a specific driver > implementation, which are support multiple formats. Typically these > map 1-to-1, but there's no such requirement in general. Hence will > refer to 'qcow3' in all its XML descriptions, and map to 'qcow2' when > talking to QEMU, or even just to 'qcow' if talking to a different impl > which supports all 3 versions in one driver. I'm not talking about the QEMU cli, but about qcow2 as the format as defined in the spec (which just happens to sit in qemu.git, but isn't qemu specific at all)

Am 04.03.2013 um 16:19 hat Daniel P. Berrange geschrieben: > On Mon, Mar 04, 2013 at 04:05:50PM +0100, Kevin Wolf wrote: >> >> I'm not talking about the QEMU cli, but about qcow2 as the format as >> defined in the spec (which just happens to sit in qemu.git, but isn't >> qemu specific at all) > > So you're saying that you consider the format name to be "qcow2" regardless > of whether the version numer field is specified as 2 or 3. Yes. > So in other words, if an application came along and required libvirt to > set format=qcow3 on its CLI, we could justifiably refuse to do that in > libvirt claiming this is not in compliance with the spec ? No, you would just check which features this image uses (which, if I understood correctly, you need to save anyway), and if a version 3 feature is among it (the basic version 3 could be represented by either a "feature flags" or "zero clusters" feature, which are what version 3 really means), pass it the 'qcow3' command line option it wants.

Of course, I would be disappointed that the tool thought it had to invent format names, but it's not really blocking any functionality. Just the same way it could happen that a tool uses different drivers for other features that we introduce. For example, imagine that we introduce a flag that modifies the L2 table structure to allow subclusters - a change that we've been discussing before and that would have a massive impact on the implementation, even though it's only a feature flag that has changed, and not the version number. Using a different driver for this looks much more likely than a different driver for version 2 and 3, which was really a quite small step.

So the main problem that I see is the assumption "different version => big change, new feature flag => small change" and as a conclusion from that "different version => possibly new driver, new feature flag => definitely only old driver". This isn't true at all. Kevin