Re: [libvirt] [Qemu-devel] Modern CPU models cannot be used with libvirt

On Thu, Mar 22, 2012 at 04:30:55PM +0200, Gleb Natapov wrote: > On Thu, Mar 22, 2012 at 10:31:21AM -0300, Eduardo Habkost wrote: > > On Thu, Mar 22, 2012 at 11:32:44AM +0200, Gleb Natapov wrote: > > > What does this mean? Will -nodefconfig disable loading of bios.bin, > > > option roms, keymaps? > > > > Correcting myself: loading of _config_ files on /usr/share. ROM images > > are opaque data to be presented to the guest somehow, just like a disk > > image or kernel binary. But maybe keymaps will become "configuration" > > someday, I really don't know. > > > Where do you draw the line between "opaque data" and configuration. CPU > models are also something that is present to a guest somehow. Just the fact that it's in a structured key=value format that Qemu itself will interpret before exposing something to the guest. Yes, it's a bit arbitrary. If we could make everything "configuration data", we would (or that's what I think Anthony is pushing for--I hope he will reply and clarify that).

> Are you > consider ROMs to be "opaque data" because they are binary and CPU models > to be config just because it is ascii file? Not just "ascii file", but structured (and relatively small) [section]key=value data. If BIOSes were not opaque binary code and could be converted to a small set of config sections and key=values just like cpudefs, one could argue that BIOSes could become configuration data, too.

> What if we pre-process CPU > models into binary for QEMU to read will it magically stop being > configuration? Doing the reverse (transforming simple [section]key=value data to a binary format) would just be silly and wouldn't gain us anything. The point here is that we (Qemu) seem to be moving towards a design where most things are structured "configuration data" that fits on a [section]key=value model, and Qemu just interprets that structured data to build a virtual machine.

(That's why I said that perhaps keymaps could become configuration someday. Because maybe they can be converted to a key=value model relatively easily)

> > > > > Doing this would make it impossible to deploy fixes to users if we evern > > > > > find out that the default configuration file had a serious bug. What if > > > > > a bug in our default configuration file has a serious security > > > > > implication? > > > > > > > > The answer to this is: if the broken templates/defaults are on > > > > /usr/share, it would be easy to deploy the fix. > > > > > > > > So, the compromise solution is: > > > > > > > > - We can move some configuration data (especially defaults/templates) > > > > to /usr/share (machine-types and CPU models could go there). This > > > > way we can easily deploy fixes to the defaults, if necessary. > > > > - To reuse Qemu models, or machine-types, and not define everything from > > > > scratch, libvirt will have to use something like: > > > > "-nodefconfig -readconfig /usr/share/qemu/cpu-models-x86.conf" > > > > > > > cpu-models-x86.conf is not a configuration file. It is hardware > > > description file. QEMU should not lose capability just because you run > > > it with -nodefconfig. -nodefconfig means that QEMU does not create > > > machine for you, but all parts needed to create a machine that would have > > > been created without -nodefconfig are still present. Not been able to > > > create Nehalem CPU after specifying -nodefconfig is the same as not been > > > able to create virtio-net i.e the bug. > > > > > > The current design direction Qemu seems to be following is different > > from that: hardware description is also considered "configuration" just > > like actual machine configuration. Anthony, please correct me if I am > > wrong. > That's a bug. Why trying to rationalize it now instead of fixing it. It's just a bug for you because you disagree with the current design. You can call it rationalization, yes; I am just accepting Anthony's proposal because it's equally good (to me) as what you are proposing. > It > was fixed in RHEL by the same person who introduced it in upstream in > the first place. He just forgot to send the fix upstream. Does bug that > is present for a long time is promoted to a feature? John didn't forget it, he knew that upstream could go in a different direction. The RHEL6 patch description has this: "Note this is intended as an interim work-around for rhel6.0. While the new location of the config file should remain the same, the mechanism to direct qemu to it will likely differ going forward."

> > > > (the item below is not something discussed on the call, just something I > > > > want to add) > > > > > > > > To make this work better, we can allow users (humans or machines) to > > > > "extend" CPU models on the config file, instead of having to define > > > > everything from scratch. So, on /etc (or on a libvirt-generated config) > > > > we could have something like: > > > > > > > > ============= > > > > [cpu] > > > > base_cpudef = Nehalem > > > > add_features = "vmx" > > > > ============= > > > > > > > > Then, as long as /usr/share/cpu-models-x86.conf is loaded, the user will > > > > be able to reuse the Nehalem CPU model provided by Qemu. > > > > > > > And if it will not be loaded? > > > > If it is not loaded, it is a configuration mistake. If you are reusing > > something defined somewhere, it would be your responsibility to make > > sure the file where the model is defined is present. On most cases you > > wouldn't use -nodefconfig and it would be shipped with Qemu and you > > shouldn't worry. If you used -nodefconfig, you load the CPU models file > > explicitly using -readconfig. > > > Regular user has no idea that we describe some cpu models in .c code and > others in config file and those that are described in config files are > disappear if -nodefconfig is used and that if they disappear they > should be loaded back and how exactly they should be loaded back. The > only user of -nodefconfig is libvirt and currently it does unexpected > things for its only user. And one could argue that this only user has wrong expectations and Qemu does it this way by design, and we would be running in circles forever.

The only problem for us is that if we keep running in circles, Qemu will keep the current design (or bug, if you want to call it), until Qemu maintainers agree to change the current behavior. I mean: you're right into arguing for it, but somehow I feel I am the wrong person to reply to your arguments, because IMO both approaches are valid. I really hope Anthony will reply to your points, too, or that we discuss that on the next Qemu call. Because if you convince only me, we would gain nothing if the patches implementing the design we agreed with get rejected. It's really a pity that your arguments weren't exposed last week during the Qemu call, when I and Anthony discussed this and agreed on the "compromise solution" I described.

> > > > > > > > > > > > But now when libvirt uses -nodefconfig, those models go away. > > > > > > -nodefconfig means start QEMU in the most minimal state possible. > > > > > > You get what you pay for if you use it. > > > > > > > > > > > > We'll have the same problem with machine configuration files. At > > > > > > some point in time, -nodefconfig will make machine models disappear. > > > > > > > > > > It shouldn't. Machine-types are defaults to be used as base, they are > > > > > not user-provided configuration. And the fact that we decided to store > > > > > some data outside of the Qemu binary is orthogonal the design decisions > > > > > in the Qemu command-line and configuration interface. > > > > > > > > So, this problem is solved if the defaults are easily found on > > > > /usr/share. > > > > > > > What problem is solved and why are we mixing machine configuration files > > > and cpu configuration files? They are different and should be treated > > > differently. > > > > This is the root of the disagreement, it seems: they are not considered > > different today. Today cpudefs are on a config file inside /etc. One > > may argue that this was a mistake in the first place, but that's the > > design we have today. > > > Machine configuration files do not exist today! Machine configuration is > done in .c code and -nodefconfig skips the code that does that, but it > does not skip the code that describes cpu models and resides in .c. So > as far as .c code is concerned machine creation and cpu description are > two different things, but for some reason the file with rest of cpu > model descriptions is not loaded. This is madness, not design. Let's try to agree on terminology here: what's "machine configuration"? I understand it as the configuration of a specific virtual machine for a specific Qemu instance (i.e. what we put on the command-line today), I am not talking about machine-types (yet ;). So, considering this definition, machine configuration exists today, there is stuff you can define on a configuration file, and you can give this configuration to -readconfig today. The config file is

unfortunately not as powerful as the command-line (some things can be set only on the command-line today), but it exists. Anthony even submitted a RFC series to make the config files as powerful as the command-line, by adding a [system] section. I don't agree completely with that specific implementation, but I agree with the direction it is taking. > > > > > -nodefconfig exists only because there is not machine > > > configuration files currently. With machine configuration files > > > libvirt does not need -nodefconfig because it can create its own machine > > > file and make QEMU use it. So specifying machine file on QEMU's command > > > line implies -nodefconfig. The option itself loses its meaning and can be > > > dropped. > > > > I think the approach today is: > There is not such thing as todays approach since there is not machine > config file today. There are config files today, and they work (but they are not as powerful as the command-line). But the interface is the one I describe below.

> > > > > - Qemu loads defaults from default config files; > > - Machine description files would be given using -readconfig and they > > would _augment_ the defaults from the default config files. > Jut make it possible to include one machine config file from another and > you do not need -nodefconfig again. That would be a valid approach too, but this is just not how the config files work today. We may change it, yes. But today the config file system is based on getting config data from multiple places (/etc/qemu/qemu.conf, extend it with /etc/qemu/target-x86_64.conf, and extend the result with the command-line arguments), instead of using includes.

> But this is completely orthogonal to > the discussion if cpu models are configuration or not. You are right. How the "machine configuration" is going to be loaded is orthogonal to the main question. The main question here is wheter cpudefs should be included on what we call "machine configuration" or not. (agreed?) -- Eduardo