Dear list, while wiring qemu-ga into libvirt I've noticed that it has ability to spawn commands inside guest. I haven't paid much attention to it then as implementing libvirt <-> qemu-ga communication was more important. But lately couple of requests on the list showed up where ability to spawn various commands inside guests would be much appreciated (e.g. when fetching some stats that HV can't know or has no support for yet - free/df/..). When it comes to spawning commands we distinguish two modes: sync and async. I think we should stick to this on the public API level too. Yet better, we can have async APIs and the sync API would just be a wrapper around async then. One picture instead of thousand words: /* obtain conn and dom somehow */ const char argv[] = {"argv1", "argv2", "argv3", ...} size_t argc = ARRAY_CARDINALITY(argv); unsigned int flags = 0; virDomainCommandPtr cmd = virDomainCommandNew("/path/to/binary", argv, argc, flags); /* Should we care about env too? */ virStreamPtr st = virSreamNew(conn, VIR_STREAM_NONBLOCK); virDomainCommandRun(dom, cmd, st, VIR_COMMAND_STRING_IO); virStreamEventAddCallback(st, VIR_STREAM_EVENT_READABLE, domainCommandCallback, ...); /* ... */ virDomainCommandJoin(dom, cmd); virStreamEventRemoveCallback(con->st) virDomainCommandFree(cmd); While qemu-ga can handle input for a binary its executing, this must be put right into JSON when constructing the qemu-ga command. It's not common that by that time users know the input they want to enter. So I have to figure out something. Opinions?

On Mon, 2016-08-08 at 16:19 +0100, Daniel P. Berrange wrote: > IOW, I don't think we should ever expose the qemu guest agent exec command > via libvirt APIs. >  > If people want a general purpose exec facility, they can trivially write > a much better exec feature via a separate virtio-serial channel (or via > the new virtio-vsock), and stil access that over libvirt by using the > virDomainOpenChannel to connect to the host side of the channel if needed Personally, I feel like most of this is way out of scope for libvirt. If you want to monitor the disk / memory usage of your guest, just run the Nagios agent or whatever in it; if you want to run random commands, just use ssh.

These are not some specific requirements that are strongly tied to virtualization, but very generic requirements that can be addressed with existing generic software.

On Mon, Aug 08, 2016 at 05:00:38PM +0200, Michal Privoznik wrote: > Dear list, > > while wiring qemu-ga into libvirt I've noticed that it has ability to > spawn commands inside guest. I haven't paid much attention to it then as > implementing libvirt <-> qemu-ga communication was more important. But > lately couple of requests on the list showed up where ability to spawn > various commands inside guests would be much appreciated (e.g. when > fetching some stats that HV can't know or has no support for yet - > free/df/..). > I never really liked the qemu guest agent ability to run arbitrary commands. It is basically re-inventing the shell but with really awful features. eg the having to provide all the input upfront, not having any way to stream large stdout/stderr data back to the host. Further, from a security POV it is really bad practice to have this feature in QEMU guest agent, as it makes it impossible to provide any kind of sane security confinment for the GA. IIRC, default Fedora SELinux policy will not even permit the exec command to be to used. Most of the QEMU GA commands have very tight scope so are easily confined, but 'exec' by its very nature wants todo anything. From that POV, a general purpose exec facility is really better suited to a separate command.

Also from an API modelling POV, exposing the guest agent exec in libvirt is pretty much giving up on any sense of API design. It'll just discourage anyone from ever writing any further special case guest agent commands with formal APIs. IOW, I don't think we should ever expose the qemu guest agent exec command via libvirt APIs.

On Tue, Aug 09, 2016 at 05:57:44PM +0800, Chen Hanxiao wrote: > > At 2016-08-08 23:19:26, "Daniel P. Berrange" <berrange(a)redhat.com&gt; wrote: > >On Mon, Aug 08, 2016 at 05:00:38PM +0200, Michal Privoznik wrote: > >> Dear list, > >> > >> while wiring qemu-ga into libvirt I've noticed that it has ability to > >> spawn commands inside guest. I haven't paid much attention to it then as > >> implementing libvirt <-> qemu-ga communication was more important. But > >> lately couple of requests on the list showed up where ability to spawn > >> various commands inside guests would be much appreciated (e.g. when > >> fetching some stats that HV can't know or has no support for yet - > >> free/df/..). > >> > > > >I never really liked the qemu guest agent ability to run arbitrary commands. > >It is basically re-inventing the shell but with really awful features. eg the > >having to provide all the input upfront, not having any way to stream large > >stdout/stderr data back to the host. > > > >Further, from a security POV it is really bad practice to have this feature > >in QEMU guest agent, as it makes it impossible to provide any kind of sane > >security confinment for the GA. IIRC, default Fedora SELinux policy will not > >even permit the exec command to be to used. Most of the QEMU GA commands have > >very tight scope so are easily confined, but 'exec' by its very nature wants > >todo anything. From that POV, a general purpose exec facility is really better > >suited to a separate command. > > Users could ban it inside VM by blacklist. > Also, with default SELinux policy, qga could not do anything more. > So it's under control. No, it really isn't satisfactory, because it does not enable you to separate responsibilities. ie the set of clients you want to grant 'exec' access to, is not the same as the set of clients you want to give general QEMU guest agent access to. By serving both use cases with the same agent, once you allow 'exec' access for one use cases, you've not opened that avenue of attack to all users of qemu guest agent.

If a general purpose 'exec' feature is desired, it can be achieved by doing something as simple as running mingetty on a new well-defined virtio-serial port. eg org.qemu.exec, and configuring that with PAM to not require a password login. That gives the client app full shell job control, I/O streaming, etc, ie all the things that would be broken/impossible with trying to wrap the qemu guest agent exec feature into an API. > >Also from an API modelling POV, exposing the guest agent exec in libvirt > >is pretty much giving up on any sense of API design. It'll just discourage > >anyone from ever writing any further special case guest agent commands > >with formal APIs. > > > >IOW, I don't think we should ever expose the qemu guest agent exec command > >via libvirt APIs. > > We've had qemuAgentCommand. > So I think it's better for a new public API. Thats in the libvirt-qemu.h file, so not something which has the same long term support guarantee as formal public APIs. So existance of that API is not justification to add a general guest exec to the main libvirt API.

At 2016-08-08 23:00:38, "Michal Privoznik" <mprivozn(a)redhat.com&gt; wrote: >Dear list, > >while wiring qemu-ga into libvirt I've noticed that it has ability to >spawn commands inside guest. I haven't paid much attention to it then as >implementing libvirt <-> qemu-ga communication was more important. But >lately couple of requests on the list showed up where ability to spawn >various commands inside guests would be much appreciated (e.g. when >fetching some stats that HV can't know or has no support for yet - >free/df/..). > >When it comes to spawning commands we distinguish two modes: sync and >async. I think we should stick to this on the public API level too. Yet >better, we can have async APIs and the sync API would just be a wrapper >around async then. > Cool! We should had a API than using qemu monitor command. ... >virStreamEventAddCallback(st, VIR_STREAM_EVENT_READABLE, >domainCommandCallback, ...); > >/* ... */ > >virDomainCommandJoin(dom, cmd); > >virStreamEventRemoveCallback(con->st) > >virDomainCommandFree(cmd); > Do we had timeout mechanism? Some command may hang for a long time in guest.