Hi, all This is a basic implementation of libvirt Native Linux KVM Tool driver. Note that this is just made with my own interest and spare time, it's not an endorsement/effort by Red Hat, and it isn't supported by Red Hat officially. Basically, the driver is designed as *stateful*, as KVM tool doesn't maintain any info about the guest except a socket which for its own IPC. And it's implemented by using KVM tool binary, which is name "kvm" currently, along with cgroup controllers "cpuacct", and "memory" support. And as one of KVM tool's pricinple is to allow both the non-root and root user to play with. The driver is designed to support root and non-root too, just like QEMU does. Example of the connection URI: virsh -c kvmtool:///system virsh -c kvmtool:///session virsh -c kvmtool+unix:///system virsh -c kvmtool+unix:///session The implementation can support more or less than 15 virsh commands currently, including basic domain cycle operations (define/undefine, start/destroy, suspend/resume, console, setmem, schedinfo, dumpxml, ,autostart, dominfo, etc.) About the domain configuration: * "kernel": must be specified as KVM tool only support boots from the kernel currently (no integration with BIOS app yet). * "disk": only virtio bus is supported, and device type must be 'disk'. * "serial/console": only one console is supported, of type serial or virtio (can extend to support multiple console as long as kvm tool supports, libvirt already supported mutiple console, see upstream commit 0873b688c). * "p9fs": only support specifying the source dir, and mount tag, only type of 'mount' is supported. * "memballoon": only virtio is supported, and there is no way to config the addr. * Multiple "disk" and "p9fs" is supported. * Graphics and network are not supported, will explain below. Please see "[PATCH 7/8]" for an example of the domain config. (which contains all the XMLs supported by current implementation). The problems of Native Linux KVM Tool from libvirt p.o.v: * Some destros package "qemu-kvm" as "kvm", also "kvm" is a long established name for "KVM" itself, so naming the project as "kvm" might be not a good idea. I assume it will be named as "kvmtool" in this implementation, never mind this if you don't like that, it can be updated easily. :-) * It still doesn't have an official package yet, even no "make install". means we have no way to check the dependancy and do the checking when 'configure'. I assume it will be installed as "/usr/bin/kvmtool" in this implementation. This is the main reason which can prevents upstream libvirt accepting the patches I guess. * Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't. * KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network. * There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running. * KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user. * kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package. * console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does? * Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly. * It will be quite desireable if kvm tool could report the capabilities, of which is lacked by QEMU forever (seems so), it causes much pain for upper layer mgmt apps. See the RFC by Daniel Berrange in QEMU list for more details: http://lists.nongnu.org/archive/html/qemu-devel/2010-06/msg00921.html As a conclusion, it seems to me that KVM tool dosesn't consider too much about the interface for upper layer management tools, (generally, upper layer apps will want good interface to set/get), which might be no good for KVM tool future development. Thoughts, and opinions? Thanks. [PATCH 1/7] kvmtool: Add configure support [PATCH] kvm tools: Introduce an ENV variable for the state dir [PATCH 2/7] kvmtool: Add documents [PATCH 3/7] kvmtool: Add new enums and error codes for the driver [PATCH 4/7] kvmtool: Add hook support for kvmtool domain [PATCH 5/7] kvmtool: Add new domain type [PATCH 6/7] conf: Set source type of the stub console [PATCH 7/7] kvmtool: Implementation for kvm tool driver Regards, Osier
Sorry, I missed the title. :( Osier On 11/11/2011 07:56 PM, Osier Yang wrote:
Hi, all
This is a basic implementation of libvirt Native Linux KVM Tool driver. Note that this is just made with my own interest and spare time, it's not an endorsement/effort by Red Hat, and it isn't supported by Red Hat officially.
Basically, the driver is designed as *stateful*, as KVM tool doesn't maintain any info about the guest except a socket which for its own IPC. And it's implemented by using KVM tool binary, which is name "kvm" currently, along with cgroup controllers "cpuacct", and "memory" support. And as one of KVM tool's pricinple is to allow both the non-root and root user to play with. The driver is designed to support root and non-root too, just like QEMU does. Example of the connection URI:
virsh -c kvmtool:///system virsh -c kvmtool:///session virsh -c kvmtool+unix:///system virsh -c kvmtool+unix:///session
The implementation can support more or less than 15 virsh commands currently, including basic domain cycle operations (define/undefine, start/destroy, suspend/resume, console, setmem, schedinfo, dumpxml, ,autostart, dominfo, etc.)
About the domain configuration: * "kernel": must be specified as KVM tool only support boots from the kernel currently (no integration with BIOS app yet).
* "disk": only virtio bus is supported, and device type must be 'disk'.
* "serial/console": only one console is supported, of type serial or virtio (can extend to support multiple console as long as kvm tool supports, libvirt already supported mutiple console, see upstream commit 0873b688c).
* "p9fs": only support specifying the source dir, and mount tag, only type of 'mount' is supported.
* "memballoon": only virtio is supported, and there is no way to config the addr.
* Multiple "disk" and "p9fs" is supported.
* Graphics and network are not supported, will explain below.
Please see "[PATCH 7/8]" for an example of the domain config. (which contains all the XMLs supported by current implementation).
The problems of Native Linux KVM Tool from libvirt p.o.v:
* Some destros package "qemu-kvm" as "kvm", also "kvm" is a long established name for "KVM" itself, so naming the project as "kvm" might be not a good idea. I assume it will be named as "kvmtool" in this implementation, never mind this if you don't like that, it can be updated easily. :-)
* It still doesn't have an official package yet, even no "make install". means we have no way to check the dependancy and do the checking when 'configure'. I assume it will be installed as "/usr/bin/kvmtool" in this implementation. This is the main reason which can prevents upstream libvirt accepting the patches I guess.
* Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't.
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
* There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running.
* KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user.
* kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package.
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
* It will be quite desireable if kvm tool could report the capabilities, of which is lacked by QEMU forever (seems so), it causes much pain for upper layer mgmt apps. See the RFC by Daniel Berrange in QEMU list for more details:
http://lists.nongnu.org/archive/html/qemu-devel/2010-06/msg00921.html
As a conclusion, it seems to me that KVM tool dosesn't consider too much about the interface for upper layer management tools, (generally, upper layer apps will want good interface to set/get), which might be no good for KVM tool future development.
Thoughts, and opinions? Thanks.
[PATCH 1/7] kvmtool: Add configure support [PATCH] kvm tools: Introduce an ENV variable for the state dir [PATCH 2/7] kvmtool: Add documents [PATCH 3/7] kvmtool: Add new enums and error codes for the driver [PATCH 4/7] kvmtool: Add hook support for kvmtool domain [PATCH 5/7] kvmtool: Add new domain type [PATCH 6/7] conf: Set source type of the stub console [PATCH 7/7] kvmtool: Implementation for kvm tool driver
Regards, Osier
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Assumes the binary of "Native Linux KVM Tool" will be named as "kvmtool", and installed to "/usr/bin". --- configure.ac | 24 ++++++++++++++++++++++++ 1 files changed, 24 insertions(+), 0 deletions(-) diff --git a/configure.ac b/configure.ac index 3b7535e..bcb2a40 100644 --- a/configure.ac +++ b/configure.ac @@ -299,6 +299,8 @@ AC_ARG_WITH([esx], AC_HELP_STRING([--with-esx], [add ESX support @<:@default=check@:>@]),[],[with_esx=check]) AC_ARG_WITH([hyperv], AC_HELP_STRING([--with-hyperv], [add Hyper-V support @<:@default=check@:>@]),[],[with_hyperv=check]) +AC_ARG_WITH([kvmtool], + AC_HELP_STRING([--with-kvmtool], [add kvm tool driver support @<:@default=yes@:>@]),[],[with_kvmtool=check]) AC_ARG_WITH([test], AC_HELP_STRING([--with-test], [add test driver support @<:@default=yes@:>@]),[],[with_test=yes]) AC_ARG_WITH([remote], @@ -485,6 +487,27 @@ if test "$with_qemu" = "yes" ; then fi AM_CONDITIONAL([WITH_QEMU], [test "$with_qemu" = "yes"]) +dnl +dnl Check if kvm tool binary available (Native Linux KVM Tool driver) +dnl +if test "$with_kvmtool" = "yes" || test "$with_kvmtool" = "check"; then + AC_PATH_PROG([KVMTOOL], [kvmtool], [], [$PATH:/sbin:/usr/sbin]) + + if test "$with_kvmtool" = "yes"; then + if test -z "$KVMTOOL"; then AC_MSG_ERROR([kvm tool binary is required for kvm tool driver]); fi + else + if test -z "$KVMTOOL"; then with_kvmtool=no; fi + + if test "$with_kvmtool" = "check"; then with_kvmtool=yes; fi + fi + + if test "$with_kvmtool" = "yes"; then + AC_DEFINE_UNQUOTED([WITH_KVMTOOL], 1, [whether kvm tool driver is enabled]) + AC_DEFINE_UNQUOTED([KVMTOOL],["$KVMTOOL"],[Location of kvm tool binary]) + fi +fi +AM_CONDITIONAL([WITH_KVMTOOL], [test "$with_kvmtool" = "yes"]) + if test "$with_test" = "yes" ; then AC_DEFINE_UNQUOTED([WITH_TEST], 1, [whether Test driver is enabled]) fi @@ -2509,6 +2532,7 @@ AC_MSG_NOTICE([ LXC: $with_lxc]) AC_MSG_NOTICE([ PHYP: $with_phyp]) AC_MSG_NOTICE([ ESX: $with_esx]) AC_MSG_NOTICE([ Hyper-V: $with_hyperv]) +AC_MSG_NOTICE([ KVMTOOL: $with_kvmtool]) AC_MSG_NOTICE([ Test: $with_test]) AC_MSG_NOTICE([ Remote: $with_remote]) AC_MSG_NOTICE([ Network: $with_network]) -- 1.7.6
Which is named as "KVMTOOL_STATE_DIR", so that the user can configure the path of state directly as he wants. --- tools/kvm/main.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/tools/kvm/main.c b/tools/kvm/main.c index 05bc82c..37b2b1d 100644 --- a/tools/kvm/main.c +++ b/tools/kvm/main.c @@ -13,7 +13,12 @@ static int handle_kvm_command(int argc, char **argv) int main(int argc, char *argv[]) { - kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH); + char *state_dir = getenv("KVMTOOL_STATE_DIR"); + + if (state_dir) + kvm__set_dir("%s", state_dir); + else + kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH); return handle_kvm_command(argc - 1, &argv[1]); } -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:00PM +0800, Osier Yang wrote:
Which is named as "KVMTOOL_STATE_DIR", so that the user can configure the path of state directly as he wants. --- tools/kvm/main.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/tools/kvm/main.c b/tools/kvm/main.c index 05bc82c..37b2b1d 100644 --- a/tools/kvm/main.c +++ b/tools/kvm/main.c @@ -13,7 +13,12 @@ static int handle_kvm_command(int argc, char **argv)
int main(int argc, char *argv[]) { - kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH); + char *state_dir = getenv("KVMTOOL_STATE_DIR"); + + if (state_dir) + kvm__set_dir("%s", state_dir); + else + kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH);
return handle_kvm_command(argc - 1, &argv[1]); }
As per my comments in the first patch, I don't think this is critical for libvirt's needs. We should just honour the default location that the KVM tool uses, rather than forcing a libvirt specific location. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2011年12月06日 22:39, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:57:00PM +0800, Osier Yang wrote:
Which is named as "KVMTOOL_STATE_DIR", so that the user can configure the path of state directly as he wants. --- tools/kvm/main.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/tools/kvm/main.c b/tools/kvm/main.c index 05bc82c..37b2b1d 100644 --- a/tools/kvm/main.c +++ b/tools/kvm/main.c @@ -13,7 +13,12 @@ static int handle_kvm_command(int argc, char **argv)
int main(int argc, char *argv[]) { - kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH); + char *state_dir = getenv("KVMTOOL_STATE_DIR"); + + if (state_dir) + kvm__set_dir("%s", state_dir); + else + kvm__set_dir("%s/%s", HOME_DIR, KVM_PID_FILE_PATH);
return handle_kvm_command(argc - 1,&argv[1]); }
As per my comments in the first patch, I don't think this is critical for libvirt's needs. We should just honour the default location that the KVM tool uses, rather than forcing a libvirt specific location.
Thanks for the guy who pushed the patch, it's already in kvmtool's source. It will be useful when we run the kvmtool process as non-priviledge user and group. Regards, Osier
The document is rather rough now, but at least contains an domain config example of all the current supported XMLs, and tells how to play with the driver. --- docs/drivers.html.in | 1 + docs/drvkvmtool.html.in | 87 +++++++++++++++++++++++++++++++++++++++++++++++ docs/index.html.in | 3 ++ docs/sitemap.html.in | 4 ++ src/README | 3 +- 5 files changed, 97 insertions(+), 1 deletions(-) create mode 100644 docs/drvkvmtool.html.in diff --git a/docs/drivers.html.in b/docs/drivers.html.in index 75038fc..249c137 100644 --- a/docs/drivers.html.in +++ b/docs/drivers.html.in @@ -29,6 +29,7 @@ <li><strong><a href="drvvmware.html">VMware Workstation/Player</a></strong></li> <li><strong><a href="drvxen.html">Xen</a></strong></li> <li><strong><a href="drvhyperv.html">Microsoft Hyper-V</a></strong></li> + <li><strong><a href="drvkvmtool.html">Native Linux KVM Tool</a></strong></li> </ul> <h2><a name="stroage">Storage drivers</a></h2> diff --git a/docs/drvkvmtool.html.in b/docs/drvkvmtool.html.in new file mode 100644 index 0000000..1b6acdf --- /dev/null +++ b/docs/drvkvmtool.html.in @@ -0,0 +1,87 @@ +<html> + <body> + <h1>KVM tool driver</h1> + + <ul id="toc"></ul> + + <p> + The libvirt KVMTOOL driver manages hypervisor Native Linux KVM Tool, + it's implemented by using command line of kvm tool binary. + </p> + + <h2><a name="project">Project Links</a></h2> + + <ul> + <li> + The <a href="git://github.com/penberg/linux-kvm.git">Native Linux KVM Tool</a> Native + Linux KVM Tool + </li> + </ul> + + <h2><a name="uris">Connections to the KVMTOOL driver</a></h2> + <p> + The libvirt KVMTOOL driver is a multi-instance driver, providing a single + system wide privileged driver (the "system" instance), and per-user + unprivileged drivers (the "session" instance). The URI driver protocol + is "kvmtool". Some example conection URIs for the libvirt driver are: + </p> + + <pre> + kvmtool:///session (local access to per-user instance) + kvmtool+unix:///session (local access to per-user instance) + + kvmtool:///system (local access to system instance) + kvmtool+unix:///system (local access to system instance) + </pre> + <p> + cgroups controllers "cpuacct", and "memory" are supported currently. + </p> + + <h3>Example config</h3> + + <pre> +<domain type='kvmtool' id='1'> + <name>vm</name> + <uuid>88bf38f1-b6ab-cfa6-ab53-4b4c0993d894</uuid> + <memory>524288</memory> + <currentMemory>524288</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='x86_64'>hvm</type> + <kernel>/var/lib/libvirt/images/bzImage</kernel> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/kvmtool</emulator> + <disk type='file' device='disk'> + <source file='/var/lib/libvirt/images/linux-0.2.img'/> + <target dev='vda' bus='virtio'/> + </disk> + <filesystem type='mount' accessmode='passthrough'> + <source dir='/tmp'/> + <target dir='/mnt'/> + </filesystem> + <console type='pty'/> + <target type='virtio' port='0'/> + </console> + <memballoon model='virtio'/> + </devices> +</domain> + </pre> + + <p> + virsh command examples: + </p> + <pre> + virsh --connect kvmtool:///system define vm.xml + virsh --connect kvmtool:///system start vm + virsh --connect kvmtool:///system destroy vm + virsh --connect kvmtool:///system undefine vm + virsh --connect kvmtool:///system console vm + </pre> + </body> +</html> diff --git a/docs/index.html.in b/docs/index.html.in index c84eb1f..fcb7b1d 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -63,6 +63,9 @@ The <a href="http://libvirt.org/drvhyperv.html">Microsoft Hyper-V</a> hypervisor </li> <li> + The <a href="http://libvirt.org/drvkvmtool.html">Native Linux KVM Tool</a> hypervisor + </li> + <li> Virtual networks using bridging, NAT, VEPA and VN-LINK. </li> <li> diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in index 1de2b20..0816190 100644 --- a/docs/sitemap.html.in +++ b/docs/sitemap.html.in @@ -210,6 +210,10 @@ <a href="drvhyperv.html">Microsoft Hyper-V</a> <span>Driver for Microsoft Hyper-V</span> </li> + <li> + <a href="drvkvmtool.html">Native Linux KVM Tool</a> + <span>Driver for Native Linux KVM Tool</span> + </li> </ul> </li> <li> diff --git a/src/README b/src/README index 00d11d1..de1720b 100644 --- a/src/README +++ b/src/README @@ -27,6 +27,7 @@ Then there are the hypervisor implementations: * esx/ - VMware ESX and GSX support using vSphere API over SOAP * hyperv/ - Microsoft Hyper-V support using WinRM + * kvm/ - Native Linux KVM Tool support using CLI * lxc/ - Linux Native Containers * openvz/ - OpenVZ containers using cli tools * phyp/ - IBM Power Hypervisor using CLI tools over SSH @@ -41,7 +42,7 @@ Then there are the hypervisor implementations: Finally some secondary drivers that are shared for several HVs. -Currently these are used by LXC, OpenVZ, QEMU, UML and Xen drivers. +Currently these are used by LXC, OpenVZ, QEMU, UML, KVMTOOL, and Xen drivers. The ESX, Hyper-V, Power Hypervisor, Remote, Test & VirtualBox drivers all implement the secondary drivers directly -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:01PM +0800, Osier Yang wrote:
The document is rather rough now, but at least contains an domain config example of all the current supported XMLs, and tells how to play with the driver. --- docs/drivers.html.in | 1 + docs/drvkvmtool.html.in | 87 +++++++++++++++++++++++++++++++++++++++++++++++ docs/index.html.in | 3 ++ docs/sitemap.html.in | 4 ++ src/README | 3 +- 5 files changed, 97 insertions(+), 1 deletions(-) create mode 100644 docs/drvkvmtool.html.in
diff --git a/docs/drivers.html.in b/docs/drivers.html.in index 75038fc..249c137 100644 --- a/docs/drivers.html.in +++ b/docs/drivers.html.in @@ -29,6 +29,7 @@ <li><strong><a href="drvvmware.html">VMware Workstation/Player</a></strong></li> <li><strong><a href="drvxen.html">Xen</a></strong></li> <li><strong><a href="drvhyperv.html">Microsoft Hyper-V</a></strong></li> + <li><strong><a href="drvkvmtool.html">Native Linux KVM Tool</a></strong></li> </ul>
<h2><a name="stroage">Storage drivers</a></h2> diff --git a/docs/drvkvmtool.html.in b/docs/drvkvmtool.html.in new file mode 100644 index 0000000..1b6acdf --- /dev/null +++ b/docs/drvkvmtool.html.in @@ -0,0 +1,87 @@ +<html> + <body> + <h1>KVM tool driver</h1> + + <ul id="toc"></ul> + + <p> + The libvirt KVMTOOL driver manages hypervisor Native Linux KVM Tool, + it's implemented by using command line of kvm tool binary. + </p> + + <h2><a name="project">Project Links</a></h2> + + <ul> + <li> + The <a href="git://github.com/penberg/linux-kvm.git">Native Linux KVM Tool</a> Native + Linux KVM Tool + </li> + </ul> + + <h2><a name="uris">Connections to the KVMTOOL driver</a></h2> + <p> + The libvirt KVMTOOL driver is a multi-instance driver, providing a single + system wide privileged driver (the "system" instance), and per-user + unprivileged drivers (the "session" instance). The URI driver protocol + is "kvmtool". Some example conection URIs for the libvirt driver are: + </p> + + <pre> + kvmtool:///session (local access to per-user instance) + kvmtool+unix:///session (local access to per-user instance) + + kvmtool:///system (local access to system instance) + kvmtool+unix:///system (local access to system instance) + </pre> + <p> + cgroups controllers "cpuacct", and "memory" are supported currently. + </p> + + <h3>Example config</h3> + + <pre> +<domain type='kvmtool' id='1'>
As mentioned in a later patch, we should just use type='kvm' here still Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
--- include/libvirt/virterror.h | 1 + src/driver.h | 1 + src/util/virterror.c | 3 +++ 3 files changed, 5 insertions(+), 0 deletions(-) diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index a8549b7..deda42d 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -84,6 +84,7 @@ typedef enum { VIR_FROM_LIBXL = 41, /* Error from libxenlight driver */ VIR_FROM_LOCKING = 42, /* Error from lock manager */ VIR_FROM_HYPERV = 43, /* Error from Hyper-V driver */ + VIR_FROM_KVMTOOL = 44, /* Error from kvm tool driver */ } virErrorDomain; diff --git a/src/driver.h b/src/driver.h index 4c14aaa..158a13c 100644 --- a/src/driver.h +++ b/src/driver.h @@ -30,6 +30,7 @@ typedef enum { VIR_DRV_VMWARE = 13, VIR_DRV_LIBXL = 14, VIR_DRV_HYPERV = 15, + VIR_DRV_KVMTOOL = 16, } virDrvNo; diff --git a/src/util/virterror.c b/src/util/virterror.c index 5006fa2..abb5b5a 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -175,6 +175,9 @@ static const char *virErrorDomainName(virErrorDomain domain) { case VIR_FROM_HYPERV: dom = "Hyper-V "; break; + case VIR_FROM_KVMTOOL: + dom = "KVMTOOL "; + break; } return(dom); } -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:02PM +0800, Osier Yang wrote:
--- include/libvirt/virterror.h | 1 + src/driver.h | 1 + src/util/virterror.c | 3 +++ 3 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index a8549b7..deda42d 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -84,6 +84,7 @@ typedef enum { VIR_FROM_LIBXL = 41, /* Error from libxenlight driver */ VIR_FROM_LOCKING = 42, /* Error from lock manager */ VIR_FROM_HYPERV = 43, /* Error from Hyper-V driver */ + VIR_FROM_KVMTOOL = 44, /* Error from kvm tool driver */ } virErrorDomain;
diff --git a/src/driver.h b/src/driver.h index 4c14aaa..158a13c 100644 --- a/src/driver.h +++ b/src/driver.h @@ -30,6 +30,7 @@ typedef enum { VIR_DRV_VMWARE = 13, VIR_DRV_LIBXL = 14, VIR_DRV_HYPERV = 15, + VIR_DRV_KVMTOOL = 16, } virDrvNo;
diff --git a/src/util/virterror.c b/src/util/virterror.c index 5006fa2..abb5b5a 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -175,6 +175,9 @@ static const char *virErrorDomainName(virErrorDomain domain) { case VIR_FROM_HYPERV: dom = "Hyper-V "; break; + case VIR_FROM_KVMTOOL: + dom = "KVMTOOL "; + break; } return(dom); }
Trivial, ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Just like QEMU and LXC, kvm driver intends to support running hook script before domain starting and after domain shutdown too. --- src/util/hooks.c | 11 ++++++++++- src/util/hooks.h | 8 ++++++++ 2 files changed, 18 insertions(+), 1 deletions(-) diff --git a/src/util/hooks.c b/src/util/hooks.c index 110a94b..765cb68 100644 --- a/src/util/hooks.c +++ b/src/util/hooks.c @@ -52,12 +52,14 @@ VIR_ENUM_DECL(virHookDaemonOp) VIR_ENUM_DECL(virHookSubop) VIR_ENUM_DECL(virHookQemuOp) VIR_ENUM_DECL(virHookLxcOp) +VIR_ENUM_DECL(virHookKvmToolOp) VIR_ENUM_IMPL(virHookDriver, VIR_HOOK_DRIVER_LAST, "daemon", "qemu", - "lxc") + "lxc", + "kvmtool") VIR_ENUM_IMPL(virHookDaemonOp, VIR_HOOK_DAEMON_OP_LAST, "start", @@ -79,6 +81,10 @@ VIR_ENUM_IMPL(virHookLxcOp, VIR_HOOK_LXC_OP_LAST, "start", "stopped") +VIR_ENUM_IMPL(virHookKvmToolOp, VIR_HOOK_KVMTOOL_OP_LAST, + "start", + "stopped") + static int virHooksFound = -1; /** @@ -230,6 +236,9 @@ virHookCall(int driver, const char *id, int op, int sub_op, const char *extra, case VIR_HOOK_DRIVER_LXC: opstr = virHookLxcOpTypeToString(op); break; + case VIR_HOOK_DRIVER_KVMTOOL: + opstr = virHookKvmToolOpTypeToString(op); + break; } if (opstr == NULL) { virHookReportError(VIR_ERR_INTERNAL_ERROR, diff --git a/src/util/hooks.h b/src/util/hooks.h index fd7411c..69081c4 100644 --- a/src/util/hooks.h +++ b/src/util/hooks.h @@ -31,6 +31,7 @@ enum virHookDriverType { VIR_HOOK_DRIVER_DAEMON = 0, /* Daemon related events */ VIR_HOOK_DRIVER_QEMU, /* QEmu domains related events */ VIR_HOOK_DRIVER_LXC, /* LXC domains related events */ + VIR_HOOK_DRIVER_KVMTOOL, /* KVMTOOL domains related events */ VIR_HOOK_DRIVER_LAST, }; @@ -67,6 +68,13 @@ enum virHookLxcOpType { VIR_HOOK_LXC_OP_LAST, }; +enum virHookKvmToolOpType { + VIR_HOOK_KVMTOOL_OP_START, /* domain is about to start */ + VIR_HOOK_KVMTOOL_OP_STOPPED, /* domain has stopped */ + + VIR_HOOK_KVMTOOL_OP_LAST, +}; + int virHookInitialize(void); int virHookPresent(int driver); -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:03PM +0800, Osier Yang wrote:
Just like QEMU and LXC, kvm driver intends to support running hook script before domain starting and after domain shutdown too. --- src/util/hooks.c | 11 ++++++++++- src/util/hooks.h | 8 ++++++++ 2 files changed, 18 insertions(+), 1 deletions(-)
diff --git a/src/util/hooks.c b/src/util/hooks.c index 110a94b..765cb68 100644 --- a/src/util/hooks.c +++ b/src/util/hooks.c @@ -52,12 +52,14 @@ VIR_ENUM_DECL(virHookDaemonOp) VIR_ENUM_DECL(virHookSubop) VIR_ENUM_DECL(virHookQemuOp) VIR_ENUM_DECL(virHookLxcOp) +VIR_ENUM_DECL(virHookKvmToolOp)
VIR_ENUM_IMPL(virHookDriver, VIR_HOOK_DRIVER_LAST, "daemon", "qemu", - "lxc") + "lxc", + "kvmtool")
VIR_ENUM_IMPL(virHookDaemonOp, VIR_HOOK_DAEMON_OP_LAST, "start", @@ -79,6 +81,10 @@ VIR_ENUM_IMPL(virHookLxcOp, VIR_HOOK_LXC_OP_LAST, "start", "stopped")
+VIR_ENUM_IMPL(virHookKvmToolOp, VIR_HOOK_KVMTOOL_OP_LAST, + "start", + "stopped") + static int virHooksFound = -1;
/** @@ -230,6 +236,9 @@ virHookCall(int driver, const char *id, int op, int sub_op, const char *extra, case VIR_HOOK_DRIVER_LXC: opstr = virHookLxcOpTypeToString(op); break; + case VIR_HOOK_DRIVER_KVMTOOL: + opstr = virHookKvmToolOpTypeToString(op); + break; } if (opstr == NULL) { virHookReportError(VIR_ERR_INTERNAL_ERROR, diff --git a/src/util/hooks.h b/src/util/hooks.h index fd7411c..69081c4 100644 --- a/src/util/hooks.h +++ b/src/util/hooks.h @@ -31,6 +31,7 @@ enum virHookDriverType { VIR_HOOK_DRIVER_DAEMON = 0, /* Daemon related events */ VIR_HOOK_DRIVER_QEMU, /* QEmu domains related events */ VIR_HOOK_DRIVER_LXC, /* LXC domains related events */ + VIR_HOOK_DRIVER_KVMTOOL, /* KVMTOOL domains related events */
VIR_HOOK_DRIVER_LAST, }; @@ -67,6 +68,13 @@ enum virHookLxcOpType { VIR_HOOK_LXC_OP_LAST, };
+enum virHookKvmToolOpType { + VIR_HOOK_KVMTOOL_OP_START, /* domain is about to start */ + VIR_HOOK_KVMTOOL_OP_STOPPED, /* domain has stopped */ + + VIR_HOOK_KVMTOOL_OP_LAST, +}; + int virHookInitialize(void);
int virHookPresent(int driver);
Trivial, ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
It's named as "kvmtool". --- src/conf/domain_conf.c | 4 +++- src/conf/domain_conf.h | 1 + 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 58f4d0f..55121d8 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -91,7 +91,8 @@ VIR_ENUM_IMPL(virDomainVirt, VIR_DOMAIN_VIRT_LAST, "hyperv", "vbox", "one", - "phyp") + "phyp", + "kvmtool") VIR_ENUM_IMPL(virDomainBoot, VIR_DOMAIN_BOOT_LAST, "fd", @@ -4018,6 +4019,7 @@ virDomainChrDefParseXML(virCapsPtr caps, if (type == NULL) { def->source.type = VIR_DOMAIN_CHR_TYPE_PTY; } else if ((def->source.type = virDomainChrTypeFromString(type)) < 0) { + VIR_WARN("type = %s", type); virDomainReportError(VIR_ERR_XML_ERROR, _("unknown type presented to host for character device: %s"), type); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a3cb834..001bc46 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -59,6 +59,7 @@ enum virDomainVirtType { VIR_DOMAIN_VIRT_VBOX, VIR_DOMAIN_VIRT_ONE, VIR_DOMAIN_VIRT_PHYP, + VIR_DOMAIN_VIRT_KVMTOOL, VIR_DOMAIN_VIRT_LAST, }; -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:04PM +0800, Osier Yang wrote:
It's named as "kvmtool". --- src/conf/domain_conf.c | 4 +++- src/conf/domain_conf.h | 1 + 2 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 58f4d0f..55121d8 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -91,7 +91,8 @@ VIR_ENUM_IMPL(virDomainVirt, VIR_DOMAIN_VIRT_LAST, "hyperv", "vbox", "one", - "phyp") + "phyp", + "kvmtool")
VIR_ENUM_IMPL(virDomainBoot, VIR_DOMAIN_BOOT_LAST, "fd", @@ -4018,6 +4019,7 @@ virDomainChrDefParseXML(virCapsPtr caps, if (type == NULL) { def->source.type = VIR_DOMAIN_CHR_TYPE_PTY; } else if ((def->source.type = virDomainChrTypeFromString(type)) < 0) { + VIR_WARN("type = %s", type); virDomainReportError(VIR_ERR_XML_ERROR, _("unknown type presented to host for character device: %s"), type); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a3cb834..001bc46 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -59,6 +59,7 @@ enum virDomainVirtType { VIR_DOMAIN_VIRT_VBOX, VIR_DOMAIN_VIRT_ONE, VIR_DOMAIN_VIRT_PHYP, + VIR_DOMAIN_VIRT_KVMTOOL,
VIR_DOMAIN_VIRT_LAST, };
IMHO this patch is not required. The domain type is refering to the hypervisor used for the domain, which is still 'kvm'. What is different here is just the userspace device model. If you look at the 3 different Xen user spaces we support, all of them use <domain type='xen'> still. So just use <domain type='kvm'> here for kvmtool. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2011年12月06日 22:46, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:57:04PM +0800, Osier Yang wrote:
It's named as "kvmtool". --- src/conf/domain_conf.c | 4 +++- src/conf/domain_conf.h | 1 + 2 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 58f4d0f..55121d8 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -91,7 +91,8 @@ VIR_ENUM_IMPL(virDomainVirt, VIR_DOMAIN_VIRT_LAST, "hyperv", "vbox", "one", - "phyp") + "phyp", + "kvmtool")
VIR_ENUM_IMPL(virDomainBoot, VIR_DOMAIN_BOOT_LAST, "fd", @@ -4018,6 +4019,7 @@ virDomainChrDefParseXML(virCapsPtr caps, if (type == NULL) { def->source.type = VIR_DOMAIN_CHR_TYPE_PTY; } else if ((def->source.type = virDomainChrTypeFromString(type))< 0) { + VIR_WARN("type = %s", type); virDomainReportError(VIR_ERR_XML_ERROR, _("unknown type presented to host for character device: %s"), type); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a3cb834..001bc46 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -59,6 +59,7 @@ enum virDomainVirtType { VIR_DOMAIN_VIRT_VBOX, VIR_DOMAIN_VIRT_ONE, VIR_DOMAIN_VIRT_PHYP, + VIR_DOMAIN_VIRT_KVMTOOL,
VIR_DOMAIN_VIRT_LAST, };
IMHO this patch is not required. The domain type is refering to the hypervisor used for the domain, which is still 'kvm'. What is different here is just the userspace device model. If you look at the 3 different Xen user spaces we support, all of them use<domain type='xen'> still. So just use<domain type='kvm'> here for kvmtool.
Make sense, agreed. Osier
As "serials[0]->source.type". This is for kvm tool driver only supports one console currently, and it setup ptys (redirect the stdout/stderr of kvm tool process to the master pty, and the slave pty path will be set to source path of the console) for connect to the guest via console. If the console config is like: <console type='pty'> <target type='serial' port='0'/> </console> Things don't work, as kvm tool driver only tries to find consoles and set the slave pty path to "consoles[0]->source.path". Though another solution is to set the slave pty path to "serials[0]->source.path" if no console of type "pty" can be found. --- src/conf/domain_conf.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 55121d8..138105e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7287,6 +7287,7 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, goto no_memory; chr->deviceType = VIR_DOMAIN_CHR_DEVICE_TYPE_CONSOLE; chr->targetType = VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL; + chr->source.type = def->serials[0]->source.type; } chr->target.port = i; -- 1.7.6
Basically, the drivers is implemented by using kvm tool binary currently, (see ./kvm help for more info). Current implementation supports define/undefine, start/destroy/, suspend/resume, connect to guest console via "virsh console", and balloon memory with with "virsh setmem" (using ./kvm balloon command). Also as it supports cgroup controllers "cpuacct", and "memory", so some other commands like "schedinfo", "memtune" can also work. Some other commands such as "domid", "domname", "dumpxml" ,"autostart", etc. are supported, as the driver is designed as a "stateful" driver, those APIs just need to talk with libvirtd simply. As Native Linux KVM Tool is designed for both non-root and root users, the driver is designed just like QEMU, supports two modes of the connection: kvmtool:///system kvmtool+unix:///system kvmtool:///session kvmtool+unix:///session An example of the domain XML (all the XMLs supported currently are listed): % virsh -c kvm:///system dumpxml kvm_test <domain type='kvmtool'> <name>kvm_test</name> <uuid>88bf38f1-b6ab-cfa6-ab53-4b4c0993d894</uuid> <memory>524288</memory> <currentMemory>524288</currentMemory> <vcpu>1</vcpu> <os> <type arch='x86_64'>hvm</type> <kernel>/boot/bzImage</kernel> <boot dev='hd'/> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/kvmtool</emulator> <disk type='file' device='disk'> <source file='/var/lib/libvirt/images/linux-0.2.img'/> <target dev='vda' bus='virtio'/> </disk> <filesystem type='mount' accessmode='passthrough'> <source dir='/tmp'/> <target dir='/mnt'/> </filesystem> <console type='pty'> <target type='serial' port='0'/> </console> <memballoon model='virtio'/> </devices> </domain> --- cfg.mk | 1 + daemon/Makefile.am | 4 + daemon/libvirtd.c | 7 + po/POTFILES.in | 2 + src/Makefile.am | 36 +- src/kvmtool/kvmtool_conf.c | 130 ++ src/kvmtool/kvmtool_conf.h | 66 + src/kvmtool/kvmtool_driver.c | 3079 ++++++++++++++++++++++++++++++++++++++++++ src/kvmtool/kvmtool_driver.h | 29 + 9 files changed, 3353 insertions(+), 1 deletions(-) create mode 100644 src/kvmtool/kvmtool_conf.c create mode 100644 src/kvmtool/kvmtool_conf.h create mode 100644 src/kvmtool/kvmtool_driver.c create mode 100644 src/kvmtool/kvmtool_driver.h diff --git a/cfg.mk b/cfg.mk index 574c7a4..d8e8fac 100644 --- a/cfg.mk +++ b/cfg.mk @@ -472,6 +472,7 @@ msg_gen_function += eventReportError msg_gen_function += ifaceError msg_gen_function += interfaceReportError msg_gen_function += iptablesError +msg_gen_function += kvmtoolError msg_gen_function += lxcError msg_gen_function += libxlError msg_gen_function += macvtapError diff --git a/daemon/Makefile.am b/daemon/Makefile.am index e8c47ae..5941c21 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -133,6 +133,10 @@ if WITH_UML libvirtd_LDADD += ../src/libvirt_driver_uml.la endif +if WITH_KVMTOOL + libvirtd_LDADD += ../src/libvirt_driver_kvmtool.la +endif + if WITH_STORAGE_DIR libvirtd_LDADD += ../src/libvirt_driver_storage.la endif diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 5e1fc96..f01fecb 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -70,6 +70,9 @@ # ifdef WITH_UML # include "uml/uml_driver.h" # endif +# ifdef WITH_KVMTOOL +# include "kvmtool/kvmtool_driver.h" +# endif # ifdef WITH_NETWORK # include "network/bridge_driver.h" # endif @@ -382,6 +385,7 @@ static void daemonInitialize(void) virDriverLoadModule("qemu"); virDriverLoadModule("lxc"); virDriverLoadModule("uml"); + virDriverLoadModule("kvmtool"); virDriverLoadModule("nwfilter"); #else # ifdef WITH_NETWORK @@ -414,6 +418,9 @@ static void daemonInitialize(void) # ifdef WITH_UML umlRegister(); # endif +# ifdef WITH_KVMTOOL + kvmtoolRegister(); +# endif #endif } diff --git a/po/POTFILES.in b/po/POTFILES.in index a3685e8..1f80e98 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -153,3 +153,5 @@ src/xenxs/xen_xm.c tools/console.c tools/libvirt-guests.init.sh tools/virsh.c +src/kvmtool/kvmtool_driver.c +src/kvmtool/kvmtool_conf.c diff --git a/src/Makefile.am b/src/Makefile.am index e931d41..d0115ac 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -442,6 +442,10 @@ HYPERV_DRIVER_EXTRA_DIST = \ hyperv/hyperv_wmi_generator.py \ $(HYPERV_DRIVER_GENERATED) +KVMTOOL_DRIVER_SOURCES = \ + kvmtool/kvmtool_conf.c kvmtool/kvmtool_conf.h \ + kvmtool/kvmtool_driver.h kvmtool/kvmtool_driver.c + NETWORK_DRIVER_SOURCES = \ network/bridge_driver.h network/bridge_driver.c @@ -906,6 +910,25 @@ endif libvirt_driver_hyperv_la_SOURCES = $(HYPERV_DRIVER_SOURCES) endif +if WITH_KVMTOOL +if WITH_DRIVER_MODULES +mod_LTLIBRARIES += libvirt_driver_kvmtool.la +else +noinst_LTLIBRARIES += libvirt_driver_kvmtool.la +# Stateful, so linked to daemon instead +#libvirt_la_BUILT_LIBADD += libvirt_driver_kvmtool.la +endif +libvirt_driver_kvmtool_la_CFLAGS = \ + -I@top_srcdir@/src/conf $(AM_CFLAGS) +libvirt_driver_kvmtool_la_LDFLAGS = $(AM_LDFLAGS) +if WITH_DRIVER_MODULES +libvirt_driver_kvmtool_la_LIBADD = ../gnulib/lib/libgnu.la +libvirt_driver_kvmtool_la_LDFLAGS += -module -avoid-version +endif +libvirt_driver_kvmtool_la_SOURCES = $(KVMTOOL_DRIVER_SOURCES) +endif + + if WITH_NETWORK if WITH_DRIVER_MODULES mod_LTLIBRARIES += libvirt_driver_network.la @@ -1101,11 +1124,12 @@ EXTRA_DIST += \ $(PHYP_DRIVER_SOURCES) \ $(VBOX_DRIVER_SOURCES) \ $(XENAPI_DRIVER_SOURCES) \ - $(LIBXL_DRIVER_SOURCES) \ + $(LIBXL_DRIVER_SOURCES) \ $(ESX_DRIVER_SOURCES) \ $(ESX_DRIVER_EXTRA_DIST) \ $(HYPERV_DRIVER_SOURCES) \ $(HYPERV_DRIVER_EXTRA_DIST) \ + $(KVMTOOL_DRIVER_SOURCES) \ $(NETWORK_DRIVER_SOURCES) \ $(INTERFACE_DRIVER_SOURCES) \ $(STORAGE_DRIVER_SOURCES) \ @@ -1554,6 +1578,11 @@ if WITH_UML $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/uml" $(MKDIR_P) "$(DESTDIR)$(localstatedir)/run/libvirt/uml" endif +if WITH_KVMTOOL + $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/kvmtool" + $(MKDIR_P) "$(DESTDIR)$(localstatedir)/run/libvirt/kvmtool" + $(MKDIR_P) "$(DESTDIR)$(localstatedir)/log/libvirt/kvmtool" +endif if WITH_NETWORK $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/network" $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/dnsmasq" @@ -1600,6 +1629,11 @@ if WITH_UML rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/uml" ||: rmdir "$(DESTDIR)$(localstatedir)/run/libvirt/uml" ||: endif +if WITH_KVMTOOL + rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/kvmtool" ||: + rmdir "$(DESTDIR)$(localstatedir)/run/libvirt/kvmtool" ||: + rmdir "$(DESTDIR)$(localstatedir)/log/libvirt/kvmtool" ||: +endif if WITH_NETWORK rm -f $(DESTDIR)$(sysconfdir)/libvirt/qemu/networks/autostart/default.xml rm -f $(DESTDIR)$(sysconfdir)/libvirt/qemu/networks/default.xml diff --git a/src/kvmtool/kvmtool_conf.c b/src/kvmtool/kvmtool_conf.c new file mode 100644 index 0000000..7ce0b61 --- /dev/null +++ b/src/kvmtool/kvmtool_conf.c @@ -0,0 +1,130 @@ +/* + * Copyright (C) 2011 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Authors: Osier Yang <jyang@redhat.com> + */ + +#include <config.h> + +#include <sys/utsname.h> + +#include "kvmtool_conf.h" +#include "nodeinfo.h" +#include "virterror_internal.h" +#include "memory.h" +#include "logging.h" +#include "uuid.h" +#include "configmake.h" + +#define VIR_FROM_THIS VIR_FROM_KVMTOOL + +static const char * +GetAlt32bitArch(const char *arch) +{ + /* Any Linux 64bit arch which has a 32bit + * personality available should be listed here */ + if (STREQ(arch, "x86_64")) + return "i686"; + if (STREQ(arch, "s390x")) + return "s390"; + if (STREQ(arch, "ppc64")) + return "ppc"; + if (STREQ(arch, "parisc64")) + return "parisc"; + if (STREQ(arch, "sparc64")) + return "sparc"; + if (STREQ(arch, "mips64")) + return "mips"; + + return NULL; +} + +/* Functions */ +virCapsPtr kvmtoolCapsInit(void) +{ + struct utsname utsname; + virCapsPtr caps; + virCapsGuestPtr guest; + const char *altArch; + + uname(&utsname); + + if ((caps = virCapabilitiesNew(utsname.machine, 0, 0)) == NULL) + goto error; + + /* Some machines have problematic NUMA toplogy causing + * unexpected failures. We don't want to break the QEMU + * driver in this scenario, so log errors & carry on + */ + if (nodeCapsInitNUMA(caps) < 0) { + virCapabilitiesFreeNUMAInfo(caps); + VIR_WARN("Failed to query host NUMA topology, disabling NUMA capabilities"); + } + + if (virGetHostUUID(caps->host.host_uuid)) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("cannot get the host uuid")); + goto error; + } + + if ((guest = virCapabilitiesAddGuest(caps, + "hvm", + utsname.machine, + sizeof(void*) == 4 ? 32 : 64, + KVMTOOL, + NULL, + 0, + NULL)) == NULL) + goto error; + + if (virCapabilitiesAddGuestDomain(guest, + "kvmtool", + NULL, + NULL, + 0, + NULL) == NULL) + goto error; + + /* On 64-bit hosts, we can use personality() to request a 32bit process */ + if ((altArch = GetAlt32bitArch(utsname.machine)) != NULL) { + if ((guest = virCapabilitiesAddGuest(caps, + "hvm", + altArch, + 32, + KVMTOOL, + NULL, + 0, + NULL)) == NULL) + goto error; + + if (virCapabilitiesAddGuestDomain(guest, + "kvmtool", + NULL, + NULL, + 0, + NULL) == NULL) + goto error; + } + + virCapabilitiesSetEmulatorRequired(caps); + + return caps; + +error: + virCapabilitiesFree(caps); + return NULL; +} diff --git a/src/kvmtool/kvmtool_conf.h b/src/kvmtool/kvmtool_conf.h new file mode 100644 index 0000000..53f10f7 --- /dev/null +++ b/src/kvmtool/kvmtool_conf.h @@ -0,0 +1,66 @@ +/* + * Copyright (C) 2010 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Authors: Osier Yang <jyang@redhat.com> + */ + +#ifndef KVMTOOL_CONF_H +# define KVMTOOL_CONF_H + +# include <config.h> + +# include "internal.h" +# include "domain_conf.h" +# include "domain_event.h" +# include "capabilities.h" +# include "threads.h" +# include "cgroup.h" +# include "configmake.h" + +# define KVMTOOL_CONFIG_DIR SYSCONFDIR "/libvirt/kvmtool" +# define KVMTOOL_STATE_DIR LOCALSTATEDIR "/run/libvirt/kvmtool" +# define KVMTOOL_LOG_DIR LOCALSTATEDIR "/log/libvirt/kvmtool" +# define KVMTOOL_AUTOSTART_DIR KVMTOOL_CONFIG_DIR "/autostart" + +typedef struct __kvmtool_driver kvmtool_driver_t; +struct __kvmtool_driver { + virMutex lock; + + int privileged; + int nextvmid; + unsigned long version; + + char *configDir; + char *autostartDir; + char *stateDir; + char *logDir; + + virCapsPtr caps; + virCgroupPtr cgroup; + virDomainObjList domains; + + virDomainEventStatePtr domainEventState; + virHashTablePtr autodestroy; +}; + +virCapsPtr kvmtoolCapsInit(void); + +# define kvmtoolError(code, ...) \ + virReportErrorHelper(VIR_FROM_KVMTOOL, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + +#endif /* KVMTOOL_CONF_H */ diff --git a/src/kvmtool/kvmtool_driver.c b/src/kvmtool/kvmtool_driver.c new file mode 100644 index 0000000..5233611 --- /dev/null +++ b/src/kvmtool/kvmtool_driver.c @@ -0,0 +1,3079 @@ +/* + * Copyright (C) 2011 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Authors: Osier Yang <jyang@redhat.com> + * + */ + +#include <config.h> + +#include <fcntl.h> +#include <sched.h> +#include <sys/utsname.h> +#include <string.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <sys/poll.h> +#include <unistd.h> +#include <wait.h> +#include <sys/stat.h> + +#include "virterror_internal.h" +#include "logging.h" +#include "datatypes.h" +#include "kvmtool_conf.h" +#include "kvmtool_driver.h" +#include "memory.h" +#include "util.h" +#include "nodeinfo.h" +#include "uuid.h" +#include "hooks.h" +#include "virfile.h" +#include "virpidfile.h" +#include "fdstream.h" +#include "domain_audit.h" + +#define VIR_FROM_THIS VIR_FROM_KVMTOOL + +#define START_POSTFIX ": starting up\n" +#define SHUTDOWN_POSTFIX ": shutting down\n" + +#define KVMTOOL_NB_MEM_PARAM 3 + +static kvmtool_driver_t *kvmtool_driver = NULL; + +typedef struct _kvmtoolDomainObjPrivate kvmtoolDomainObjPrivate; +typedef kvmtoolDomainObjPrivate *kvmtoolDomainObjPrivatePtr; +struct _kvmtoolDomainObjPrivate { + /* For future use, as long as kvmtool tool provide APIs to talk + * with the guest socket directly + */ + int monitor; +}; + +static int kvmtoolStartup(int privileged); +static int kvmtoolShutdown(void); + +static void kvmtoolDomainEventFlush(int timer, void *opaque); +static void kvmtoolDomainEventQueue(kvmtool_driver_t *driver, + virDomainEventPtr event); + +static int kvmtoolProcessAutoDestroyInit(kvmtool_driver_t *driver); +static void kvmtoolProcessAutoDestroyRun(kvmtool_driver_t *driver, + virConnectPtr conn); +static void kvmtoolProcessAutoDestroyShutdown(kvmtool_driver_t *driver); +static int kvmtoolProcessAutoDestroyAdd(kvmtool_driver_t *driver, + virDomainObjPtr vm, + virConnectPtr conn); +static int kvmtoolProcessAutoDestroyRemove(kvmtool_driver_t *driver, + virDomainObjPtr vm); + +static void +kvmtoolDriverLock(kvmtool_driver_t *driver) +{ + virMutexLock(&driver->lock); +} +static void +kvmtoolDriverUnlock(kvmtool_driver_t *driver) +{ + virMutexUnlock(&driver->lock); +} + +static void * +kvmtoolDomainObjPrivateAlloc(void) +{ + kvmtoolDomainObjPrivatePtr priv; + + if (VIR_ALLOC(priv) < 0) + return NULL; + + priv->monitor = -1; + + return priv; +} + +static void +kvmtoolDomainObjPrivateFree(void *data) +{ + kvmtoolDomainObjPrivatePtr priv = data; + + VIR_FREE(priv); +} + +static virDrvOpenStatus +kvmtoolOpen(virConnectPtr conn, + virConnectAuthPtr auth ATTRIBUTE_UNUSED, + unsigned int flags) +{ + virCheckFlags(VIR_CONNECT_RO, VIR_DRV_OPEN_ERROR); + + /* Verify uri was specified */ + if (conn->uri == NULL) { + if (kvmtool_driver == NULL) + return VIR_DRV_OPEN_DECLINED; + + conn->uri = xmlParseURI(kvmtool_driver->privileged ? + "kvmtool:///system": + "kvmtool:///session"); + if (!conn->uri) { + virReportOOMError(); + return VIR_DRV_OPEN_ERROR; + } + } else { + if (conn->uri->scheme == NULL || + STRNEQ(conn->uri->scheme, "kvmtool")) + return VIR_DRV_OPEN_DECLINED; + + /* Leave for remote driver */ + if (conn->uri->server != NULL) + return VIR_DRV_OPEN_DECLINED; + + if (conn->uri->path == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("no KVMTOOL URI path given, try %s"), + kvmtool_driver->privileged ? + "kvmtool:///system" : + "kvmtool:///session"); + return VIR_DRV_OPEN_ERROR; + } + + if (kvmtool_driver->privileged) { + if (STRNEQ (conn->uri->path, "/system") && + STRNEQ (conn->uri->path, "/session")) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("unexpected KVMTOOL URI path blu '%s', try kvmtool:///system"), + conn->uri->path); + return VIR_DRV_OPEN_ERROR; + } + } else { + if (STRNEQ (conn->uri->path, "/session")) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("unexpected KVMTOOL URI path bla '%s', try kvmtool:///session"), + conn->uri->path); + return VIR_DRV_OPEN_ERROR; + } + } + + /* URI was good, but driver isn't active */ + if (kvmtool_driver == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("kvmtool state driver is not active")); + return VIR_DRV_OPEN_ERROR; + } + } + + conn->privateData = kvmtool_driver; + return VIR_DRV_OPEN_SUCCESS; +} + +static int +kvmtoolClose(virConnectPtr conn) +{ + kvmtool_driver_t *driver = conn->privateData; + + kvmtoolDriverLock(driver); + virDomainEventCallbackListRemoveConn(conn, + driver->domainEventState->callbacks); + kvmtoolProcessAutoDestroyRun(driver, conn); + kvmtoolDriverUnlock(driver); + + conn->privateData = NULL; + return 0; +} + +static int +kvmtoolIsSecure(virConnectPtr conn ATTRIBUTE_UNUSED) +{ + /* Trivially secure, since always inside the daemon */ + return 1; +} + + +static int +kvmtoolIsEncrypted(virConnectPtr conn ATTRIBUTE_UNUSED) +{ + /* Not encrypted, but remote driver takes care of that */ + return 0; +} + +static char * +kvmtoolGetCapabilities(virConnectPtr conn) +{ + kvmtool_driver_t *driver = conn->privateData; + char *xml; + + kvmtoolDriverLock(driver); + if ((xml = virCapabilitiesFormatXML(driver->caps)) == NULL) + virReportOOMError(); + kvmtoolDriverUnlock(driver); + + return xml; +} + +static virDomainPtr +kvmtoolDomainLookupByID(virConnectPtr conn, int id) +{ + kvmtool_driver_t *driver = conn->privateData; + virDomainObjPtr vm; + virDomainPtr dom = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByID(&driver->domains, id); + kvmtoolDriverUnlock(driver); + + if (!vm) { + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching id %d"), id); + goto cleanup; + } + + dom = virGetDomain(conn, vm->def->name, vm->def->uuid); + if (dom) + dom->id = vm->def->id; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return dom; +} + +static virDomainPtr +kvmtoolDomainLookupByUUID(virConnectPtr conn, + const unsigned char *uuid) +{ + kvmtool_driver_t *driver = conn->privateData; + virDomainObjPtr vm; + virDomainPtr dom = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + dom = virGetDomain(conn, vm->def->name, vm->def->uuid); + if (dom) + dom->id = vm->def->id; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return dom; +} + +static virDomainPtr +kvmtoolDomainLookupByName(virConnectPtr conn, + const char *name) +{ + kvmtool_driver_t *driver = conn->privateData; + virDomainObjPtr vm; + virDomainPtr dom = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByName(&driver->domains, name); + kvmtoolDriverUnlock(driver); + if (!vm) { + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching name '%s'"), name); + goto cleanup; + } + + dom = virGetDomain(conn, vm->def->name, vm->def->uuid); + if (dom) + dom->id = vm->def->id; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return dom; +} + + +static int +kvmtoolDomainIsActive(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr obj; + int ret = -1; + + kvmtoolDriverLock(driver); + obj = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + if (!obj) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + ret = virDomainObjIsActive(obj); + +cleanup: + if (obj) + virDomainObjUnlock(obj); + return ret; +} + + +static int +kvmtoolDomainIsPersistent(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr obj; + int ret = -1; + + kvmtoolDriverLock(driver); + obj = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + if (!obj) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + ret = obj->persistent; + +cleanup: + if (obj) + virDomainObjUnlock(obj); + return ret; +} + +static int +kvmtoolDomainIsUpdated(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr obj; + int ret = -1; + + kvmtoolDriverLock(driver); + obj = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + if (!obj) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + ret = obj->updated; + +cleanup: + if (obj) + virDomainObjUnlock(obj); + return ret; +} + +static int +kvmtoolListDomains(virConnectPtr conn, int *ids, int nids) { + kvmtool_driver_t *driver = conn->privateData; + int n; + + kvmtoolDriverLock(driver); + n = virDomainObjListGetActiveIDs(&driver->domains, ids, nids); + kvmtoolDriverUnlock(driver); + + return n; +} + +static int +kvmtoolNumOfDomains(virConnectPtr conn) { + kvmtool_driver_t *driver = conn->privateData; + int n; + + kvmtoolDriverLock(driver); + n = virDomainObjListNumOfDomains(&driver->domains, 1); + kvmtoolDriverUnlock(driver); + + return n; +} + +static int +kvmtoolListDefinedDomains(virConnectPtr conn, + char **const names, int nnames) { + kvmtool_driver_t *driver = conn->privateData; + int n; + + kvmtoolDriverLock(driver); + n = virDomainObjListGetInactiveNames(&driver->domains, names, nnames); + kvmtoolDriverUnlock(driver); + + return n; +} + + +static int +kvmtoolNumOfDefinedDomains(virConnectPtr conn) { + kvmtool_driver_t *driver = conn->privateData; + int n; + + kvmtoolDriverLock(driver); + n = virDomainObjListNumOfDomains(&driver->domains, 0); + kvmtoolDriverUnlock(driver); + + return n; +} + +static virDomainPtr +kvmtoolDomainDefine(virConnectPtr conn, const char *xml) +{ + kvmtool_driver_t *driver = conn->privateData; + virDomainDefPtr def = NULL; + virDomainObjPtr vm = NULL; + virDomainPtr dom = NULL; + virDomainEventPtr event = NULL; + int dupVM; + + kvmtoolDriverLock(driver); + if (!(def = virDomainDefParseString(driver->caps, xml, + 1 << VIR_DOMAIN_VIRT_KVMTOOL, + VIR_DOMAIN_XML_INACTIVE))) + goto cleanup; + + if ((dupVM = virDomainObjIsDuplicate(&driver->domains, def, 0)) < 0) + goto cleanup; + + if (!(vm = virDomainAssignDef(driver->caps, + &driver->domains, def, false))) + goto cleanup; + + def = NULL; + vm->persistent = 1; + + if (virDomainSaveConfig(driver->configDir, + vm->newDef ? vm->newDef : vm->def) < 0) { + virDomainRemoveInactive(&driver->domains, vm); + vm = NULL; + goto cleanup; + } + + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_DEFINED, + !dupVM ? + VIR_DOMAIN_EVENT_DEFINED_ADDED : + VIR_DOMAIN_EVENT_DEFINED_UPDATED); + + dom = virGetDomain(conn, vm->def->name, vm->def->uuid); + if (dom) + dom->id = vm->def->id; + +cleanup: + virDomainDefFree(def); + if (vm) + virDomainObjUnlock(vm); + if (event) + kvmtoolDomainEventQueue(driver, event); + kvmtoolDriverUnlock(driver); + return dom; +} + +static int +kvmtoolDomainUndefineFlags(virDomainPtr dom, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainEventPtr event = NULL; + int ret = -1; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!vm->persistent) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Cannot undefine transient domain")); + goto cleanup; + } + + if (virDomainDeleteConfig(driver->configDir, + driver->autostartDir, + vm) < 0) + goto cleanup; + + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_UNDEFINED, + VIR_DOMAIN_EVENT_UNDEFINED_REMOVED); + + if (virDomainObjIsActive(vm)) { + vm->persistent = 0; + } else { + virDomainRemoveInactive(&driver->domains, vm); + vm = NULL; + } + + ret = 0; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + if (event) + kvmtoolDomainEventQueue(driver, event); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int kvmtoolDomainUndefine(virDomainPtr dom) +{ + return kvmtoolDomainUndefineFlags(dom, 0); +} + +static int +kvmtoolDomainGetInfo(virDomainPtr dom, + virDomainInfoPtr info) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virCgroupPtr cgroup = NULL; + int ret = -1; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + info->state = virDomainObjGetState(vm, NULL); + + if (!virDomainObjIsActive(vm)) { + info->memory = vm->def->mem.cur_balloon; + } else { + /* XXX: Should query the balloon information instead + * as long as kvmtool tool supports it + */ + info->memory = 0; + } + + if (driver->cgroup) { + if (virCgroupForDomain(driver->cgroup, + vm->def->name, + &cgroup, 0) != 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Unable to get cgroup for %s"), vm->def->name); + goto cleanup; + } + + if (virCgroupGetCpuacctUsage(cgroup, &(info->cpuTime)) < 0) { + kvmtoolError(VIR_ERR_OPERATION_FAILED, + "%s", _("Cannot read cputime for domain")); + goto cleanup; + } + } else { + info->cpuTime = 0; + } + + info->maxMem = vm->def->mem.max_balloon; + info->nrVirtCpu = vm->def->vcpus; + ret = 0; + +cleanup: + kvmtoolDriverUnlock(driver); + if (cgroup) + virCgroupFree(&cgroup); + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static int +kvmtoolDomainGetState(virDomainPtr dom, + int *state, + int *reason, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + int ret = -1; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + *state = virDomainObjGetState(vm, reason); + ret = 0; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static char * +kvmtoolGetOSType(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + char *ret = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + ret = strdup(vm->def->os.type); + + if (ret == NULL) + virReportOOMError(); + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +/* Returns max memory in kb, or 0 if error */ +static unsigned long +kvmtoolDomainGetMaxMemory(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + unsigned long ret = 0; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + ret = vm->def->mem.max_balloon; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static int +kvmtoolDomainSetMemoryFlags(virDomainPtr dom, + unsigned long newmem, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainDefPtr persistentDef = NULL; + int ret = -1, r; + bool isActive; + virCommandPtr cmd = NULL; + char *errbuf = NULL; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_DOMAIN_MEM_MAXIMUM, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + isActive = virDomainObjIsActive(vm); + + if (flags == VIR_DOMAIN_AFFECT_CURRENT) { + if (isActive) + flags = VIR_DOMAIN_AFFECT_LIVE; + else + flags = VIR_DOMAIN_AFFECT_CONFIG; + } + + if (flags == VIR_DOMAIN_MEM_MAXIMUM) { + if (isActive) + flags |= VIR_DOMAIN_MEM_MAXIMUM; + else + flags |= VIR_DOMAIN_MEM_MAXIMUM; + } + + if (!isActive && (flags & VIR_DOMAIN_AFFECT_LIVE)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("domain is not running")); + goto cleanup; + } + + if (flags & VIR_DOMAIN_AFFECT_CONFIG) { + if (!vm->persistent) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot change persistent config of a " + "transient domain")); + goto cleanup; + } + if (!(persistentDef = virDomainObjGetPersistentDef(driver->caps, vm))) + goto cleanup; + } + + if (flags & VIR_DOMAIN_MEM_MAXIMUM) { + /* Resize the maximum memory */ + if (flags & VIR_DOMAIN_AFFECT_LIVE) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot resize the maximum memory on an " + "active domain")); + goto cleanup; + } + + if (flags & VIR_DOMAIN_AFFECT_CONFIG) { + persistentDef->mem.max_balloon = newmem; + if (persistentDef->mem.cur_balloon > newmem) + persistentDef->mem.cur_balloon = newmem; + ret = virDomainSaveConfig(driver->configDir, persistentDef); + goto cleanup; + } + } else { + /* Resize the current memory */ + if (newmem > vm->def->mem.max_balloon) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("cannot set memory higher than max memory")); + goto cleanup; + } + + if (flags & VIR_DOMAIN_AFFECT_LIVE) { + cmd = virCommandNew(vm->def->emulator); + virCommandAddEnvPassCommon(cmd); + virCommandAddEnvFormat(cmd, "KVMTOOL_STATE_DIR=%s", driver->stateDir); + virCommandAddArgList(cmd, "balloon", "--name", vm->def->name, NULL); + + if (newmem > vm->def->mem.cur_balloon) + virCommandAddArg(cmd, "--inflate"); + else + virCommandAddArg(cmd, "--deflate"); + + /* kvmtool balloon command use MB. */ + virCommandAddArgFormat(cmd, "%lu", newmem / 1024); + + virCommandSetErrorBuffer(cmd, &errbuf); + + r = virCommandRun(cmd, NULL); + + virDomainAuditMemory(vm, vm->def->mem.cur_balloon, newmem, "update", + r == 1); + if (r < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Failed to balloon memory: %s"), + errbuf); + goto cleanup; + } + } + + if (flags & VIR_DOMAIN_AFFECT_CONFIG) { + persistentDef->mem.cur_balloon = newmem; + ret = virDomainSaveConfig(driver->configDir, persistentDef); + goto cleanup; + } + } + + ret = 0; + +cleanup: + VIR_FREE(errbuf); + virCommandFree(cmd); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int +kvmtoolDomainSetMemory(virDomainPtr dom, unsigned long newmem) +{ + return kvmtoolDomainSetMemoryFlags(dom, newmem, VIR_DOMAIN_AFFECT_LIVE); +} + +static int +kvmtoolDomainSetMaxMemory(virDomainPtr dom, unsigned long memory) +{ + return kvmtoolDomainSetMemoryFlags(dom, memory, VIR_DOMAIN_MEM_MAXIMUM); +} + +static int +kvmtoolDomainSetMemoryParameters(virDomainPtr dom, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + int i; + virCgroupPtr cgroup = NULL; + virDomainObjPtr vm = NULL; + int ret = -1; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (vm == NULL) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("cannot find cgroup for domain %s"), vm->def->name); + goto cleanup; + } + + ret = 0; + for (i = 0; i < nparams; i++) { + virTypedParameterPtr param = ¶ms[i]; + + if (STREQ(param->field, VIR_DOMAIN_MEMORY_HARD_LIMIT)) { + int rc; + if (param->type != VIR_TYPED_PARAM_ULLONG) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("invalid type for memory hard_limit tunable, expected a 'ullong'")); + ret = -1; + continue; + } + + rc = virCgroupSetMemoryHardLimit(cgroup, params[i].value.ul); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to set memory hard_limit tunable")); + ret = -1; + } + } else if (STREQ(param->field, VIR_DOMAIN_MEMORY_SOFT_LIMIT)) { + int rc; + if (param->type != VIR_TYPED_PARAM_ULLONG) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("invalid type for memory soft_limit tunable, expected a 'ullong'")); + ret = -1; + continue; + } + + rc = virCgroupSetMemorySoftLimit(cgroup, params[i].value.ul); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to set memory soft_limit tunable")); + ret = -1; + } + } else if (STREQ(param->field, VIR_DOMAIN_MEMORY_SWAP_HARD_LIMIT)) { + int rc; + if (param->type != VIR_TYPED_PARAM_ULLONG) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("invalid type for swap_hard_limit tunable, expected a 'ullong'")); + ret = -1; + continue; + } + + rc = virCgroupSetMemSwapHardLimit(cgroup, params[i].value.ul); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to set swap_hard_limit tunable")); + ret = -1; + } + } else if (STREQ(param->field, VIR_DOMAIN_MEMORY_MIN_GUARANTEE)) { + kvmtoolError(VIR_ERR_INVALID_ARG, + _("Memory tunable `%s' not implemented"), param->field); + ret = -1; + } else { + kvmtoolError(VIR_ERR_INVALID_ARG, + _("Parameter `%s' not supported"), param->field); + ret = -1; + } + } + +cleanup: + if (cgroup) + virCgroupFree(&cgroup); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int +kvmtoolDomainGetMemoryParameters(virDomainPtr dom, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + int i; + virCgroupPtr cgroup = NULL; + virDomainObjPtr vm = NULL; + unsigned long long val; + int ret = -1; + int rc; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (vm == NULL) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if ((*nparams) == 0) { + /* Current number of memory parameters supported by cgroups */ + *nparams = KVMTOOL_NB_MEM_PARAM; + ret = 0; + goto cleanup; + } + if ((*nparams) < KVMTOOL_NB_MEM_PARAM) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("Invalid parameter count")); + goto cleanup; + } + + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Unable to get cgroup for %s"), vm->def->name); + goto cleanup; + } + + for (i = 0; i < KVMTOOL_NB_MEM_PARAM; i++) { + virTypedParameterPtr param = ¶ms[i]; + val = 0; + param->value.ul = 0; + param->type = VIR_TYPED_PARAM_ULLONG; + + switch(i) { + case 0: /* fill memory hard limit here */ + rc = virCgroupGetMemoryHardLimit(cgroup, &val); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to get memory hard limit")); + goto cleanup; + } + if (virStrcpyStatic(param->field, VIR_DOMAIN_MEMORY_HARD_LIMIT) == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Field memory hard limit too long for destination")); + goto cleanup; + } + param->value.ul = val; + break; + + case 1: /* fill memory soft limit here */ + rc = virCgroupGetMemorySoftLimit(cgroup, &val); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to get memory soft limit")); + goto cleanup; + } + if (virStrcpyStatic(param->field, VIR_DOMAIN_MEMORY_SOFT_LIMIT) == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Field memory soft limit too long for destination")); + goto cleanup; + } + param->value.ul = val; + break; + + case 2: /* fill swap hard limit here */ + rc = virCgroupGetMemSwapHardLimit(cgroup, &val); + if (rc != 0) { + virReportSystemError(-rc, "%s", + _("unable to get swap hard limit")); + goto cleanup; + } + if (virStrcpyStatic(param->field, VIR_DOMAIN_MEMORY_SWAP_HARD_LIMIT) == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Field swap hard limit too long for destination")); + goto cleanup; + } + param->value.ul = val; + break; + + default: + break; + /* should not hit here */ + } + } + + *nparams = KVMTOOL_NB_MEM_PARAM; + ret = 0; + +cleanup: + if (cgroup) + virCgroupFree(&cgroup); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static char * +kvmtoolDomainGetXMLDesc(virDomainPtr dom, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + char *ret = NULL; + + /* Flags checked by virDomainDefFormat */ + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + ret = virDomainDefFormat((flags & VIR_DOMAIN_XML_INACTIVE) && + vm->newDef ? vm->newDef : vm->def, + flags); +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + + +static int +kvmtoolProcessAutoDestroyInit(kvmtool_driver_t *driver) +{ + if (!(driver->autodestroy = virHashCreate(5, NULL))) + return -1; + + return 0; +} + +static int +kvmtoolDomainDestroyHelper(kvmtool_driver_t *driver, + virDomainObjPtr vm, + virDomainShutoffReason reason) +{ + virCgroupPtr cgroup; + kvmtoolDomainObjPrivatePtr priv = vm->privateData; + int ret = -1; + virCommandPtr cmd = NULL; + char *logfile = NULL; + int logfd = -1; + const char *timestamp = NULL; + char ebuf[1024]; + char *sockpath = NULL; + char *errbuf = NULL; + + cmd = virCommandNew(vm->def->emulator); + virCommandAddEnvPassCommon(cmd); + virCommandAddEnvFormat(cmd, "KVMTOOL_STATE_DIR=%s", driver->stateDir); + virCommandAddArgList(cmd, "stop", "-n", vm->def->name, NULL); + virCommandSetErrorBuffer(cmd, &errbuf); + + if (virAsprintf(&logfile, "%s/%s.log", driver->logDir, + vm->def->name) < 0) { + virReportOOMError(); + goto cleanup; + } + + if ((logfd = open(logfile, O_WRONLY | O_APPEND)) < 0) { + virReportSystemError(errno, + _("Failed to open '%s'"), + logfile); + goto cleanup; + } + + if ((timestamp = virTimestamp()) == NULL) { + virReportOOMError(); + goto cleanup; + } + if (safewrite(logfd, timestamp, strlen(timestamp)) < 0 || + safewrite(logfd, SHUTDOWN_POSTFIX, strlen(SHUTDOWN_POSTFIX)) < 0) { + VIR_WARN("Unable to write timestamp to logfile: %s", + virStrerror(errno, ebuf, sizeof ebuf)); + } + + if (priv->monitor) + VIR_FORCE_CLOSE(priv->monitor); + + if (virCommandRun(cmd, NULL) < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Failed to destroy domain '%s': %s"), + vm->def->name, errbuf); + goto cleanup; + } + + if (virAsprintf(&sockpath, "%s/%s.sock", driver->stateDir, + vm->def->name) < 0) { + virReportOOMError(); + goto cleanup; + } + + /* It's up to kvmtool to remove the socket. What we can do + * is just to report a warning if it still exists. */ + if (virFileExists(sockpath)) + VIR_WARN("The domain socket still exists after destroyed"); + + /* now that we know it's stopped call the hook if present */ + if (virHookPresent(VIR_HOOK_DRIVER_KVMTOOL)) { + char *xml = virDomainDefFormat(vm->def, 0); + + /* we can't stop the operation even if the script raised an error */ + virHookCall(VIR_HOOK_DRIVER_KVMTOOL, vm->def->name, + VIR_HOOK_KVMTOOL_OP_STOPPED, VIR_HOOK_SUBOP_END, NULL, xml); + VIR_FREE(xml); + } + + /* Stop autodestroy in case guest is restarted */ + kvmtoolProcessAutoDestroyRemove(driver, vm); + + virDomainDeleteConfig(driver->stateDir, NULL, vm); + + virDomainObjSetState(vm, VIR_DOMAIN_SHUTOFF, reason); + + vm->def->id = -1; + priv->monitor = -1; + + if (driver->cgroup && + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) == 0) { + virCgroupRemove(cgroup); + virCgroupFree(&cgroup); + } + + if (vm->newDef) { + virDomainDefFree(vm->def); + vm->def = vm->newDef; + vm->def->id = -1; + vm->newDef = NULL; + } + + ret = 0; + +cleanup: + virCommandFree(cmd); + VIR_FREE(logfile); + VIR_FREE(timestamp); + VIR_FREE(sockpath); + VIR_FREE(errbuf); + if (logfd) + VIR_FORCE_CLOSE(logfd); + return ret; +} + +struct kvmtoolProcessAutoDestroyData { + kvmtool_driver_t *driver; + virConnectPtr conn; +}; + +static void +kvmtoolProcessAutoDestroyDom(void *payload, + const void *name, + void *opaque) +{ + struct kvmtoolProcessAutoDestroyData *data = opaque; + virConnectPtr conn = payload; + const char *uuidstr = name; + unsigned char uuid[VIR_UUID_BUFLEN]; + virDomainObjPtr dom; + virDomainEventPtr event = NULL; + + VIR_DEBUG("conn=%p uuidstr=%s thisconn=%p", conn, uuidstr, data->conn); + + if (data->conn != conn) + return; + + if (virUUIDParse(uuidstr, uuid) < 0) { + VIR_WARN("Failed to parse %s", uuidstr); + return; + } + + if (!(dom = virDomainFindByUUID(&data->driver->domains, + uuid))) { + VIR_DEBUG("No domain object to kill"); + return; + } + + VIR_DEBUG("Killing domain"); + kvmtoolDomainDestroyHelper(data->driver, dom, VIR_DOMAIN_SHUTOFF_DESTROYED); + virDomainAuditStop(dom, "destroyed"); + event = virDomainEventNewFromObj(dom, + VIR_DOMAIN_EVENT_STOPPED, + VIR_DOMAIN_EVENT_STOPPED_DESTROYED); + + if (dom && !dom->persistent) + virDomainRemoveInactive(&data->driver->domains, dom); + + if (dom) + virDomainObjUnlock(dom); + if (event) + kvmtoolDomainEventQueue(data->driver, event); + virHashRemoveEntry(data->driver->autodestroy, uuidstr); +} + +/* + * Precondition: driver is locked + */ +static void +kvmtoolProcessAutoDestroyRun(kvmtool_driver_t *driver, virConnectPtr conn) +{ + struct kvmtoolProcessAutoDestroyData data = { + driver, conn + }; + VIR_DEBUG("conn=%p", conn); + virHashForEach(driver->autodestroy, kvmtoolProcessAutoDestroyDom, &data); +} + +static void +kvmtoolProcessAutoDestroyShutdown(kvmtool_driver_t *driver) +{ + virHashFree(driver->autodestroy); +} + +static int +kvmtoolProcessAutoDestroyAdd(kvmtool_driver_t *driver, + virDomainObjPtr vm, + virConnectPtr conn) +{ + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(vm->def->uuid, uuidstr); + + VIR_DEBUG("vm=%s uuid=%s conn=%p", vm->def->name, uuidstr, conn); + + if (virHashAddEntry(driver->autodestroy, uuidstr, conn) < 0) + return -1; + return 0; +} + +static int +kvmtoolProcessAutoDestroyRemove(kvmtool_driver_t *driver, + virDomainObjPtr vm) +{ + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(vm->def->uuid, uuidstr); + + VIR_DEBUG("vm=%s uuid=%s", vm->def->name, uuidstr); + + if (virHashRemoveEntry(driver->autodestroy, uuidstr) < 0) + return -1; + return 0; +} + +static int +kvmtoolConnectDomainSocket(kvmtool_driver_t * driver, + virDomainObjPtr vm) +{ + char *sockpath = NULL; + int fd; + struct sockaddr_un addr; + + if (virAsprintf(&sockpath, "%s/%s.sock", + driver->stateDir, vm->def->name) < 0) { + virReportOOMError(); + return -1; + } + + if ((fd = socket(PF_UNIX, SOCK_STREAM, 0)) < 0) { + virReportSystemError(errno, "%s", + _("Failed to create client socket")); + goto error; + } + + memset(&addr, 0, sizeof(addr)); + addr.sun_family = AF_UNIX; + if (virStrcpyStatic(addr.sun_path, sockpath) == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Socket path %s too big for destination"), sockpath); + goto error; + } + + if (connect(fd, (struct sockaddr *) &addr, sizeof(addr)) < 0) { + virReportSystemError(errno, _("Failed to connect to guest " + "socket '%s'"), sockpath); + goto error; + } + + VIR_FREE(sockpath); + return fd; + +error: + VIR_FREE(sockpath); + VIR_FORCE_CLOSE(fd); + return -1; +} + +static virCommandPtr +kvmtoolBuildCommandLine(kvmtool_driver_t *driver ATTRIBUTE_UNUSED, + virDomainObjPtr vm, + int logfd, + int masterPty) +{ + virCommandPtr cmd = NULL; + int i; + + cmd = virCommandNew(vm->def->emulator); + + virCommandAddEnvPassCommon(cmd); + + /* kvmtool use $HOME/.kvmtool_tools as the state dir by default */ + virCommandAddEnvFormat(cmd, "KVMTOOL_STATE_DIR=%s", driver->stateDir); + + virCommandAddArg(cmd, "run"); + virCommandAddArgList(cmd, "--name", vm->def->name, NULL); + + if (!vm->def->os.kernel) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("'kernel' must be specified")); + goto cleanup; + } + virCommandAddArgList(cmd, "--kernel", vm->def->os.kernel, NULL); + + if (!vm->def->ndisks) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("No disk is specified")); + goto cleanup; + } + + if (vm->def->os.initrd) + virCommandAddArgList(cmd, "--initrd", vm->def->os.initrd, NULL); + + if (vm->def->os.cmdline) + virCommandAddArgList(cmd, "--params", vm->def->os.cmdline, NULL); + + if (vm->def->mem.cur_balloon) { + virCommandAddArg(cmd, "--mem"); + virCommandAddArgFormat(cmd, "%lu", vm->def->mem.cur_balloon / 1024); + } + + if (vm->def->vcpus) { + virCommandAddArg(cmd, "--cpus"); + virCommandAddArgFormat(cmd, "%u", vm->def->vcpus); + } + + for (i = 0; i < vm->def->ndisks; i++) { + if (vm->def->disks[i]->bus != VIR_DOMAIN_DISK_BUS_VIRTIO) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("disk bus type must be 'virtio'")); + goto cleanup; + } + + if (vm->def->disks[i]->device != VIR_DOMAIN_DISK_DEVICE_DISK) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("'device' type of disk must be 'disk'")); + goto cleanup; + } + + if (vm->def->disks[i]->type != VIR_DOMAIN_DISK_TYPE_FILE) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("disk type must be 'file'")); + goto cleanup; + } + virCommandAddArgList(cmd, "--disk", vm->def->disks[0]->src, NULL); + } + + if (vm->def->memballoon) { + if (vm->def->memballoon->model != VIR_DOMAIN_MEMBALLOON_MODEL_NONE && + vm->def->memballoon->model != VIR_DOMAIN_MEMBALLOON_MODEL_VIRTIO) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("memballoon 'model' must be 'virtio'")); + goto cleanup; + } + + virCommandAddArg(cmd, "--balloon"); + } + + for (i = 0; i < vm->def->nfss; i++) { + if (vm->def->fss[i]->type != VIR_DOMAIN_FS_TYPE_MOUNT) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Only supports mount filesystem type")); + goto cleanup; + } + + virCommandAddArg(cmd, "--9p"); + virCommandAddArgFormat(cmd, "%s,%s", vm->def->fss[i]->src, + vm->def->fss[i]->dst); + } + + if (vm->def->nconsoles > 1) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Only one console is supported")); + goto cleanup; + } + + /* XXX: I'm lost in the XMLs for char devices, the logic + * might need to improve here. + */ + if (vm->def->consoles) { + switch (vm->def->consoles[0]->targetType) { + case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_VIRTIO: + virCommandAddArgList(cmd, "--console", "virtio", NULL); + break; + case VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL: + virCommandAddArgList(cmd, "--console", "serial", NULL); + break; + default: + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("target type of console must be 'virtio' or 'serial'")); + goto cleanup; + } + } else { + if (vm->def->nserials != 0) { + if (vm->def->nserials > 1) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Only one serial port is supported")); + goto cleanup; + } else { + virCommandAddArgList(cmd, "--console", "serial", NULL); + } + } + } + + virCommandSetInputFD(cmd, masterPty); + virCommandSetOutputFD(cmd, &masterPty); + virCommandSetErrorFD(cmd, &logfd); + virCommandDaemonize(cmd); + + return cmd; +cleanup: + if (logfd) + VIR_FORCE_CLOSE(logfd); + if (masterPty) + VIR_FORCE_CLOSE(masterPty); + virCommandFree(cmd); + return NULL; +} + +static bool +kvmtoolCgroupControllerActive(kvmtool_driver_t *driver, + int controller) +{ + if (driver->cgroup == NULL) + return false; + if (controller < 0 || controller >= VIR_CGROUP_CONTROLLER_LAST) + return false; + if (!virCgroupMounted(driver->cgroup, controller)) + return false; + return true; +} + +static int +kvmtoolRemoveCgroup(kvmtool_driver_t *driver, + virDomainObjPtr vm, + int quiet) +{ + virCgroupPtr cgroup; + int rc; + + if (driver->cgroup == NULL) + return 0; /* Not supported, so claim success */ + + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0); + if (rc != 0) { + if (!quiet) + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Unable to find cgroup for %s"), + vm->def->name); + return rc; + } + + rc = virCgroupRemove(cgroup); + virCgroupFree(&cgroup); + return rc; +} + +static int +kvmtoolSetupCgroup(kvmtool_driver_t *driver, + virDomainObjPtr vm) +{ + virCgroupPtr cgroup = NULL; + int rc; + + if (driver->cgroup == NULL) + return 0; /* Not supported, so claim success */ + + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 1); + if (rc != 0) { + virReportSystemError(-rc, + _("Unable to create cgroup for %s"), + vm->def->name); + goto cleanup; + } + + if (vm->def->blkio.weight != 0) { + if (kvmtoolCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_BLKIO)) { + rc = virCgroupSetBlkioWeight(cgroup, vm->def->blkio.weight); + if(rc != 0) { + virReportSystemError(-rc, + _("Unable to set io weight for domain %s"), + vm->def->name); + goto cleanup; + } + } else { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Block I/O tuning is not available on this host")); + } + } + + if (vm->def->mem.hard_limit != 0 || + vm->def->mem.soft_limit != 0 || + vm->def->mem.swap_hard_limit != 0) { + if (kvmtoolCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_MEMORY)) { + if (vm->def->mem.hard_limit != 0) { + rc = virCgroupSetMemoryHardLimit(cgroup, vm->def->mem.hard_limit); + if (rc != 0) { + virReportSystemError(-rc, + _("Unable to set memory hard limit for domain %s"), + vm->def->name); + goto cleanup; + } + } + if (vm->def->mem.soft_limit != 0) { + rc = virCgroupSetMemorySoftLimit(cgroup, vm->def->mem.soft_limit); + if (rc != 0) { + virReportSystemError(-rc, + _("Unable to set memory soft limit for domain %s"), + vm->def->name); + goto cleanup; + } + } + + if (vm->def->mem.swap_hard_limit != 0) { + rc = virCgroupSetMemSwapHardLimit(cgroup, vm->def->mem.swap_hard_limit); + if (rc != 0) { + virReportSystemError(-rc, + _("Unable to set swap hard limit for domain %s"), + vm->def->name); + goto cleanup; + } + } + } else { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Memory cgroup is not available on this host")); + } + } + + if (vm->def->cputune.shares != 0) { + if (kvmtoolCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPU)) { + rc = virCgroupSetCpuShares(cgroup, vm->def->cputune.shares); + if(rc != 0) { + virReportSystemError(-rc, + _("Unable to set io cpu shares for domain %s"), + vm->def->name); + goto cleanup; + } + } else { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, + _("CPU tuning is not available on this host")); + } + } + + virCgroupFree(&cgroup); + return 0; + +cleanup: + if (cgroup) { + virCgroupRemove(cgroup); + virCgroupFree(&cgroup); + } + return -1; +} + +/** + * kvmtoolDomainStartHelper: + * @conn: pointer to connection + * @driver: pointer to driver structure + * @vm: pointer to virtual machine structure + * @autoDestroy: mark the domain for auto destruction + * @reason: reason for switching vm to running state + * + * Starts a vm + * + * Returns 0 on success or -1 in case of error + */ +static int +kvmtoolDomainStartHelper(virConnectPtr conn, + kvmtool_driver_t * driver, + virDomainObjPtr vm, + bool autoDestroy, + virDomainRunningReason reason) +{ + int ret = -1; + int masterPty; + char *slavePty = NULL; + char *logfile = NULL; + int logfd = -1; + off_t pos = -1; + char ebuf[1024]; + const char *timestamp = NULL; + virCommandPtr cmd = NULL; + kvmtoolDomainObjPrivatePtr priv = vm->privateData; + int retries = 100; + char *sockpath = NULL; + + if (!driver->cgroup) { + VIR_WARN("cgroup is not mounted"); + } else { + if (!kvmtoolCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPUACCT)) { + VIR_WARN("cgroup cpuacct controller is not mounted"); + } + if (!kvmtoolCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_MEMORY)) { + VIR_WARN("cgroup cpuacct controller is not mounted"); + } + } + + if (virAsprintf(&logfile, "%s/%s.log", + driver->logDir, vm->def->name) < 0) { + virReportOOMError(); + return -1; + } + + if ((logfd = open(logfile, O_WRONLY | O_APPEND | O_CREAT, + S_IRUSR|S_IWUSR)) < 0) { + virReportSystemError(errno, + _("Failed to open '%s'"), + logfile); + goto cleanup; + } + + /* Open master pty */ + if (virFileOpenTty(&masterPty, &slavePty, 1) < 0) { + virReportSystemError(errno, "%s", + _("Failed to allocate tty")); + goto cleanup; + } + + VIR_DEBUG("masterPty = %d, slavePty = %s", masterPty, slavePty); + + if (vm->def->consoles) { + if (vm->def->consoles[0]->source.type != VIR_DOMAIN_CHR_TYPE_PTY) { + kvmtoolError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Only PTY console type is supported")); + goto cleanup; + } + + VIR_FREE(vm->def->consoles[0]->source.data.file.path); + vm->def->consoles[0]->source.data.file.path = slavePty; + + VIR_FREE(vm->def->consoles[0]->info.alias); + if (virAsprintf(&vm->def->consoles[0]->info.alias, "console%d", 0) < 0) { + virReportOOMError(); + goto cleanup; + } + } + + kvmtoolRemoveCgroup(driver, vm, 1); + + /* Now that we know it is about to start call the hook if present */ + if (virHookPresent(VIR_HOOK_DRIVER_KVMTOOL)) { + char *xml = virDomainDefFormat(vm->def, 0); + int hookret; + + hookret = virHookCall(VIR_HOOK_DRIVER_KVMTOOL, vm->def->name, + VIR_HOOK_KVMTOOL_OP_START, VIR_HOOK_SUBOP_BEGIN, NULL, xml); + VIR_FREE(xml); + + /* If the script raised an error abort the launch. */ + if (hookret < 0) + goto cleanup; + } + + /* Log timestamp */ + if ((timestamp = virTimestamp()) == NULL) { + virReportOOMError(); + goto cleanup; + } + if (safewrite(logfd, timestamp, strlen(timestamp)) < 0 || + safewrite(logfd, START_POSTFIX, strlen(START_POSTFIX)) < 0) { + VIR_WARN("Unable to write timestamp to logfile: %s", + virStrerror(errno, ebuf, sizeof ebuf)); + } + + if (!(cmd = kvmtoolBuildCommandLine(driver, vm, logfd, masterPty))) + goto cleanup; + + /* Log generated command line */ + virCommandWriteArgLog(cmd, logfd); + + if ((pos = lseek(logfd, 0, SEEK_END)) < 0) + VIR_WARN("Unable to seek to end of logfile: %s", + virStrerror(errno, ebuf, sizeof ebuf)); + + if (virAsprintf(&sockpath, "%s/%s.sock", driver->stateDir, + vm->def->name) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (virCommandRun(cmd, NULL) < 0) + goto cleanup; + + /* Wait for guest socket shows up. */ + while (!virFileExists(sockpath) && retries) { + usleep(100*100); + retries--; + } + + /* Check if could connect to the kvmtool guest socket */ + if ((priv->monitor = kvmtoolConnectDomainSocket(driver, vm)) < 0) + goto cleanup; + + if (kvmtoolSetupCgroup(driver, vm) < 0) + goto cleanup; + + vm->def->id = driver->nextvmid++; + + virDomainObjSetState(vm, VIR_DOMAIN_RUNNING, reason); + + if (autoDestroy && + kvmtoolProcessAutoDestroyAdd(driver, vm, conn) < 0) + goto cleanup; + + if (virDomainObjSetDefTransient(driver->caps, vm, false) < 0) + goto cleanup; + + /* Write domain status to disk. */ + if (virDomainSaveStatus(driver->caps, driver->stateDir, vm) < 0) + goto cleanup; + + ret = 0; + +cleanup: + virCommandFree(cmd); + if (ret == -1) + VIR_FORCE_CLOSE(priv->monitor); + VIR_FREE(logfile); + VIR_FREE(sockpath); + VIR_FREE(timestamp); + return ret; +} + +/** + * kvmtoolDomainStartWithFlags: + * @dom: domain to start + * @flags: Must be 0 for now + * + * Looks up domain and starts it. + * + * Returns 0 on success or -1 in case of error + */ +static int +kvmtoolDomainStartWithFlags(virDomainPtr dom, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainEventPtr event = NULL; + int ret = -1; + + virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (virDomainObjIsActive(vm)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain is already running")); + goto cleanup; + } + + ret = kvmtoolDomainStartHelper(dom->conn, driver, vm, + (flags & VIR_DOMAIN_START_AUTODESTROY), + VIR_DOMAIN_RUNNING_BOOTED); + + if (ret == 0) { + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_STARTED, + VIR_DOMAIN_EVENT_STARTED_BOOTED); + virDomainAuditStart(vm, "booted", true); + } else { + virDomainAuditStart(vm, "booted", false); + } + +cleanup: + if (vm) + virDomainObjUnlock(vm); + if (event) + kvmtoolDomainEventQueue(driver, event); + kvmtoolDriverUnlock(driver); + return ret; +} + +/** + * kvmtoolDomainStart: + * @dom: domain to start + * + * Looks up domain and starts it. + * + * Returns 0 on success or -1 in case of error + */ +static int kvmtoolDomainStart(virDomainPtr dom) +{ + return kvmtoolDomainStartWithFlags(dom, 0); +} + +/** + * kvmtoolDomainCreateAndStart: + * @conn: pointer to connection + * @xml: XML definition of domain + * @flags: Must be 0 for now + * + * Creates a domain based on xml and starts it + * + * Returns 0 on success or -1 in case of error + */ +static virDomainPtr +kvmtoolDomainCreateAndStart(virConnectPtr conn, + const char *xml, + unsigned int flags) { + kvmtool_driver_t *driver = conn->privateData; + virDomainObjPtr vm = NULL; + virDomainDefPtr def; + virDomainPtr dom = NULL; + virDomainEventPtr event = NULL; + + virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, NULL); + + kvmtoolDriverLock(driver); + if (!(def = virDomainDefParseString(driver->caps, xml, + 1 << VIR_DOMAIN_VIRT_KVMTOOL, + VIR_DOMAIN_XML_INACTIVE))) + goto cleanup; + + if (virDomainObjIsDuplicate(&driver->domains, def, 1) < 0) + goto cleanup; + + if (!(vm = virDomainAssignDef(driver->caps, + &driver->domains, def, false))) + goto cleanup; + def = NULL; + + if (kvmtoolDomainStartHelper(conn, driver, vm, + (flags & VIR_DOMAIN_START_AUTODESTROY), + VIR_DOMAIN_RUNNING_BOOTED) < 0) { + virDomainAuditStart(vm, "booted", false); + virDomainRemoveInactive(&driver->domains, vm); + vm = NULL; + goto cleanup; + } + + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_STARTED, + VIR_DOMAIN_EVENT_STARTED_BOOTED); + virDomainAuditStart(vm, "booted", true); + + dom = virGetDomain(conn, vm->def->name, vm->def->uuid); + if (dom) + dom->id = vm->def->id; + +cleanup: + virDomainDefFree(def); + if (vm) + virDomainObjUnlock(vm); + if (event) + kvmtoolDomainEventQueue(driver, event); + kvmtoolDriverUnlock(driver); + return dom; +} + + +static int +kvmtoolDomainEventRegister(virConnectPtr conn, + virConnectDomainEventCallback callback, + void *opaque, + virFreeCallback freecb) +{ + kvmtool_driver_t *driver = conn->privateData; + int ret; + + kvmtoolDriverLock(driver); + ret = virDomainEventCallbackListAdd(conn, + driver->domainEventState->callbacks, + callback, opaque, freecb); + kvmtoolDriverUnlock(driver); + + return ret; +} + + +static int +kvmtoolDomainEventDeregister(virConnectPtr conn, + virConnectDomainEventCallback callback) +{ + kvmtool_driver_t *driver = conn->privateData; + int ret; + + kvmtoolDriverLock(driver); + ret = virDomainEventStateDeregister(conn, + driver->domainEventState, + callback); + kvmtoolDriverUnlock(driver); + + return ret; +} + + +static int +kvmtoolDomainEventRegisterAny(virConnectPtr conn, + virDomainPtr dom, + int eventID, + virConnectDomainEventGenericCallback callback, + void *opaque, + virFreeCallback freecb) +{ + kvmtool_driver_t *driver = conn->privateData; + int ret; + + kvmtoolDriverLock(driver); + ret = virDomainEventCallbackListAddID(conn, + driver->domainEventState->callbacks, + dom, eventID, + callback, opaque, freecb); + kvmtoolDriverUnlock(driver); + + return ret; +} + + +static int +kvmtoolDomainEventDeregisterAny(virConnectPtr conn, + int callbackID) +{ + kvmtool_driver_t *driver = conn->privateData; + int ret; + + kvmtoolDriverLock(driver); + ret = virDomainEventStateDeregisterAny(conn, + driver->domainEventState, + callbackID); + kvmtoolDriverUnlock(driver); + + return ret; +} + + +static void +kvmtoolDomainEventDispatchFunc(virConnectPtr conn, + virDomainEventPtr event, + virConnectDomainEventGenericCallback cb, + void *cbopaque, + void *opaque) +{ + kvmtool_driver_t *driver = opaque; + + /* Drop the lock whle dispatching, for sake of re-entrancy */ + kvmtoolDriverUnlock(driver); + virDomainEventDispatchDefaultFunc(conn, event, cb, cbopaque, NULL); + kvmtoolDriverLock(driver); +} + + +static void +kvmtoolDomainEventFlush(int timer ATTRIBUTE_UNUSED, + void *opaque) +{ + kvmtool_driver_t *driver = opaque; + + kvmtoolDriverLock(driver); + virDomainEventStateFlush(driver->domainEventState, + kvmtoolDomainEventDispatchFunc, + driver); + kvmtoolDriverUnlock(driver); +} + + +/* driver must be locked before calling */ +static void +kvmtoolDomainEventQueue(kvmtool_driver_t *driver, + virDomainEventPtr event) +{ + virDomainEventStateQueue(driver->domainEventState, event); +} + +/** + * kvmtoolDomainDestroyFlags: + * @dom: pointer to domain to destroy + * @flags: an OR'ed set of virDomainDestroyFlags + * + * Sends SIGKILL to container root process to terminate the container + * + * Returns 0 on success or -1 in case of error + */ +static int +kvmtoolDomainDestroyFlags(virDomainPtr dom, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainEventPtr event = NULL; + int ret = -1; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!virDomainObjIsActive(vm)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain is not running")); + goto cleanup; + } + + ret = kvmtoolDomainDestroyHelper(driver, vm, VIR_DOMAIN_SHUTOFF_DESTROYED); + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_STOPPED, + VIR_DOMAIN_EVENT_STOPPED_DESTROYED); + virDomainAuditStop(vm, "destroyed"); + if (!vm->persistent) { + virDomainRemoveInactive(&driver->domains, vm); + vm = NULL; + } + +cleanup: + if (vm) + virDomainObjUnlock(vm); + if (event) + kvmtoolDomainEventQueue(driver, event); + kvmtoolDriverUnlock(driver); + return ret; +} + +/** + * kvmtoolDomainDestroy: + * @dom: pointer to domain to destroy + * + * Sends SIGKILL to container root process to terminate the container + * + * Returns 0 on success or -1 in case of error + */ +static int +kvmtoolDomainDestroy(virDomainPtr dom) +{ + return kvmtoolDomainDestroyFlags(dom, 0); +} + +struct kvmtoolAutostartData { + kvmtool_driver_t *driver; + virConnectPtr conn; +}; + +static void +kvmtoolAutostartDomain(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virDomainObjPtr vm = payload; + const struct kvmtoolAutostartData *data = opaque; + + virDomainObjLock(vm); + if (vm->autostart && + !virDomainObjIsActive(vm)) { + int ret = kvmtoolDomainStartHelper(data->conn, data->driver, vm, false, + VIR_DOMAIN_RUNNING_BOOTED); + virDomainAuditStart(vm, "booted", ret >= 0); + if (ret < 0) { + virErrorPtr err = virGetLastError(); + VIR_ERROR(_("Failed to autostart VM '%s': %s"), + vm->def->name, + err ? err->message : ""); + } else { + virDomainEventPtr event = + virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_STARTED, + VIR_DOMAIN_EVENT_STARTED_BOOTED); + if (event) + kvmtoolDomainEventQueue(data->driver, event); + } + } + virDomainObjUnlock(vm); +} + +static void +kvmtoolAutostartConfigs(kvmtool_driver_t *driver) +{ + /* XXX: Figure out a better way todo this. The domain + * startup code needs a connection handle in order + * to lookup the bridge associated with a virtual + * network + */ + virConnectPtr conn = virConnectOpen(driver->privileged ? + "kvmtool:///system" : + "kvmtool:///session"); + /* Ignoring NULL conn which is mostly harmless here */ + + struct kvmtoolAutostartData data = { driver, conn }; + + kvmtoolDriverLock(driver); + virHashForEach(driver->domains.objs, kvmtoolAutostartDomain, &data); + kvmtoolDriverUnlock(driver); + + if (conn) + virConnectClose(conn); +} + +static void +kvmtoolReconnectVM(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virDomainObjPtr vm = payload; + kvmtool_driver_t *driver = opaque; + kvmtoolDomainObjPrivatePtr priv; + virCgroupPtr cgroup = NULL; + + virDomainObjLock(vm); + VIR_DEBUG("Reconnect %d %d %d\n", vm->def->id, vm->pid, vm->state.state); + + priv = vm->privateData; + + if ((priv->monitor = kvmtoolConnectDomainSocket(driver, vm)) < 0) + goto cleanup; + + vm->def->id = driver->nextvmid++; + virDomainObjSetState(vm, VIR_DOMAIN_RUNNING, VIR_DOMAIN_RUNNING_UNKNOWN); + + virDomainObjUnlock(vm); + return; + +cleanup: + kvmtoolProcessAutoDestroyRemove(driver, vm); + + virDomainDeleteConfig(driver->stateDir, NULL, vm); + + virDomainObjSetState(vm, VIR_DOMAIN_SHUTOFF, VIR_DOMAIN_SHUTOFF_FAILED); + + vm->def->id = -1; + priv->monitor = -1; + + if (driver->cgroup && + virCgroupForDomain(driver->cgroup, vm->def->name, + &cgroup, 0) == 0) { + virCgroupRemove(cgroup); + virCgroupFree(&cgroup); + } + + if (vm->newDef) { + virDomainDefFree(vm->def); + vm->def = vm->newDef; + vm->def->id = -1; + vm->newDef = NULL; + } + + virDomainAuditStop(vm, "failed"); + virDomainObjUnlock(vm); + return; +} + +#define KVM_DEV "/dev/kvm" + +static int +kvmtoolStartup(int privileged) +{ + int ret = -1; + + if (access(KVM_DEV, F_OK) != 0) { + VIR_INFO("Host doesn't support hardware virt: %s is " + " not available", KVM_DEV); + return 0; + } + + if (VIR_ALLOC(kvmtool_driver) < 0) { + return -1; + } + if (virMutexInit(&kvmtool_driver->lock) < 0) { + VIR_FREE(kvmtool_driver); + return -1; + } + kvmtoolDriverLock(kvmtool_driver); + + kvmtool_driver->privileged = privileged; + kvmtool_driver->nextvmid = 1; + + if (virDomainObjListInit(&kvmtool_driver->domains) < 0) + goto cleanup; + + kvmtool_driver->domainEventState = virDomainEventStateNew(kvmtoolDomainEventFlush, + kvmtool_driver, + NULL, + true); + if (!kvmtool_driver->domainEventState) + goto cleanup; + + ret = virCgroupForDriver("kvmtool", &kvmtool_driver->cgroup, privileged, 1); + if (ret < 0) { + char buf[1024] ATTRIBUTE_UNUSED; + VIR_DEBUG("Unable to create cgroup for KVMTOOL driver: %s", + virStrerror(-ret, buf, sizeof(buf))); + /* Don't abort startup. We will explicitly report to + * the user when they try to start a VM + */ + } + + /* Setup the directories */ + if (privileged) { + if ((kvmtool_driver->configDir = strdup(KVMTOOL_CONFIG_DIR)) == NULL) + goto out_of_memory; + + if ((kvmtool_driver->stateDir = strdup(KVMTOOL_STATE_DIR)) == NULL) + goto out_of_memory; + + if ((kvmtool_driver->logDir = strdup(KVMTOOL_LOG_DIR)) == NULL) + goto out_of_memory; + + if ((kvmtool_driver->autostartDir = strdup(KVMTOOL_AUTOSTART_DIR)) == NULL) + goto out_of_memory; + } else { + uid_t uid = geteuid(); + char *base = NULL; + char *userdir = virGetUserDirectory(uid); + if (!userdir) + goto cleanup; + + if (virAsprintf(&kvmtool_driver->logDir, + "%s/.libvirt/kvmtool/log", userdir) == -1) { + VIR_FREE(userdir); + goto out_of_memory; + } + + if (virAsprintf(&base, "%s/.libvirt", userdir) == -1) { + VIR_FREE(userdir); + goto out_of_memory; + } + VIR_FREE(userdir); + + if (virAsprintf(&kvmtool_driver->stateDir, "%s/kvmtool/run", base) == -1) { + VIR_FREE(base); + goto out_of_memory; + } + + if (virAsprintf(&kvmtool_driver->configDir, "%s/kvmtool", base) == -1) { + VIR_FREE(base); + goto out_of_memory; + } + + if (virAsprintf(&kvmtool_driver->autostartDir, + "%s/kvmtool/autostart", base) == -1) { + VIR_FREE(base); + goto out_of_memory; + } + } + + if (virFileMakePath(kvmtool_driver->logDir) < 0) { + virReportSystemError(errno, + _("Failed to create log directory '%s'"), + kvmtool_driver->logDir); + goto cleanup; + } + + if (virFileMakePath(kvmtool_driver->stateDir) < 0) { + virReportSystemError(errno, + _("Failed to create log directory '%s'"), + kvmtool_driver->logDir); + goto cleanup; + } + + if (virFileMakePath(kvmtool_driver->configDir) < 0) { + virReportSystemError(errno, + _("Failed to create log directory '%s'"), + kvmtool_driver->logDir); + goto cleanup; + } + + if (virFileMakePath(kvmtool_driver->autostartDir) < 0) { + virReportSystemError(errno, + _("Failed to create log directory '%s'"), + kvmtool_driver->logDir); + goto cleanup; + } + + if ((kvmtool_driver->caps = kvmtoolCapsInit()) == NULL) + goto cleanup; + + kvmtool_driver->caps->privateDataAllocFunc = kvmtoolDomainObjPrivateAlloc; + kvmtool_driver->caps->privateDataFreeFunc = kvmtoolDomainObjPrivateFree; + + if (kvmtoolProcessAutoDestroyInit(kvmtool_driver) < 0) + goto cleanup; + + /* Get all the running persistent or transient configs first */ + if (virDomainLoadAllConfigs(kvmtool_driver->caps, + &kvmtool_driver->domains, + kvmtool_driver->stateDir, + NULL, + 1, 1 << VIR_DOMAIN_VIRT_KVMTOOL, + NULL, NULL) < 0) + goto cleanup; + + virHashForEach(kvmtool_driver->domains.objs, kvmtoolReconnectVM, kvmtool_driver); + + /* Then inactive persistent configs */ + if (virDomainLoadAllConfigs(kvmtool_driver->caps, + &kvmtool_driver->domains, + kvmtool_driver->configDir, + kvmtool_driver->autostartDir, + 0, 1 << VIR_DOMAIN_VIRT_KVMTOOL, + NULL, NULL) < 0) + goto cleanup; + + kvmtoolDriverUnlock(kvmtool_driver); + + kvmtoolAutostartConfigs(kvmtool_driver); + + return 0; + +out_of_memory: + virReportOOMError(); +cleanup: + kvmtoolDriverUnlock(kvmtool_driver); + kvmtoolShutdown(); + return -1; +} + +static void kvmtoolNotifyLoadDomain(virDomainObjPtr vm, int newVM, void *opaque) +{ + kvmtool_driver_t *driver = opaque; + + if (newVM) { + virDomainEventPtr event = + virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_DEFINED, + VIR_DOMAIN_EVENT_DEFINED_ADDED); + if (event) + kvmtoolDomainEventQueue(driver, event); + } +} + +/** + * kvmtoolReload: + * + * Function to restart the KVMTOOL driver, it will recheck the configuration + * files and perform autostart + */ +static int +kvmtoolReload(void) { + if (!kvmtool_driver) + return 0; + + kvmtoolDriverLock(kvmtool_driver); + virDomainLoadAllConfigs(kvmtool_driver->caps, + &kvmtool_driver->domains, + kvmtool_driver->configDir, + kvmtool_driver->autostartDir, + 0, 1 << VIR_DOMAIN_VIRT_KVMTOOL, + kvmtoolNotifyLoadDomain, kvmtool_driver); + kvmtoolDriverUnlock(kvmtool_driver); + + kvmtoolAutostartConfigs(kvmtool_driver); + + return 0; +} + +static int +kvmtoolShutdown(void) +{ + if (kvmtool_driver == NULL) + return -1; + + kvmtoolDriverLock(kvmtool_driver); + virDomainObjListDeinit(&kvmtool_driver->domains); + virDomainEventStateFree(kvmtool_driver->domainEventState); + + kvmtoolProcessAutoDestroyShutdown(kvmtool_driver); + + virCapabilitiesFree(kvmtool_driver->caps); + VIR_FREE(kvmtool_driver->configDir); + VIR_FREE(kvmtool_driver->autostartDir); + VIR_FREE(kvmtool_driver->stateDir); + VIR_FREE(kvmtool_driver->logDir); + kvmtoolDriverUnlock(kvmtool_driver); + virMutexDestroy(&kvmtool_driver->lock); + VIR_FREE(kvmtool_driver); + + return 0; +} + +/** + * kvmtoolIsActive: + * + * Checks if the KVMTOOL daemon is active, i.e. has an active domain + * + * Returns 1 if active, 0 otherwise + */ +static int +kvmtoolIsActive(void) { + int active; + + if (kvmtool_driver == NULL) + return(0); + + kvmtoolDriverLock(kvmtool_driver); + active = virDomainObjListNumOfDomains(&kvmtool_driver->domains, 1); + kvmtoolDriverUnlock(kvmtool_driver); + + return active; +} + +/* + * kvmtoolGetVersion + * + * XXX: This API might be just broken currently, as kvmtool tool still + * doesn't have a formal version, the upstream kvmtool tool will output + * things like "3.0.rc5.873.gb73216" by "./kvmtool version". + */ +static int +kvmtoolGetVersion(virConnectPtr conn, unsigned long *version) +{ + kvmtool_driver_t *driver = conn->privateData; + virCommandPtr cmd = NULL; + char *outbuf = NULL; + char *errbuf = NULL; + const char *emulator = NULL; + struct utsname ut; + struct stat sb; + char *p = NULL; + int ret = -1; + int i; + + uname(&ut); + + for (i = 0; i < driver->caps->nguests; i++) { + if (STREQ(driver->caps->guests[i]->arch.name, ut.machine)) { + emulator = driver->caps->guests[i]->arch.defaultInfo.emulator; + break; + } + } + + if (!emulator) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("can't find the emulator for '%s'"), + ut.machine); + goto cleanup; + } + + if (stat(emulator, &sb) < 0) { + virReportSystemError(errno, + _("Cannot stat KVMTOOL binary %s"), + emulator); + goto cleanup; + } + + cmd = virCommandNewArgList(emulator, "version", NULL); + virCommandAddEnvPassCommon(cmd); + virCommandSetOutputBuffer(cmd, &outbuf); + virCommandSetErrorBuffer(cmd, &errbuf); + + if (virCommandRun(cmd, NULL) < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Failed to get kvmtool version: %s"), + errbuf); + goto cleanup; + } + + p = strrchr(outbuf, ' '); + p++; + + if (virParseVersionString(p, version, true) < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, _("Unknown release: %s"), p); + goto cleanup; + } + + ret = 0; +cleanup: + VIR_FREE(outbuf); + VIR_FREE(errbuf); + return ret; +} + +static char *kvmtoolGetSchedulerType(virDomainPtr domain ATTRIBUTE_UNUSED, + int *nparams) +{ + char *schedulerType = NULL; + + if (nparams) + *nparams = 1; + + schedulerType = strdup("posix"); + + if (schedulerType == NULL) + virReportOOMError(); + + return schedulerType; +} + +static int +kvmtoolSetSchedulerParametersFlags(virDomainPtr domain, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + kvmtool_driver_t *driver = domain->conn->privateData; + int i; + virCgroupPtr group = NULL; + virDomainObjPtr vm = NULL; + int ret = -1; + + virCheckFlags(0, -1); + + if (driver->cgroup == NULL) + return -1; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, domain->uuid); + + if (vm == NULL) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(domain->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) + goto cleanup; + + for (i = 0; i < nparams; i++) { + virTypedParameterPtr param = ¶ms[i]; + + if (STRNEQ(param->field, "cpu_shares")) { + kvmtoolError(VIR_ERR_INVALID_ARG, + _("Invalid parameter `%s'"), param->field); + goto cleanup; + } + + if (param->type != VIR_TYPED_PARAM_ULLONG) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("Invalid type for cpu_shares tunable, expected a 'ullong'")); + goto cleanup; + } + + int rc = virCgroupSetCpuShares(group, params[i].value.ul); + if (rc != 0) { + virReportSystemError(-rc, _("failed to set cpu_shares=%llu"), + params[i].value.ul); + goto cleanup; + } + + vm->def->cputune.shares = params[i].value.ul; + } + ret = 0; + +cleanup: + kvmtoolDriverUnlock(driver); + virCgroupFree(&group); + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static int +kvmtoolSetSchedulerParameters(virDomainPtr domain, + virTypedParameterPtr params, + int nparams) +{ + return kvmtoolSetSchedulerParametersFlags(domain, params, nparams, 0); +} + +static int +kvmtoolGetSchedulerParametersFlags(virDomainPtr domain, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + kvmtool_driver_t *driver = domain->conn->privateData; + virCgroupPtr group = NULL; + virDomainObjPtr vm = NULL; + unsigned long long val; + int ret = -1; + + virCheckFlags(0, -1); + + if (driver->cgroup == NULL) + return -1; + + if (*nparams < 1) { + kvmtoolError(VIR_ERR_INVALID_ARG, "%s", + _("Invalid parameter count")); + return -1; + } + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, domain->uuid); + + if (vm == NULL) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(domain->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) + goto cleanup; + + if (virCgroupGetCpuShares(group, &val) != 0) + goto cleanup; + params[0].value.ul = val; + if (virStrcpyStatic(params[0].field, "cpu_shares") == NULL) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Field cpu_shares too big for destination")); + goto cleanup; + } + params[0].type = VIR_TYPED_PARAM_ULLONG; + + *nparams = 1; + ret = 0; + +cleanup: + kvmtoolDriverUnlock(driver); + virCgroupFree(&group); + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static int +kvmtoolGetSchedulerParameters(virDomainPtr domain, + virTypedParameterPtr params, + int *nparams) +{ + return kvmtoolGetSchedulerParametersFlags(domain, params, nparams, 0); +} + +static int +kvmtoolDomainGetAutostart(virDomainPtr dom, + int *autostart) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + int ret = -1; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + kvmtoolDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + *autostart = vm->autostart; + ret = 0; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + +static int +kvmtoolDomainSetAutostart(virDomainPtr dom, + int autostart) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + char *configFile = NULL, *autostartLink = NULL; + int ret = -1; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!vm->persistent) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Cannot set autostart for transient domain")); + goto cleanup; + } + + autostart = (autostart != 0); + + if (vm->autostart == autostart) { + ret = 0; + goto cleanup; + } + + configFile = virDomainConfigFile(driver->configDir, + vm->def->name); + if (configFile == NULL) + goto cleanup; + autostartLink = virDomainConfigFile(driver->autostartDir, + vm->def->name); + if (autostartLink == NULL) + goto cleanup; + + if (autostart) { + if (virFileMakePath(driver->autostartDir) < 0) { + virReportSystemError(errno, + _("Cannot create autostart directory %s"), + driver->autostartDir); + goto cleanup; + } + + if (symlink(configFile, autostartLink) < 0) { + virReportSystemError(errno, + _("Failed to create symlink '%s to '%s'"), + autostartLink, configFile); + goto cleanup; + } + } else { + if (unlink(autostartLink) < 0 && errno != ENOENT && errno != ENOTDIR) { + virReportSystemError(errno, + _("Failed to delete symlink '%s'"), + autostartLink); + goto cleanup; + } + } + + vm->autostart = autostart; + ret = 0; + +cleanup: + VIR_FREE(configFile); + VIR_FREE(autostartLink); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int +kvmtoolDomainSuspend(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainEventPtr event = NULL; + int ret = -1; + virCommandPtr cmd = NULL; + char *errbuf = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!virDomainObjIsActive(vm)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain is not running")); + goto cleanup; + } + + if (virDomainObjGetState(vm, NULL) != VIR_DOMAIN_PAUSED) { + cmd = virCommandNew(vm->def->emulator); + virCommandAddEnvPassCommon(cmd); + virCommandAddEnvFormat(cmd, "KVMTOOL_STATE_DIR=%s", driver->stateDir); + virCommandAddArgList(cmd, "pause", "--name", vm->def->name, NULL); + + virCommandSetErrorBuffer(cmd, &errbuf); + + if (virCommandRun(cmd, NULL) < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Failed to suspend domain '%s': %s"), + vm->def->name, errbuf); + goto cleanup; + } + + virDomainObjSetState(vm, VIR_DOMAIN_PAUSED, VIR_DOMAIN_PAUSED_USER); + + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_SUSPENDED, + VIR_DOMAIN_EVENT_SUSPENDED_PAUSED); + } + + if (virDomainSaveStatus(driver->caps, driver->stateDir, vm) < 0) + goto cleanup; + ret = 0; + +cleanup: + VIR_FREE(errbuf); + if (event) + kvmtoolDomainEventQueue(driver, event); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int +kvmtoolDomainResume(virDomainPtr dom) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm; + virDomainEventPtr event = NULL; + int ret = -1; + virCommandPtr cmd = NULL; + char *errbuf = NULL; + + kvmtoolDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("No domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!virDomainObjIsActive(vm)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain is not running")); + goto cleanup; + } + + if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_PAUSED) { + virDomainObjSetState(vm, VIR_DOMAIN_RUNNING, + VIR_DOMAIN_RUNNING_UNPAUSED); + cmd = virCommandNew(vm->def->emulator); + virCommandAddEnvPassCommon(cmd); + virCommandAddEnvFormat(cmd, "KVMTOOL_STATE_DIR=%s", driver->stateDir); + virCommandAddArgList(cmd, "resume", "--name", vm->def->name, NULL); + + virCommandSetErrorBuffer(cmd, &errbuf); + + if (virCommandRun(cmd, NULL) < 0) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("Failed to resume domain '%s': %s"), + vm->def->name, errbuf); + goto cleanup; + } + + event = virDomainEventNewFromObj(vm, + VIR_DOMAIN_EVENT_RESUMED, + VIR_DOMAIN_EVENT_RESUMED_UNPAUSED); + } + + if (virDomainSaveStatus(driver->caps, driver->stateDir, vm) < 0) + goto cleanup; + ret = 0; + +cleanup: + virCommandFree(cmd); + VIR_FREE(errbuf); + if (event) + kvmtoolDomainEventQueue(driver, event); + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +static int +kvmtoolDomainOpenConsole(virDomainPtr dom, + const char *dev_name, + virStreamPtr st, + unsigned int flags) +{ + kvmtool_driver_t *driver = dom->conn->privateData; + virDomainObjPtr vm = NULL; + char uuidstr[VIR_UUID_STRING_BUFLEN]; + int ret = -1; + virDomainChrDefPtr chr = NULL; + + virCheckFlags(0, -1); + + kvmtoolDriverLock(driver); + virUUIDFormat(dom->uuid, uuidstr); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + if (!vm) { + kvmtoolError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + if (!virDomainObjIsActive(vm)) { + kvmtoolError(VIR_ERR_OPERATION_INVALID, "%s", + _("domain is not running")); + goto cleanup; + } + + if (dev_name) { + if (vm->def->consoles[0]->info.alias && + STREQ(vm->def->consoles[0]->info.alias, dev_name)) { + chr = vm->def->consoles[0]; + } + } else { + if (vm->def->consoles) + chr = vm->def->consoles[0]; + else if (vm->def->nserials) + chr = vm->def->serials[0]; + } + + if (!chr) { + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("cannot find console device '%s'"), + dev_name ? dev_name : _("default")); + goto cleanup; + } + + if (chr->source.type != VIR_DOMAIN_CHR_TYPE_PTY) { + VIR_WARN("%d", chr->source.type); + kvmtoolError(VIR_ERR_INTERNAL_ERROR, + _("character device %s is not using a PTY"), dev_name); + goto cleanup; + } + + if (virFDStreamOpenFile(st, chr->source.data.file.path, + 0, 0, O_RDWR) < 0) + goto cleanup; + + ret = 0; +cleanup: + if (vm) + virDomainObjUnlock(vm); + kvmtoolDriverUnlock(driver); + return ret; +} + +/* Function Tables */ +static virDriver kvmtoolDriver = { + .no = VIR_DRV_KVMTOOL, + .name = "KVMTOOL", + .open = kvmtoolOpen, /* 0.4.2 */ + .close = kvmtoolClose, /* 0.4.2 */ + .version = kvmtoolGetVersion, /* 0.4.6 */ + .getHostname = virGetHostname, /* 0.6.3 */ + .nodeGetInfo = nodeGetInfo, /* 0.6.5 */ + .getCapabilities = kvmtoolGetCapabilities, /* 0.6.5 */ + .listDomains = kvmtoolListDomains, /* 0.4.2 */ + .numOfDomains = kvmtoolNumOfDomains, /* 0.4.2 */ + .domainCreateXML = kvmtoolDomainCreateAndStart, /* 0.4.4 */ + .domainLookupByID = kvmtoolDomainLookupByID, /* 0.4.2 */ + .domainLookupByUUID = kvmtoolDomainLookupByUUID, /* 0.4.2 */ + .domainLookupByName = kvmtoolDomainLookupByName, /* 0.4.2 */ + .domainSuspend = kvmtoolDomainSuspend, /* 0.7.2 */ + .domainResume = kvmtoolDomainResume, /* 0.7.2 */ + .domainDestroy = kvmtoolDomainDestroy, /* 0.4.4 */ + .domainDestroyFlags = kvmtoolDomainDestroyFlags, /* 0.9.4 */ + .domainGetOSType = kvmtoolGetOSType, /* 0.4.2 */ + .domainGetMaxMemory = kvmtoolDomainGetMaxMemory, /* 0.7.2 */ + .domainSetMaxMemory = kvmtoolDomainSetMaxMemory, /* 0.7.2 */ + .domainSetMemory = kvmtoolDomainSetMemory, /* 0.7.2 */ + .domainSetMemoryFlags = kvmtoolDomainSetMemoryFlags, /* 0.9.0 */ + .domainSetMemoryParameters = kvmtoolDomainSetMemoryParameters, /* 0.8.5 */ + .domainGetMemoryParameters = kvmtoolDomainGetMemoryParameters, /* 0.8.5 */ + .domainGetInfo = kvmtoolDomainGetInfo, /* 0.4.2 */ + .domainGetState = kvmtoolDomainGetState, /* 0.9.2 */ + .domainGetXMLDesc = kvmtoolDomainGetXMLDesc, /* 0.4.2 */ + .listDefinedDomains = kvmtoolListDefinedDomains, /* 0.4.2 */ + .numOfDefinedDomains = kvmtoolNumOfDefinedDomains, /* 0.4.2 */ + .domainCreate = kvmtoolDomainStart, /* 0.4.4 */ + .domainCreateWithFlags = kvmtoolDomainStartWithFlags, /* 0.8.2 */ + .domainDefineXML = kvmtoolDomainDefine, /* 0.4.2 */ + .domainUndefine = kvmtoolDomainUndefine, /* 0.4.2 */ + .domainUndefineFlags = kvmtoolDomainUndefineFlags, /* 0.9.4 */ + .domainGetAutostart = kvmtoolDomainGetAutostart, /* 0.7.0 */ + .domainSetAutostart = kvmtoolDomainSetAutostart, /* 0.7.0 */ + .domainGetSchedulerType = kvmtoolGetSchedulerType, /* 0.5.0 */ + .domainGetSchedulerParameters = kvmtoolGetSchedulerParameters, /* 0.5.0 */ + .domainGetSchedulerParametersFlags = kvmtoolGetSchedulerParametersFlags, /* 0.9.2 */ + .domainSetSchedulerParameters = kvmtoolSetSchedulerParameters, /* 0.5.0 */ + .domainSetSchedulerParametersFlags = kvmtoolSetSchedulerParametersFlags, /* 0.9.2 */ + .nodeGetCPUStats = nodeGetCPUStats, /* 0.9.3 */ + .nodeGetMemoryStats = nodeGetMemoryStats, /* 0.9.3 */ + .nodeGetCellsFreeMemory = nodeGetCellsFreeMemory, /* 0.6.5 */ + .nodeGetFreeMemory = nodeGetFreeMemory, /* 0.6.5 */ + .domainEventRegister = kvmtoolDomainEventRegister, /* 0.7.0 */ + .domainEventDeregister = kvmtoolDomainEventDeregister, /* 0.7.0 */ + .isEncrypted = kvmtoolIsEncrypted, /* 0.7.3 */ + .isSecure = kvmtoolIsSecure, /* 0.7.3 */ + .domainIsActive = kvmtoolDomainIsActive, /* 0.7.3 */ + .domainIsPersistent = kvmtoolDomainIsPersistent, /* 0.7.3 */ + .domainIsUpdated = kvmtoolDomainIsUpdated, /* 0.8.6 */ + .domainEventRegisterAny = kvmtoolDomainEventRegisterAny, /* 0.8.0 */ + .domainEventDeregisterAny = kvmtoolDomainEventDeregisterAny, /* 0.8.0 */ + .domainOpenConsole = kvmtoolDomainOpenConsole, /* 0.8.6 */ +}; + +static virStateDriver kvmtoolStateDriver = { + .name = "KVMTOOL", + .initialize = kvmtoolStartup, + .cleanup = kvmtoolShutdown, + .active = kvmtoolIsActive, + .reload = kvmtoolReload, +}; + +int kvmtoolRegister(void) +{ + virRegisterDriver(&kvmtoolDriver); + virRegisterStateDriver(&kvmtoolStateDriver); + return 0; +} diff --git a/src/kvmtool/kvmtool_driver.h b/src/kvmtool/kvmtool_driver.h new file mode 100644 index 0000000..84cc3db --- /dev/null +++ b/src/kvmtool/kvmtool_driver.h @@ -0,0 +1,29 @@ +/* + * Copyright (C) 2011 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Authors: Osier Yang <jyang@redhat.com> + */ + +#ifndef KVMTOOL_DRIVER_H +# define KVMTOOL_DRIVER_H + +# include <config.h> + +/* Function declarations */ +int kvmtoolRegister(void); + +#endif /* KVMTOOL_DRIVER_H */ -- 1.7.6
On Fri, Nov 11, 2011 at 07:57:06PM +0800, Osier Yang wrote:
Basically, the drivers is implemented by using kvm tool binary currently, (see ./kvm help for more info).
Current implementation supports define/undefine, start/destroy/, suspend/resume, connect to guest console via "virsh console", and balloon memory with with "virsh setmem" (using ./kvm balloon command). Also as it supports cgroup controllers "cpuacct", and "memory", so some other commands like "schedinfo", "memtune" can also work. Some other commands such as "domid", "domname", "dumpxml" ,"autostart", etc. are supported, as the driver is designed as a "stateful" driver, those APIs just need to talk with libvirtd simply.
As Native Linux KVM Tool is designed for both non-root and root users, the driver is designed just like QEMU, supports two modes of the connection:
kvmtool:///system kvmtool+unix:///system
kvmtool:///session kvmtool+unix:///session
An example of the domain XML (all the XMLs supported currently are listed):
% virsh -c kvm:///system dumpxml kvm_test <domain type='kvmtool'> <name>kvm_test</name> <uuid>88bf38f1-b6ab-cfa6-ab53-4b4c0993d894</uuid> <memory>524288</memory> <currentMemory>524288</currentMemory> <vcpu>1</vcpu> <os> <type arch='x86_64'>hvm</type> <kernel>/boot/bzImage</kernel> <boot dev='hd'/> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/kvmtool</emulator> <disk type='file' device='disk'> <source file='/var/lib/libvirt/images/linux-0.2.img'/> <target dev='vda' bus='virtio'/> </disk> <filesystem type='mount' accessmode='passthrough'> <source dir='/tmp'/> <target dir='/mnt'/> </filesystem> <console type='pty'> <target type='serial' port='0'/> </console> <memballoon model='virtio'/> </devices> </domain> --- cfg.mk | 1 + daemon/Makefile.am | 4 + daemon/libvirtd.c | 7 + po/POTFILES.in | 2 + src/Makefile.am | 36 +- src/kvmtool/kvmtool_conf.c | 130 ++ src/kvmtool/kvmtool_conf.h | 66 + src/kvmtool/kvmtool_driver.c | 3079 ++++++++++++++++++++++++++++++++++++++++++ src/kvmtool/kvmtool_driver.h | 29 +
My main suggestion here would be to split up the kvmtool_driver.c file into 3 parts as we did with the QEMU driver. kvmtool_driver.c -> Basic libvirt API glue kvmtool_command.c -> ARGV generation kvmtool_process.c -> KVMtool process start/stop/autostart/autodestroy Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2011年12月06日 22:55, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:57:06PM +0800, Osier Yang wrote:
Basically, the drivers is implemented by using kvm tool binary currently, (see ./kvm help for more info).
Current implementation supports define/undefine, start/destroy/, suspend/resume, connect to guest console via "virsh console", and balloon memory with with "virsh setmem" (using ./kvm balloon command). Also as it supports cgroup controllers "cpuacct", and "memory", so some other commands like "schedinfo", "memtune" can also work. Some other commands such as "domid", "domname", "dumpxml" ,"autostart", etc. are supported, as the driver is designed as a "stateful" driver, those APIs just need to talk with libvirtd simply.
As Native Linux KVM Tool is designed for both non-root and root users, the driver is designed just like QEMU, supports two modes of the connection:
kvmtool:///system kvmtool+unix:///system
kvmtool:///session kvmtool+unix:///session
An example of the domain XML (all the XMLs supported currently are listed):
% virsh -c kvm:///system dumpxml kvm_test <domain type='kvmtool'> <name>kvm_test</name> <uuid>88bf38f1-b6ab-cfa6-ab53-4b4c0993d894</uuid> <memory>524288</memory> <currentMemory>524288</currentMemory> <vcpu>1</vcpu> <os> <type arch='x86_64'>hvm</type> <kernel>/boot/bzImage</kernel> <boot dev='hd'/> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/kvmtool</emulator> <disk type='file' device='disk'> <source file='/var/lib/libvirt/images/linux-0.2.img'/> <target dev='vda' bus='virtio'/> </disk> <filesystem type='mount' accessmode='passthrough'> <source dir='/tmp'/> <target dir='/mnt'/> </filesystem> <console type='pty'> <target type='serial' port='0'/> </console> <memballoon model='virtio'/> </devices> </domain> --- cfg.mk | 1 + daemon/Makefile.am | 4 + daemon/libvirtd.c | 7 + po/POTFILES.in | 2 + src/Makefile.am | 36 +- src/kvmtool/kvmtool_conf.c | 130 ++ src/kvmtool/kvmtool_conf.h | 66 + src/kvmtool/kvmtool_driver.c | 3079 ++++++++++++++++++++++++++++++++++++++++++ src/kvmtool/kvmtool_driver.h | 29 +
My main suggestion here would be to split up the kvmtool_driver.c file into 3 parts as we did with the QEMU driver.
kvmtool_driver.c -> Basic libvirt API glue kvmtool_command.c -> ARGV generation kvmtool_process.c -> KVMtool process start/stop/autostart/autodestroy
Agreed. As a early thinking, kvmtool might has APIs exposed in future, how should we plan for it? A new driver just like libxl for XEN? or new backend driver like what we do for xm, xend for XEN driver? Should we consider the expansibility currently if we tend to use backend drivers? Regards, Osier
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
Hi, all
This is a basic implementation of libvirt Native Linux KVM Tool driver. Note that this is just made with my own interest and spare time, it's not an endorsement/effort by Red Hat, and it isn't supported by Red Hat officially.
Basically, the driver is designed as *stateful*, as KVM tool doesn't maintain any info about the guest except a socket which for its own IPC. And it's implemented by using KVM tool binary, which is name "kvm" currently, along with cgroup controllers "cpuacct", and "memory" support. And as one of KVM tool's pricinple is to allow both the non-root and root user to play with. The driver is designed to support root and non-root too, just like QEMU does. Example of the connection URI:
virsh -c kvmtool:///system virsh -c kvmtool:///session virsh -c kvmtool+unix:///system virsh -c kvmtool+unix:///session
The implementation can support more or less than 15 virsh commands currently, including basic domain cycle operations (define/undefine, start/destroy, suspend/resume, console, setmem, schedinfo, dumpxml, ,autostart, dominfo, etc.)
About the domain configuration: * "kernel": must be specified as KVM tool only support boots from the kernel currently (no integration with BIOS app yet).
* "disk": only virtio bus is supported, and device type must be 'disk'.
* "serial/console": only one console is supported, of type serial or virtio (can extend to support multiple console as long as kvm tool supports, libvirt already supported mutiple console, see upstream commit 0873b688c).
* "p9fs": only support specifying the source dir, and mount tag, only type of 'mount' is supported.
* "memballoon": only virtio is supported, and there is no way to config the addr.
* Multiple "disk" and "p9fs" is supported.
* Graphics and network are not supported, will explain below.
Please see "[PATCH 7/8]" for an example of the domain config. (which contains all the XMLs supported by current implementation).
The problems of Native Linux KVM Tool from libvirt p.o.v:
* Some destros package "qemu-kvm" as "kvm", also "kvm" is a long established name for "KVM" itself, so naming the project as "kvm" might be not a good idea. I assume it will be named as "kvmtool" in this implementation, never mind this if you don't like that, it can be updated easily. :-)
Yeah, naming the binary 'kvm' is just madness. I'd strongly recommend using 'kvmtool' as the binary name to avoid confusion with existing 'kvm' binaries based on QEMU.
* It still doesn't have an official package yet, even no "make install". means we have no way to check the dependancy and do the checking when 'configure'. I assume it will be installed as "/usr/bin/kvmtool" in this implementation. This is the main reason which can prevents upstream libvirt accepting the patches I guess.
Ok, not really a problem - we do similar for the regular QEMU driver.
* Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't.
Being able to specify a VNC port of libvirt's choosing is pretty much mandatory to be able to support that.In addition being able to specify the bind address is important to be able to control security. eg to only bind to 127.0.0.1, or only to certain NICs in a multi-NIC host.
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
* There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running.
Being able to reliably detect shutdown/exit of the KVM too is a very important tasks, and we can't rely on waitpid/SIG_CHLD because we want to daemonize all instances wrt libvirtd. In the QEMU driver we keep open a socket to the monitor, and when we see an I/O error / POLLHUP on the socket we know that QEMU has quit. What is this guest socket used for ? Could libvirt keep open a connection to it ? One other option would be to use inotify to watch for deletion of the guest socket in the filesystem. This is sortof what we do with the UML driver.
* KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user.
What does it do with the state dir ? Is that just for storing the guest socket ? With QEMU we chose $HOME/.libvirt/qemu or /var/run/libvirt because there was no policy set by QEMU itself. If KVM tool has a policy for where it stores its state, we should just use that, and not try to force it into a libvirt specific location. In a privileged libvirtd instace, we should aim to still have kvmtool itself run as an unprivilegd user / group , eg 'kvmtool:kvmtool' And we could set the home dir of that user to /var/lib/kvmtool
* kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package.
The version numbers libvirt reports for hypervisors are pretty meaningless really. In that example you give I'd just report '3.0' as the version from libvirt. Anything that relies on these versions from libvirt is doomed to be broken anyway.
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are - Mapping between PIDs and vCPU threads - Current balloon driver value Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, 2011-12-06 at 14:38 +0000, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
* Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't.
Being able to specify a VNC port of libvirt's choosing is pretty much mandatory to be able to support that.In addition being able to specify the bind address is important to be able to control security. eg to only bind to 127.0.0.1, or only to certain NICs in a multi-NIC host.
I'll add that feature in the next couple of days.
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
KVM tool currently creates and configures the TAP devices it uses, it shouldn't be an issue to have it use a TAP fd passed to it either. How does libvirt do it? Create a /dev/tapX on it's own and pass the fd to the hypervisor?
* There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running.
Being able to reliably detect shutdown/exit of the KVM too is a very important tasks, and we can't rely on waitpid/SIG_CHLD because we want to daemonize all instances wrt libvirtd.
In the QEMU driver we keep open a socket to the monitor, and when we see an I/O error / POLLHUP on the socket we know that QEMU has quit.
What is this guest socket used for ? Could libvirt keep open a connection to it ?
It's being used for communication with the IPC sub-commands (like 'kvm list', 'kvm debug', etc). It's basically the server in a server-client setup used to signal the hypervisor to do things. Theres also no problem with keeping an open connection to it.
One other option would be to use inotify to watch for deletion of the guest socket in the filesystem. This is sortof what we do with the UML driver.
* KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user.
What does it do with the state dir ? Is that just for storing the guest socket ?
afaik that patch should be already in as well. It does two things in the state dir: - Store sockets. - KVM tools has a feature which lets a user boot a guest based on virtio-9p which lets him see a system which is an exact copy of the host. This makes testing of programs and sandboxing very easy. The state files required for that are stored in that dir as well.
With QEMU we chose $HOME/.libvirt/qemu or /var/run/libvirt because there was no policy set by QEMU itself. If KVM tool has a policy for where it stores its state, we should just use that, and not try to force it into a libvirt specific location.
In a privileged libvirtd instace, we should aim to still have kvmtool itself run as an unprivilegd user / group , eg 'kvmtool:kvmtool' And we could set the home dir of that user to /var/lib/kvmtool
* kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package.
The version numbers libvirt reports for hypervisors are pretty meaningless really. In that example you give I'd just report '3.0' as the version from libvirt. Anything that relies on these versions from libvirt is doomed to be broken anyway.
The version is just a 'git describe' of the kernel tree in which it was built, so if you build KVM tools from an "official" kernel tree* you'll also get pretty versions :) * After KVM tools is merged...
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
KVM tools does a redirection to a PTY, which at that point could be redirected to anywhere the user wants. What features might be interesting to do on top of that?
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are
- Mapping between PIDs and vCPU threads - Current balloon driver value
Those are pretty easily added using the IPC interface I've mentioned above. For example, 'kvm balloon' and 'kvm stat' will return a lot of info out of the balloon driver (including the memory stats VQ - which afaik we're probably the only ones who actually do that (but I might be wrong) :) Any other commands are added on-demand. Just let us know what you want to see there. -- Sasha.
On Wed, Dec 07, 2011 at 08:21:06AM +0200, Sasha Levin wrote:
On Tue, 2011-12-06 at 14:38 +0000, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
KVM tool currently creates and configures the TAP devices it uses, it shouldn't be an issue to have it use a TAP fd passed to it either.
How does libvirt do it? Create a /dev/tapX on it's own and pass the fd to the hypervisor?
Yes, libvirt opens a /dev/tap device (or a macvtap device for VEPA mode), adds it to the neccessary bridge, and/or configures VEPA, etc and then passes the FD to the hypervisor, with a ARGV parameter to tell the HV which FD is being passed.
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
KVM tools does a redirection to a PTY, which at that point could be redirected to anywhere the user wants.
What features might be interesting to do on top of that?
I presume that Osier is just comparing with the features QEMU has available for chardevs config, which include - PTYs - UNIX sockets - TCP sockets - UDP sockets - FIFO pipe - Plain file (output only obviously, but useful for logging) libvirt doesn't specifically need any of them, but it can support those options if they exist.
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are
- Mapping between PIDs and vCPU threads - Current balloon driver value
Those are pretty easily added using the IPC interface I've mentioned above. For example, 'kvm balloon' and 'kvm stat' will return a lot of info out of the balloon driver (including the memory stats VQ - which afaik we're probably the only ones who actually do that (but I might be wrong) :)
Ok, that sounds sufficient for the balloon info. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2011年12月07日 17:16, Daniel P. Berrange wrote:
On Wed, Dec 07, 2011 at 08:21:06AM +0200, Sasha Levin wrote:
On Tue, 2011-12-06 at 14:38 +0000, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
KVM tool currently creates and configures the TAP devices it uses, it shouldn't be an issue to have it use a TAP fd passed to it either.
How does libvirt do it? Create a /dev/tapX on it's own and pass the fd to the hypervisor?
Yes, libvirt opens a /dev/tap device (or a macvtap device for VEPA mode), adds it to the neccessary bridge, and/or configures VEPA, etc and then passes the FD to the hypervisor, with a ARGV parameter to tell the HV which FD is being passed.
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
KVM tools does a redirection to a PTY, which at that point could be redirected to anywhere the user wants.
What features might be interesting to do on top of that?
I presume that Osier is just comparing with the features QEMU has available for chardevs config, which include
- PTYs - UNIX sockets - TCP sockets - UDP sockets - FIFO pipe - Plain file (output only obviously, but useful for logging)
Yes, that's what I meant. :-)
libvirt doesn't specifically need any of them, but it can support those options if they exist.
Yes, these won't prevent us, I just meant it will be great if they are supported.
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are
- Mapping between PIDs and vCPU threads - Current balloon driver value
Those are pretty easily added using the IPC interface I've mentioned above. For example, 'kvm balloon' and 'kvm stat' will return a lot of info out of the balloon driver (including the memory stats VQ - which afaik we're probably the only ones who actually do that (but I might be wrong) :)
Ok, that sounds sufficient for the balloon info.
Regards, Daniel
On 2011年12月07日 14:21, Sasha Levin wrote:
On Tue, 2011-12-06 at 14:38 +0000, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
* Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't.
Being able to specify a VNC port of libvirt's choosing is pretty much mandatory to be able to support that.In addition being able to specify the bind address is important to be able to control security. eg to only bind to 127.0.0.1, or only to certain NICs in a multi-NIC host.
I'll add that feature in the next couple of days.
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
KVM tool currently creates and configures the TAP devices it uses, it shouldn't be an issue to have it use a TAP fd passed to it either.
How does libvirt do it? Create a /dev/tapX on it's own and pass the fd to the hypervisor?
* There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running.
Being able to reliably detect shutdown/exit of the KVM too is a very important tasks, and we can't rely on waitpid/SIG_CHLD because we want to daemonize all instances wrt libvirtd.
In the QEMU driver we keep open a socket to the monitor, and when we see an I/O error / POLLHUP on the socket we know that QEMU has quit.
What is this guest socket used for ? Could libvirt keep open a connection to it ?
It's being used for communication with the IPC sub-commands (like 'kvm list', 'kvm debug', etc). It's basically the server in a server-client setup used to signal the hypervisor to do things.
Theres also no problem with keeping an open connection to it.
I'll update the codes to use it.
One other option would be to use inotify to watch for deletion of the guest socket in the filesystem. This is sortof what we do with the UML driver.
* KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user.
What does it do with the state dir ? Is that just for storing the guest socket ?
afaik that patch should be already in as well.
It does two things in the state dir:
- Store sockets. - KVM tools has a feature which lets a user boot a guest based on virtio-9p which lets him see a system which is an exact copy of the host. This makes testing of programs and sandboxing very easy. The state files required for that are stored in that dir as well.
With QEMU we chose $HOME/.libvirt/qemu or /var/run/libvirt because there was no policy set by QEMU itself. If KVM tool has a policy for where it stores its state, we should just use that, and not try to force it into a libvirt specific location.
In a privileged libvirtd instace, we should aim to still have kvmtool itself run as an unprivilegd user / group , eg 'kvmtool:kvmtool' And we could set the home dir of that user to /var/lib/kvmtool
* kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package.
The version numbers libvirt reports for hypervisors are pretty meaningless really. In that example you give I'd just report '3.0' as the version from libvirt. Anything that relies on these versions from libvirt is doomed to be broken anyway.
The version is just a 'git describe' of the kernel tree in which it was built, so if you build KVM tools from an "official" kernel tree* you'll also get pretty versions :)
* After KVM tools is merged...
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
KVM tools does a redirection to a PTY, which at that point could be redirected to anywhere the user wants.
What features might be interesting to do on top of that?
No others except the console. But as we have already implemented the console support using pty in libvirt self. So temporarily wont's use it.
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are
- Mapping between PIDs and vCPU threads - Current balloon driver value
Those are pretty easily added using the IPC interface I've mentioned above. For example, 'kvm balloon' and 'kvm stat' will return a lot of info out of the balloon driver (including the memory stats VQ - which afaik we're probably the only ones who actually do that (but I might be wrong) :)
Any other commands are added on-demand. Just let us know what you want to see there.
On 2011年12月06日 22:38, Daniel P. Berrange wrote:
On Fri, Nov 11, 2011 at 07:56:58PM +0800, Osier Yang wrote:
Hi, all
This is a basic implementation of libvirt Native Linux KVM Tool driver. Note that this is just made with my own interest and spare time, it's not an endorsement/effort by Red Hat, and it isn't supported by Red Hat officially.
Basically, the driver is designed as *stateful*, as KVM tool doesn't maintain any info about the guest except a socket which for its own IPC. And it's implemented by using KVM tool binary, which is name "kvm" currently, along with cgroup controllers "cpuacct", and "memory" support. And as one of KVM tool's pricinple is to allow both the non-root and root user to play with. The driver is designed to support root and non-root too, just like QEMU does. Example of the connection URI:
virsh -c kvmtool:///system virsh -c kvmtool:///session virsh -c kvmtool+unix:///system virsh -c kvmtool+unix:///session
The implementation can support more or less than 15 virsh commands currently, including basic domain cycle operations (define/undefine, start/destroy, suspend/resume, console, setmem, schedinfo, dumpxml, ,autostart, dominfo, etc.)
About the domain configuration: * "kernel": must be specified as KVM tool only support boots from the kernel currently (no integration with BIOS app yet).
* "disk": only virtio bus is supported, and device type must be 'disk'.
* "serial/console": only one console is supported, of type serial or virtio (can extend to support multiple console as long as kvm tool supports, libvirt already supported mutiple console, see upstream commit 0873b688c).
* "p9fs": only support specifying the source dir, and mount tag, only type of 'mount' is supported.
* "memballoon": only virtio is supported, and there is no way to config the addr.
* Multiple "disk" and "p9fs" is supported.
* Graphics and network are not supported, will explain below.
Please see "[PATCH 7/8]" for an example of the domain config. (which contains all the XMLs supported by current implementation).
The problems of Native Linux KVM Tool from libvirt p.o.v:
* Some destros package "qemu-kvm" as "kvm", also "kvm" is a long established name for "KVM" itself, so naming the project as "kvm" might be not a good idea. I assume it will be named as "kvmtool" in this implementation, never mind this if you don't like that, it can be updated easily. :-)
Yeah, naming the binary 'kvm' is just madness. I'd strongly recommend using 'kvmtool' as the binary name to avoid confusion with existing 'kvm' binaries based on QEMU.
* It still doesn't have an official package yet, even no "make install". means we have no way to check the dependancy and do the checking when 'configure'. I assume it will be installed as "/usr/bin/kvmtool" in this implementation. This is the main reason which can prevents upstream libvirt accepting the patches I guess.
Ok, not really a problem - we do similar for the regular QEMU driver.
* Lacks of options for user's configuration, such as "-vnc", there is no option for user to configure the properties for the "vnc", such as the port. It hides things, doesn't provide ways to query the properties too, this causes problems for libvirt to add the vnc support, as vnc clients such as virt-manager, virt-viewer, have no way to connect the guest. Even vncviewer can't.
Being able to specify a VNC port of libvirt's choosing is pretty much mandatory to be able to support that.In addition being able to specify the bind address is important to be able to control security. eg to only bind to 127.0.0.1, or only to certain NICs in a multi-NIC host.
* KVM tool manages the network completely itself (with DHCP support?), no way to configure, except specify the modes (user|tap|none). I have not test it yet, but it should need explicit script to setup the network rules(e.g. NAT) for the guest access outside world. Anyway, there is no way for libvirt to control the guest network.
If KVM tool support TAP devices, can't be do whatever we like with that just by passing in a configured TAP device from libvir ?
* There is a gap about the domain status between KVM tool and libvirt, it's caused by KVM tool unlink()s the guest socket when user exits from console (both text and graphic), but libvirt still think the guest is running.
Being able to reliably detect shutdown/exit of the KVM too is a very important tasks, and we can't rely on waitpid/SIG_CHLD because we want to daemonize all instances wrt libvirtd.
In the QEMU driver we keep open a socket to the monitor, and when we see an I/O error / POLLHUP on the socket we know that QEMU has quit.
What is this guest socket used for ? Could libvirt keep open a connection to it ?
One other option would be to use inotify to watch for deletion of the guest socket in the filesystem. This is sortof what we do with the UML driver.
* KVM tool uses $HOME/.kvm_tool as the state dir, and no way to configure, I made a small patch to allow KVM tool accept a ENV variable, which is "KVM_STATE_DIR", it's used across the driver. I made a simple patch against kvm tool to let the whole patches work. See "[PATCH] kvm tools.....". As generally we want the state dir of a driver can be "/var/run/libvirt/kvmtool/..." for root user or "$HOME/.libvirt/kvmtool/run" for non-root user.
What does it do with the state dir ? Is that just for storing the guest socket ?
With QEMU we chose $HOME/.libvirt/qemu or /var/run/libvirt because there was no policy set by QEMU itself. If KVM tool has a policy for where it stores its state, we should just use that, and not try to force it into a libvirt specific location.
In a privileged libvirtd instace, we should aim to still have kvmtool itself run as an unprivilegd user / group , eg 'kvmtool:kvmtool' And we could set the home dir of that user to /var/lib/kvmtool
Is there any harm if we force it into /var/run/libvirt/kvmtool? The only harm I can think of the user who uses kvmtool directly might be confused after turning into using libvirt.
* kvmtoolGetVersion is just broken now, as what "./kvm version" returns is something like "3.0.rc5.873.gb73216", however, libvirt wants things like "2.6.40.6-0". This might be not a problem as long as KVM tool has a official package.
The version numbers libvirt reports for hypervisors are pretty meaningless really. In that example you give I'd just report '3.0' as the version from libvirt. Anything that relies on these versions from libvirt is doomed to be broken anyway.
Okay, I thought the API wants the complete version. :-)
* console connection is implemented by setup ptys in libvirt, stdout/stderr of kvm tool process is redirected to the master pty, and libvirt connects to the slave pty. This works fine now, but it might be better if kvm tool could provide more advanced console mechanisms. Just like QEMU does?
This sounds good enough for now.
* Not much ways existed yet for external apps or user to query the guest informations. But this might be changed soon per KVM tool grows up quickly.
What sort of guest info are you thinking about ? The most immediate pieces of info I can imagine we need are
- Mapping between PIDs and vCPU threads - Current balloon driver value
My mainly concern was the balloon driver value, we want to display the value in domain's XML. Didn't think of the mapping between PIDs and vCPU threads, we might also want to see the disk info / stats. But that's not what can prevent us going forward. By the way, nobody is interested in kvmtool privodes a way to for external apps to get the capabilities? Do we still want to suffer from parsing the capabilities ourselves in future just like what we do for qemu? :-) Regards, Osier
On Fri, 2011-12-09 at 20:30 +0800, Osier Yang wrote:
By the way, nobody is interested in kvmtool privodes a way to for external apps to get the capabilities? Do we still want to suffer from parsing the capabilities ourselves in future just like what we do for qemu? :-)
I think the feature makes sense especially if it simplifies libvirt. Pekka
participants (4)
-
Daniel P. Berrange -
Osier Yang -
Pekka Enberg -
Sasha Levin