Re: [libvirt] [PATCH v2] Make LXC container startup/shutdown/I/O more robust

The current LXC I/O controller looks for HUP to detect when a guest has quit. This isn't reliable as during initial bootup it is possible that 'init' will close the console and let mingetty re-open it. The shutdown of containers was also flakey because it only killed the libvirt I/O controller and expected container processes to gracefully follow. Change the I/O controller such that when it see HUP or an I/O error, it uses kill($PID, 0) to see if the process has really quit. Change the container shutdown sequence to use the virCgroupKillPainfully function to ensure every really goes away This change makes the use of the 'cpu', 'cpuacct' and 'memory' cgroups controllers compulsory with LXC

* docs/drvlxc.html.in: Document that certain cgroups controllers are now mandatory * src/lxc/lxc_controller.c: Check if PID is still alive before quitting on I/O error/HUP * src/lxc/lxc_driver.c: Use virCgroupKillPainfully --- docs/drvlxc.html.in | 18 +++++ src/lxc/lxc_controller.c | 42 +++++++++--- src/lxc/lxc_driver.c | 155 ++++++++++++++++++---------------------------- 3 files changed, 110 insertions(+), 105 deletions(-) diff --git a/docs/drvlxc.html.in b/docs/drvlxc.html.in index 35058c4..3e715b1 100644 --- a/docs/drvlxc.html.in +++ b/docs/drvlxc.html.in @@ -9,6 +9,24 @@ light-weight "application container" which does not have it's own root image. Y start it using </p> +<h2>Cgroups Requirements</h2> + +<p> +The libvirt LXC driver requires that certain cgroups controllers are +mounted on the host OS. The minimum required controllers are 'cpuacct', +'memory' and 'devices', while recommended extra controllers are +'cpu', 'freezer' and 'blkio'. The /etc/cgconfig.conf &amp; cgconfig +init service used to mount cgroups at host boot time. To manually +mount them use.

NB, the blkio controller in some kernels will not +allow creation of nested sub-directories which will prevent correct +operation of the libvirt LXC driver. On such kernels the blkio controller +must not be mounted. +</p> + +<pre> + # mount -t cgroup cgroup /dev/cgroup -o cpuacct,memory,devices,cpu,freezer,blkio +</pre> + <h3>Example config version 1</h3> <p></p> <pre> diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index d2b113c..61e21c3 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -341,7 +352,8 @@ ignorable_epoll_accept_errno(int errnum) static int lxcControllerMain(int monitor, int client, int appPty, - int contPty) + int contPty, + pid_t container) { int rc = -1; int epollFd; @@ -447,7 +459,13 @@ static int lxcControllerMain(int monitor, ++numActive; } } else if (epollEvent.events & EPOLLHUP) { - VIR_DEBUG("EPOLLHUP from fd %d", epollEvent.data.fd); + if (lxcPidGone(container)) + goto cleanup; + curFdOff = epollEvent.data.fd == appPty ? 0 : 1; + if (fdArray[curFdOff].active) { + fdArray[curFdOff].active = 0; + --numActive; + } continue;

} else { lxcError(VIR_ERR_INTERNAL_ERROR,

diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 5b6f784..7aaa5cd 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -952,36 +952,16 @@ cleanup: * @driver: pointer to driver structure * @vm: pointer to VM to clean up * - * waitpid() on the container process. kill and wait the tty process - * This is called by both lxcDomainDestroy and lxcSigHandler when a - * container exits. - * - * Returns 0 on success or -1 in case of error + * Cleanout resources associated with the now dead VM + *

*/ -static int lxcVmCleanup(lxc_driver_t *driver, +static void lxcVmCleanup(lxc_driver_t *driver, virDomainObjPtr vm) {

@@ -2844,7 +2811,7 @@ static virDriver lxcDriver = { lxcDomainLookupByName, /* domainLookupByName */ lxcDomainSuspend, /* domainSuspend */ lxcDomainResume, /* domainResume */ - lxcDomainShutdown, /* domainShutdown */ + NULL, /* domainShutdown */ NULL, /* domainReboot */ lxcDomainDestroy, /* domainDestroy */ lxcGetOSType, /* domainGetOSType */