[libvirt] [PATCH] lxc: Add virCgroupSetOwner()
From: Richard Weinberger <richard@nod.at> Add a new helper function to change the permissions of a control group. This function is needed for user namespaces, we need to chmod() the cgroup to the initial uid/gid such that systemd is allowed to use the cgroup. Only the systemd controller is made accessible to the container. Others must remain read-only since it is generally not safe to delegate resource controller write access to unprivileged processes. Signed-off-by: Richard Weinberger <richard@nod.at> --- src/libvirt_private.syms | 1 + src/lxc/lxc_cgroup.c | 9 ++++++++ src/util/vircgroup.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++ src/util/vircgroup.h | 5 +++++ 4 files changed, 69 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0b28bac..cfa9f75 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1056,6 +1056,7 @@ virCgroupSetMemory; virCgroupSetMemoryHardLimit; virCgroupSetMemorySoftLimit; virCgroupSetMemSwapHardLimit; +virCgroupSetOwner; virCgroupSupportsCpuBW; diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index cc0d5e8..0d0d9c0 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -484,6 +484,15 @@ virCgroupPtr virLXCCgroupCreate(virDomainDefPtr def) &cgroup) < 0) goto cleanup; + /* setup control group permissions for user namespace */ + if (def->idmap.uidmap) { + if (virCgroupSetOwner(cgroup, + def->idmap.uidmap[0].target, + def->idmap.gidmap[0].target, + (1 << VIR_CGROUP_CONTROLLER_SYSTEMD))) + goto cleanup; + } + cleanup: return cgroup; } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index a6d60c5..2dc6986 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -3253,6 +3253,60 @@ cleanup: } +int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers) +{ + size_t i; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + char *base, *entry; + DIR *dh; + struct dirent *de; + + if (!((1 << i) & controllers)) + continue; + + if (!cgroup->controllers[i].mountPoint) + continue; + + if (virAsprintf(&base, "%s%s", cgroup->controllers[i].mountPoint, + cgroup->controllers[i].placement) < 0) { + virReportOOMError(); + return -1; + } + + dh = opendir(base); + while ((de = readdir(dh)) != NULL) { + if (STREQ(de->d_name, ".") || + STREQ(de->d_name, "..")) + continue; + + if (virAsprintf(&entry, "%s/%s", base, de->d_name) < 0) { + VIR_FREE(base); + virReportOOMError(); + } + + if (chown(entry, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + entry, uid, gid); + + VIR_FREE(entry); + } + closedir(dh); + + if (chown(base, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + base, uid, gid); + + VIR_FREE(base); + } + + return 0; +} + + /** * virCgroupSupportsCpuBW(): * Check whether the host supports CFS bandwidth. diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index a70eb18..38d94f3 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -225,4 +225,9 @@ int virCgroupIsolateMount(virCgroupPtr group, bool virCgroupSupportsCpuBW(virCgroupPtr cgroup); +int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers); + #endif /* __VIR_CGROUP_H__ */ -- 1.8.5.3
On Thu, Feb 13, 2014 at 05:15:22PM +0000, Daniel P. Berrange wrote:
From: Richard Weinberger <richard@nod.at>
Add a new helper function to change the permissions of a control group. This function is needed for user namespaces, we need to chmod() the cgroup to the initial uid/gid such that systemd is allowed to use the cgroup.
Only the systemd controller is made accessible to the container. Others must remain read-only since it is generally not safe to delegate resource controller write access to unprivileged processes.
Signed-off-by: Richard Weinberger <richard@nod.at> --- src/libvirt_private.syms | 1 + src/lxc/lxc_cgroup.c | 9 ++++++++ src/util/vircgroup.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++ src/util/vircgroup.h | 5 +++++ 4 files changed, 69 insertions(+)
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0b28bac..cfa9f75 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1056,6 +1056,7 @@ virCgroupSetMemory; virCgroupSetMemoryHardLimit; virCgroupSetMemorySoftLimit; virCgroupSetMemSwapHardLimit; +virCgroupSetOwner; virCgroupSupportsCpuBW;
diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index cc0d5e8..0d0d9c0 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -484,6 +484,15 @@ virCgroupPtr virLXCCgroupCreate(virDomainDefPtr def) &cgroup) < 0) goto cleanup;
+ /* setup control group permissions for user namespace */ + if (def->idmap.uidmap) { + if (virCgroupSetOwner(cgroup, + def->idmap.uidmap[0].target, + def->idmap.gidmap[0].target, + (1 << VIR_CGROUP_CONTROLLER_SYSTEMD)))
This should be "if (virCgroupSetOwner() < 0)" to go with the rest.
+ goto cleanup; + } +
virCgroupNewMachine() guarantees that the cgroup is NULL in case of an error, but you don't guarantee that in virCgroupSetOwner(), so the errors from it won't propagate anywhere, because you don't return NULL from this function.
cleanup: return cgroup; } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index a6d60c5..2dc6986 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -3253,6 +3253,60 @@ cleanup: }
+int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers) +{ + size_t i; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + char *base, *entry; + DIR *dh; + struct dirent *de; + + if (!((1 << i) & controllers)) + continue; + + if (!cgroup->controllers[i].mountPoint) + continue; + + if (virAsprintf(&base, "%s%s", cgroup->controllers[i].mountPoint, + cgroup->controllers[i].placement) < 0) { + virReportOOMError();
Double OOM reporting.
+ return -1; + } + + dh = opendir(base); + while ((de = readdir(dh)) != NULL) { + if (STREQ(de->d_name, ".") || + STREQ(de->d_name, "..")) + continue; + + if (virAsprintf(&entry, "%s/%s", base, de->d_name) < 0) { + VIR_FREE(base); + virReportOOMError();
Same here, plus you continue the loop and don't return -1.
+ } + + if (chown(entry, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + entry, uid, gid);
Indentation's off and you continue the loop again.
+ + VIR_FREE(entry); + } + closedir(dh); + + if (chown(base, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + base, uid, gid);
Again reporting an error, but returning 0 even in case of an error.
+ + VIR_FREE(base); + } + + return 0; +} + + /** * virCgroupSupportsCpuBW(): * Check whether the host supports CFS bandwidth. diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index a70eb18..38d94f3 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -225,4 +225,9 @@ int virCgroupIsolateMount(virCgroupPtr group,
bool virCgroupSupportsCpuBW(virCgroupPtr cgroup);
+int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers); + #endif /* __VIR_CGROUP_H__ */ -- 1.8.5.3
Am 14.02.2014 08:10, schrieb Martin Kletzander:
On Thu, Feb 13, 2014 at 05:15:22PM +0000, Daniel P. Berrange wrote:
From: Richard Weinberger <richard@nod.at>
Add a new helper function to change the permissions of a control group. This function is needed for user namespaces, we need to chmod() the cgroup to the initial uid/gid such that systemd is allowed to use the cgroup.
Only the systemd controller is made accessible to the container. Others must remain read-only since it is generally not safe to delegate resource controller write access to unprivileged processes.
Signed-off-by: Richard Weinberger <richard@nod.at> --- src/libvirt_private.syms | 1 + src/lxc/lxc_cgroup.c | 9 ++++++++ src/util/vircgroup.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++ src/util/vircgroup.h | 5 +++++ 4 files changed, 69 insertions(+)
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0b28bac..cfa9f75 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1056,6 +1056,7 @@ virCgroupSetMemory; virCgroupSetMemoryHardLimit; virCgroupSetMemorySoftLimit; virCgroupSetMemSwapHardLimit; +virCgroupSetOwner; virCgroupSupportsCpuBW;
diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index cc0d5e8..0d0d9c0 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -484,6 +484,15 @@ virCgroupPtr virLXCCgroupCreate(virDomainDefPtr def) &cgroup) < 0) goto cleanup;
+ /* setup control group permissions for user namespace */ + if (def->idmap.uidmap) { + if (virCgroupSetOwner(cgroup, + def->idmap.uidmap[0].target, + def->idmap.gidmap[0].target, + (1 << VIR_CGROUP_CONTROLLER_SYSTEMD)))
This should be "if (virCgroupSetOwner() < 0)" to go with the rest.
Ok.
+ goto cleanup; + } +
virCgroupNewMachine() guarantees that the cgroup is NULL in case of an error, but you don't guarantee that in virCgroupSetOwner(), so the errors from it won't propagate anywhere, because you don't return NULL from this function.
Do we really want to treat a failed chown() as fatal error?
cleanup: return cgroup; } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index a6d60c5..2dc6986 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -3253,6 +3253,60 @@ cleanup: }
+int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers) +{ + size_t i; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + char *base, *entry; + DIR *dh; + struct dirent *de; + + if (!((1 << i) & controllers)) + continue; + + if (!cgroup->controllers[i].mountPoint) + continue; + + if (virAsprintf(&base, "%s%s", cgroup->controllers[i].mountPoint, + cgroup->controllers[i].placement) < 0) { + virReportOOMError();
Double OOM reporting.
Ahh, virAsprintf() already reports the error...
+ return -1; + } + + dh = opendir(base); + while ((de = readdir(dh)) != NULL) { + if (STREQ(de->d_name, ".") || + STREQ(de->d_name, "..")) + continue; + + if (virAsprintf(&entry, "%s/%s", base, de->d_name) < 0) { + VIR_FREE(base); + virReportOOMError();
Same here, plus you continue the loop and don't return -1.
Ok!
+ } + + if (chown(entry, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + entry, uid, gid);
Indentation's off and you continue the loop again.
I continue here by design because I don't treat a failed chown() as fatal error.
+ + VIR_FREE(entry); + } + closedir(dh); + + if (chown(base, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + base, uid, gid);
Again reporting an error, but returning 0 even in case of an error.
Same here. Thanks, //richard
On Fri, Feb 14, 2014 at 08:47:37AM +0100, Richard Weinberger wrote:
Am 14.02.2014 08:10, schrieb Martin Kletzander:
On Thu, Feb 13, 2014 at 05:15:22PM +0000, Daniel P. Berrange wrote:
From: Richard Weinberger <richard@nod.at>
Add a new helper function to change the permissions of a control group. This function is needed for user namespaces, we need to chmod() the cgroup to the initial uid/gid such that systemd is allowed to use the cgroup.
Only the systemd controller is made accessible to the container. Others must remain read-only since it is generally not safe to delegate resource controller write access to unprivileged processes.
Signed-off-by: Richard Weinberger <richard@nod.at> --- src/libvirt_private.syms | 1 + src/lxc/lxc_cgroup.c | 9 ++++++++ src/util/vircgroup.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++ src/util/vircgroup.h | 5 +++++ 4 files changed, 69 insertions(+)
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0b28bac..cfa9f75 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1056,6 +1056,7 @@ virCgroupSetMemory; virCgroupSetMemoryHardLimit; virCgroupSetMemorySoftLimit; virCgroupSetMemSwapHardLimit; +virCgroupSetOwner; virCgroupSupportsCpuBW;
diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index cc0d5e8..0d0d9c0 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -484,6 +484,15 @@ virCgroupPtr virLXCCgroupCreate(virDomainDefPtr def) &cgroup) < 0) goto cleanup;
+ /* setup control group permissions for user namespace */ + if (def->idmap.uidmap) { + if (virCgroupSetOwner(cgroup, + def->idmap.uidmap[0].target, + def->idmap.gidmap[0].target, + (1 << VIR_CGROUP_CONTROLLER_SYSTEMD)))
This should be "if (virCgroupSetOwner() < 0)" to go with the rest.
Ok.
+ goto cleanup; + } +
virCgroupNewMachine() guarantees that the cgroup is NULL in case of an error, but you don't guarantee that in virCgroupSetOwner(), so the errors from it won't propagate anywhere, because you don't return NULL from this function.
Do we really want to treat a failed chown() as fatal error?
I'm not saying either way, but if you're not using the error (or you don't want that error to be used, than don't report it with virReportError() and use VIR_WARN() for example. However, if the called function should report an error and this is the only case which should not do it (an exception), then reset the error at least.
cleanup: return cgroup; } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index a6d60c5..2dc6986 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -3253,6 +3253,60 @@ cleanup: }
+int virCgroupSetOwner(virCgroupPtr cgroup, + uid_t uid, + gid_t gid, + int controllers) +{ + size_t i; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + char *base, *entry; + DIR *dh; + struct dirent *de; + + if (!((1 << i) & controllers)) + continue; + + if (!cgroup->controllers[i].mountPoint) + continue; + + if (virAsprintf(&base, "%s%s", cgroup->controllers[i].mountPoint, + cgroup->controllers[i].placement) < 0) { + virReportOOMError();
Double OOM reporting.
Ahh, virAsprintf() already reports the error...
+ return -1; + } + + dh = opendir(base); + while ((de = readdir(dh)) != NULL) { + if (STREQ(de->d_name, ".") || + STREQ(de->d_name, "..")) + continue; + + if (virAsprintf(&entry, "%s/%s", base, de->d_name) < 0) { + VIR_FREE(base); + virReportOOMError();
Same here, plus you continue the loop and don't return -1.
Ok!
+ } + + if (chown(entry, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + entry, uid, gid);
Indentation's off and you continue the loop again.
I continue here by design because I don't treat a failed chown() as fatal error.
+ + VIR_FREE(entry); + } + closedir(dh); + + if (chown(base, uid, gid) < 0) + virReportSystemError(errno, _("cannot chown '%s' to (%u, %u)"), + base, uid, gid);
Again reporting an error, but returning 0 even in case of an error.
Same here.
Thanks, //richard
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
this patch works for me. container is starting fine. but a login (ssh/console) is not possible. host: centos6 kernel: 3.13.2 libvirt: 1.2.1 (+ lxc: Add virCgroupSetOwner()) container logs says agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot set process group: Inappropriate ioctl for device in my container: # ls -la /dev/ total 4 drwxr-xr-x 3 root root 320 Feb 14 10:06 . dr-xr-xr-x 20 root root 4096 Feb 13 10:06 .. lrwxrwxrwx 1 root root 10 Feb 14 10:06 console -> /dev/pts/0 lrwxrwxrwx 1 root root 13 Feb 14 10:06 fd -> /proc/self/fd crw-rw-rw- 1 root root 1, 7 Feb 14 10:06 full crw-rw-rw- 1 root root 1, 3 Feb 14 10:06 null crw-rw-rw- 1 root root 5, 2 Feb 14 10:06 ptmx drwxr-xr-x 2 root root 0 Feb 14 10:06 pts crw-rw-rw- 1 root root 1, 8 Feb 14 10:06 random lrwxrwxrwx 1 root root 15 Feb 14 10:06 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 root root 15 Feb 14 10:06 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 root root 15 Feb 14 10:06 stdout -> /proc/self/fd/1 crw-rw-rw- 1 root root 5, 0 Feb 14 10:06 tty lrwxrwxrwx 1 root root 10 Feb 14 10:06 tty1 -> /dev/pts/0 crw-rw-rw- 1 root root 1, 9 Feb 14 10:06 urandom crw-rw-rw- 1 root root 1, 5 Feb 14 10:06 zero # ls -la /dev/pts/ total 0 drwxr-xr-x 2 root root 0 Feb 14 10:06 . drwxr-xr-x 3 root root 320 Feb 14 10:06 .. crw--w---- 1 root root 136, 0 Feb 14 10:07 0 crw-rw-rw- 1 root root 5, 2 Feb 14 10:07 ptmx "chgrp tty /dev/pts/0" and then "exec /sbin/init systemd.log_level=debug" has no effect. still the same error "cannot get controlling tty: Operation not permitted" -- Software is like sex, it's better when it's free!
Am 14.02.2014 11:21, schrieb Stephan Sachse:
this patch works for me. container is starting fine. but a login (ssh/console) is not possible.
Thanks for testing! But I fear my patch is not the culprit for your login issues.
host: centos6 kernel: 3.13.2 libvirt: 1.2.1 (+ lxc: Add virCgroupSetOwner())
container logs says
agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot set process group: Inappropriate ioctl for device
Is this really the reason why the login fails? Does getty not start or does the login fail? I've never tested an upstart based distro maybe it is somehow confused. :) Can you please collect more details why a login via ssh is failing? Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units... Thanks, //richard
agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot set process group: Inappropriate ioctl for device
Is this really the reason why the login fails? Does getty not start or does the login fail?
yes, the getty is started 100000 26665 0.0 0.0 6412 792 pts/0 Ss+ 13:22 0:00 /sbin/agetty --noclear tty1 100000 26667 0.0 0.0 6412 848 ? Ss 13:22 0:00 /sbin/agetty --noclear -s console 115200 38400 9600 and i can connect with "fedora1 console" virsh # console fedora1 Verbunden mit der Domain: fedora1 Escape-Zeichen ist ^] Fedora release 20 (Heisenbug) Kernel 3.13.2-2.el6.x86_64 on an x86_64 (tty1) fedora1 login: put the console is "crazy". wrong line breaks after hit enter. first two logins fails with wrong log entries. the password is displayed as login. the third attemot works. then i have a mix of console and login promt. every 10 secons i hit enter i see a passwort promt. but i can still use the shell. but imo this is a other problem. there are messages about systemd --user can not connect the dbus socket in /run/user/0/dbus/user_bus_socket
I've never tested an upstart based distro maybe it is somehow confused. :)
forgot to say the container is fedora 20
Can you please collect more details why a login via ssh is failing?
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug" /stephan -- Software is like sex, it's better when it's free!
Am 14.02.2014 13:42, schrieb Stephan Sachse:
agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot set process group: Inappropriate ioctl for device
Is this really the reason why the login fails? Does getty not start or does the login fail?
yes, the getty is started
100000 26665 0.0 0.0 6412 792 pts/0 Ss+ 13:22 0:00 /sbin/agetty --noclear tty1 100000 26667 0.0 0.0 6412 848 ? Ss 13:22 0:00 /sbin/agetty --noclear -s console 115200 38400 9600
and i can connect with "fedora1 console"
virsh # console fedora1 Verbunden mit der Domain: fedora1 Escape-Zeichen ist ^]
Fedora release 20 (Heisenbug) Kernel 3.13.2-2.el6.x86_64 on an x86_64 (tty1)
fedora1 login:
put the console is "crazy". wrong line breaks after hit enter. first two logins fails with wrong log entries. the password is displayed as login. the third attemot works. then i have a mix of console and login promt. every 10 secons i hit enter i see a passwort promt. but i can still use the shell. but imo this is a other problem. there are messages about systemd --user can not connect the dbus socket in /run/user/0/dbus/user_bus_socket
Looks like you have a getty on /dev/console _and_ /dev/tty1 ...both are symlinks to the same pts fake console.
I've never tested an upstart based distro maybe it is somehow confused. :)
forgot to say the container is fedora 20
Ah, thought it is CentOS 6 too.
Can you please collect more details why a login via ssh is failing?
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Hmmm, no idea so far. Have you disabled pam_loginuid? Maybe SELinux hates you too... Thanks, //richard
Am 14.02.2014 13:42, schrieb Stephan Sachse:
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug"
/stephan
I have the same problem on a slightly different setup. Both host and guest are Archlinux with systemd-208, libvirt-git with the chown() patches. LXC Console login works fine (and I do not have such issues with messed up console), but login via SSH fails with the exact same symptoms. ~tom
On Fri, Feb 14, 2014 at 02:17:24PM +0100, Tom Kuther wrote:
Am 14.02.2014 13:42, schrieb Stephan Sachse:
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug"
/stephan
I have the same problem on a slightly different setup. Both host and guest are Archlinux with systemd-208, libvirt-git with the chown() patches.
LXC Console login works fine (and I do not have such issues with messed up console), but login via SSH fails with the exact same symptoms.
Most likely is the pam_loginuid module denying access. Sadly I find debugging PAM a complete pain - if anyone knows how to make it spew logs for each module executed and then accept/reject state, that'd be awesome for troubleshooting this. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Am 14.02.2014 14:32, schrieb Daniel P. Berrange:
On Fri, Feb 14, 2014 at 02:17:24PM +0100, Tom Kuther wrote:
Am 14.02.2014 13:42, schrieb Stephan Sachse:
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug"
/stephan
I have the same problem on a slightly different setup. Both host and guest are Archlinux with systemd-208, libvirt-git with the chown() patches.
LXC Console login works fine (and I do not have such issues with messed up console), but login via SSH fails with the exact same symptoms.
Most likely is the pam_loginuid module denying access. Sadly I find debugging PAM a complete pain - if anyone knows how to make it spew logs for each module executed and then accept/reject state, that'd be awesome for troubleshooting this.
Daniel
I have that disabled. As I wrote in the other mail, it seems to be a funny DNS lookup problem. Setting UseDNS=no in sshd_config fixes it. But there is a more general problem with local LAN DNS lookup. I do have set the router's DNS server in /etc/resolv.conf, yet I cannot reach any clients on the LAN using their lan hostname.domainname - this works fine when not using user namespace. Using their IP works, Internet DNS lookup works, too. Interface type for the container is bridge via the hosts's br0. No idea if this could be a libvirt, kernel or systemd problem. ~tom
Am 14.02.2014 14:17, schrieb Tom Kuther:
Am 14.02.2014 13:42, schrieb Stephan Sachse:
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug"
/stephan
I have the same problem on a slightly different setup. Both host and guest are Archlinux with systemd-208, libvirt-git with the chown() patches.
LXC Console login works fine (and I do not have such issues with messed up console), but login via SSH fails with the exact same symptoms.
~tom
Setting UseDNS=no in the container's sshd_config fixes this. I have no idea why that happens. DNS lookups generally do work in the container. ~tom
On Fri, Feb 14, 2014 at 2:17 PM, Tom Kuther <tom@kuther.net> wrote:
Am 14.02.2014 13:42, schrieb Stephan Sachse:
set LogLevel to DEBUG3. keyexchange is down. put then hangs for some time und sshd dies
sshd[269]: debug1: KEX done [preauth] sshd[269]: debug1: userauth-request for user root service ssh-connection method none [preauth] sshd[269]: debug1: attempt 0 failures 0 [preauth] sshd[269]: debug3: mm_getpwnamallow entering [preauth] sshd[269]: debug3: mm_request_send entering: type 8 [preauth] sshd[269]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] sshd[269]: debug3: mm_request_receive_expect entering: type 9 [preauth] sshd[269]: debug3: mm_request_receive entering [preauth] sshd[269]: debug3: mm_request_receive entering sshd[269]: debug3: monitor_read: checking request 8 sshd[269]: debug3: mm_answer_pwnamallow sshd[269]: debug3: Trying to reverse map address 10.1.25.151. systemd[1]: Received SIGCHLD from PID 270 (sshd). systemd[1]: Got SIGCHLD for process 270 (sshd) systemd[1]: Child 270 died (code=killed, status=15/TERM)
Also keep in mind that running a compete distro within LXC + user namespaces requires some changes. Like disabling pam_loginuid.so in pam. For systemd distros you have to remove OOMScoreAdjust= and CapabilityBoundingSet= from all units...
yes, i know. i have no errors from systemd, all looks fine with "exec /sbin/init systemd.log_level=debug"
/stephan
I have the same problem on a slightly different setup. Both host and guest are Archlinux with systemd-208, libvirt-git with the chown() patches.
LXC Console login works fine (and I do not have such issues with messed up console), but login via SSH fails with the exact same symptoms.
~tom
Please don't crop CC lists. -- Thanks, //richard
On Fri, Feb 14, 2014 at 01:42:25PM +0100, Stephan Sachse wrote:
agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot get controlling tty: Operation not permitted agetty[38]: /dev/tty1: cannot set process group: Inappropriate ioctl for device
Is this really the reason why the login fails? Does getty not start or does the login fail?
yes, the getty is started
100000 26665 0.0 0.0 6412 792 pts/0 Ss+ 13:22 0:00 /sbin/agetty --noclear tty1 100000 26667 0.0 0.0 6412 848 ? Ss 13:22 0:00 /sbin/agetty --noclear -s console 115200 38400 9600
and i can connect with "fedora1 console"
virsh # console fedora1 Verbunden mit der Domain: fedora1 Escape-Zeichen ist ^]
Fedora release 20 (Heisenbug) Kernel 3.13.2-2.el6.x86_64 on an x86_64 (tty1)
fedora1 login:
put the console is "crazy". wrong line breaks after hit enter. first two logins fails with wrong log entries. the password is displayed as login. the third attemot works. then i have a mix of console and login promt. every 10 secons i hit enter i see a passwort promt. but i can still use the shell. but imo this is a other problem. there are messages about systemd --user can not connect the dbus socket in /run/user/0/dbus/user_bus_socket
I believe you might need a newer systemd or libvirt. Libvirt creates /dev/ttyN from 1-> number of <console> elements in the XML, and also makes /dev/console a symlink to /dev/ttyN. If you see double logins this is a good sign that an agetty has been started on both /dev/tty1 and /dev/console. A month or so back we made libvirt set "container_ttys" to list all /dev/ttyN devices except for /dev/tty1. systemd GIT will honour this env var when deciding how to spawn agettys on the /dev/ttyN devices. That said unless you've done some custom config I'd not expect an outdated systemd to launch anything on /dev/ttyN devices, only /dev/console. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
next "problem": on centos6 there is no systemd. the init of the cgroups is handled by cgconfig. per default cgroups are mounted below /cgroup. i change /etc/cgconfig.conf to mount all cgroups below /sys/fs/cgroup. but thats not enough for the systemd in the container. i must add ""name=systemd" = /sys/fs/cgroup/systemd;" to /etc/cgconfig.conf. is this secure? is it ok to run more then one systemd container with this config? maybe libvirt should init the name=systemd thing. /stephan -- Software is like sex, it's better when it's free!
participants (6)
-
Daniel P. Berrange -
Martin Kletzander -
Richard Weinberger -
Richard Weinberger -
Stephan Sachse -
Tom Kuther