Re: [libvirt] possible 0.9.8 regression?
xhu wrote:
Hi All,
I've noticed a regression in libvirt 0.9.8 on some of my kvm test machines
# virsh start opensuse12 error: Failed to start domain opensuse12 error: Cannot open network interface control socket: Permission denied For I can't reproduce it on my machine with 0.9.8, can you provide me
On 12/16/2011 11:33 AM, Jim Fehlig wrote: the detailed steps?
Nothing special, basic domain config using file-backed disk and connecting to a bridge.
Also your os, libvirt, qemu-kvm and kernel version?
Yeah, it has something to do with the kernel, glibc, or other such component. qemu-kvm isn't the problem as the error occurs before it is invoked. kernel 3.1.0, glibc 2.14.1 (openSUSE12.1): With libvirt 0.9.7, starting the domain works. This version of libvirt opens control socket with 'socket(AF_INET, SOCK_STREAM, 0)'. With libvirt 0.9.8, the domain does not start. In this version, the control socket is opened with 'socket(AF_PACKET, SOCK_DGRAM, 0)', which fails with EACCES. kernel 3.0.13, glibc 2.11.3 (SLES11 SP2): Regression between libvirt 0.9.7 and 0.9.8 not observed. Initially, I assumed the bug was in glibc. But I can open packet(7) sockets in a test program running as uid=euid=0, just not within libvirtd running with same privileges. Regards, Jim
Thanks!
Opening a control socket for setting MAC addr, etc. failed with EACCES. In 0.9.7, the socket was opened with domain AF_INET, type SOCK_STREAM, which of course works on this system. In 0.9.8, the socket is opened with AF_PACKET, SOCK_DGRAM. Interestingly, a small test program calling 'socket(AF_PACKET, SOCK_DGRAM, 0)' works on this system.
libvirt is built with '--without-capng --without-apparmor --without-selinux' and libvirtd is running with uid=euid=0.
I'm really baffled why this fails in libvirtd but works otherwise. Any ideas?
Thanks, Jim
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Tue, Dec 20, 2011 at 08:59:48AM -0700, Jim Fehlig wrote:
xhu wrote:
Hi All,
I've noticed a regression in libvirt 0.9.8 on some of my kvm test machines
# virsh start opensuse12 error: Failed to start domain opensuse12 error: Cannot open network interface control socket: Permission denied For I can't reproduce it on my machine with 0.9.8, can you provide me
On 12/16/2011 11:33 AM, Jim Fehlig wrote: the detailed steps?
Nothing special, basic domain config using file-backed disk and connecting to a bridge.
Also your os, libvirt, qemu-kvm and kernel version?
Yeah, it has something to do with the kernel, glibc, or other such component. qemu-kvm isn't the problem as the error occurs before it is invoked.
kernel 3.1.0, glibc 2.14.1 (openSUSE12.1): With libvirt 0.9.7, starting the domain works. This version of libvirt opens control socket with 'socket(AF_INET, SOCK_STREAM, 0)'. With libvirt 0.9.8, the domain does not start. In this version, the control socket is opened with 'socket(AF_PACKET, SOCK_DGRAM, 0)', which fails with EACCES.
kernel 3.0.13, glibc 2.11.3 (SLES11 SP2): Regression between libvirt 0.9.7 and 0.9.8 not observed.
Initially, I assumed the bug was in glibc. But I can open packet(7) sockets in a test program running as uid=euid=0, just not within libvirtd running with same privileges.
Interesting, this is very bizarre. I assume that if you patch libvirt 0.9.8 to use AF_INET again, it'll work fine ? Is there any other access control mechanism in force like SELinux or AppArmour ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Daniel P. Berrange wrote:
On Tue, Dec 20, 2011 at 08:59:48AM -0700, Jim Fehlig wrote:
xhu wrote:
On 12/16/2011 11:33 AM, Jim Fehlig wrote:
Hi All,
I've noticed a regression in libvirt 0.9.8 on some of my kvm test machines
# virsh start opensuse12 error: Failed to start domain opensuse12 error: Cannot open network interface control socket: Permission denied
For I can't reproduce it on my machine with 0.9.8, can you provide me the detailed steps?
Nothing special, basic domain config using file-backed disk and connecting to a bridge.
Also your os, libvirt, qemu-kvm and kernel version?
Yeah, it has something to do with the kernel, glibc, or other such component. qemu-kvm isn't the problem as the error occurs before it is invoked.
kernel 3.1.0, glibc 2.14.1 (openSUSE12.1): With libvirt 0.9.7, starting the domain works. This version of libvirt opens control socket with 'socket(AF_INET, SOCK_STREAM, 0)'. With libvirt 0.9.8, the domain does not start. In this version, the control socket is opened with 'socket(AF_PACKET, SOCK_DGRAM, 0)', which fails with EACCES.
kernel 3.0.13, glibc 2.11.3 (SLES11 SP2): Regression between libvirt 0.9.7 and 0.9.8 not observed.
Initially, I assumed the bug was in glibc. But I can open packet(7) sockets in a test program running as uid=euid=0, just not within libvirtd running with same privileges.
Interesting, this is very bizarre. I assume that if you patch libvirt 0.9.8 to use AF_INET again, it'll work fine ?
Yes, it is bizarre and yes, using AF_INET works.
Is there any other access control mechanism in force like SELinux or AppArmour ?
No, which is why I'm rather confused... Thanks, Jim
On Tue, Dec 20, 2011 at 12:07:00PM -0700, Jim Fehlig wrote:
Daniel P. Berrange wrote:
On Tue, Dec 20, 2011 at 08:59:48AM -0700, Jim Fehlig wrote:
xhu wrote:
On 12/16/2011 11:33 AM, Jim Fehlig wrote:
Hi All,
I've noticed a regression in libvirt 0.9.8 on some of my kvm test machines
# virsh start opensuse12 error: Failed to start domain opensuse12 error: Cannot open network interface control socket: Permission denied
For I can't reproduce it on my machine with 0.9.8, can you provide me the detailed steps?
Nothing special, basic domain config using file-backed disk and connecting to a bridge.
Also your os, libvirt, qemu-kvm and kernel version?
Yeah, it has something to do with the kernel, glibc, or other such component. qemu-kvm isn't the problem as the error occurs before it is invoked.
kernel 3.1.0, glibc 2.14.1 (openSUSE12.1): With libvirt 0.9.7, starting the domain works. This version of libvirt opens control socket with 'socket(AF_INET, SOCK_STREAM, 0)'. With libvirt 0.9.8, the domain does not start. In this version, the control socket is opened with 'socket(AF_PACKET, SOCK_DGRAM, 0)', which fails with EACCES.
kernel 3.0.13, glibc 2.11.3 (SLES11 SP2): Regression between libvirt 0.9.7 and 0.9.8 not observed.
Initially, I assumed the bug was in glibc. But I can open packet(7) sockets in a test program running as uid=euid=0, just not within libvirtd running with same privileges.
Interesting, this is very bizarre. I assume that if you patch libvirt 0.9.8 to use AF_INET again, it'll work fine ?
Yes, it is bizarre and yes, using AF_INET works.
Is there any other access control mechanism in force like SELinux or AppArmour ?
No, which is why I'm rather confused...
Do you have a stack trace for the socket() call which generates EACCESS ? I'm wondering if there is any chance that the call is being made during the startup of QEMU inbetween fork() & exec() where we might have already dropped some capabilities ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Daniel P. Berrange wrote:
On Tue, Dec 20, 2011 at 12:07:00PM -0700, Jim Fehlig wrote:
Daniel P. Berrange wrote:
On Tue, Dec 20, 2011 at 08:59:48AM -0700, Jim Fehlig wrote:
xhu wrote:
On 12/16/2011 11:33 AM, Jim Fehlig wrote:
Hi All,
I've noticed a regression in libvirt 0.9.8 on some of my kvm test machines
# virsh start opensuse12 error: Failed to start domain opensuse12 error: Cannot open network interface control socket: Permission denied
For I can't reproduce it on my machine with 0.9.8, can you provide me the detailed steps?
Nothing special, basic domain config using file-backed disk and connecting to a bridge.
Also your os, libvirt, qemu-kvm and kernel version?
Yeah, it has something to do with the kernel, glibc, or other such component. qemu-kvm isn't the problem as the error occurs before it is invoked.
kernel 3.1.0, glibc 2.14.1 (openSUSE12.1): With libvirt 0.9.7, starting the domain works. This version of libvirt opens control socket with 'socket(AF_INET, SOCK_STREAM, 0)'. With libvirt 0.9.8, the domain does not start. In this version, the control socket is opened with 'socket(AF_PACKET, SOCK_DGRAM, 0)', which fails with EACCES.
kernel 3.0.13, glibc 2.11.3 (SLES11 SP2): Regression between libvirt 0.9.7 and 0.9.8 not observed.
Initially, I assumed the bug was in glibc. But I can open packet(7) sockets in a test program running as uid=euid=0, just not within libvirtd running with same privileges.
Interesting, this is very bizarre. I assume that if you patch libvirt 0.9.8 to use AF_INET again, it'll work fine ?
Yes, it is bizarre and yes, using AF_INET works.
Is there any other access control mechanism in force like SELinux or AppArmour ?
No, which is why I'm rather confused...
Do you have a stack trace for the socket() call which generates EACCESS ? I'm wondering if there is any chance that the call is being made during the startup of QEMU inbetween fork() & exec() where we might have already dropped some capabilities ?
I don't think this is the case. IIUC, the qemu process hasn't been started yet: Breakpoint 1, virNetDevSetupControlFull (ifname=0x7fca880e7b90 "vnet0", ifr=0x7fca92900c90, domain=17, type=2) at util/virnetdev.c:55 55 { (gdb) n 58 memset(ifr, 0, sizeof(*ifr)); (gdb) 60 if (virStrcpyStatic(ifr->ifr_name, ifname) == NULL) { (gdb) 67 fd = socket(domain, type, 0); (gdb) 68 if (fd < 0) { (gdb) p fd $1 = -1 (gdb) p errno $2 = 13 (gdb) bt #0 virNetDevSetupControlFull (ifname=0x7fca880e7b90 "vnet0", ifr=0x7fca92900c90, domain=17, type=2) at util/virnetdev.c:68 #1 0x00007fca99c36d4a in virNetDevSetupControl (ifname=0x7fca880e7b90 "vnet0", ifr=0x7fca92900c90) at util/virnetdev.c:88 #2 0x00007fca99c36eab in virNetDevSetMAC (ifname=0x7fca880e7b90 "vnet0", macaddr=0x7fca92900db0 "\376T") at util/virnetdev.c:154 #3 0x00007fca99c3bdef in virNetDevTapCreateInBridgePort (brname=0x7fca880e7b70 "br0", ifname=0x7fca880e9b78, macaddr=0x7fca92900db0 "\376T", vnet_hdr=1, up=true, tapfd=0x7fca92900d8c) at util/virnetdevtap.c:279 #4 0x000000000047e6f5 in qemuNetworkIfaceConnect (def=0x7fca880e9190, conn=0x7fca84000ae0, driver=0x7fca88068b10, net=0x7fca880e9b20, qemuCaps=0x7fca880e96e0) at qemu/qemu_command.c:249 #5 0x000000000048a7d7 in qemuBuildCommandLine (conn=0x7fca84000ae0, driver=0x7fca88068b10, def=0x7fca880e9190, monitor_chr=0x7fca880e6180, monitor_json=true, qemuCaps=0x7fca880e96e0, migrateFrom=0x0, migrateFd=-1, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE) at qemu/qemu_command.c:4374 #6 0x00000000004af22a in qemuProcessStart (conn=0x7fca84000ae0, driver=0x7fca88068b10, vm=0x7fca880e9e30, migrateFrom=0x0, start_paused=false, autodestroy=false, stdin_fd=-1, stdin_path=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE) at qemu/qemu_process.c:3094 #7 0x000000000046692b in qemuDomainObjStart (conn=0x7fca84000ae0, driver=0x7fca88068b10, vm=0x7fca880e9e30, flags=0) at qemu/qemu_driver.c:4704 #8 0x0000000000466bba in qemuDomainStartWithFlags (dom=0x7fca880e6260, flags=0) at qemu/qemu_driver.c:4761 #9 0x0000000000466c4e in qemuDomainStart (dom=0x7fca880e6260) at qemu/qemu_driver.c:4780 #10 0x00007fca99cc6d2a in virDomainCreate (domain=0x7fca880e6260) at libvirt.c:7629 #11 0x0000000000427b9f in remoteDispatchDomainCreate (server=0x7b6c60, client=0x7be250, msg=0x800440, rerr=0x7fca92901bb0, args=0x7fca880e62a0) at remote_dispatch.h:794 #12 0x0000000000427aa6 in remoteDispatchDomainCreateHelper (server=0x7b6c60, client=0x7be250, msg=0x800440, rerr=0x7fca92901bb0, args=0x7fca880e62a0, ret=0x7fca880e6240) at remote_dispatch.h:772 #13 0x00007fca99d21ef4 in virNetServerProgramDispatchCall (prog=0x7c0210, server=0x7b6c60, client=0x7be250, msg=0x800440) at rpc/virnetserverprogram.c:416 #14 0x00007fca99d21a8a in virNetServerProgramDispatch (prog=0x7c0210, server=0x7b6c60, client=0x7be250, msg=0x800440) at rpc/virnetserverprogram.c:289 #15 0x00007fca99d1b287 in virNetServerHandleJob (jobOpaque=0x79bac0, opaque=0x7b6c60) at rpc/virnetserver.c:164 #16 0x00007fca99c29892 in virThreadPoolWorker (opaque=0x7c01a0) at util/threadpool.c:144 #17 0x00007fca99c292bb in virThreadHelper (data=0x7bf9d0) at util/threads-pthread.c:157 #18 0x00007fca96025f05 in start_thread () from /lib64/libpthread.so.0 #19 0x00007fca9596153d in clone () from /lib64/libc.so.6 Thanks, Jim
participants (2)
-
Daniel P. Berrange -
Jim Fehlig