[libvirt] how to enable ceph-rbd in libvirt.
1.my libvirtd version is 0.9.10. i rebuild qemu with --enable-rbd. it seems not work if guest-os has a rbd disk. -----rbd disk info ---- <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> 2.-----------------------error log--------- LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name xpSP3 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=26,id=hostnet0 -device e! 1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=6 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 10:37:24.215+0000: shutting down
On 10/10/2012 07:52 PM, yue wrote:
1.my libvirtd version is 0.9.10. i rebuild qemu with --enable-rbd. it seems not work if guest-os has a rbd disk. -----rbd disk info ---- <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> 2.-----------------------error log--------- LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name xpSP3 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=26,id=hostnet0 -device! e! 1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=6 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 10:37:24.215+0000: shutting down
You may need this patch to libvirt: http://www.redhat.com/archives/libvir-list/2012-May/msg00003.html This is included in libvirt 0.9.12. Josh
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error. At 2012-10-11 11:05:46,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 07:52 PM, yue wrote:
1.my libvirtd version is 0.9.10. i rebuild qemu with --enable-rbd. it seems not work if guest-os has a rbd disk. -----rbd disk info ---- <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> 2.-----------------------error log--------- LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name xpSP3 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=26,id=hostnet0 -devic! e e! 1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=6 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 10:37:24.215+0000: shutting down
You may need this patch to libvirt: http://www.redhat.com/archives/libvir-list/2012-May/msg00003.html
This is included in libvirt 0.9.12.
Josh
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either. Josh
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3 -------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log--- [root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev! =hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netd! ev! =hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)? Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
hi, i add debug options related to rbd in xpSP3.xml. but error seems same with privious. ----libvirtd.log------------------ erfaces 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:29.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:29.266+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:30.267+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 is tainted: high-privileges 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : failed to connect to monitor socket: No such process 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : Failed to connect monitor for xpSP3 2012-10-12 13:25:30.943+0000: 9021: error : qemuProcessWaitForMonitor:1634 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory ---------xpSP3.log---is attached.--- 2012-10-12 13:11:33.133+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,driv! e=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=1 is tainted: high-privileges 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed fd 18 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 24 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run hook 0x7faa10a62030 0x7faa160d4370 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : Obtaining domain lock 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockProcessStart:164 : plugin=0x7faa080020a0 dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:123 : plugin=0x7faa080020a0 dom=0x7faa0804af40 withResources=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:104 : key=uuid type=uuid value=b45bd66a-6700-4905-c5f4-4c799413d7b7 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:100 : key=name type=string value=xpSP3 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=id type=uint value=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=pid type=uint value=7818 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:135 : Adding leases 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:140 : Adding disks 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerAddDisk:86 : Add disk /var/lib/libvirt/images/xpSP3.img 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAddResource:333 : lock=0x7faa08060820 type=0 name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : lock=0x7faa08060820 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : Moving process to cgroup 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 0:cpu at /cgroup/cpu in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 3:memory at /cgroup/memory in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 4:devices at /cgroup/devices in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 5:freezer at /cgroup/freezer in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 6:blkio at /cgroup/blkio in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : Make group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/memory/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/devices/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1848 : Setting CPU affinity 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified cpuset 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : Setting up security labelling 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : User record for user '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 : Group record for group '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : Hook complete ret=0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : Done hook 0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : Notifying parent for handshake start on 21 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : Waiting on parent for handshake complete on 22 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 21 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 22 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : Hook is done 0 char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-12 13:11:33.557+0000: shutting down ---- At 2012-10-12 00:04:19,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,net! dev! =hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx\;none:mon_host=192.168.10.4\:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)?
Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
Sorry, I forgot libvirt/qemu eats stderr. The extra logging wasn't captured. Could you try: name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_file=/tmp/qemu-rbd.log' If /tmp/qemu-rbd.log isn't full of debug messages, make sure selinux or apparmor aren't preventing qemu from writing to it. Josh On 2012-10-11 22:31, yue wrote:
hi, i add debug options related to rbd in xpSP3.xml. but error seems same with privious. ----libvirtd.log------------------ erfaces 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:29.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:29.266+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this f unction is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:30.267+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 is tainted: high-privileges 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : failed to connect to monitor socket: No such process 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : Failed to connect monitor for xpSP3 2012-10-12 13:25:30.943+0000: 9021: error : qemuProcessWaitForMonitor:1634 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory
---------xpSP3.log---is attached.--- 2012-10-12 13:11:33.133+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/ qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=19,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=1 is tainted: high-privileges 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed fd 18 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 24 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run hook 0x7faa10a62030 0x7faa160d4370 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : Obtaining domain lock 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockProcessStart:164 : plugin=0x7faa080020a0 dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:123 : plugin=0x7faa080020a0 dom=0x7faa0804af40 withResources=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:104 : key=uuid type=uuid value=b45bd66a-6700-4905-c5f4-4c799413d7b7 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:100 : key=name type=string value=xpSP3 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=id type=uint value=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=pid type=uint value=7818 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:135 : Adding leases 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:140 : Adding disks 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerAddDisk:86 : Add disk /var/lib/libvirt/images/xpSP3.img 2012-10-12 13:11:33.146+0000: 7818: debug : virLockMan agerAddResource:333 : lock=0x7faa08060820 type=0 name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : lock=0x7faa08060820 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : Moving process to cgroup 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 0:cpu at /cgroup/cpu in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/ma pping 3:memory at /cgroup/memory in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 4:devices at /cgroup/devices in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 5:freezer at /cgroup/freezer in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 6:blkio at /cgroup/blkio in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : Make group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/memory/libvirt/ qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/devices/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1848 : Setting CPU affinity 2 012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified cpuset 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : Setting up security labelling 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : User record for user '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 : Group record for group '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : Hook complete ret=0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : Done hook 0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : Notifying parent for handshake start on 21 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : Waiting on parent for handshake complete on 22 2012-10-12 13:11:33.157+0 000: 7818: debug : virFileClose:72 : Closed fd 21 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 22 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : Hook is done 0 char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-12 13:11:33.557+0000: shutting down
----
At 2012-10-12 00:04:19,"Josh Durgi n" <josh.durgin@inktank.com> wrote:
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789, if=none,id=drive-virtio-disk1,format=raw -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev!
=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)?
Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
i try your proposal. but there are no any output in qemu-rbd.log. selinux has been disabled(then reboot my OS). i do not know apparmor. my OS has no rpm of apparmor. my os is centps-6.2. 2.6.32-220.el6.x86_64. 1.would you give me the options of libvirt configure. i rebuild it. 2.if there a other way to confirm whether libvirt support rbd.other way to find root cause. thanks. ----rbd disk---- <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='admin'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=20:debug_auth=20:log_file=/tmp/qemu-rbd.log'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> -------------------- At 2012-10-12 15:06:45,"Josh Durgin" <josh.durgin@inktank.com> wrote:
Sorry, I forgot libvirt/qemu eats stderr. The extra logging wasn't captured.
Could you try:
name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_file=/tmp/qemu-rbd.log'
If /tmp/qemu-rbd.log isn't full of debug messages, make sure selinux or apparmor aren't preventing qemu from writing to it.
Josh
On 2012-10-11 22:31, yue wrote:
hi, i add debug options related to rbd in xpSP3.xml. but error seems same with privious. ----libvirtd.log------------------ erfaces 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:29.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:29.266+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this f unction is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:30.267+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 is tainted: high-privileges 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : failed to connect to monitor socket: No such process 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : Failed to connect monitor for xpSP3 2012-10-12 13:25:30.943+0000: 9021: error : qemuProcessWaitForMonitor:1634 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory
---------xpSP3.log---is attached.--- 2012-10-12 13:11:33.133+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/ qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=19,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=1 is tainted: high-privileges 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed fd 18 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 24 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run hook 0x7faa10a62030 0x7faa160d4370 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : Obtaining domain lock 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockProcessStart:164 : plugin=0x7faa080020a0 dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:123 : plugin=0x7faa080020a0 dom=0x7faa0804af40 withResources=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:104 : key=uuid type=uuid value=b45bd66a-6700-4905-c5f4-4c799413d7b7 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:100 : key=name type=string value=xpSP3 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=id type=uint value=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=pid type=uint value=7818 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:135 : Adding leases 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:140 : Adding disks 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerAddDisk:86 : Add disk /var/lib/libvirt/images/xpSP3.img 2012-10-12 13:11:33.146+0000: 7818: debug : virLockMan agerAddResource:333 : lock=0x7faa08060820 type=0 name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : lock=0x7faa08060820 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : Moving process to cgroup 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 0:cpu at /cgroup/cpu in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/ma pping 3:memory at /cgroup/memory in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 4:devices at /cgroup/devices in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 5:freezer at /cgroup/freezer in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 6:blkio at /cgroup/blkio in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : Make group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/memory/libvirt/ qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/devices/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1848 : Setting CPU affinity 2 012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified cpuset 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : Setting up security labelling 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : User record for user '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 : Group record for group '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : Hook complete ret=0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : Done hook 0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : Notifying parent for handshake start on 21 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : Waiting on parent for handshake complete on 22 2012-10-12 13:11:33.157+0 000: 7818: debug : virFileClose:72 : Closed fd 21 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 22 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : Hook is done 0 char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-12 13:11:33.557+0000: shutting down
----
At 2012-10-12 00:04:19,"Josh Durgi n" <josh.durgin@inktank.com> wrote:
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789, if=none,id=drive-virtio-disk1,format=raw -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev!
=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)?
Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote:
hi, now i rebuild libvirtd, version 0.10.2. and define ceph secret ,define ceph storage-pool. all fine up to now. i operate guest-os through virt-manager-GUI . i add a rbd-file(qemu-img create before) from ceph-rbd-pool, click 'finish',it pop error "NoneType object has no attribute startswith", someone say it is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
It sounds like the qemu binary that libvirt is using isn't one compiled with rbd support. You can check by running: qemu-kvm -drive format=? rbd should appear in the list of supported formats. Josh On 10/13/2012 01:20 AM, yue wrote:
i try your proposal. but there are no any output in qemu-rbd.log. selinux has been disabled(then reboot my OS). i do not know apparmor. my OS has no rpm of apparmor. my os is centps-6.2. 2.6.32-220.el6.x86_64. 1.would you give me the options of libvirt configure. i rebuild it. 2.if there a other way to confirm whether libvirt support rbd.other way to find root cause.
thanks.
----rbd disk---- <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='admin'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=20:debug_auth=20:log_file=/tmp/qemu-rbd.log'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> --------------------
At 2012-10-12 15:06:45,"Josh Durgin" <josh.durgin@inktank.com> wrote:
Sorry, I forgot libvirt/qemu eats stderr. The extra logging wasn't captured.
Could you try:
name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_file=/tmp/qemu-rbd.log'
If /tmp/qemu-rbd.log isn't full of debug messages, make sure selinux or apparmor aren't preventing qemu from writing to it.
Josh
On 2012-10-11 22:31, yue wrote:
hi, i add debug options related to rbd in xpSP3.xml. but error seems same with privious. ----libvirtd.log------------------ erfaces 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:29.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:29.266+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this f unction is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:30.267+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 is tainted: high-privileges 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : failed to connect to monitor socket: No such process 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : Failed to connect monitor for xpSP3 2012-10-12 13:25:30.943+0000: 9021: error : qemuProcessWaitForMonitor:1634 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory
---------xpSP3.log---is attached.--- 2012-10-12 13:11:33.133+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/ qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=19,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=1 is tainted: high-privileges 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed fd 18 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 24 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run hook 0x7faa10a62030 0x7faa160d4370 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : Obtaining domain lock 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockProcessStart:164 : plugin=0x7faa080020a0 dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:123 : plugin=0x7faa080020a0 dom=0x7faa0804af40 withResources=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:104 : key=uuid type=uuid value=b45bd66a-6700-4905-c5f4-4c799413d7b7 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:100 : key=name type=string value=xpSP3 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=id type=uint value=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=pid type=uint value=7818 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:135 : Adding leases 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:140 : Adding disks 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerAddDisk:86 : Add disk /var/lib/libvirt/images/xpSP3.img 2012-10-12 13:11:33.146+0000: 7818: debug : virLockMan agerAddResource:333 : lock=0x7faa08060820 type=0 name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : lock=0x7faa08060820 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : Moving process to cgroup 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 0:cpu at /cgroup/cpu in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/ma pping 3:memory at /cgroup/memory in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 4:devices at /cgroup/devices in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 5:freezer at /cgroup/freezer in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 6:blkio at /cgroup/blkio in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : Make group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/memory/libvirt/ qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/devices/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1848 : Setting CPU affinity 2 012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified cpuset 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : Setting up security labelling 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : User record for user '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 : Group record for group '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : Hook complete ret=0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : Done hook 0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : Notifying parent for handshake start on 21 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : Waiting on parent for handshake complete on 22 2012-10-12 13:11:33.157+0 000: 7818: debug : virFileClose:72 : Closed fd 21 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 22 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : Hook is done 0 char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-12 13:11:33.557+0000: shutting down
----
At 2012-10-12 00:04:19,"Josh Durgi n" <josh.durgin@inktank.com> wrote:
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789, if=none,id=drive-virtio-disk1,format=raw -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev!
=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)?
Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote:
On 10/10/2012 10:52 PM, yue wrote: > hi, now i rebuild libvirtd, version 0.10.2. and define ceph > secret > ,define ceph storage-pool. all fine up to now. > i operate guest-os through virt-manager-GUI . i add a > rbd-file(qemu-img create before) from ceph-rbd-pool, click > 'finish',it > pop error "NoneType object has no attribute startswith", someone > say it > is python error.
python-virtinst (and thus virt-manager) don't support the rbd storage pool. They don't know how to configure rbd disks yet either.
Josh
hahha, you are right. my qemu-kvm is installed from yum.repo ,not compiled by me. i always think qemu-kvm and qemu-system-x86_64 are not the same thing. actully they are the same thing. i replace /usr/libexec/qemu-kvm with my qemu-system-x86_64,now it works. i have a question. what the difference between qemu.git and qemu-kvm.git? any one is ok for kvm? thanks. At 2012-10-13 17:31:25,"Josh Durgin" <josh.durgin@inktank.com> wrote:
It sounds like the qemu binary that libvirt is using isn't one compiled with rbd support. You can check by running:
qemu-kvm -drive format=?
rbd should appear in the list of supported formats.
Josh
On 10/13/2012 01:20 AM, yue wrote:
i try your proposal. but there are no any output in qemu-rbd.log. selinux has been disabled(then reboot my OS). i do not know apparmor. my OS has no rpm of apparmor. my os is centps-6.2. 2.6.32-220.el6.x86_64. 1.would you give me the options of libvirt configure. i rebuild it. 2.if there a other way to confirm whether libvirt support rbd.other way to find root cause.
thanks.
----rbd disk---- <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='admin'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=20:debug_auth=20:log_file=/tmp/qemu-rbd.log'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> --------------------
At 2012-10-12 15:06:45,"Josh Durgin" <josh.durgin@inktank.com> wrote:
Sorry, I forgot libvirt/qemu eats stderr. The extra logging wasn't captured.
Could you try:
name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_file=/tmp/qemu-rbd.log'
If /tmp/qemu-rbd.log isn't full of debug messages, make sure selinux or apparmor aren't preventing qemu from writing to it.
Josh
On 2012-10-11 22:31, yue wrote:
hi, i add debug options related to rbd in xpSP3.xml. but error seems same with privious. ----libvirtd.log------------------ erfaces 2012-10-12 13:25:28.267+0000: 9023: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:29.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this function is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:29.266+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:29.267+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.264+0000: 9025: error : virConnectNumOfInterfaces:11014 : this f unction is not supported by the connection driver: virConnectNumOfInterfaces 2012-10-12 13:25:30.267+0000: 9023: error : virConnectNumOfDefinedInterfaces:11098 : this function is not supported by the connection driver: virConnectNumOfDefinedInterfaces 2012-10-12 13:25:30.269+0000: 9022: error : virNodeNumOfDevices:13984 : this function is not supported by the connection driver: virNodeNumOfDevices 2012-10-12 13:25:30.523+0000: 9021: warning : qemuDomainObjTaint:1364 : Domain id=1 name='xpSP3' uuid=b45bd66a-6700-4905-c5f4-4c799413d7b7 is tainted: high-privileges 2012-10-12 13:25:30.943+0000: 9021: error : qemuMonitorOpenUnix:293 : failed to connect to monitor socket: No such process 2012-10-12 13:25:30.943+0000: 9021: info : qemuConnectMonitor:1247 : Failed to connect monitor for xpSP3 2012-10-12 13:25:30.943+0000: 9021: error : qemuProcessWaitForMonitor:1634 : internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory
---------xpSP3.log---is attached.--- 2012-10-12 13:11:33.133+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/ qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=19,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=1 is tainted: high-privileges 2012-10-12 13:11:33.144+0000: 7815: debug : virFileClose:72 : Closed fd 18 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 24 2012-10-12 13:11:33.145+0000: 7815: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.146+0000: 7818: debug : virCommandHook:2058 : Run hook 0x7faa10a62030 0x7faa160d4370 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2589 : Obtaining domain lock 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockProcessStart:164 : plugin=0x7faa080020a0 dom=0x7faa0804af40 paused=1 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:123 : plugin=0x7faa080020a0 dom=0x7faa0804af40 withResources=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerPluginGetDriver:280 : plugin=0x7faa080020a0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerNew:304 : driver=0x3712e9cb20 type=0 nparams=4 params=0x7faa160d3830 flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:104 : key=uuid type=uuid value=b45bd66a-6700-4905-c5f4-4c799413d7b7 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:100 : key=name type=string value=xpSP3 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=id type=uint value=1 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerLogParams:88 : key=pid type=uint value=7818 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:135 : Adding leases 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerNew:140 : Adding disks 2012-10-12 13:11:33.146+0000: 7818: debug : virDomainLockManagerAddDisk:86 : Add disk /var/lib/libvirt/images/xpSP3.img 2012-10-12 13:11:33.146+0000: 7818: debug : virLockMan agerAddResource:333 : lock=0x7faa08060820 type=0 name=/var/lib/libvirt/images/xpSP3.img nparams=0 params=(nil) flags=0 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerAcquire:350 : lock=0x7faa08060820 state='(null)' flags=3 fd=0x7faa160d3998 2012-10-12 13:11:33.146+0000: 7818: debug : virLockManagerFree:387 : lock=0x7faa08060820 2012-10-12 13:11:33.146+0000: 7818: debug : qemuProcessHook:2614 : Moving process to cgroup 2012-10-12 13:11:33.146+0000: 7818: debug : virCgroupNew:617 : New group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.151+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 0:cpu at /cgroup/cpu in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/ma pping 3:memory at /cgroup/memory in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 4:devices at /cgroup/devices in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 5:freezer at /cgroup/freezer in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupDetect:273 : Detected mount/mapping 6:blkio at /cgroup/blkio in 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:535 : Make group /libvirt/qemu/xpSP3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpu/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuacct/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/cpuset/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/memory/libvirt/ qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/devices/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/freezer/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupMakeGroup:560 : Make controller /cgroup/blkio/libvirt/qemu/xpSP3/ 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpu/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuacct/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/cpuset/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/memory/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/devices/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/freezer/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : virCgroupSetValueStr:331 : Set value '/cgroup/blkio/libvirt/qemu/xpSP3/tasks' to '7818' 2012-10-12 13:11:33.156+0000: 7818: debug : virFileClose:72 : Closed fd 3 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1848 : Setting CPU affinity 2 012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessInitCpuAffinity:1883 : Set CPU affinity with specified cpuset 2012-10-12 13:11:33.156+0000: 7818: debug : qemuProcessHook:2626 : Setting up security labelling 2012-10-12 13:11:33.157+0000: 7818: debug : virGetUserIDByName:2539 : User record for user '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virGetGroupIDByName:2621 : Group record for group '0' does not exist 2012-10-12 13:11:33.157+0000: 7818: debug : virSecurityDACSetProcessLabel:861 : Dropping privileges of DEF to 0:0 2012-10-12 13:11:33.157+0000: 7818: debug : qemuProcessHook:2633 : Hook complete ret=0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2060 : Done hook 0 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2074 : Notifying parent for handshake start on 21 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2098 : Waiting on parent for handshake complete on 22 2012-10-12 13:11:33.157+0 000: 7818: debug : virFileClose:72 : Closed fd 21 2012-10-12 13:11:33.157+0000: 7818: debug : virFileClose:72 : Closed fd 22 2012-10-12 13:11:33.157+0000: 7818: debug : virCommandHook:2119 : Hook is done 0 char device redirected to /dev/pts/4 qemu-kvm: -drive
file=rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough,aio=native: could not open disk image
rbd:cloud/testrbd:debug_rbd=20:debug_monc=20:debug_auth=20:log_to_stderr=2:id=admin:key=AQCp1GFQMMOrCxAAEb4jxgUF9BuSBkQIigFi/Q==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-12 13:11:33.557+0000: shutting down
----
At 2012-10-12 00:04:19,"Josh Durgi n" <josh.durgin@inktank.com> wrote:
On 10/11/2012 12:00 AM, yue wrote:
how should i do? i configurate guest-os.xml ,add rbd disk manually. then virsh start xpSP3
That's the right way to do it.
-------rbd disk------------ <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writethrough' io='native'/> <auth username='cloud'> <secret type='ceph' uuid='7a91dc24-b072-43c4-98fb-4b2415322b0f'/> </auth> <source protocol='rbd' name='cloud/testrbd'> <host name='192.168.10.4' port='6789'/> </source> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ---error log---
[root@KVM-ip11 git]# virsh start xpSP3 error: Failed to start domain xpSP3 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw,cache=writethrough,aio=native: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory ---xpSP3.log----- 2012-10-11 14:57:58.118+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name xpSP3 -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid b45bd66a-6700-4905-c5f4-4c799413d7b7 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xpSP3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=localtime,driftfix=slew -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/xpSP3.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789, if=none,id=drive-virtio-disk1,format=raw -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev!
=hostnet0,id=net0,mac=52:54:00:2a:8c:48,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Domain id=3 is tainted: high-privileges char device redirected to /dev/pts/4 qemu-kvm: -drive file=rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789,if=none,id=drive-virtio-disk1,format=raw: could not open disk image rbd:cloud/testrbd:id=cloud:key=AQCGbGRQ+M+NGBAATtylZNiSxqCTQ4uaApd+9w==:auth_supported=cephx;none:mon_host=192.168.10.4:6789: No such file or directory 2012-10-11 14:57:58.548+0000: shutting down
It's unclear where the actual problem is - could you try again with name='cloud/testrbd:debug_ms=1:debug_rbd=20:debug_monc=10:log_to_stderr=true' and post the instance log (i.e. xpSP3.log)?
Josh
At 2012-10-11 14:48:43,"Josh Durgin" <josh.durgin@inktank.com> wrote: > On 10/10/2012 10:52 PM, yue wrote: >> hi, now i rebuild libvirtd, version 0.10.2. and define ceph >> secret >> ,define ceph storage-pool. all fine up to now. >> i operate guest-os through virt-manager-GUI . i add a >> rbd-file(qemu-img create before) from ceph-rbd-pool, click >> 'finish',it >> pop error "NoneType object has no attribute startswith", someone >> say it >> is python error. > > python-virtinst (and thus virt-manager) don't support the rbd > storage > pool. They don't know how to configure rbd disks yet either. > > Josh
On 2012-10-13 10:49, libvirt@163.com wrote:
hahha, you are right. my qemu-kvm is installed from yum.repo ,not compiled by me. i always think qemu-kvm and qemu-system-x86_64 are not the same thing. actully they are the same thing. i replace /usr/libexec/qemu-kvm with my qemu-system-x86_64,now it works.
i have a question. what the difference between qemu.git and qemu-kvm.git? any one is ok for kvm?
There's little difference these days. This wiki page summarizes the differences as of last May: http://wiki.qemu.org/KVM#qemu-kvm_fork_for_x86
participants (3)
-
Josh Durgin -
libvirt＠163.com -
yue