Re: [libvirt] Rudimentary (basic) s390x architecture functions for libvirt
A few more details for s390x: why you can't set param '-S' and need '-chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons' on s390x: 1) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" => screenshot 1 2) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" => screenshot 2 3) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons => RUN 4) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons => screenshot 4 Regards, Patrick Siegl Am 25.01.2011 19:37, schrieb Justin Clift:
On 26/01/2011, at 5:11 AM, Patrick Siegl wrote:
<screenshot_paramS.png> Interestingly, I'm guessing that's the QEMU "monitor console".
Apparently it's a key feature of QEMU, but it's not something I've personally had anything to do with (yet). :)
Just mentioning, in case it's a useful thought. The guys on the mailing list have a lot deeper knowledge about this stuff that me. ;)
Regards and best wishes,
Justin Clift
On Tue, Jan 25, 2011 at 08:34:03PM +0100, Patrick Siegl wrote:
A few more details for s390x: why you can't set param '-S' and need '-chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons' on s390x:
1) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 1
2) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 2
3) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> RUN
4) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> screenshot 4
These screenshots are all showing the same thing, but for different reasons. IIUC, there is no 'VGA' graphics adapter in s390 world, hence you need some form of text console. This is what virtio-serial-s390 is providing for you. This does not need to be hardcoded in the QEMU arg generator. You can set this in the XML <console type='vc'> <target type='virtio'/> </console> When you add the '-S' argument this tells QEMU to launch the binary but *NOT* run the guest. When libvirt uses '-S', it launches QEMU runs some monitor commands, and then runs 'cont' to start the guest running. When you're running QEMU manually in your demo above, I assume you never told QEMU to start the CPUs, hence you see the screenshot 4. If you had issued 'cont' on the monitor it would run normally as per example 3. Regards, Daniel
Thanks for the good advice regarding the 'qemu monitor' and starting virtual machines with 'cont' on this monitor. Thank telnet, virtual machines can start with param '-S' on s390x. :-) In the 'qemu_command.c' I had commented out so much because you build a virtual machine with the PCI-Bus. But on s390x there ist only the s390-virtio - Bus which you can only activate but can't properly configure. I will try to correct my last patch within the next days and hopefully you will accept it :-) Regards Patrick Siegl Am 26.01.2011 13:41, schrieb Daniel P. Berrange:
On Tue, Jan 25, 2011 at 08:34:03PM +0100, Patrick Siegl wrote:
A few more details for s390x: why you can't set param '-S' and need '-chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons' on s390x:
1) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 1
2) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 2 3) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> RUN
4) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> screenshot 4 These screenshots are all showing the same thing, but for different reasons.
IIUC, there is no 'VGA' graphics adapter in s390 world, hence you need some form of text console. This is what virtio-serial-s390 is providing for you. This does not need to be hardcoded in the QEMU arg generator. You can set this in the XML
<console type='vc'> <target type='virtio'/> </console>
When you add the '-S' argument this tells QEMU to launch the binary but *NOT* run the guest. When libvirt uses '-S', it launches QEMU runs some monitor commands, and then runs 'cont' to start the guest running. When you're running QEMU manually in your demo above, I assume you never told QEMU to start the CPUs, hence you see the screenshot 4. If you had issued 'cont' on the monitor it would run normally as per example 3.
Regards, Daniel
Am 26.01.2011 13:41, schrieb Daniel P. Berrange:
On Tue, Jan 25, 2011 at 08:34:03PM +0100, Patrick Siegl wrote:
A few more details for s390x: why you can't set param '-S' and need '-chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons' on s390x:
1) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 1
2) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 2 3) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> RUN
4) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> screenshot 4 These screenshots are all showing the same thing, but for different reasons.
IIUC, there is no 'VGA' graphics adapter in s390 world, hence you need some form of text console. This is what virtio-serial-s390 is providing for you. This does not need to be hardcoded in the QEMU arg generator. You can set this in the XML
<console type='vc'> <target type='virtio'/> </console>
When you add the '-S' argument this tells QEMU to launch the binary but *NOT* run the guest. When libvirt uses '-S', it launches QEMU runs some monitor commands, and then runs 'cont' to start the guest running. When you're running QEMU manually in your demo above, I assume you never told QEMU to start the CPUs, hence you see the screenshot 4. If you had issued 'cont' on the monitor it would run normally as per example 3.
Regards, Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
I have overhauled my last patch (for libvirt KVM support on s390x). You will find the new one as attachment. There is only one problem, which I can't get fixed. You wrote following: "This is expected change, since s390 isn't PCI based, but what addressing scheme is used for s390 devices to uniquely and stablly identify them. eg, we need to make sure that qemu -drive AAA -drive BBB -drive CCC (monitor) drive_del BBB results except same guest visible ABI as qemu -drive AAA -drive CCC" This is a small problem under s390x, because if you try to add ',addr=' to a device, qemu prints: "Property '<device>.addr' not found". There are a few more details: - netdev isn't supported, because netdev needs param 'bus='. If you try to run this with BUS s390-virtio, netdev prints, that this is not supported. (everything should work with virtio, so if you enable '<model type="virtio"/>', it works fine) - You have to set 'ports' for the controller. If you don't do this, the virtual machine doesn't find VDA and so you can't start it. Maybe this is because of the flags '-nodefconfig' and '-nodefaults'. The maximum count of ports is 31 (the qemu implementation kills everything which is higher than 32). - Also 'vectors' aren't supported for the controller. If you therefor try to set a value, the virtual machine doesn't start. - It is not very nice within the libvirt implementation ('qemu_command.c') when every device gets PCI as address type (the only way to fix this in a relative moderat time, is to overwrite it with s390-virtio, if it runs on s390x). - The last point is that I haven't found a way for parsing the cpu MHz count. Within the patch there is some comment about it. Everything which I recognized for building an virtual machine on s390x is written into the XML file. Regards, Patrick Siegl
On Sun, Jan 30, 2011 at 03:56:15PM +0100, Patrick Siegl wrote:
Am 26.01.2011 13:41, schrieb Daniel P. Berrange:
On Tue, Jan 25, 2011 at 08:34:03PM +0100, Patrick Siegl wrote:
A few more details for s390x: why you can't set param '-S' and need '-chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons' on s390x:
1) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 1
2) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw"
=> screenshot 2 3) qemu-system-s390x -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> RUN
4) qemu-system-s390x -S -M s390-virtio -vnc :0 -enable-kvm -smp 1 -m 1024 -k de -kernel /boot/image-2.6.34-tue01 -initrd /boot/initrd-2.6.34-tue01_GUEST -drive file=/data/sle11.img -append "root=/dev/vda1 rw" -chardev vc,id=cons -device virtio-serial-s390 -device virtconsole,chardev=cons
=> screenshot 4 These screenshots are all showing the same thing, but for different reasons.
IIUC, there is no 'VGA' graphics adapter in s390 world, hence you need some form of text console. This is what virtio-serial-s390 is providing for you. This does not need to be hardcoded in the QEMU arg generator. You can set this in the XML
<console type='vc'> <target type='virtio'/> </console>
When you add the '-S' argument this tells QEMU to launch the binary but *NOT* run the guest. When libvirt uses '-S', it launches QEMU runs some monitor commands, and then runs 'cont' to start the guest running. When you're running QEMU manually in your demo above, I assume you never told QEMU to start the CPUs, hence you see the screenshot 4. If you had issued 'cont' on the monitor it would run normally as per example 3.
Regards, Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
I have overhauled my last patch (for libvirt KVM support on s390x). You will find the new one as attachment. There is only one problem, which I can't get fixed. You wrote following:
"This is expected change, since s390 isn't PCI based, but what addressing scheme is used for s390 devices to uniquely and stablly identify them.
eg, we need to make sure that
qemu -drive AAA -drive BBB -drive CCC (monitor) drive_del BBB
results except same guest visible ABI as
qemu -drive AAA -drive CCC"
This is a small problem under s390x, because if you try to add ',addr=' to a device, qemu prints: "Property '<device>.addr' not found".
Can you show the output of # qemu -monitor stdio (qemu) info qtree And also # qemu -monitor stdio -nodefconfig -nodefaults (qemu) info qtree
There are a few more details: - netdev isn't supported, because netdev needs param 'bus='. If you try to run this with BUS s390-virtio, netdev prints, that this is not supported. (everything should work with virtio, so if you enable '<model type="virtio"/>', it works fine)
This sounds rather like a bug in QEMU, but it might be a mistake in the way libvirt uses netdev.
- You have to set 'ports' for the controller. If you don't do this, the virtual machine doesn't find VDA and so you can't start it. Maybe this is because of the flags '-nodefconfig' and '-nodefaults'. The maximum count of ports is 31 (the qemu implementation kills everything which is higher than 32).
Hmm, qemu has a default value for 'ports' which ought to be being applied regardless of x86 vs s390.
- Also 'vectors' aren't supported for the controller. If you therefor try to set a value, the virtual machine doesn't start.
Not sure why that is. It could be that vectors is something specific to the PCI impl.
- It is not very nice within the libvirt implementation ('qemu_command.c') when every device gets PCI as address type (the only way to fix this in a relative moderat time, is to overwrite it with s390-virtio, if it runs on s390x).
Yes, this is the thing that needs the most work to come to a proper solution in libvirt. I'd like to understand more about s390 from the 'info qtree' data requested above to see what we need todo... Daniel
Am 01.02.2011 12:55, schrieb Daniel P. Berrange:
On Sun, Jan 30, 2011 at 03:56:15PM +0100, Patrick Siegl wrote:
I have overhauled my last patch (for libvirt KVM support on s390x). You will find the new one as attachment. There is only one problem, which I can't get fixed. You wrote following:
"This is expected change, since s390 isn't PCI based, but what addressing scheme is used for s390 devices to uniquely and stablly identify them.
eg, we need to make sure that
qemu -drive AAA -drive BBB -drive CCC (monitor) drive_del BBB
results except same guest visible ABI as
qemu -drive AAA -drive CCC"
This is a small problem under s390x, because if you try to add ',addr=' to a device, qemu prints: "Property '<device>.addr' not found". Can you show the output of
# qemu -monitor stdio (qemu) info qtree
And also
# qemu -monitor stdio -nodefconfig -nodefaults (qemu) info qtree
That's the output of your recommended commands. You have to pass urgently '--enable-kvm' on s390! boromir:/ # qemu-system-s390x -monitor stdio The S390 target only works with KVM enabled boromir:/ # qemu-system-s390x --enable-kvm -monitor stdio VNC server running on `::1:5900' QEMU 0.12.50 monitor - type 'help' for more information (qemu) info qtree bus: main-system-bus type System dev: s390-virtio-bridge, id "" bus: s390-virtio type s390-virtio dev: virtio-serial-s390, id "" dev-prop: max_ports = 31 bus: virtio-serial-bus.0 type virtio-serial-bus dev: virtconsole, id "" dev-prop: is_console = 1 dev-prop: chardev = virtcon0 dev-prop: name = <null> dev-prop-int: id: 0 dev-prop-int: guest_connected: 1 dev-prop-int: host_connected: 1 dev: virtio-net-s390, id "" dev-prop: mac = 52:54:00:12:34:56 dev-prop: vlan = 0 dev-prop: netdev = <null> (qemu) quit boromir:/ # qemu-system-s390x --enable-kvm -monitor stdio -nodefconfig -nodefaults VNC server running on `::1:5900' QEMU 0.12.50 monitor - type 'help' for more information (qemu) info qtree bus: main-system-bus type System dev: s390-virtio-bridge, id "" bus: s390-virtio type s390-virtio (qemu) quit boromir:/ #
There are a few more details: - netdev isn't supported, because netdev needs param 'bus='. If you try to run this with BUS s390-virtio, netdev prints, that this is not supported. (everything should work with virtio, so if you enable '<model type="virtio"/>', it works fine) This sounds rather like a bug in QEMU, but it might be a mistake in the way libvirt uses netdev.
- You have to set 'ports' for the controller. If you don't do this, the virtual machine doesn't find VDA and so you can't start it. Maybe this is because of the flags '-nodefconfig' and '-nodefaults'. The maximum count of ports is 31 (the qemu implementation kills everything which is higher than 32). Hmm, qemu has a default value for 'ports' which ought to be being applied regardless of x86 vs s390.
I think that the output above (of your recommended commands) corresponds with my assumption. If you look into the qemu implementation of max_ports in "qemu-0.13.0/hw/s390-virtio-bus.c" (http://wiki.qemu.org/Download => qemu-0.13.0.tar.gz) you will find following which underlines also my assumption with maximum of 31: static VirtIOS390DeviceInfo s390_virtio_serial = { .init = s390_virtio_serial_init, .qdev.name = "virtio-serial-s390", .qdev.alias = "virtio-serial", .qdev.size = sizeof(VirtIOS390Device), .qdev.props = (Property[]) { DEFINE_PROP_UINT32("max_ports", VirtIOS390Device, max_virtserial_ports, 31), DEFINE_PROP_END_OF_LIST(), }, };
- Also 'vectors' aren't supported for the controller. If you therefor try to set a value, the virtual machine doesn't start. Not sure why that is. It could be that vectors is something specific to the PCI impl. Maybe your are right. Test qemu command line shows following:
boromir:/data # qemu-system-s390x -enable-kvm -device virtio-serial-s390,id=virtio-serial0,max_ports=16,vectors=1,bus=s390-virtio qemu-system-s390x: -device virtio-serial-s390,id=virtio-serial0,max_ports=16,vectors=1,bus=s390-virtio: Property 'virtio-serial-s390.vectors' not found
- It is not very nice within the libvirt implementation ('qemu_command.c') when every device gets PCI as address type (the only way to fix this in a relative moderat time, is to overwrite it with s390-virtio, if it runs on s390x). Yes, this is the thing that needs the most work to come to a proper solution in libvirt. I'd like to understand more about s390 from the 'info qtree' data requested above to see what we need todo...
If you looked at my last patch, there where only few points where I used the C preprocessor. The point is that I don't know how I should rewrite the functions for having an arch informationen within it, so that it can be accepted. Regards, Patrick Siegl
participants (2)
-
Daniel P. Berrange -
Patrick Siegl