Re: [libvirt] Live migration woes
by Lance Albertson
> On Fri, Feb 05, 2010 at 03:21:32PM -0500, Thomas Sjolshagen wrote:
>>> is anyone using libvirt-git and qemu-kvm post-0.11 and can live
>>> migrate VMs successfully for- and backwards? qemu-kvm-0.11 seems
>>> to be the latest release were live migration works, and I'm
>>> trying to figure out if its a kvm or libvirt problem, since there
>>> were other things (balloon parameter f.e.) that were changed with
>>> qemu-kvm-0.12 and caused troubles when using libvirt.
>>
>> Yes.
>>
>> I'm using the virt-preview repo (rawhide) and I just noticed,
>> today, after re-upgrading to 0.12.2-5 (since -4 has a rather nasty
>> IO problem for virtio devices against raw image files) that my
>> live-migrations fail with the following error message on the
>> destination system (i.e. the system the I'm trying to migrate to):
>
>> /var/log/libvirt/qemu/<guest>.log: Unknown savevm section
>> -2038417646 load of migration failed
>
> This looks like a new QEMU bug to me
I can confirm this bug is happening on Ganeti as well [1]. It appears
that if you sync the time on the guest VM before migrating back to the
other node it will "succeed" most of the time. Has anyone opened a bug
for this on the qemu-kvm bug tracker yet? I was about to do that myself.
Thanks-
[1]
http://groups.google.com/group/ganeti/browse_thread/thread/9b2c556557e749cf
--
Lance Albertson lance <at> osuosl.org
Systems Administrator / Architect Open Source Lab
Network Services Oregon State University
Work: 541-737-9975 PGP Key: 0x27F4B742
GPG Fingerprint 0423 92F3 544A 1282 5AB1 4D07 416F A15D 27F4 B742
14 years, 10 months
[libvirt] Inbound NAT and iptables rules
by Karl Vogel
This issue has been brought up a few times, but I haven't found any real
solution yet. The problem is with the way libvirt adds iptables rules.
I have a KVM Host with 1 public IP address, on which a few virtual
machines run. I want to be able to direct certain ports on the public IP
to certain virtual machine's internal IP.
For example:
allow conneciton to 'public-ip' tcp port: 80 to 'vm1'
Which is possible using following rules:
$ iptables -I FORWARD 1 -d vm1 -p tcp --dport 80 -j ACCEPT
$ iptables -t nat -A PREROUTING -p tcp -d public-ip --dport 80 -j DNAT
--to-destination vm1
However the problem is that libvirt _always_ adds it's rules to the top
of the FORWARD rules, hence it blocks any 'custom' rules to the VM machines.
ie.
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
39 2964 ACCEPT all -- * virbr0 0.0.0.0/0
192.168.122.0/24 state RELATED,ESTABLISHED
39 2964 ACCEPT all -- virbr0 * 192.168.122.0/24
0.0.0.0/0
2 656 ACCEPT all -- virbr0 virbr0 0.0.0.0/0
0.0.0.0/0
2 120 REJECT all -- * virbr0 0.0.0.0/0
0.0.0.0/0 reject-with icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0
0.0.0.0/0 reject-with icmp-port-unreachable
Only established connections are allowed with the libvirt rules, hence
the forwarded connection to port 80 will be rejected by the 4th libvirt
rule.
The only way to make it work is to insert the rule _after_ libvirt has
started the network.. and if the libvirt network ever gets restarted,
the rule will be disabled since libvirt inserts it's rules at the top of
the chain.
Is there a proper way to achieve this with libvirt ?!
14 years, 10 months
[libvirt] qemu can't parse floppy cmdline
by Cole Robinson
Hi Dan,
With qemu-0.12.2-5.fc13.x86_64 in fedora, libvirt is generating a cmdline
which qemu-kvm can't parse. XML is:
<domain type='kvm'>
<name>winxp_32</name>
<uuid>3485e26d-c370-c0b0-32e2-a041a12f5cfc</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64' machine='pc-0.11'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='localtime'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu'/>
<source file='/mnt/devel/vms/disks/winxp_32.qcow2.img'/>
<target dev='hda' bus='ide'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu'/>
<source file='/mnt/devel/media/win_2003_64.iso'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='1' unit='0'/>
</disk>
<disk type='file' device='cdrom'>
<target dev='hdb' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='0' unit='1'/>
</disk>
<disk type='file' device='floppy'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/1.img'/>
<target dev='fda' bus='fdc'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<controller type='ide' index='0'/>
<controller type='fdc' index='0'/>
<interface type='network'>
<mac address='52:54:00:5d:46:22'/>
<source network='default'/>
<target dev='vnet0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target port='0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'/>
<sound model='es1370'/>
<video>
<model type='vga' vram='9216' heads='1'/>
</video>
</devices>
</domain>
The generated cmdline is:
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none
/usr/bin/qemu-kvm -S -M pc-0.11 -enable-kvm -m 1024 -smp
1,sockets=1,cores=1,threads=1 -name winxp_32 -uuid
3485e26d-c370-c0b0-32e2-a041a12f5cfc -nodefaults -chardev
socket,id=monitor,path=/var/lib/libvirt/qemu/winxp_32.monitor,server,nowait
-mon chardev=monitor,mode=readline -localtime -boot c -drive
file=/mnt/devel/vms/disks/winxp_32.qcow2.img,if=none,id=drive-ide0-0-0,boot=on
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive
file=/mnt/devel/media/win_2003_64.iso,if=none,media=cdrom,id=drive-ide0-1-0
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
if=none,media=cdrom,id=drive-ide0-0-1 -device
ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive
file=/var/lib/libvirt/images/1.img,if=none,id=drive-fdc0-0-0,format=raw
-global isa-fdc,driveA=drive-fdc0-0-0 -device
rtl8139,vlan=0,id=net0,mac=52:54:00:5d:46:22,bus=pci.0,addr=0x4 -net
tap,fd=38,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device
isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0
-vga std -device ES1370,id=sound0,bus=pci.0,addr=0x5 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3
can't parse: "isa-fdc,driveA=drive-fdc0-0-0"
If I drop the floppy dev and controller, the guest boots, but a few warnings
are printed. Not sure if any of this reflects qemu bugs or libvirt bugs:
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none
/usr/bin/qemu-kvm -S -M pc-0.11 -enable-kvm -m 1024 -smp
1,sockets=1,cores=1,threads=1 -name winxp_32 -uuid
3485e26d-c370-c0b0-32e2-a041a12f5cfc -nodefaults -chardev
socket,id=monitor,path=/var/lib/libvirt/qemu/winxp_32.monitor,server,nowait
-mon chardev=monitor,mode=readline -localtime -boot c -drive
file=/mnt/devel/vms/disks/winxp_32.qcow2.img,if=none,id=drive-ide0-0-0,boot=on
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive
file=/mnt/devel/media/win_2003_64.iso,if=none,media=cdrom,id=drive-ide0-1-0
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
if=none,media=cdrom,id=drive-ide0-0-1 -device
ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -device
rtl8139,vlan=0,id=net0,mac=52:54:00:5d:46:22,bus=pci.0,addr=0x4 -net
tap,fd=37,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device
isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0
-vga std -device ES1370,id=sound0,bus=pci.0,addr=0x5 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3
char device redirected to /dev/pts/2
Warning: vlan 0 with no nics
Option 'ipv4': Use 'on' or 'off'
Failed to parse "yes" for "dummy.ipv4"
Thanks,
Cole
14 years, 10 months
[libvirt] [PATCH 0/9] Remove conn parameter from several functions
by Matthias Bolte
This set of patches removes the conn parameter from several internal functions
where it was used of error reporting only. Since errors are reported to an
error object in thread local storage the conn parameter is not necessary for
this anymore.
Matthias
14 years, 10 months
[libvirt] RFC: solving problem with qemu domain save on root-squashing NFS
by Laine Stump
(I had sent this message last week as a response to a message in a patch
thread, but thought today that possibly some mail readers would bury it,
so I'm sending it again disconnected from any existing thread. I can
send the patches I mention below to anyone interested in playing around
with them, but they're not useful since they still fail at the final
step...)
On 01/24/2010 11:57 AM, Dan Kenigsberg wrote:
> On Wed, Jan 20, 2010 at 03:14:57PM +0000, Daniel P. Berrange wrote:
>> This patch series does some work on te security drivers, and the QEMU
>> code
>> for managing DAC permissions on files.
>>
>> The core goal is to turn the QEMU driver DAC file management code into a
>> security driver. Instead of QEMU calling into the SELinux/AppArmour
>> drivers
>> directly, a stacked driver module is introduced. This delegates all
>> operations
>> to first the QEMU DAC driver, and then the main SELinux/AppArmour
>> driver.
>> The end result is that all the permissions management code is removed
>> from
>> the QEMU driver, and we're left with just simple security driver calls.
>>
>> In the process of this a number of flaws in the current hotplug code
>> were
>> found, and code was generally tidied up with a view to making it
>> easier to
>> manage.
>>
>> Finally, we add the ability to turn off the QEMU DAC file managment
>> code,
>> and also deal gracefully with failures to change ownership (eg on NFS
>> with
>> root squash, or readonly FS).
> Thanks for this series. However, it seems that we still have a problem
> when trying to save domain to a root-squashing nfs export.
> When using qemu directly, as a user with write permissions to that
> export, there is no problem. When using libvirt, libvirt tries to write
> its own state to the target file. I would not want to pre-create the
> target file as world redable.
>
> How about performing open(path, O_CREAT|O_TRUNC|O_WRONLY,
> S_IRUSR|S_IWUSR)) with the euid of the qemu process?
I came up with a patch that creates the domain state file by first
forking, followed by the child process doing setuid(qemu) and all the
write-to-the-file stuff that creates the file (similar to the recent
virFileCreate patch). This works all the way up through qemu's appending
its state onto the file.
The trouble comes when qemudDomainSave calls
securityDriver->domainRestoreSavedStateLabel() (in this case,
qemuSecurityStackedRestoreSavedStateLabel()). That function first calls
the primary security driver's qemuSecurityDACRestoreSavedStateLabel(),
which will fail because it attempts to chown the file to 0,0 (to prevent
other qemu instances from messing with it). That can be disabled by
setting "dynamic_ownership=0" in qemu.conf (needed in several other
places when stuff is stored on root-squashing NFS anyway :-/). Then the
stacked security driver calls the secondary security driver's
SELinuxRestoreSavedStateLabel(), which also fails because of an SELinux
function (matchpathcon) returning failure. We could ignore this failure,
but that would create an unnecessary security hole for all those
installations that *aren't* storing their domains on root-squashing NFS.
So I *think* what I'm looking for a good way to avoid the SELinux stuff
when the destination file is located on a root-squashing NFS share, but
not ignore it otherwise (I believe the consensus is that we *don't* want
to require another configuration knob if at all possible, since that
would not only lead to extra complexity of config, but also to
uninformed users turning it off when not necessary). Does anyone have
any good ideas for doing this (or for a different approach)? (I suppose
I could always attempt the entire operation assuming not, then redo with
fork/setuid if the first attempt failed, but that seems less than elegant).
14 years, 10 months
[libvirt] Live migration woes
by Thomas Treutner
Hi,
is anyone using libvirt-git and qemu-kvm post-0.11 and can live migrate VMs
successfully for- and backwards? qemu-kvm-0.11 seems to be the latest release
were live migration works, and I'm trying to figure out if its a kvm or
libvirt problem, since there were other things (balloon parameter f.e.) that
were changed with qemu-kvm-0.12 and caused troubles when using libvirt.
thx,t
14 years, 10 months
[libvirt] Regarding lxc driver for libvirt.
by Kumar L Srikanth-B22348
Hi,
I am new to libvirt.
I want to create a Domain using libvirt XML. In order to mount the
host's '/home/srikanth' directory to the new container's '/' directory,
my XML format is shown below:
<domain type='lxc' id='1'>
<name>container1_vm</name>
<memory>500000</memory>
<os>
<type>exe</type>
<init>/bin/sh</init>
</os>
<vcpu>1</vcpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/libvirt_lxc</emulator>
<filesystem type='mount'>
<source dir='/home/srikanth'/>
<target dir='/'/>
</filesystem>
<console type='pty' />
</devices>
</domain>
With the above libvirt XML, Domain is defining, but not starting. When I
issue the start command it's saying "Domain started", but showing "shut
off" status. If I changed the target directory(<traget dir='/'/>) from
'/' to '/home/container1'(<traget dir='/home/container1'/>), the domain
is starting normally and I am able to see the contents in the target
directory.
Can you please let me know, how can I set the target directory to '/'?
By the way, I am using libvirt version o.7.6.
Regards,
Srikanth.
14 years, 10 months
