10:42 a.m.
Hi all.
Short summary on DV's request ;)
I ran into a problem migrating kvm machines with libvirt-0.6.5:
1) At first, using the same syntax for the migrateuri as with xen (just the
IP) did not work... looking into the source code (! ;) ), I found a
different syntax for qemu.
2) using tcp://<ip>:<port> just produced an 'unknown failure' on the
receiving side:
root@loadgen137:~> virsh -c qemu:///system migrate --live kvm-testnode-vnode3
qemu+tcp://10.192.11.136/system?no_verify=1 tcp://10.192.11.136:12345
error: Unknown failure
(Note: it was working like a charm when I eliminated the migrateuri
altogether, but this was not an option in all my work environments,
since one of them doesn't feature DNS, breaking migration on the
hostname alone (and keeping /etc/hosts in sync on n machines is
something I'd like to avoid, if possible ;) ))
3) removing the case distinction and the handling of the migrateuri in
the qemudDomainMigratePrepare2 function in qemu_driver.c entirely
(if-statement, and full else-part) solved both my issues.
---
The issue seems to be current with 0.7.1 as well (at least I did receive
the same error message - I did not investigate further)
---
My 2ct: ;)
I think that a wrapper like libvirt should consequently abstract
lower level details (such as 'qemu', or 'xen') as much as possible
(that's what libvirt is for, I guess :) ).
Since the removal of the URI handling solves both the issue _and_
eliminates a case distinction between xen and qemu (painlessly, as
far as I can see), I'd suggest perhaps considering the removal. :)
As to the sending side specification of transfer ports: I think there
should always be a possiblity to leave port selection up to libvirt, as
the sending side does not necessarily an oversight over the free ports
on the receiving side. (I feel that the reason for the different syntax
including mandatory :port part was motivated by the wish to allow users
to specify ports on the sending side)
Cheers,
Gregor Schaffrath.
6:46 a.m.
On Wed, Sep 16, 2009 at 05:42:50PM +0200, Gregor Schaffrath wrote:
Hi all.
Short summary on DV's request ;)
I ran into a problem migrating kvm machines with libvirt-0.6.5:
1) At first, using the same syntax for the migrateuri as with xen (just the
IP) did not work... looking into the source code (! ;) ), I found a
different syntax for qemu.
The URI schemes should be listed in the driver capabilities XML.
The reason they are different is that they are two different ways
of doing migration.
We are working on a new tunnelled migration scheme that will be
uniform across drivers.
2) using tcp://<ip>:<port> just produced an 'unknown
failure' on the
receiving side:
root@loadgen137:~> virsh -c qemu:///system migrate --live kvm-testnode-vnode3
qemu+tcp://10.192.11.136/system?no_verify=1 tcp://10.192.11.136:12345
error: Unknown failure
(Note: it was working like a charm when I eliminated the migrateuri
altogether,
Hmm, try tcp:10.192.11.136:12345 instead - for some unknown reason
it is not using correct URI formats.
3) removing the case distinction and the handling of the migrateuri
in
the qemudDomainMigratePrepare2 function in qemu_driver.c entirely
(if-statement, and full else-part) solved both my issues.
I don't know what exactly you removed, by you'll almost certainly have
broken something else.
daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:24 p.m.
On Mon, Sep 21, 2009 at 12:46:46PM +0100, Daniel P. Berrange wrote:
On Wed, Sep 16, 2009 at 05:42:50PM +0200, Gregor Schaffrath wrote:
> Hi all.
>
> Short summary on DV's request ;)
>
> I ran into a problem migrating kvm machines with libvirt-0.6.5:
>
> 1) At first, using the same syntax for the migrateuri as with xen (just the
> IP) did not work... looking into the source code (! ;) ), I found a
> different syntax for qemu.
The URI schemes should be listed in the driver capabilities XML.
The reason they are different is that they are two different ways
of doing migration.
We are working on a new tunnelled migration scheme that will be
uniform across drivers.
ic - To be honest, I was confused by the migrateuri anyhow,
since I
considered the situation where libvirt traffic is tunneled via SSH, but
Xen-/KVM-/foo communication may be direct a rather rare exception (or am
I mistaken?) (I understood that this was the rationale behind the
hostname-Query in the first place)
> 2) using tcp://<ip>:<port> just produced an 'unknown failure' on
the
> receiving side:
> root@loadgen137:~> virsh -c qemu:///system migrate --live kvm-testnode-vnode3
qemu+tcp://10.192.11.136/system?no_verify=1 tcp://10.192.11.136:12345
> error: Unknown failure
> (Note: it was working like a charm when I eliminated the migrateuri
> altogether,
Hmm, try tcp:10.192.11.136:12345 instead - for some unknown reason
it is not using correct URI formats.
works indeed :)
> 3) removing the case distinction and the handling of the migrateuri in
> the qemudDomainMigratePrepare2 function in qemu_driver.c entirely
> (if-statement, and full else-part) solved both my issues.
I don't know what exactly you removed, by you'll almost certainly have
broken something else.
the part doesn't do much besides setting the port - the hostname is even
ignored ;) ... and the error comes from the receiving side - not the
sending one.
Therefore as far as I see, the only thing broken is that now the sending
side can't choose the listening port number on the receiving side (is
this a debugging feature?)
Cheers & thx for the response,
Gregor.
---snip---
root@loadgen137:/usr/src# diff libvirt-0.6.5/src/qemu_driver.c
libvirt-0.6.5.modified/src/qemu_driver.c
4862c4862
< if (uri_in == NULL) {
---
//if (uri_in == NULL) {
4878c4878
< } else {
---
//} else {
4883c4883
< if (!STRPREFIX (uri_in, "tcp:")) {
---
/* if (!STRPREFIX (uri_in, "tcp:")) {
4887c4887
< }
---
}*/
4890,4892c4890,4892
< p = strrchr (uri_in, ':');
< p++; /* definitely has a ':' in it, see above */
< this_port = virParseNumber (&p);
---
// p = strrchr (uri_in, ':');
// p++; /* definitely has a ':' in it, see above */
/* this_port = virParseNumber (&p);
4898c4898
< }
---
--snip---
}*/
daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:05 a.m.
Gregor Schaffrath wrote:
On Mon, Sep 21, 2009 at 12:46:46PM +0100, Daniel P. Berrange wrote:
> On Wed, Sep 16, 2009 at 05:42:50PM +0200, Gregor Schaffrath wrote:
>> Hi all.
>>
>> Short summary on DV's request ;)
>>
>> I ran into a problem migrating kvm machines with libvirt-0.6.5:
>>
>> 1) At first, using the same syntax for the migrateuri as with xen (just the
>> IP) did not work... looking into the source code (! ;) ), I found a
>> different syntax for qemu.
> The URI schemes should be listed in the driver capabilities XML.
> The reason they are different is that they are two different ways
> of doing migration.
>
> We are working on a new tunnelled migration scheme that will be
> uniform across drivers.
ic - To be honest, I was confused by the migrateuri anyhow, since I
considered the situation where libvirt traffic is tunneled via SSH, but
Xen-/KVM-/foo communication may be direct a rather rare exception (or am
I mistaken?) (I understood that this was the rationale behind the
hostname-Query in the first place)
Well, the rationale is that you may have two paths to get to a machine, and you
may only want to allow migration traffic on one of them (say, a direct
cross-over) since the data goes across unencrypted. So you might have a machine
with eth0 and eth1, where eth0 is exposed to the world, and you have libvirtd
listening on eth0. But then when you actually do the migration, you want it to
send the data across on eth1.
Note also that libvirt traffic tunnelled via ssh isn't the only method, you can
also attach to libvirtd via TLS and TCP (with SASL encryption).
>> 2) using tcp://<ip>:<port> just produced an 'unknown failure'
on the
>> receiving side:
>> root@loadgen137:~> virsh -c qemu:///system migrate --live kvm-testnode-vnode3
qemu+tcp://10.192.11.136/system?no_verify=1 tcp://10.192.11.136:12345
>> error: Unknown failure
>> (Note: it was working like a charm when I eliminated the migrateuri
>> altogether,
> Hmm, try tcp:10.192.11.136:12345 instead - for some unknown reason
> it is not using correct URI formats.
works indeed :)
>> 3) removing the case distinction and the handling of the migrateuri in
>> the qemudDomainMigratePrepare2 function in qemu_driver.c entirely
>> (if-statement, and full else-part) solved both my issues.
> I don't know what exactly you removed, by you'll almost certainly have
> broken something else.
the part doesn't do much besides setting the port - the hostname is even
ignored ;) ... and the error comes from the receiving side - not the
sending one.
Therefore as far as I see, the only thing broken is that now the sending
side can't choose the listening port number on the receiving side (is
this a debugging feature?)
Not exactly a debugging feature, more a "give more control to the admin". If
you do not supply a migrateuri, then libvirtd will choose a port between 49152
and 49216. However, if you don't want to open up all of those ports on your
firewall, you can specify a migrateuri to say "use *this* port", and then you
only have to open up one port in the firewall. So we do need to allow the
migrateuri, and removing it isn't really feasible.
The tunnelled migration stuff should make this a bit easier, although we'll
still have to allow the migrateuri type stuff for the dual-homed situation.
--
Chris Lalancette
3:27 a.m.
On Wed, Sep 23, 2009 at 10:05:36AM +0200, Chris Lalancette wrote:
Gregor Schaffrath wrote:
> On Mon, Sep 21, 2009 at 12:46:46PM +0100, Daniel P. Berrange wrote:
>> On Wed, Sep 16, 2009 at 05:42:50PM +0200, Gregor Schaffrath wrote:
>>> Hi all.
>>>
>>> Short summary on DV's request ;)
>>>
>>> I ran into a problem migrating kvm machines with libvirt-0.6.5:
>>>
>>> 1) At first, using the same syntax for the migrateuri as with xen (just the
>>> IP) did not work... looking into the source code (! ;) ), I found a
>>> different syntax for qemu.
>> The URI schemes should be listed in the driver capabilities XML.
>> The reason they are different is that they are two different ways
>> of doing migration.
>>
>> We are working on a new tunnelled migration scheme that will be
>> uniform across drivers.
> ic - To be honest, I was confused by the migrateuri anyhow, since I
> considered the situation where libvirt traffic is tunneled via SSH, but
> Xen-/KVM-/foo communication may be direct a rather rare exception (or am
> I mistaken?) (I understood that this was the rationale behind the
> hostname-Query in the first place)
Well, the rationale is that you may have two paths to get to a machine, and you
may only want to allow migration traffic on one of them (say, a direct
cross-over) since the data goes across unencrypted. So you might have a machine
with eth0 and eth1, where eth0 is exposed to the world, and you have libvirtd
listening on eth0. But then when you actually do the migration, you want it to
send the data across on eth1.
More critically that than, you cannot assume that the libvirt URI
used to connect to the machine actually has a hostname that refers
to the machine in question, eg
ssh -L 22:src.virt.machine:9001 -L 22:dst.virt.machine:9002 some.gateway.machine
Now do migration using....
virsh -c xen+ssh://localhost:9001/ migrate --live guest xen+ssh://localhost:9002/
...what hostname exactly are you going to expect the migration data
to go over ? It certainly won't be 'localhost'. We have no choice but
todo the hostname query on the target machine to discover the real
primary hostname - the libvirt connection URIs are useless for
determining the hostname. If the user then wants a different NIC that
is not associated with the primary hostname we have to use the extra
migrateuri field to supply that data as there's no other way to get it.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:34 a.m.
On Wed, Sep 23, 2009 at 10:05:36AM +0200, Chris Lalancette wrote:
>> We are working on a new tunnelled migration scheme that will
be
>> uniform across drivers.
> ic - To be honest, I was confused by the migrateuri anyhow, since I
> considered the situation where libvirt traffic is tunneled via SSH, but
> Xen-/KVM-/foo communication may be direct a rather rare exception (or am
> I mistaken?) (I understood that this was the rationale behind the
> hostname-Query in the first place)
Well, the rationale is that you may have two paths to get to a machine, and you
may only want to allow migration traffic on one of them (say, a direct
cross-over) since the data goes across unencrypted. So you might have a machine
with eth0 and eth1, where eth0 is exposed to the world, and you have libvirtd
listening on eth0. But then when you actually do the migration, you want it to
send the data across on eth1.
Note also that libvirt traffic tunnelled via ssh isn't the only method, you can
also attach to libvirtd via TLS and TCP (with SASL encryption).
Hm - I acknowledge
that there might be such situation, so you want to
have this feature.
And as long as there's a way around the assumption that the remote hostname -
especially without a domain part - is resolvable at the sending side, my only
concern would be a unified migrateuri syntax, which seems to be on the
way :) .
I guess my actual confusion is rather about the choice of the default
behaviour, than the feature's existence (since I never had the
split-path-situation in this context, but definitely had an environment where
`hostname` output would not be resolvable, and I deemed the latter more probable
than the prior) - but that's certainly debatable.
> the part doesn't do much besides setting the port - the
hostname is even
> ignored ;) ... and the error comes from the receiving side - not the
> sending one.
>
> Therefore as far as I see, the only thing broken is that now the sending
> side can't choose the listening port number on the receiving side (is
> this a debugging feature?)
Not exactly a debugging feature, more a "give more control to the admin". If
you do not supply a migrateuri, then libvirtd will choose a port between 49152
and 49216. However, if you don't want to open up all of those ports on your
firewall, you can specify a migrateuri to say "use *this* port", and then you
only have to open up one port in the firewall. So we do need to allow the
migrateuri, and removing it isn't really feasible.
Again acknowledged, but then
I would request a possibility to
specify the IP, while leaving the port choice up to the receiving side
(basically making the port specification optional rather than
mandatory).
My rationale for the request would be that when you consider scripted
(i.e. automated) management of virtual nodes along with the possibility
of several concurrent migrations, the port choice on the sending side
is likely to turn out awkward, even though you may have an environment
without DNS, or with dual-homing (and therefore need to specify the
migrateuri).
The tunnelled migration stuff should make this a bit easier, although we'll
still have to allow the migrateuri type stuff for the dual-homed situation.
Hm - I
don't know what exactly you have in mind (not familiar with the
plans). But I'd like to bring forward the point that as a user I was quite
confused, because I intuitively expected a different default behaviour
than the one libvirt currently exhibits (i.e., I was not prepared to get
a 'hostname could not be resolved' type of error, when I specified an IP
as migration destination ;) ).
Cheers,
Gregor.
--
Chris Lalancette
5538
days inactive
5546
days old
5 comments
3 participants
participants (3)
-
Chris Lalancette -
Daniel P. Berrange -
Gregor Schaffrath