12:25 p.m.
So I have an existing QEMU/KVM setup with networks connected to a bridge
which is routed/forwarded to the host's other interfaces. I'm trying
to switch to administering at least *some* of its VMs through libvirt
(probably not all as I use QEMU snapshots extensively for some of them
and libvirt doesn't support that yet, if ever). So libvirt has to be
able to start guests connected to a particular host bridge, and give them
appropriate IP addresses.
I thought it was easy to teach libvirt about this:
spindle:/etc/libvirt/qemu/networks# cat /etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<uuid>ee573497-1d74-473b-a2cb-9916229209b8</uuid>
<forward mode='route'/>
<bridge name='vm-net' stp='on' delay='0' />
<ip address='192.168.20.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.20.1' end='192.168.20.254' />
</dhcp>
</ip>
</network>
However, there appears to be no way to say 'this is what the network is
already like'. That network is considered 'inactive' and can't be used by
any guests, and if I try to make it active, I get this:
virsh # net-start default
error: Failed to start network default
error: cannot create bridge 'vm-net': File exists
Of course it bloody can't create that bridge: it's already there, has an
IP address on the host, and has the host routing packets to it. There
appears to be no option to allow libvirt to assign IPs on the host...
... should I fix that, 'net-start' tries to update iptables rules!
How should I put this: I do not *not not* want libvirt pissing with the
firewall in any way at all. If I want firewall rules, I'll create them.
But there's no way to tell it 'hands off! This network is already active,
don't try to *make* it active!'
... at least, there's none in the documentation, and there's none in the
code that I can see.
Or is there?
12:42 p.m.
On Thu, Nov 26, 2009 at 06:25:07PM +0000, Nix wrote:
I thought it was easy to teach libvirt about this:
spindle:/etc/libvirt/qemu/networks# cat /etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<uuid>ee573497-1d74-473b-a2cb-9916229209b8</uuid>
<forward mode='route'/>
<bridge name='vm-net' stp='on' delay='0' />
<ip address='192.168.20.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.20.1' end='192.168.20.254' />
</dhcp>
</ip>
</network>
However, there appears to be no way to say 'this is what the network is
already like'. That network is considered 'inactive' and can't be used
by
any guests, and if I try to make it active, I get this:
virsh # net-start default
error: Failed to start network default
error: cannot create bridge 'vm-net': File exists
Of course it bloody can't create that bridge: it's already there, has an
IP address on the host, and has the host routing packets to it. There
appears to be no option to allow libvirt to assign IPs on the host...
... should I fix that, 'net-start' tries to update iptables rules!
How should I put this: I do not *not not* want libvirt pissing with the
firewall in any way at all. If I want firewall rules, I'll create them.
But there's no way to tell it 'hands off! This network is already active,
don't try to *make* it active!'
If you don't want libvirt to create the bridge + setup IPtables rules
then don't use the net-XXX commands / XML. That functionality is
not there for pointing libvirt to existing bridge devices.
If you already have a bridge configured, then just point the guest
directly at that bridge by name.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:06 p.m.
On 26 Nov 2009, Daniel P. Berrange verbalised:
If you don't want libvirt to create the bridge + setup IPtables
rules
then don't use the net-XXX commands / XML. That functionality is
not there for pointing libvirt to existing bridge devices.
Ah. All comes clear.
If you already have a bridge configured, then just point the guest
directly at that bridge by name.
That seems to work: I thought it didn't, but that was my DHCP server
deciding to go catatonic on me.
The only remaining obvious problem is virt-manager-related, I think:
this log spam:
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfInterfaces
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfDefinedInterfaces
Emitting this *once a second* seems a bit... extreme. Can't it try once
and give up?
3:53 a.m.
On Fri, Nov 27, 2009 at 12:06:13AM +0000, Nix wrote:
On 26 Nov 2009, Daniel P. Berrange verbalised:
> If you don't want libvirt to create the bridge + setup IPtables rules
> then don't use the net-XXX commands / XML. That functionality is
> not there for pointing libvirt to existing bridge devices.
Ah. All comes clear.
> If you already have a bridge configured, then just point the guest
> directly at that bridge by name.
That seems to work: I thought it didn't, but that was my DHCP server
deciding to go catatonic on me.
The only remaining obvious problem is virt-manager-related, I think:
this log spam:
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfInterfaces
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfDefinedInterfaces
Emitting this *once a second* seems a bit... extreme. Can't it try once
and give up?
libvirtd always logs all errors, so if you want these to go away, then
virt-manager will have to stop issuing these APi calls. Best file a bug
against virt-manager about this, or raise it on the virt-manager mailing
list (virt-tools-list(a)redhat.com)
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:22 a.m.
New subject: [libvirt] how do I stop libvirt futzing with my network configuration?
On 11/26/2009 07:06 PM, Nix wrote:
The only remaining obvious problem is virt-manager-related, I think:
this log spam:
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfInterfaces
Nov 27 00:04:38 spindle err: 00:04:38.977: error : virLibConnError:383 : this function is
not supported by the hypervisor: virConnectNumOfDefinedInterfaces
Emitting this *once a second* seems a bit... extreme. Can't it try once
and give up?
Sounds like you are using a recent upstream virt-manager checkout? This was a
bug I just fixed yesterday (the fix is actually in virtinst, so you'll want to
pull there):
http://hg.fedorahosted.org/hg/python-virtinst/rev/05adb0b921e2
- Cole
2:45 p.m.
New subject: [libvirt] how do I stop libvirt futzing with my network configuration?
On 30 Nov 2009, Cole Robinson spake thusly:
Sounds like you are using a recent upstream virt-manager checkout?
This was a
bug I just fixed yesterday (the fix is actually in virtinst, so you'll want to
pull there):
Ah, I love preemptive bugfixing. Looks fixed, thanks! :)
3:10 p.m.
On 26 Nov 2009, Daniel P. Berrange spake thusly:
On Thu, Nov 26, 2009 at 06:25:07PM +0000, Nix wrote:
> However, there appears to be no way to say 'this is what the network is
> already like'. That network is considered 'inactive' and can't be
used by
> any guests, and if I try to make it active, I get this:
>
> virsh # net-start default
> error: Failed to start network default
> error: cannot create bridge 'vm-net': File exists
>
> Of course it bloody can't create that bridge: it's already there, has an
> IP address on the host, and has the host routing packets to it. There
> appears to be no option to allow libvirt to assign IPs on the host...
>
> ... should I fix that, 'net-start' tries to update iptables rules!
> How should I put this: I do not *not not* want libvirt pissing with the
> firewall in any way at all. If I want firewall rules, I'll create them.
> But there's no way to tell it 'hands off! This network is already active,
> don't try to *make* it active!'
If you don't want libvirt to create the bridge + setup IPtables rules
then don't use the net-XXX commands / XML. That functionality is
not there for pointing libvirt to existing bridge devices.
If you already have a bridge configured, then just point the guest
directly at that bridge by name.
OK, I still can't make this work: it worked briefly but then stopped.
As far as I can tell tools like virt-manager are unwilling to *let* you
connect to a network considered 'inactive', and networks are only
considered active if they have a configuration file under
/var/run/libvirt/network. These files are only created if libvirt has
created the bridge itself as well. If no networks are considerd active,
virt-manager won't let you create a guest at all: it insists on trying
to start the sodding network, and when that fails doesn't let you get
any further.
So as far as I can tell, if you don't want libvirt creating all your
bridges for you, you may as well give up hope of using virt-manager, or
start hacking all this stuff out of the source.
I hoped I could use libvirt in conjunction with raw qemu. So much for
that, it seems :( it *really* wants to take over the world...
(aside: ideally I should not have to spend half an hour crawling around
the source to figure this out. The only other program I've ever seen
that was this hard to set up was Oracle! Whole *Linux distros* take less
work than this. I have half a dozen patches I'll send your way, but I
wasn't going to send any of them until I'd actually managed to get a VM
working. I got one up last night, somehow -- I no longer have any idea
how, obviously one network had somehow got marked active -- whereupon
KVM fell over. *sigh*)
3:56 a.m.
On Sat, Nov 28, 2009 at 09:10:28PM +0000, Nix wrote:
On 26 Nov 2009, Daniel P. Berrange spake thusly:
> On Thu, Nov 26, 2009 at 06:25:07PM +0000, Nix wrote:
>> However, there appears to be no way to say 'this is what the network is
>> already like'. That network is considered 'inactive' and can't
be used by
>> any guests, and if I try to make it active, I get this:
>>
>> virsh # net-start default
>> error: Failed to start network default
>> error: cannot create bridge 'vm-net': File exists
>>
>> Of course it bloody can't create that bridge: it's already there, has
an
>> IP address on the host, and has the host routing packets to it. There
>> appears to be no option to allow libvirt to assign IPs on the host...
>>
>> ... should I fix that, 'net-start' tries to update iptables rules!
>> How should I put this: I do not *not not* want libvirt pissing with the
>> firewall in any way at all. If I want firewall rules, I'll create them.
>> But there's no way to tell it 'hands off! This network is already
active,
>> don't try to *make* it active!'
>
> If you don't want libvirt to create the bridge + setup IPtables rules
> then don't use the net-XXX commands / XML. That functionality is
> not there for pointing libvirt to existing bridge devices.
>
> If you already have a bridge configured, then just point the guest
> directly at that bridge by name.
OK, I still can't make this work: it worked briefly but then stopped.
As far as I can tell tools like virt-manager are unwilling to *let* you
connect to a network considered 'inactive', and networks are only
considered active if they have a configuration file under
/var/run/libvirt/network. These files are only created if libvirt has
created the bridge itself as well. If no networks are considerd active,
virt-manager won't let you create a guest at all: it insists on trying
to start the sodding network, and when that fails doesn't let you get
any further.
These files are all related to the libvirt net-XXX commands, so not
things that are relevant to what you are trying todo.
So as far as I can tell, if you don't want libvirt creating all
your
bridges for you, you may as well give up hope of using virt-manager, or
start hacking all this stuff out of the source.
virt-manager supports two primary networking modes described here:
http://wiki.libvirt.org/page/Networking
The first is the NAT based networking as managed by the net-XXX commands
that you don't want to use. The second is bridging of a physical ethXX
device to share it with a guest
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
4:18 p.m.
On 30 Nov 2009, Daniel P. Berrange spake thusly:
On Sat, Nov 28, 2009 at 09:10:28PM +0000, Nix wrote:
> So as far as I can tell, if you don't want libvirt creating all your
> bridges for you, you may as well give up hope of using virt-manager, or
> start hacking all this stuff out of the source.
virt-manager supports two primary networking modes described here:
http://wiki.libvirt.org/page/Networking
The first is the NAT based networking as managed by the net-XXX commands
that you don't want to use. The second is bridging of a physical ethXX
device to share it with a guest
So you have a choice of letting libvirt mess with your host's
networking, or messing with its physical ethernet configuration
yourself? You can't just use ordinary routing?
Blast. :/
7:29 a.m.
New subject: [libvirt] how do I stop libvirt futzing with my network configuration?
On 11/28/2009 04:10 PM, Nix wrote:
On 26 Nov 2009, Daniel P. Berrange spake thusly:
> On Thu, Nov 26, 2009 at 06:25:07PM +0000, Nix wrote:
>> However, there appears to be no way to say 'this is what the network is
>> already like'. That network is considered 'inactive' and can't be
used by
>> any guests, and if I try to make it active, I get this:
>>
>> virsh # net-start default
>> error: Failed to start network default
>> error: cannot create bridge 'vm-net': File exists
>>
>> Of course it bloody can't create that bridge: it's already there, has an
>> IP address on the host, and has the host routing packets to it. There
>> appears to be no option to allow libvirt to assign IPs on the host...
>>
>> ... should I fix that, 'net-start' tries to update iptables rules!
>> How should I put this: I do not *not not* want libvirt pissing with the
>> firewall in any way at all. If I want firewall rules, I'll create them.
>> But there's no way to tell it 'hands off! This network is already
active,
>> don't try to *make* it active!'
>
> If you don't want libvirt to create the bridge + setup IPtables rules
> then don't use the net-XXX commands / XML. That functionality is
> not there for pointing libvirt to existing bridge devices.
>
> If you already have a bridge configured, then just point the guest
> directly at that bridge by name.
OK, I still can't make this work: it worked briefly but then stopped.
As far as I can tell tools like virt-manager are unwilling to *let* you
connect to a network considered 'inactive', and networks are only
considered active if they have a configuration file under
/var/run/libvirt/network. These files are only created if libvirt has
created the bridge itself as well. If no networks are considerd active,
virt-manager won't let you create a guest at all: it insists on trying
to start the sodding network, and when that fails doesn't let you get
any further.
virt-manager should be fixed here, it should offer some fallback (at least 'no
networking') if it can't start the virtual network.
But if you don't want the virtual network at all and it can't be started
without erroring, you want to virsh net-undefine it, or delete it in
virt-manager via:
Edit->Host Details->Networks
So as far as I can tell, if you don't want libvirt creating all
your
bridges for you, you may as well give up hope of using virt-manager, or
start hacking all this stuff out of the source.
If you create a manual bridge, virt-manager should see it. However, the
virtinst bug I mentioned in my other email was conspiring to make this not
work for you. Pull from virtinst, and your manually created bridge devices
should show up in virt-manager.
- Cole
4:17 p.m.
New subject: [libvirt] [PATCH] Fix executable name and version parsing for KVM 0.11.0-stable.
---
src/qemu/qemu_conf.c | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)
On 30 Nov 2009, Cole Robinson uttered the following:
If you create a manual bridge, virt-manager should see it. However,
the
virtinst bug I mentioned in my other email was conspiring to make this not
work for you. Pull from virtinst, and your manually created bridge devices
should show up in virt-manager.
Pulled. Bridges still show up as 'not bridged', inactive.
I'm giving up. I've spent a week of free time fighting libvirt and
virt-manager and they simply don't seem to be ready for anything but the
simplest of use cases: by comparison I had qemu pretty much working from
the source in about twenty minutes, including reading the manuals from
zero qemu knowledge.
Maybe libvirt works better if you're using one of the subset of popular
distros for which netcf knows how to parse and manipulate the network
config (something which no serious site is going to allow in any case,
because they have their own network control scripts). Maybe libvirt and
virt-manager are only meant to be useful for single-user desktops and
very small sites, in which case it would be nice to, y'know, *say* that
somewhere, perhaps along with a big banner saying "don't bother using
the code from upstream if you're not running unmodified Fedora or RHEL",
so people don't waste their time trying to make it work otherwise. Maybe
I overstate the case, but I don't think by much. I've looked at libvirt
support in a pile of not-completely-mainstream distros now and I haven't
yet found one in which networking works without manual bringup, because
netcf fails to parse the distro config files (yes, I know Augeas support
is possible, but not many people have written that and nobody but Debian
has contributed it upstream: a shame, Augeas is *lovely*). And that then
lands me in the same situation I've just been trying and failing to get
out of.
Whenever anything goes wrong in the virtualization layer I have to fall
back to raw qemu for monitor access in any case, because for reasons
incomprehensible to me libvirt actually bans me from accessing the
monitor or even adding a -serial mon:telnet stanza (or *anything*
user-definable) to the command line: and it insists on using VNC
consoles so I can't use the SDL console's monitor either (though this is
reasonable: you want to be able to stop virt-manager and leave the
emulator running, which VNC makes easier). So the ability to ignore the
nuts and bolts of your virtualization system's command line is
effectively lost the first time anything goes wrong, and you have to
keep everything working without libvirt as well. (Of course the qemu
guys assume you have monitor access when trying to debug problems.)
While I wish I *could* use libvirt (the ability to start VMs as a
regular user with working tun/tap alone would be excellent), it's just
not remotely ready for serious use yet, though it's a very promising
start and has all the infrastructure that it needs to be something very
nice indeed in the future. I think I'll hack up something much simpler
for now which just kicks up tun/tap devices as root and passes their fds
down to setuid() children that run specified qemus as a regular
user. You'd think someone would have written this already...
Have a patch to be going on with, anyway: libvirt can't parse the
version output of qemu-kvm 0.11.0-stable, and doesn't know the name of
its x86_64 upstream binary. (I'm not certain that this is right:
I'm not sure it can distinguish between KMM and non-KVM. I'm equally
unsure that this matters anymore now that enough KVM support to be
getting on with is upstream.)
diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c
index 8f8d490..10efdae 100644
--- a/src/qemu/qemu_conf.c
+++ b/src/qemu/qemu_conf.c
@@ -642,7 +642,8 @@ qemudCapsInitGuest(virCapsPtr caps,
if (STREQ(info->arch, hostmachine) ||
(STREQ(hostmachine, "x86_64") && STREQ(info->arch,
"i686"))) {
const char *const kvmbins[] = { "/usr/bin/qemu-kvm", /* Fedora */
- "/usr/bin/kvm" }; /* Upstream .spec */
+ "/usr/bin/kvm", /* Upstream .spec */
+ "/usr/bin/qemu-system-x86_64" }; /*
Upstream sources */
for (i = 0; i < ARRAY_CARDINALITY(kvmbins); ++i) {
if (access(kvmbins[i], X_OK) == 0 &&
@@ -953,6 +954,7 @@ static unsigned int qemudComputeCmdFlags(const char *help,
* in parenthesis as follows:
* - qemu-kvm-x.y.z in stable releases
* - kvm-XX for kvm versions up to kvm-85
+ * - kvm-devel for kvm 0.11.0-stable
* - qemu-kvm-devel-XX for kvm version kvm-86 and later
*
* For qemu-kvm versions before 0.10.z, we need to detect
@@ -962,6 +964,7 @@ static unsigned int qemudComputeCmdFlags(const char *help,
*/
#define QEMU_VERSION_STR "QEMU PC emulator version"
#define QEMU_KVM_VER_PREFIX "(qemu-kvm-"
+#define KVM_GIT_STABLE_VER_PREFIX "(kvm-devel)"
#define KVM_VER_PREFIX "(kvm-"
#define SKIP_BLANKS(p) do { while ((*(p) == ' ') || (*(p) == '\t'))
(p)++; } while (0)
@@ -1005,6 +1008,9 @@ int qemudParseHelpStr(const char *help,
if (STRPREFIX(p, QEMU_KVM_VER_PREFIX)) {
*is_kvm = 1;
p += strlen(QEMU_KVM_VER_PREFIX);
+ } else if (STRPREFIX(p, KVM_GIT_STABLE_VER_PREFIX)) {
+ *is_kvm = 1;
+ p += strlen(KVM_GIT_STABLE_VER_PREFIX);
} else if (STRPREFIX(p, KVM_VER_PREFIX)) {
int ret;
--
1.6.5.3.100.g75959
4:49 p.m.
New subject: [libvirt] Re: [PATCH] Fix executable name and version parsing for KVM 0.11.0-stable.
On 11/30/2009 05:17 PM, Nix wrote:
---
src/qemu/qemu_conf.c | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)
On 30 Nov 2009, Cole Robinson uttered the following:
> If you create a manual bridge, virt-manager should see it. However, the
> virtinst bug I mentioned in my other email was conspiring to make this not
> work for you. Pull from virtinst, and your manually created bridge devices
> should show up in virt-manager.
Pulled. Bridges still show up as 'not bridged', inactive.
It might just be unfortunate timing. If you look at the recent virt-manager
commits (as in, within the past week), the bridging code was largely
refactored to deal with HAL deprecation, and optionally use libvirt nodedev
and interface (aka netcf) APIs if available. This may very well have caused
regressions. Might be worth a shot to try the 0.8.0 release and see if you
still hit issues.
Also, can you provide the output of:
~/.virt-manager/virt-manager.log
brctl show
virsh iface-list --all
- Cole
5472
days inactive
5476
days old
11 comments
3 participants
participants (3)
-
Cole Robinson -
Daniel P. Berrange -
Nix