Re: [libvirt-users] [libvirt] configuring network interface
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem. a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1 b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config # The loopback network interface auto lo iface lo inet loopback auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0 c) this is the xml guest configuration related to the network. <interface type='bridge'> <source bridge='br0'/> </interface> d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..)*. I don't know how to achieve the guest don't behave like this and create always its network interface *with a fixed name, for example, eth0). Why I ask that? if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either. The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface. I hope I've explained better the problem I have. Thank you very much, you've already helped too much to better understand the different cases of network configuration Regards. 2011/2/10 Marcela Castro León <mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump <laine@laine.org>
(For future questions such as this, you may find you get a better response
(and lower signal-noise ratio) by asking on the recently -created libvirt-users@redhat.com list, as libvir-list has a very large volume of patches and discussion about the internals of libvirt)
On 02/08/2011 12:05 PM, Marcela Castro León wrote:
Hello
I've a lan of virtual machine configured with bridge. I've a private
lan of 4 machine, but one of them have 2 lan-card with access to internet.
Evertything works almost fine, except that I've configured on each guest the static configuration of the bridge with the ip address, like this:
auto lo iface lo inet loopback
auto eth6 iface eth6 inet static address 192.168.1.41 network 192.168.1.0 netmask 255.255.255.0 gateway 192.168.1.10
So are you saying that you're attempting to give each guest the *same* IP address? There's no way that can possibly work, even if each is connected to a separate virtual network or bridge. Since the host's IP stack can "see" all of the guests' network traffic, regardless of the connection mode you choose, each must have a unique IP address even if they can't see each other.
But each time I, for example, define the guest, it start with a ascending number of interface: eth6,eth7, eth8, etc, Now is initializing with eth12.
There must be a mixup in the explanation - I'm understanding that you put the same config file in each guest, but each guest, when it starts, shows a different ethernet interface in its ifconfig. That doesn't seem right, because none of the guests know about each other. Or are you saying something different? Where are you seeing the "eth6", "eth7", ... "eth12"? In the guest XML config maybe?
So, it doesn't get the manual configuration I've done on the /etc/network/interfaces. How can avoid this way of work of the number of ethernet interface the guest start?
Why are you doing a manual/static definition on the guest at all? It's simpler to just leave the guest to automatically configure its eth0, and if you really want a particular IP address for a particular guest, handle that with a <host> element in the <dhcp> section of the virtual network the guest is connected to (or, if you're using a pure bridging setup, then in the static hosts config of the dhcp server on that network).
A way i'm trying to solve is configuring the interface trough the guest xml definition,
The guest xml interface configuration cannot be used to setup the IP address, route, etc config of the guest. It is only used for two things:
1) configuring what kind of hardware appears on the guest (but not what name the guest gives to it!)
2) configuring how that hardware is connected to the physical network (usually either via an existing bridge interface on the host (in order for the guest to appear as if its directly connected to the physical network), or via a "virtual network" created by libvirt (which is really just a bridge that's not directly connected to any physical interface, relying instead on the host's IP stack to route packets from the guest out to the physical network.)
but the operation "define" on virsh accepted without error the xml i've done, but the dumpxml doesn't show them, so, it doesn't work at all. I've followed the instruction that said the manual "Application development guide - a guide to application development with libvirt" on page 71.
That document is targeted towards software developers writing applications to manage virtual guests using the libvirt API, not for system administrators who are configuring virtual guests manually (or, more likely, by using software that uses the libvirt API, eg virsh or virt-manager). It likely contains more detail and less overview than what you need, which is probably what led to the confusion. I've included a pointer to a wiki page below that is more the appropriate level of information (Justin, if the docs you were working on are somewhere easily accessible, you can point to those instead.)
The whole definition of the VM is attached, but the related to interface is: <interface type='bridge'> <source bridge='br0'/> <target dev='vnet0'/> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> <bridge stp="off" delay="0.01"> <interface type="ethernet" name="eth12"> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> </interface> </bridge> </interface>
You are confusing the guest interface config (that is part of the guest's XML config, and described above), and host physical interface config (that is not in any way associated with guest config, but instead used to configure the host's physical network interfaces, bridges, bonds, and vlan interfaces; and nearly as important, is not supported on Ubuntu anyway).
Actual guest interface config is *much* simpler than that.
But after define, the dumpxml only show this part: <interface type='bridge'> <mac address='52:54:00:3c:92:9a'/> <source bridge='br0'/> </interface>
That's because everything else you've put in your guest config is not actually a part of a valid guest interface config, and so is ignored.
I would appreciate any help to solve the question. The problem I've is I lost the access to the guest every time the libvirt change the name of the interface eth1,2,3, etc.
In this case I think it may be useful to start over. You've somehow been tricked into misunderstanding the way that network configuration in libvirt works, confusing host interface config with guest interface config, and the resulting config bears little resemblance to what would actually work.
== The first step is to read this page:
http://wiki.libvirt.org/page/Networking
to get a basic overview.
== Second step: you should decide which of the three methods of connecting to guests to the physical network you want to use:
1) NATed 2) routed 3) bridged
The simplest is (1), and that will probably work just fine as long as 1) you don't need incoming connections to the guests, and 2) you're not concerned about getting the last fraction of an ounce of performance out of the network.
Depending on which of those 3 you decide to use:
== Third step: follow the instructions to either create a bridge interface on the host connected to a physical network interface (for (3)), or to create a libvirt virtual network using one of the two modes ((1) or (2) - actually the stock libvirt config already has a "default" network that is probably adequate for (1), and unless you understand IP routing very well, you'll likely want to avoid (2)).
Once the physical network is setup:
== Fourth step: define your guests. In each guest, you'll have an <interface> element, but it will be much simpler than what you've previously tried. For bridged mode, the interface section would look like this:
<interface type='bridge'> <source bridge='br0'/> </interface>
Depending on the guest, you may want to specify what type of hardware is presented to the guest (default is rtl, which almost always works, but is not the best perfrorming), in particular, if the guest supports virtio-net, you'll want to add this into the <interface> element:
<model type='virtio'/>
For NATed or routed mode, the guest's interface definition would look like this:
<interface type='network'> <source network='default'/> </interface>
(replace "default" with the name of the network you've defined, if you've defined your own). In this case also, you can add a <model type='...'/> if appropriate.
Note that I don't put anything in the <interface> element about MAC address, target device, or bus address. Those are all setup automatically by libvirt the first time you start the guest, and it will make sure that the guests don't interfere with each other.
== Fifth step: only if you need a particular guest to have a particular fixed IP address, either 1) edit the network config file on the guest, 2) add a static host to the libvirt virtual network definition (giving the MAC address provided to the guest by libvirt during the initial guest startup) with a <host> element added to the <dhcp> element of the network; details are here:
http://www.libvirt.org/formatnetwork.html
or 3) if you're using the bridged method to connect to the physical network, again either configure the guest's own config file, or add a static host definition to the dhcp server running on your physical network.
If you still have problems don't hesitate to send another mail (although, as I said at the top, sending to libvirt-users@redhat.com may yield better results than libvir-list@redhat.com).
Your problem is related to udev : every reboot of VM, udev in VM checks the MAC address of the ethernet interface of the system to associate it with a static name (eth0, eth1...). If you change the MAC address of the Ethernet interface of your VM, it will detect a new interface and associate it with a new name (ethx+1). In order to avoid it : try not to change the MAC address of your VM ethernet device (you can fix it with libvirt xml), or deactivate this mecanism in the VM : search in /etc/udev/rules.d/70-persistent-net.rules : it calls /lib/udev/write_net_rules at every reboot, you could chmod -x this script in order to avoid it. Tanguy 2011/2/10 Marcela Castro León <mcastrol@gmail.com>:
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem.
a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1
b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config
# The loopback network interface auto lo iface lo inet loopback
auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0
c) this is the xml guest configuration related to the network.
<interface type='bridge'> <source bridge='br0'/> </interface>
d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..). I don't know how to achieve the guest don't behave like this and create always its network interface with a fixed name, for example, eth0).
Why I ask that?
if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either.
The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface.
I hope I've explained better the problem I have.
Thank you very much, you've already helped too much to better understand the different cases of network configuration
Regards.
2011/2/10 Marcela Castro León <mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump <laine@laine.org>
(For future questions such as this, you may find you get a better response (and lower signal-noise ratio) by asking on the recently -created libvirt-users@redhat.com list, as libvir-list has a very large volume of patches and discussion about the internals of libvirt)
On 02/08/2011 12:05 PM, Marcela Castro León wrote:
Hello
I've a lan of virtual machine configured with bridge. I've a private lan of 4 machine, but one of them have 2 lan-card with access to internet.
Evertything works almost fine, except that I've configured on each guest the static configuration of the bridge with the ip address, like this:
auto lo iface lo inet loopback
auto eth6 iface eth6 inet static address 192.168.1.41 network 192.168.1.0 netmask 255.255.255.0 gateway 192.168.1.10
So are you saying that you're attempting to give each guest the *same* IP address? There's no way that can possibly work, even if each is connected to a separate virtual network or bridge. Since the host's IP stack can "see" all of the guests' network traffic, regardless of the connection mode you choose, each must have a unique IP address even if they can't see each other.
But each time I, for example, define the guest, it start with a ascending number of interface: eth6,eth7, eth8, etc, Now is initializing with eth12.
There must be a mixup in the explanation - I'm understanding that you put the same config file in each guest, but each guest, when it starts, shows a different ethernet interface in its ifconfig. That doesn't seem right, because none of the guests know about each other. Or are you saying something different? Where are you seeing the "eth6", "eth7", ... "eth12"? In the guest XML config maybe?
So, it doesn't get the manual configuration I've done on the /etc/network/interfaces. How can avoid this way of work of the number of ethernet interface the guest start?
Why are you doing a manual/static definition on the guest at all? It's simpler to just leave the guest to automatically configure its eth0, and if you really want a particular IP address for a particular guest, handle that with a <host> element in the <dhcp> section of the virtual network the guest is connected to (or, if you're using a pure bridging setup, then in the static hosts config of the dhcp server on that network).
A way i'm trying to solve is configuring the interface trough the guest xml definition,
The guest xml interface configuration cannot be used to setup the IP address, route, etc config of the guest. It is only used for two things:
1) configuring what kind of hardware appears on the guest (but not what name the guest gives to it!)
2) configuring how that hardware is connected to the physical network (usually either via an existing bridge interface on the host (in order for the guest to appear as if its directly connected to the physical network), or via a "virtual network" created by libvirt (which is really just a bridge that's not directly connected to any physical interface, relying instead on the host's IP stack to route packets from the guest out to the physical network.)
but the operation "define" on virsh accepted without error the xml i've done, but the dumpxml doesn't show them, so, it doesn't work at all. I've followed the instruction that said the manual "Application development guide - a guide to application development with libvirt" on page 71.
That document is targeted towards software developers writing applications to manage virtual guests using the libvirt API, not for system administrators who are configuring virtual guests manually (or, more likely, by using software that uses the libvirt API, eg virsh or virt-manager). It likely contains more detail and less overview than what you need, which is probably what led to the confusion. I've included a pointer to a wiki page below that is more the appropriate level of information (Justin, if the docs you were working on are somewhere easily accessible, you can point to those instead.)
The whole definition of the VM is attached, but the related to interface is: <interface type='bridge'> <source bridge='br0'/> <target dev='vnet0'/> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> <bridge stp="off" delay="0.01"> <interface type="ethernet" name="eth12"> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> </interface> </bridge> </interface>
You are confusing the guest interface config (that is part of the guest's XML config, and described above), and host physical interface config (that is not in any way associated with guest config, but instead used to configure the host's physical network interfaces, bridges, bonds, and vlan interfaces; and nearly as important, is not supported on Ubuntu anyway).
Actual guest interface config is *much* simpler than that.
But after define, the dumpxml only show this part: <interface type='bridge'> <mac address='52:54:00:3c:92:9a'/> <source bridge='br0'/> </interface>
That's because everything else you've put in your guest config is not actually a part of a valid guest interface config, and so is ignored.
I would appreciate any help to solve the question. The problem I've is I lost the access to the guest every time the libvirt change the name of the interface eth1,2,3, etc.
In this case I think it may be useful to start over. You've somehow been tricked into misunderstanding the way that network configuration in libvirt works, confusing host interface config with guest interface config, and the resulting config bears little resemblance to what would actually work.
== The first step is to read this page:
http://wiki.libvirt.org/page/Networking
to get a basic overview.
== Second step: you should decide which of the three methods of connecting to guests to the physical network you want to use:
1) NATed 2) routed 3) bridged
The simplest is (1), and that will probably work just fine as long as 1) you don't need incoming connections to the guests, and 2) you're not concerned about getting the last fraction of an ounce of performance out of the network.
Depending on which of those 3 you decide to use:
== Third step: follow the instructions to either create a bridge interface on the host connected to a physical network interface (for (3)), or to create a libvirt virtual network using one of the two modes ((1) or (2) - actually the stock libvirt config already has a "default" network that is probably adequate for (1), and unless you understand IP routing very well, you'll likely want to avoid (2)).
Once the physical network is setup:
== Fourth step: define your guests. In each guest, you'll have an <interface> element, but it will be much simpler than what you've previously tried. For bridged mode, the interface section would look like this:
<interface type='bridge'> <source bridge='br0'/> </interface>
Depending on the guest, you may want to specify what type of hardware is presented to the guest (default is rtl, which almost always works, but is not the best perfrorming), in particular, if the guest supports virtio-net, you'll want to add this into the <interface> element:
<model type='virtio'/>
For NATed or routed mode, the guest's interface definition would look like this:
<interface type='network'> <source network='default'/> </interface>
(replace "default" with the name of the network you've defined, if you've defined your own). In this case also, you can add a <model type='...'/> if appropriate.
Note that I don't put anything in the <interface> element about MAC address, target device, or bus address. Those are all setup automatically by libvirt the first time you start the guest, and it will make sure that the guests don't interfere with each other.
== Fifth step: only if you need a particular guest to have a particular fixed IP address, either 1) edit the network config file on the guest, 2) add a static host to the libvirt virtual network definition (giving the MAC address provided to the guest by libvirt during the initial guest startup) with a <host> element added to the <dhcp> element of the network; details are here:
http://www.libvirt.org/formatnetwork.html
or 3) if you're using the bridged method to connect to the physical network, again either configure the guest's own config file, or add a static host definition to the dhcp server running on your physical network.
If you still have problems don't hesitate to send another mail (although, as I said at the top, sending to libvirt-users@redhat.com may yield better results than libvir-list@redhat.com).
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- ---------------------------------------- Tanguy ROZIER tanguy@rozier.eu +33 6 07 26 46 68 ----------------------------------------
On 02/10/2011 08:50 AM, Tanguy ROZIER wrote:
Your problem is related to udev : every reboot of VM, udev in VM checks the MAC address of the ethernet interface of the system to associate it with a static name (eth0, eth1...). If you change the MAC address of the Ethernet interface of your VM, it will detect a new interface and associate it with a new name (ethx+1). In order to avoid it : try not to change the MAC address of your VM ethernet device (you can fix it with libvirt xml),
When a domain is defined with an interface, if no mac address is supplied, libvirt will automatically generate one and write that generated mac back to the config, so that the next time the guest boots it will be presented with an ethernet having the same mac address as the previous time. How are you defining your guests? Are you perhaps using "virsh create" (but using an existing image file) to create a transient guest each time you want to boot the guest, rather than doing "virsh define" once to create a persistent guest, then using "virsh start" each time you want to boot it? You should be doing the latter - if you continuously use "virsh create", libvirt will think that it's a "throw away" guest, and won't save the config (or any changes to it), so the next time you do "virsh create", the disk image will have memory of the previous run (by having "eth0" assigned to some particular mac address), but libvirt won't have any memory of it (it thinks you're creating a brand new guest), so it will generate a new mac address for the interface, the guest will boot, see a "new" interface, notice that "eth0" is already taken, so it will use "eth1", and so on. If you actually are using "virsh define" to define a persistent guest, do a "virsh dumpxml" of the guest after you've started and destroyed it, and see if there is a mac address in the interface config. If not, that's a bug.
or deactivate this mecanism in the VM : search in /etc/udev/rules.d/70-persistent-net.rules : it calls /lib/udev/write_net_rules at every reboot, you could chmod -x this script in order to avoid it.
This may work (or may not - I haven't investigated), but it isn't the right way to fix this problem. It's dealing with the symptom rather than the cause.
Tanguy
2011/2/10 Marcela Castro León<mcastrol@gmail.com>:
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem.
a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1
b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config
# The loopback network interface auto lo iface lo inet loopback
auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0
c) this is the xml guest configuration related to the network.
<interface type='bridge'> <source bridge='br0'/> </interface>
d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..). I don't know how to achieve the guest don't behave like this and create always its network interface with a fixed name, for example, eth0).
Why I ask that?
if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either.
The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface.
I hope I've explained better the problem I have.
Thank you very much, you've already helped too much to better understand the different cases of network configuration
Regards.
2011/2/10 Marcela Castro León<mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump<laine@laine.org>
(For future questions such as this, you may find you get a better response (and lower signal-noise ratio) by asking on the recently -created libvirt-users@redhat.com list, as libvir-list has a very large volume of patches and discussion about the internals of libvirt)
On 02/08/2011 12:05 PM, Marcela Castro León wrote:
Hello I've a lan of virtual machine configured with bridge. I've a private lan of 4 machine, but one of them have 2 lan-card with access to internet.
Evertything works almost fine, except that I've configured on each guest the static configuration of the bridge with the ip address, like this:
auto lo iface lo inet loopback
auto eth6 iface eth6 inet static address 192.168.1.41 network 192.168.1.0 netmask 255.255.255.0 gateway 192.168.1.10
So are you saying that you're attempting to give each guest the *same* IP address? There's no way that can possibly work, even if each is connected to a separate virtual network or bridge. Since the host's IP stack can "see" all of the guests' network traffic, regardless of the connection mode you choose, each must have a unique IP address even if they can't see each other.
But each time I, for example, define the guest, it start with a ascending number of interface: eth6,eth7, eth8, etc, Now is initializing with eth12.
There must be a mixup in the explanation - I'm understanding that you put the same config file in each guest, but each guest, when it starts, shows a different ethernet interface in its ifconfig. That doesn't seem right, because none of the guests know about each other. Or are you saying something different? Where are you seeing the "eth6", "eth7", ... "eth12"? In the guest XML config maybe?
So, it doesn't get the manual configuration I've done on the /etc/network/interfaces. How can avoid this way of work of the number of ethernet interface the guest start?
Why are you doing a manual/static definition on the guest at all? It's simpler to just leave the guest to automatically configure its eth0, and if you really want a particular IP address for a particular guest, handle that with a<host> element in the<dhcp> section of the virtual network the guest is connected to (or, if you're using a pure bridging setup, then in the static hosts config of the dhcp server on that network).
A way i'm trying to solve is configuring the interface trough the guest xml definition,
The guest xml interface configuration cannot be used to setup the IP address, route, etc config of the guest. It is only used for two things:
1) configuring what kind of hardware appears on the guest (but not what name the guest gives to it!)
2) configuring how that hardware is connected to the physical network (usually either via an existing bridge interface on the host (in order for the guest to appear as if its directly connected to the physical network), or via a "virtual network" created by libvirt (which is really just a bridge that's not directly connected to any physical interface, relying instead on the host's IP stack to route packets from the guest out to the physical network.)
but the operation "define" on virsh accepted without error the xml i've done, but the dumpxml doesn't show them, so, it doesn't work at all. I've followed the instruction that said the manual "Application development guide - a guide to application development with libvirt" on page 71.
That document is targeted towards software developers writing applications to manage virtual guests using the libvirt API, not for system administrators who are configuring virtual guests manually (or, more likely, by using software that uses the libvirt API, eg virsh or virt-manager). It likely contains more detail and less overview than what you need, which is probably what led to the confusion. I've included a pointer to a wiki page below that is more the appropriate level of information (Justin, if the docs you were working on are somewhere easily accessible, you can point to those instead.)
The whole definition of the VM is attached, but the related to interface is: <interface type='bridge'> <source bridge='br0'/> <target dev='vnet0'/> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> <bridge stp="off" delay="0.01"> <interface type="ethernet" name="eth12"> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> </interface> </bridge> </interface>
You are confusing the guest interface config (that is part of the guest's XML config, and described above), and host physical interface config (that is not in any way associated with guest config, but instead used to configure the host's physical network interfaces, bridges, bonds, and vlan interfaces; and nearly as important, is not supported on Ubuntu anyway).
Actual guest interface config is *much* simpler than that.
But after define, the dumpxml only show this part: <interface type='bridge'> <mac address='52:54:00:3c:92:9a'/> <source bridge='br0'/> </interface>
That's because everything else you've put in your guest config is not actually a part of a valid guest interface config, and so is ignored.
I would appreciate any help to solve the question. The problem I've is I lost the access to the guest every time the libvirt change the name of the interface eth1,2,3, etc.
In this case I think it may be useful to start over. You've somehow been tricked into misunderstanding the way that network configuration in libvirt works, confusing host interface config with guest interface config, and the resulting config bears little resemblance to what would actually work.
== The first step is to read this page:
http://wiki.libvirt.org/page/Networking
to get a basic overview.
== Second step: you should decide which of the three methods of connecting to guests to the physical network you want to use:
1) NATed 2) routed 3) bridged
The simplest is (1), and that will probably work just fine as long as 1) you don't need incoming connections to the guests, and 2) you're not concerned about getting the last fraction of an ounce of performance out of the network.
Depending on which of those 3 you decide to use:
== Third step: follow the instructions to either create a bridge interface on the host connected to a physical network interface (for (3)), or to create a libvirt virtual network using one of the two modes ((1) or (2) - actually the stock libvirt config already has a "default" network that is probably adequate for (1), and unless you understand IP routing very well, you'll likely want to avoid (2)).
Once the physical network is setup:
== Fourth step: define your guests. In each guest, you'll have an <interface> element, but it will be much simpler than what you've previously tried. For bridged mode, the interface section would look like this:
<interface type='bridge'> <source bridge='br0'/> </interface>
Depending on the guest, you may want to specify what type of hardware is presented to the guest (default is rtl, which almost always works, but is not the best perfrorming), in particular, if the guest supports virtio-net, you'll want to add this into the<interface> element:
<model type='virtio'/>
For NATed or routed mode, the guest's interface definition would look like this:
<interface type='network'> <source network='default'/> </interface>
(replace "default" with the name of the network you've defined, if you've defined your own). In this case also, you can add a<model type='...'/> if appropriate.
Note that I don't put anything in the<interface> element about MAC address, target device, or bus address. Those are all setup automatically by libvirt the first time you start the guest, and it will make sure that the guests don't interfere with each other.
== Fifth step: only if you need a particular guest to have a particular fixed IP address, either 1) edit the network config file on the guest, 2) add a static host to the libvirt virtual network definition (giving the MAC address provided to the guest by libvirt during the initial guest startup) with a<host> element added to the<dhcp> element of the network; details are here:
http://www.libvirt.org/formatnetwork.html
or 3) if you're using the bridged method to connect to the physical network, again either configure the guest's own config file, or add a static host definition to the dhcp server running on your physical network.
If you still have problems don't hesitate to send another mail (although, as I said at the top, sending to libvirt-users@redhat.com may yield better results than libvir-list@redhat.com).
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
OK so he should fix the problem with libvirt to associate a fixed MAC to the interface of the VM : <interface type='bridge'> <source bridge='br0'/> <mac address="00:11:22:33:44:55"/> <----------this is here : you must *define* the MAC address of your interface </interface> Tanguy 2011/2/10 Laine Stump <laine@laine.org>:
On 02/10/2011 08:50 AM, Tanguy ROZIER wrote:
Your problem is related to udev : every reboot of VM, udev in VM checks the MAC address of the ethernet interface of the system to associate it with a static name (eth0, eth1...). If you change the MAC address of the Ethernet interface of your VM, it will detect a new interface and associate it with a new name (ethx+1). In order to avoid it : try not to change the MAC address of your VM ethernet device (you can fix it with libvirt xml),
When a domain is defined with an interface, if no mac address is supplied, libvirt will automatically generate one and write that generated mac back to the config, so that the next time the guest boots it will be presented with an ethernet having the same mac address as the previous time.
How are you defining your guests? Are you perhaps using "virsh create" (but using an existing image file) to create a transient guest each time you want to boot the guest, rather than doing "virsh define" once to create a persistent guest, then using "virsh start" each time you want to boot it? You should be doing the latter - if you continuously use "virsh create", libvirt will think that it's a "throw away" guest, and won't save the config (or any changes to it), so the next time you do "virsh create", the disk image will have memory of the previous run (by having "eth0" assigned to some particular mac address), but libvirt won't have any memory of it (it thinks you're creating a brand new guest), so it will generate a new mac address for the interface, the guest will boot, see a "new" interface, notice that "eth0" is already taken, so it will use "eth1", and so on.
If you actually are using "virsh define" to define a persistent guest, do a "virsh dumpxml" of the guest after you've started and destroyed it, and see if there is a mac address in the interface config. If not, that's a bug.
or deactivate this mecanism in the VM : search in /etc/udev/rules.d/70-persistent-net.rules : it calls /lib/udev/write_net_rules at every reboot, you could chmod -x this script in order to avoid it.
This may work (or may not - I haven't investigated), but it isn't the right way to fix this problem. It's dealing with the symptom rather than the cause.
Tanguy
2011/2/10 Marcela Castro León<mcastrol@gmail.com>:
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem.
a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1
b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config
# The loopback network interface auto lo iface lo inet loopback
auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0
c) this is the xml guest configuration related to the network.
<interface type='bridge'> <source bridge='br0'/> </interface>
d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..). I don't know how to achieve the guest don't behave like this and create always its network interface with a fixed name, for example, eth0).
Why I ask that?
if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either.
The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface.
I hope I've explained better the problem I have.
Thank you very much, you've already helped too much to better understand the different cases of network configuration
Regards.
2011/2/10 Marcela Castro León<mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump<laine@laine.org>
(For future questions such as this, you may find you get a better response (and lower signal-noise ratio) by asking on the recently -created libvirt-users@redhat.com list, as libvir-list has a very large volume of patches and discussion about the internals of libvirt)
On 02/08/2011 12:05 PM, Marcela Castro León wrote:
Hello I've a lan of virtual machine configured with bridge. I've a private lan of 4 machine, but one of them have 2 lan-card with access to internet.
Evertything works almost fine, except that I've configured on each guest the static configuration of the bridge with the ip address, like this:
auto lo iface lo inet loopback
auto eth6 iface eth6 inet static address 192.168.1.41 network 192.168.1.0 netmask 255.255.255.0 gateway 192.168.1.10
So are you saying that you're attempting to give each guest the *same* IP address? There's no way that can possibly work, even if each is connected to a separate virtual network or bridge. Since the host's IP stack can "see" all of the guests' network traffic, regardless of the connection mode you choose, each must have a unique IP address even if they can't see each other.
But each time I, for example, define the guest, it start with a ascending number of interface: eth6,eth7, eth8, etc, Now is initializing with eth12.
There must be a mixup in the explanation - I'm understanding that you put the same config file in each guest, but each guest, when it starts, shows a different ethernet interface in its ifconfig. That doesn't seem right, because none of the guests know about each other. Or are you saying something different? Where are you seeing the "eth6", "eth7", ... "eth12"? In the guest XML config maybe?
So, it doesn't get the manual configuration I've done on the /etc/network/interfaces. How can avoid this way of work of the number of ethernet interface the guest start?
Why are you doing a manual/static definition on the guest at all? It's simpler to just leave the guest to automatically configure its eth0, and if you really want a particular IP address for a particular guest, handle that with a<host> element in the<dhcp> section of the virtual network the guest is connected to (or, if you're using a pure bridging setup, then in the static hosts config of the dhcp server on that network).
A way i'm trying to solve is configuring the interface trough the guest xml definition,
The guest xml interface configuration cannot be used to setup the IP address, route, etc config of the guest. It is only used for two things:
1) configuring what kind of hardware appears on the guest (but not what name the guest gives to it!)
2) configuring how that hardware is connected to the physical network (usually either via an existing bridge interface on the host (in order for the guest to appear as if its directly connected to the physical network), or via a "virtual network" created by libvirt (which is really just a bridge that's not directly connected to any physical interface, relying instead on the host's IP stack to route packets from the guest out to the physical network.)
but the operation "define" on virsh accepted without error the xml i've done, but the dumpxml doesn't show them, so, it doesn't work at all. I've followed the instruction that said the manual "Application development guide - a guide to application development with libvirt" on page 71.
That document is targeted towards software developers writing applications to manage virtual guests using the libvirt API, not for system administrators who are configuring virtual guests manually (or, more likely, by using software that uses the libvirt API, eg virsh or virt-manager). It likely contains more detail and less overview than what you need, which is probably what led to the confusion. I've included a pointer to a wiki page below that is more the appropriate level of information (Justin, if the docs you were working on are somewhere easily accessible, you can point to those instead.)
The whole definition of the VM is attached, but the related to interface is: <interface type='bridge'> <source bridge='br0'/> <target dev='vnet0'/> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> <bridge stp="off" delay="0.01"> <interface type="ethernet" name="eth12"> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> </interface> </bridge> </interface>
You are confusing the guest interface config (that is part of the guest's XML config, and described above), and host physical interface config (that is not in any way associated with guest config, but instead used to configure the host's physical network interfaces, bridges, bonds, and vlan interfaces; and nearly as important, is not supported on Ubuntu anyway).
Actual guest interface config is *much* simpler than that.
But after define, the dumpxml only show this part: <interface type='bridge'> <mac address='52:54:00:3c:92:9a'/> <source bridge='br0'/> </interface>
That's because everything else you've put in your guest config is not actually a part of a valid guest interface config, and so is ignored.
I would appreciate any help to solve the question. The problem I've is I lost the access to the guest every time the libvirt change the name of the interface eth1,2,3, etc.
In this case I think it may be useful to start over. You've somehow been tricked into misunderstanding the way that network configuration in libvirt works, confusing host interface config with guest interface config, and the resulting config bears little resemblance to what would actually work.
== The first step is to read this page:
http://wiki.libvirt.org/page/Networking
to get a basic overview.
== Second step: you should decide which of the three methods of connecting to guests to the physical network you want to use:
1) NATed 2) routed 3) bridged
The simplest is (1), and that will probably work just fine as long as 1) you don't need incoming connections to the guests, and 2) you're not concerned about getting the last fraction of an ounce of performance out of the network.
Depending on which of those 3 you decide to use:
== Third step: follow the instructions to either create a bridge interface on the host connected to a physical network interface (for (3)), or to create a libvirt virtual network using one of the two modes ((1) or (2) - actually the stock libvirt config already has a "default" network that is probably adequate for (1), and unless you understand IP routing very well, you'll likely want to avoid (2)).
Once the physical network is setup:
== Fourth step: define your guests. In each guest, you'll have an <interface> element, but it will be much simpler than what you've previously tried. For bridged mode, the interface section would look like this:
<interface type='bridge'> <source bridge='br0'/> </interface>
Depending on the guest, you may want to specify what type of hardware is presented to the guest (default is rtl, which almost always works, but is not the best perfrorming), in particular, if the guest supports virtio-net, you'll want to add this into the<interface> element:
<model type='virtio'/>
For NATed or routed mode, the guest's interface definition would look like this:
<interface type='network'> <source network='default'/> </interface>
(replace "default" with the name of the network you've defined, if you've defined your own). In this case also, you can add a<model type='...'/> if appropriate.
Note that I don't put anything in the<interface> element about MAC address, target device, or bus address. Those are all setup automatically by libvirt the first time you start the guest, and it will make sure that the guests don't interfere with each other.
== Fifth step: only if you need a particular guest to have a particular fixed IP address, either 1) edit the network config file on the guest, 2) add a static host to the libvirt virtual network definition (giving the MAC address provided to the guest by libvirt during the initial guest startup) with a<host> element added to the<dhcp> element of the network; details are here:
http://www.libvirt.org/formatnetwork.html
or 3) if you're using the bridged method to connect to the physical network, again either configure the guest's own config file, or add a static host definition to the dhcp server running on your physical network.
If you still have problems don't hesitate to send another mail (although, as I said at the top, sending to libvirt-users@redhat.com may yield better results than libvir-list@redhat.com).
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- ---------------------------------------- Tanguy ROZIER tanguy@rozier.eu +33 6 07 26 46 68 ----------------------------------------
On 02/10/2011 09:51 AM, Tanguy ROZIER wrote:
OK so he should fix the problem with libvirt to associate a fixed MAC to the interface of the VM :
<interface type='bridge'> <source bridge='br0'/> <mac address="00:11:22:33:44:55"/> <----------this is here : you must *define* the MAC address of your interface </interface>
As I said in my last mail, if you do not manually define a MAC address for an interface, libvirt will automatically generate a random mac address, and save it into the domain config for you, so that the 2nd and subsequent times you start the guest, it will use the same mac address. So, there is no requirement that you manually define the MAC address. For example, I just edited an existing domain config with "virsh edit domainname" and added this: <interface type='network'> <source network='default'/> <model type='virtio'/> </interface> Then I immediately ran "virsh dumpxml domainname", and the interface config had been magically transformed: <interface type='network'> <mac address='52:54:00:e2:75:a4'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> This mac address (and pci addressing information) is stored in the persistent config for the guest, so they will be the same every time the guest is started. If this is not happening, it's due to 1 of 3 reasons: 1) You are using a *very* old libvirt. I'm fairly certain, that it has auto-generated mac addresses at least since I became involved with libvirt 2 years ago (and possibly it's always been that way). 2) The guests are being "virsh create"ed as transient guests, so that the auto-generated mac address is never saved. "virsh create" is the wrong tool for this job, a combination of "virsh define" followed by "virsh start" would be more appropriate. 3) there is a bug somewhere.
2011/2/10 Laine Stump<laine@laine.org>:
Your problem is related to udev : every reboot of VM, udev in VM checks the MAC address of the ethernet interface of the system to associate it with a static name (eth0, eth1...). If you change the MAC address of the Ethernet interface of your VM, it will detect a new interface and associate it with a new name (ethx+1). In order to avoid it : try not to change the MAC address of your VM ethernet device (you can fix it with libvirt xml), When a domain is defined with an interface, if no mac address is supplied,
On 02/10/2011 08:50 AM, Tanguy ROZIER wrote: libvirt will automatically generate one and write that generated mac back to the config, so that the next time the guest boots it will be presented with an ethernet having the same mac address as the previous time.
How are you defining your guests? Are you perhaps using "virsh create" (but using an existing image file) to create a transient guest each time you want to boot the guest, rather than doing "virsh define" once to create a persistent guest, then using "virsh start" each time you want to boot it? You should be doing the latter - if you continuously use "virsh create", libvirt will think that it's a "throw away" guest, and won't save the config (or any changes to it), so the next time you do "virsh create", the disk image will have memory of the previous run (by having "eth0" assigned to some particular mac address), but libvirt won't have any memory of it (it thinks you're creating a brand new guest), so it will generate a new mac address for the interface, the guest will boot, see a "new" interface, notice that "eth0" is already taken, so it will use "eth1", and so on.
If you actually are using "virsh define" to define a persistent guest, do a "virsh dumpxml" of the guest after you've started and destroyed it, and see if there is a mac address in the interface config. If not, that's a bug.
or deactivate this mecanism in the VM : search in /etc/udev/rules.d/70-persistent-net.rules : it calls /lib/udev/write_net_rules at every reboot, you could chmod -x this script in order to avoid it. This may work (or may not - I haven't investigated), but it isn't the right way to fix this problem. It's dealing with the symptom rather than the cause.
Tanguy
2011/2/10 Marcela Castro León<mcastrol@gmail.com>:
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem.
a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1
b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config
# The loopback network interface auto lo iface lo inet loopback
auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0
c) this is the xml guest configuration related to the network.
<interface type='bridge'> <source bridge='br0'/> </interface>
d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..). I don't know how to achieve the guest don't behave like this and create always its network interface with a fixed name, for example, eth0).
Why I ask that?
if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either.
The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface.
I hope I've explained better the problem I have.
Thank you very much, you've already helped too much to better understand the different cases of network configuration
Regards.
2011/2/10 Marcela Castro León<mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump<laine@laine.org>
(For future questions such as this, you may find you get a better response (and lower signal-noise ratio) by asking on the recently -created libvirt-users@redhat.com list, as libvir-list has a very large volume of patches and discussion about the internals of libvirt)
On 02/08/2011 12:05 PM, Marcela Castro León wrote:
Hello I've a lan of virtual machine configured with bridge. I've a private lan of 4 machine, but one of them have 2 lan-card with access to internet.
Evertything works almost fine, except that I've configured on each guest the static configuration of the bridge with the ip address, like this:
auto lo iface lo inet loopback
auto eth6 iface eth6 inet static address 192.168.1.41 network 192.168.1.0 netmask 255.255.255.0 gateway 192.168.1.10
So are you saying that you're attempting to give each guest the *same* IP address? There's no way that can possibly work, even if each is connected to a separate virtual network or bridge. Since the host's IP stack can "see" all of the guests' network traffic, regardless of the connection mode you choose, each must have a unique IP address even if they can't see each other.
But each time I, for example, define the guest, it start with a ascending number of interface: eth6,eth7, eth8, etc, Now is initializing with eth12.
There must be a mixup in the explanation - I'm understanding that you put the same config file in each guest, but each guest, when it starts, shows a different ethernet interface in its ifconfig. That doesn't seem right, because none of the guests know about each other. Or are you saying something different? Where are you seeing the "eth6", "eth7", ... "eth12"? In the guest XML config maybe?
So, it doesn't get the manual configuration I've done on the /etc/network/interfaces. How can avoid this way of work of the number of ethernet interface the guest start?
Why are you doing a manual/static definition on the guest at all? It's simpler to just leave the guest to automatically configure its eth0, and if you really want a particular IP address for a particular guest, handle that with a<host> element in the<dhcp> section of the virtual network the guest is connected to (or, if you're using a pure bridging setup, then in the static hosts config of the dhcp server on that network).
A way i'm trying to solve is configuring the interface trough the guest xml definition,
The guest xml interface configuration cannot be used to setup the IP address, route, etc config of the guest. It is only used for two things:
1) configuring what kind of hardware appears on the guest (but not what name the guest gives to it!)
2) configuring how that hardware is connected to the physical network (usually either via an existing bridge interface on the host (in order for the guest to appear as if its directly connected to the physical network), or via a "virtual network" created by libvirt (which is really just a bridge that's not directly connected to any physical interface, relying instead on the host's IP stack to route packets from the guest out to the physical network.)
but the operation "define" on virsh accepted without error the xml i've done, but the dumpxml doesn't show them, so, it doesn't work at all. I've followed the instruction that said the manual "Application development guide - a guide to application development with libvirt" on page 71.
That document is targeted towards software developers writing applications to manage virtual guests using the libvirt API, not for system administrators who are configuring virtual guests manually (or, more likely, by using software that uses the libvirt API, eg virsh or virt-manager). It likely contains more detail and less overview than what you need, which is probably what led to the confusion. I've included a pointer to a wiki page below that is more the appropriate level of information (Justin, if the docs you were working on are somewhere easily accessible, you can point to those instead.)
The whole definition of the VM is attached, but the related to interface is: <interface type='bridge'> <source bridge='br0'/> <target dev='vnet0'/> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> <bridge stp="off" delay="0.01"> <interface type="ethernet" name="eth12"> <protocol> <ip address="192.168.1.41" prefix="24"/> <route gateway="192.168.1.10"/> </protocol> </interface> </bridge> </interface>
You are confusing the guest interface config (that is part of the guest's XML config, and described above), and host physical interface config (that is not in any way associated with guest config, but instead used to configure the host's physical network interfaces, bridges, bonds, and vlan interfaces; and nearly as important, is not supported on Ubuntu anyway).
Actual guest interface config is *much* simpler than that.
But after define, the dumpxml only show this part: <interface type='bridge'> <mac address='52:54:00:3c:92:9a'/> <source bridge='br0'/> </interface>
That's because everything else you've put in your guest config is not actually a part of a valid guest interface config, and so is ignored.
I would appreciate any help to solve the question. The problem I've is I lost the access to the guest every time the libvirt change the name of the interface eth1,2,3, etc.
In this case I think it may be useful to start over. You've somehow been tricked into misunderstanding the way that network configuration in libvirt works, confusing host interface config with guest interface config, and the resulting config bears little resemblance to what would actually work.
== The first step is to read this page:
http://wiki.libvirt.org/page/Networking
to get a basic overview.
== Second step: you should decide which of the three methods of connecting to guests to the physical network you want to use:
1) NATed 2) routed 3) bridged
The simplest is (1), and that will probably work just fine as long as 1) you don't need incoming connections to the guests, and 2) you're not concerned about getting the last fraction of an ounce of performance out of the network.
Depending on which of those 3 you decide to use:
== Third step: follow the instructions to either create a bridge interface on the host connected to a physical network interface (for (3)), or to create a libvirt virtual network using one of the two modes ((1) or (2) - actually the stock libvirt config already has a "default" network that is probably adequate for (1), and unless you understand IP routing very well, you'll likely want to avoid (2)).
Once the physical network is setup:
== Fourth step: define your guests. In each guest, you'll have an <interface> element, but it will be much simpler than what you've previously tried. For bridged mode, the interface section would look like this:
<interface type='bridge'> <source bridge='br0'/> </interface>
Depending on the guest, you may want to specify what type of hardware is presented to the guest (default is rtl, which almost always works, but is not the best perfrorming), in particular, if the guest supports virtio-net, you'll want to add this into the<interface> element:
<model type='virtio'/>
For NATed or routed mode, the guest's interface definition would look like this:
<interface type='network'> <source network='default'/> </interface>
(replace "default" with the name of the network you've defined, if you've defined your own). In this case also, you can add a<model type='...'/> if appropriate.
Note that I don't put anything in the<interface> element about MAC address, target device, or bus address. Those are all setup automatically by libvirt the first time you start the guest, and it will make sure that the guests don't interfere with each other.
== Fifth step: only if you need a particular guest to have a particular fixed IP address, either 1) edit the network config file on the guest, 2) add a static host to the libvirt virtual network definition (giving the MAC address provided to the guest by libvirt during the initial guest startup) with a<host> element added to the<dhcp> element of the network; details are here:
http://www.libvirt.org/formatnetwork.html
or 3) if you're using the bridged method to connect to the physical network, again either configure the guest's own config file, or add a static host definition to the dhcp server running on your physical network.
If you still have problems don't hesitate to send another mail (although, as I said at the top, sending to libvirt-users@redhat.com may yield better results than libvir-list@redhat.com).
libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Hello It's working now. I've put the mac address in the xml definition. Thank you very much, Regards. 2011/2/10 Laine Stump <laine@laine.org>
On 02/10/2011 09:51 AM, Tanguy ROZIER wrote:
OK so he should fix the problem with libvirt to associate a fixed MAC to the interface of the VM :
<interface type='bridge'> <source bridge='br0'/> <mac address="00:11:22:33:44:55"/> <----------this is here : you must *define* the MAC address of your interface </interface>
As I said in my last mail, if you do not manually define a MAC address for an interface, libvirt will automatically generate a random mac address, and save it into the domain config for you, so that the 2nd and subsequent times you start the guest, it will use the same mac address. So, there is no requirement that you manually define the MAC address.
For example, I just edited an existing domain config with "virsh edit domainname" and added this:
<interface type='network'> <source network='default'/> <model type='virtio'/> </interface>
Then I immediately ran "virsh dumpxml domainname", and the interface config had been magically transformed:
<interface type='network'> <mac address='52:54:00:e2:75:a4'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface>
This mac address (and pci addressing information) is stored in the persistent config for the guest, so they will be the same every time the guest is started.
If this is not happening, it's due to 1 of 3 reasons:
1) You are using a *very* old libvirt. I'm fairly certain, that it has auto-generated mac addresses at least since I became involved with libvirt 2 years ago (and possibly it's always been that way).
2) The guests are being "virsh create"ed as transient guests, so that the auto-generated mac address is never saved. "virsh create" is the wrong tool for this job, a combination of "virsh define" followed by "virsh start" would be more appropriate.
3) there is a bug somewhere.
2011/2/10 Laine Stump<laine@laine.org>:
On 02/10/2011 08:50 AM, Tanguy ROZIER wrote:
Your problem is related to udev : every reboot of VM, udev in VM checks the MAC address of the ethernet interface of the system to associate it with a static name (eth0, eth1...). If you change the MAC address of the Ethernet interface of your VM, it will detect a new interface and associate it with a new name (ethx+1). In order to avoid it : try not to change the MAC address of your VM ethernet device (you can fix it with libvirt xml),
When a domain is defined with an interface, if no mac address is supplied, libvirt will automatically generate one and write that generated mac back to the config, so that the next time the guest boots it will be presented with an ethernet having the same mac address as the previous time.
How are you defining your guests? Are you perhaps using "virsh create" (but using an existing image file) to create a transient guest each time you want to boot the guest, rather than doing "virsh define" once to create a persistent guest, then using "virsh start" each time you want to boot it? You should be doing the latter - if you continuously use "virsh create", libvirt will think that it's a "throw away" guest, and won't save the config (or any changes to it), so the next time you do "virsh create", the disk image will have memory of the previous run (by having "eth0" assigned to some particular mac address), but libvirt won't have any memory of it (it thinks you're creating a brand new guest), so it will generate a new mac address for the interface, the guest will boot, see a "new" interface, notice that "eth0" is already taken, so it will use "eth1", and so on.
If you actually are using "virsh define" to define a persistent guest, do a "virsh dumpxml" of the guest after you've started and destroyed it, and see if there is a mac address in the interface config. If not, that's a bug.
or deactivate this mecanism in the VM : search in
/etc/udev/rules.d/70-persistent-net.rules : it calls /lib/udev/write_net_rules at every reboot, you could chmod -x this script in order to avoid it.
This may work (or may not - I haven't investigated), but it isn't the right way to fix this problem. It's dealing with the symptom rather than the cause.
Tanguy
2011/2/10 Marcela Castro León<mcastrol@gmail.com>:
Hello Laine: I've revised the configuration and followed all your instructions. I'll try to give you a better explain of my problem.
a) I need a bridged (3) configuration, because I have to connect the guests from each others. (via ssh and execute mpi). I've three hosts, each of one has at least a guest. I'll only explain about the configuration of one of them, because the problem is only related a one configuration. hostname=rionegro ip=192.168.1.4 guest-name=rnompi1
b) I've configured host bridge following step by step ther instruction on http://wiki.libvirt.org/page/Networking this is the /etc/network/config
# The loopback network interface auto lo iface lo inet loopback
auto br0 iface br0 inet static address 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.10 bridge_ports eth0 bridge_stp on bridge_fd 0 bridge_maxwait 0
c) this is the xml guest configuration related to the network.
<interface type='bridge'> <source bridge='br0'/> </interface>
d) But the guest doesn't get any IP automatically, and create an interface with a variable name (eth4, eth5,etch6..). I don't know how to achieve the guest don't behave like this and create always its network interface with a fixed name, for example, eth0).
Why I ask that?
if the guest created an interface with a fixed name, I would configure and static ip for the it. I'm attaching a printscreen "rnompi1.jpg" that show what ifconfig ahow on the guest (eth14 and without any IP). It doesn't get their host either.
The only way I achieve to solve but "temporally" until reboot is configuring the guest statically using the name of the interface that has just up.In that case every works fine, and the guest can connect to all the other guests on the lan and others can connect to it. But this only works until reboot or reconfigure the guest, because each time the name of interface change, and my configuration I had done of /etc/network/interface didn't work any more because is related to another interface.
I hope I've explained better the problem I have.
Thank you very much, you've already helped too much to better understand the different cases of network configuration
Regards.
2011/2/10 Marcela Castro León<mcastrol@gmail.com>
Hello Laine: Thank you very much. I've showed only one of the guest, each one have it's own ip address. Anyway, I'll follow your advise of revise all the configuration after reading the doc. Thank you very much again. Regards.
2011/2/8 Laine Stump<laine@laine.org>
> (For future questions such as this, you may find you get a better > response (and lower signal-noise ratio) by asking on the recently > -created > libvirt-users@redhat.com list, as libvir-list has a very large > volume > of > patches and discussion about the internals of libvirt) > > On 02/08/2011 12:05 PM, Marcela Castro León wrote: > > Hello > I've a lan of virtual machine configured with bridge. I've a private > lan > of 4 machine, but one of them have 2 lan-card with access to > internet. > > Evertything works almost fine, except that I've configured on each > guest > the static configuration of the bridge with the ip address, like > this: > > auto lo > iface lo inet loopback > > auto eth6 > iface eth6 inet static > address 192.168.1.41 > network 192.168.1.0 > netmask 255.255.255.0 > gateway 192.168.1.10 > > > So are you saying that you're attempting to give each guest the > *same* > IP > address? There's no way that can possibly work, even if each is > connected to > a separate virtual network or bridge. Since the host's IP stack can > "see" > all of the guests' network traffic, regardless of the connection mode > you > choose, each must have a unique IP address even if they can't see > each > other. > > > > But each time I, for example, define the guest, it start with a > ascending > number of interface: eth6,eth7, eth8, etc, Now is initializing with > eth12. > > There must be a mixup in the explanation - I'm understanding that you > put > the same config file in each guest, but each guest, when it starts, > shows a > different ethernet interface in its ifconfig. That doesn't seem > right, > because none of the guests know about each other. Or are you saying > something different? Where are you seeing the "eth6", "eth7", ... > "eth12"? > In the guest XML config maybe? > > So, it doesn't get the manual configuration I've done on the > /etc/network/interfaces. > How can avoid this way of work of the number of ethernet interface > the > guest start? > > Why are you doing a manual/static definition on the guest at all? > It's > simpler to just leave the guest to automatically configure its eth0, > and if > you really want a particular IP address for a particular guest, > handle > that > with a<host> element in the<dhcp> section of the virtual > network the > guest > is connected to (or, if you're using a pure bridging setup, then in > the > static hosts config of the dhcp server on that network). > > > A way i'm trying to solve is configuring the interface trough the > guest > xml definition, > > The guest xml interface configuration cannot be used to setup the IP > address, route, etc config of the guest. It is only used for two > things: > > 1) configuring what kind of hardware appears on the guest (but not > what > name the guest gives to it!) > > 2) configuring how that hardware is connected to the physical network > (usually either via an existing bridge interface on the host (in > order > for > the guest to appear as if its directly connected to the physical > network), > or via a "virtual network" created by libvirt (which is really just a > bridge > that's not directly connected to any physical interface, relying > instead on > the host's IP stack to route packets from the guest out to the > physical > network.) > > but the operation "define" on virsh accepted without error the xml > i've > done, but the dumpxml doesn't show them, so, it doesn't work at all. > I've followed the instruction that said the manual "Application > development guide - a guide to application development with libvirt" > on > page > 71. > > > That document is targeted towards software developers writing > applications to manage virtual guests using the libvirt API, not for > system > administrators who are configuring virtual guests manually (or, more > likely, > by using software that uses the libvirt API, eg virsh or > virt-manager). > It > likely contains more detail and less overview than what you need, > which > is > probably what led to the confusion. I've included a pointer to a wiki > page > below that is more the appropriate level of information (Justin, if > the > docs > you were working on are somewhere easily accessible, you can point to > those > instead.) > > > The whole definition of the VM is attached, but the related to > interface > is: > <interface type='bridge'> > <source bridge='br0'/> > <target dev='vnet0'/> > <protocol> > <ip address="192.168.1.41" prefix="24"/> > <route gateway="192.168.1.10"/> > </protocol> > <bridge stp="off" delay="0.01"> > <interface type="ethernet" name="eth12"> > <protocol> > <ip address="192.168.1.41" prefix="24"/> > <route gateway="192.168.1.10"/> > </protocol> > </interface> > </bridge> > </interface> > > > You are confusing the guest interface config (that is part of the > guest's > XML config, and described above), and host physical interface config > (that > is not in any way associated with guest config, but instead used to > configure the host's physical network interfaces, bridges, bonds, and > vlan > interfaces; and nearly as important, is not supported on Ubuntu > anyway). > > Actual guest interface config is *much* simpler than that. > > > > But after define, the dumpxml only show this part: > <interface type='bridge'> > <mac address='52:54:00:3c:92:9a'/> > <source bridge='br0'/> > </interface> > > > That's because everything else you've put in your guest config is not > actually a part of a valid guest interface config, and so is ignored. > > > > I would appreciate any help to solve the question. The problem I've > is > I > lost the access to the guest every time the libvirt change the name > of > the > interface eth1,2,3, etc. > > In this case I think it may be useful to start over. You've somehow > been > tricked into misunderstanding the way that network configuration in > libvirt > works, confusing host interface config with guest interface config, > and > the > resulting config bears little resemblance to what would actually > work. > > == > The first step is to read this page: > > http://wiki.libvirt.org/page/Networking > > to get a basic overview. > > == > Second step: you should decide which of the three methods of > connecting > to guests to the physical network you want to use: > > 1) NATed > 2) routed > 3) bridged > > The simplest is (1), and that will probably work just fine as long as > 1) > you don't need incoming connections to the guests, and 2) you're not > concerned about getting the last fraction of an ounce of performance > out of > the network. > > Depending on which of those 3 you decide to use: > > == > Third step: follow the instructions to either create a bridge > interface > on the host connected to a physical network interface (for (3)), or > to > create a libvirt virtual network using one of the two modes ((1) or > (2) > - > actually the stock libvirt config already has a "default" network > that > is > probably adequate for (1), and unless you understand IP routing very > well, > you'll likely want to avoid (2)). > > Once the physical network is setup: > > == > Fourth step: define your guests. In each guest, you'll have an > <interface> element, but it will be much simpler than what you've > previously > tried. For bridged mode, the interface section would look like this: > > <interface type='bridge'> > <source bridge='br0'/> > </interface> > > Depending on the guest, you may want to specify what type of hardware > is > presented to the guest (default is rtl, which almost always works, > but > is > not the best perfrorming), in particular, if the guest supports > virtio-net, > you'll want to add this into the<interface> element: > > <model type='virtio'/> > > For NATed or routed mode, the guest's interface definition would look > like this: > > <interface type='network'> > <source network='default'/> > </interface> > > (replace "default" with the name of the network you've defined, if > you've > defined your own). In this case also, you can add a<model > type='...'/> > if > appropriate. > > Note that I don't put anything in the<interface> element about MAC > address, target device, or bus address. Those are all setup > automatically by > libvirt the first time you start the guest, and it will make sure > that > the > guests don't interfere with each other. > > == > Fifth step: only if you need a particular guest to have a particular > fixed IP address, either 1) edit the network config file on the > guest, > 2) > add a static host to the libvirt virtual network definition (giving > the > MAC > address provided to the guest by libvirt during the initial guest > startup) > with a<host> element added to the<dhcp> element of the network; > details > are here: > > http://www.libvirt.org/formatnetwork.html > > or 3) if you're using the bridged method to connect to the physical > network, again either configure the guest's own config file, or add a > static > host definition to the dhcp server running on your physical network. > > If you still have problems don't hesitate to send another mail > (although, > as I said at the top, sending to libvirt-users@redhat.com may yield > better > results than libvir-list@redhat.com). > _______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
participants (3)
-
Laine Stump -
Marcela Castro León -
Tanguy ROZIER