[libvirt] Integrating LiSA into libvirt
My name is Andreea and I am a member of LiSA team. LiSA stands for Linux Switching Appliance and offers a software switch that switches packets at Layer 2 and Layer 3. It is an open source project under development, you can find more about it from lisa.mindbit.ro or from https://github.com/lisa-project/. One of the development directions of LiSA is to integrate it into libvirt. First I'll state the motivation for such an integration and then I will describe a use case. The main reasons for extending libvirt network component with LiSA are related to the fact that different hypervisors offer different switching engines with different capabilities. By integrating LiSA into libvirt, the virtual machines would connect to LiSA switch ports as they were linked through a physical switch. Hence, with LiSA, the switching engine would offer the same switching capabilities independent of hypervisor. Moreover, if I understand correctly, in libvirt switching configurations are possible either through a set of commands in virsh or directly within a XML file. One of LiSA components is swcli that is a CLI for configuring and retrieving configurations of the software switch. Through swcli, the switching configurations (including VLANs, trunk or access mode interfaces etc.) are made through a CISCO-IOS like set of commands. I think this is a nice-to-have because it would separate complex network configurations from NIC configuration. That is, through libvirt the machines would just connect to the switch and other switching configurations, simple or complex, would be made from swcli. In other words, the XML file would remain simple and readable while the user would still be able to make switching specific configurations. In conclusion, LiSA and libvirt would make a great team for building a complex networking model, without complicating the setup of a virtual network. Scenario of where LiSA fits into libvirt =========================== I thought at the following scenario: two virtual machines linked together through a LiSA switch, i.e. the LiSA switch being the physical machine running LiSA. We, LiSA team, started by studying the virt-manager interface and figured out that an entry in the "Source device" drop down list when configuring a domain NIC should suffice our goal for the moment. When applying these changes, the XML configuration file should contain an interface tag of type "lisa-switch" or something similar. Therefore, a patch to libvirt for integrating LiSA would contain: - the implementation of two functions - virNetDevLisaAddPort - virNetDevLisaRemovePort - and a new interface type (XML level) for network configuration. I would like to have your feedback on the ideas above. Thanks, Andreea
On Sun, Sep 15, 2013 at 01:05:15PM -0700, Andreea Hodea wrote:
My name is Andreea and I am a member of LiSA team. LiSA stands for Linux Switching Appliance and offers a software switch that switches packets at Layer 2 and Layer 3. It is an open source project under development, you can find more about it from lisa.mindbit.ro or from https://github.com/lisa-project/.
One of the development directions of LiSA is to integrate it into libvirt. First I'll state the motivation for such an integration and then I will describe a use case.
Looking at the website it seems that it requires custom kernel branches, and the most recent kernel branch I see, with patches from last 2 months, is based on the pretty old 2.6.32 release ? The website also indicates the kernel<->userspace API is not ABI stable. I must say I don't know all that much about networking. My main uninformed question would be how does this LiSA kernel support differ from/relate to OpenVSwitch which seems to be the technology the kernel community has decided upon for next generation network capabilities in Linux. I'm rather loathe consider support for new Linux networking features based on out-of-tree kernel patches. Reards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, 2013-09-16 at 11:02 +0100, Daniel P. Berrange wrote:
Looking at the website it seems that it requires custom kernel branches, and the most recent kernel branch I see, with patches from last 2 months, is based on the pretty old 2.6.32 release ? The website also indicates the kernel<->userspace API is not ABI stable.
Unfortunately, our website is not up-to-date with the latest development effort that has been put into the source code. We've recently changed LiSA to be switching engine agnostic. Now it can work with the "bridge" and "8021q" modules, which are part of the stock linux kernel. Our custom kernel module is "yet another engine" that LiSA supports, but it's not a requirement.
I must say I don't know all that much about networking. My main uninformed question would be how does this LiSA kernel support differ from/relate to OpenVSwitch which seems to be the technology the kernel community has decided upon for next generation network capabilities in Linux. I'm rather loathe consider support for new Linux networking features based on out-of-tree kernel patches.
I completely understand your point, but maybe you'll reconsider now that you know an out-of-tree patch is not actually required. As for our out-of-tree module, we actually tried to push it upstream a couple of years ago, but the kernel networking team said that it didn't make sense to add a new module with similar functionality to already existing modules and that we should have fixed those instead (if something had been wrong with them). That was even before OpenVSwitch was released as open source. I'm actually disappointed about this. LiSA was out in 2005 - that's 4 years before OpenVSwitch, but nobody cared. Regards, Radu
On 09/16/2013 06:32 AM, Radu Rendec wrote:
On Mon, 2013-09-16 at 11:02 +0100, Daniel P. Berrange wrote:
Looking at the website it seems that it requires custom kernel branches, and the most recent kernel branch I see, with patches from last 2 months, is based on the pretty old 2.6.32 release ? The website also indicates the kernel<->userspace API is not ABI stable. Unfortunately, our website is not up-to-date with the latest development effort that has been put into the source code.
We've recently changed LiSA to be switching engine agnostic. Now it can work with the "bridge" and "8021q" modules, which are part of the stock linux kernel.
1) What are the advantages of using LiSA vs. using the standard Linux host bridge without LiSA? vs. openvswitch? 2) Since LiSA can use the standard Linux host bridge, is it possible to use the LiSA userspace utilities on a bridge that was created in the traditional manner (i.e. one of the bridge's that libvirt creates today)? This may actually provide all the functionality that you're looking for. (I can imagine though that, for example, if LiSA is able to support per-port vlan configuration, it might be nice to be able to translate that information from the interface (or network or portgroup) config into the proper LiSA command/ioctl/whatever to setup the vlan tag for a port.) 3) Since LiSA is apparently using standard tap devices for all its connections, if libvirt did add any sort of support for it, I think it should be in a form parallel to the openvswitch/802.1Qb[gh] support, which continues to use <interface type='bridge'> (or <interface type='network'>) with switching technology-specific details in the <virtualport type='<openvswitch|8021Qbg|8021Qbh)'> sub-element of the <interface> or <network>.
Our custom kernel module is "yet another engine" that LiSA supports, but it's not a requirement.
I must say I don't know all that much about networking. My main uninformed question would be how does this LiSA kernel support differ from/relate to OpenVSwitch which seems to be the technology the kernel community has decided upon for next generation network capabilities in Linux. I'm rather loathe consider support for new Linux networking features based on out-of-tree kernel patches. I completely understand your point, but maybe you'll reconsider now that you know an out-of-tree patch is not actually required.
Purely from the point of view of "let's see if we've abstracted the networking well enough to cleanly support a new technology" this would be interesting. But I am in agreement with Dan that we need to be sure that it is something that is "standards track", has good momentum, has a stable ABI, and offers a definite gain in functionality and/or user-base prior to committing to supporting it in libvirt - once something goes into the libvirt API, we guarantee that it will continue to be supported effectively "forever", so we can't make the decision too lightly.
As for our out-of-tree module, we actually tried to push it upstream a couple of years ago, but the kernel networking team said that it didn't make sense to add a new module with similar functionality to already existing modules and that we should have fixed those instead (if something had been wrong with them).
That was even before OpenVSwitch was released as open source. I'm actually disappointed about this. LiSA was out in 2005 - that's 4 years before OpenVSwitch, but nobody cared.
Yes, unfortunately marketing is very often just as important as (or even more important than) the quality of the code.
On Wed, 2013-09-18 at 09:25 -0400, Laine Stump wrote:
On 09/16/2013 06:32 AM, Radu Rendec wrote:
We've recently changed LiSA to be switching engine agnostic. Now it can work with the "bridge" and "8021q" modules, which are part of the stock linux kernel.
1) What are the advantages of using LiSA vs. using the standard Linux host bridge without LiSA? vs. openvswitch?
The main advantage is the LiSA CLI, which provides a unified environment for managing the virtual switch, very similar to a real (hardware) switch. It may also be very useful in "no-such-usual" scenarios, for instance when you want to configure a (8021q) tagged port for a VM NIC or configure link aggregation for 2 VM NICs - I can explain this more thoroughly, if needed. Since LiSA is now a switching engine abstraction, it may provide support in libvirt for other switching technologies, without having to integrate each of them into libvirt directly. Obviously there's no advantage in the switching process itself when the bridge module is used.
2) Since LiSA can use the standard Linux host bridge, is it possible to use the LiSA userspace utilities on a bridge that was created in the traditional manner (i.e. one of the bridge's that libvirt creates today)? This may actually provide all the functionality that you're looking for.
Yes, theoretically it can. But (when used with bridge+8021q) LiSA creates its own bridges and automatically sets them up as needed.
(I can imagine though that, for example, if LiSA is able to support per-port vlan configuration, it might be nice to be able to translate that information from the interface (or network or portgroup) config into the proper LiSA command/ioctl/whatever to setup the vlan tag for a port.)
Perfect example! If you want to setup a port in tagged mode, and provide access for vlans 2 and 3, for instance, all you have to do with LiSA is: switchport mode trunk switchport trunk allowed vlans 2,3 Assuming that the corresponding VM port is seen as tap0 in the host machine, LiSA would automatically create tap0.2 and tap0.3 (8021q devices) and add them to the appropriate bridges.
3) Since LiSA is apparently using standard tap devices for all its connections, if libvirt did add any sort of support for it, I think it should be in a form parallel to the openvswitch/802.1Qb[gh] support, which continues to use <interface type='bridge'> (or <interface type='network'>) with switching technology-specific details in the <virtualport type='<openvswitch|8021Qbg|8021Qbh)'> sub-element of the <interface> or <network>.
Honestly, I don't think I know enough about libvirt internals to have an opinion about this. However, if we get to the point where we implement any support for LiSA in libvirt, we'll discuss any technical details on the list before actually implementing anything.
I completely understand your point, but maybe you'll reconsider now that you know an out-of-tree patch is not actually required.
Purely from the point of view of "let's see if we've abstracted the networking well enough to cleanly support a new technology" this would be interesting. But I am in agreement with Dan that we need to be sure that it is something that is "standards track", has good momentum, has a stable ABI, and offers a definite gain in functionality and/or user-base prior to committing to supporting it in libvirt - once something goes into the libvirt API, we guarantee that it will continue to be supported effectively "forever", so we can't make the decision too lightly.
This is like a chicken and egg problem. LiSA (or any project, after all) cannot have a large user base unless it's known to the public and people can easily experiment with it. I cannot go out and say "hey, here's this new thing you can play with, but you need to apply this out-of-tree patch, recompile libvirt, mess up your whole environment and spend 3 days wondering why it doesn't work out of the box". Nobody would even try that. On the other hand, I understand your point about not rushing into integrating new technology. I can guarantee a stable ABI and long-term support, but I don't expect you (or anybody else) to take this for granted. Perhaps a good approach is to adopt LiSA as "experimental technology" and distribute it accordingly - for instance in "playground" distros like Fedora. I'm really not sure what can be done (or not), so some good advice in this direction would be highly appreciated.
As for our out-of-tree module, we actually tried to push it upstream a couple of years ago, but the kernel networking team said that it didn't make sense to add a new module with similar functionality to already existing modules and that we should have fixed those instead (if something had been wrong with them).
That was even before OpenVSwitch was released as open source. I'm actually disappointed about this. LiSA was out in 2005 - that's 4 years before OpenVSwitch, but nobody cared.
Yes, unfortunately marketing is very often just as important as (or even more important than) the quality of the code.
I agree. After all, size does matter :) Meaning that OpenVSwitch was (is?) developped/supported by Citrix, and the LiSA team is just a small team of programmers.
participants (4)
-
Andreea Hodea -
Daniel P. Berrange -
Laine Stump -
Radu Rendec