[libvirt] Libvirt and IPSec (was: What about Trusted Virtual Domains???)
Hi to everyone! First of all, sorry for the thread subject change. Due to the several issues of the Libvirt implementation of the Trusted Virtual Domains (TVD), I decided to approach the topic in a modular manner. I think that the first step should be to define the IPSec support or, more in general, the secure tunnel support for Libvirt. I see the implementation divided in two step: 1. define a new driver called 'sectunnel' which describes a generic secure tunnel that will be established using several technologies (for now using only ipsec) 2. modify the existing 'network' driver by adding the possibility to specify the 'sectunnel' that the network have to use in the virtual network definition As an example, you can see below a possible XML definition of the network which use a secure tunnel and the corresponding 'sectunnel' XML definition: NETWORK DEFINITION ================== <network> <name>sec-net</name> <uuid>3e3fce45-4f53-4fa7-bb32-11f34168b82b</uuid> <bridge name='virbr0' /> <domain name='example' /> ... <sectunnel name='sec-tun' /> <--(specify the 'sectunnel' to use) </network> SECTUNNEL DEFINITION ==================== <sectunnel type='ipsec'> <name>sec-tun</name> <uuid>8b7fd1b0-4463-43b7-8b6e-8006344aeb66</uuid> <!-- Security Association definitions --> <sa> <secret uuid='...' /> <--(specify the 'secret' which contains the pre-shared key) </sa> <!-- Security Policy definitions --> <sp> <src_range address='10.0.0.1' prefixlen='30' port='5000' /> <dst_range address='10.0.0.2' prefixlen='30' port='5000' /> <upperspec protocol='any' /> <policy direction='out' action='ipsec'> <rule protocol='esp' mode='tunnel' level='require'> <src address='192.168.0.1' port='55055' /> <dst address='192.168.0.2' port='55055' /> </rule> </policy> </sp> <sp> <src_range address='10.0.0.2' prefixlen='30' port='5000' /> <dst_range address='10.0.0.1' prefixlen='30' port='5000' /> <upperspec protocol='any' /> <policy direction='in' action='ipsec'> <rule protocol='esp' mode='tunnel' level='require'> <src address='192.168.0.2' port='55055' /> <dst address='192.168.0.1' port='55055' /> </rule> </policy> </sp> </sectunnel> As you can see in the 'sectunnel' XML definition, I use a 'secret' element. This element is a Libvirt secret [1] and it stores the pre-shared key used by IPSec to establish the Security Associations (SA). Obviously this feature requires to define a new usage category in the 'secret' driver definition. Another possible way to establish the SA is to use the X.509 certificates. To this purpose, I think that the certificates already used by Libvirt to setup SSL/TLS remote connections, might be used. That's all! :-) What do you think about this possible IPSec implementation? Thanks in advance for the replies! Best regards, PAOLO LINK LIST --------- [1] http://libvirt.org/formatsecret.html -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Mobile: +39 (333) 527 3593 Email: paolo.smiraglia@polito.it
On 04/06/2011 03:10 PM, Paolo Smiraglia wrote:
Hi to everyone! First of all, sorry for the thread subject change.
Due to the several issues of the Libvirt implementation of the Trusted Virtual Domains (TVD), I decided to approach the topic in a modular manner.
Hi Paolo, so basically this is about IPSec driver implementation to the libvirt ? I don't think the idea is bad however I'm not working on libvirt too much so you should ask libvirt guys about they opinion. Since I think implementing this may be a good idea I'm putting DV to the CC list for his opinion on this matter. Daniel, if you are not the right person to ask, please tell me however I see Paolo's thread is not having any reply and I think the idea is not bad so I'd like to ask about your opinion on this matter. Thanks, Michal -- Michal Novotny <minovotn@redhat.com>, RHCE Virtualization Team (xen userspace), Red Hat
Hi Michael! Thanks for the reply. Comments are inline.
Hi Paolo, so basically this is about IPSec driver implementation to the libvirt ?
Exactly...
I don't think the idea is bad however I'm not working on libvirt too much so you should ask libvirt guys about they opinion.
Since I think implementing this may be a good idea I'm putting DV to the CC list for his opinion on this matter.
Thanks for the "support of the cause". ;-) See you, Paolo -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Polytechnic University of Turin Email: paolo.smiraglia@polito.it
On 04/12/2011 01:23 PM, Paolo Smiraglia wrote:
Hi Michael! Thanks for the reply. Comments are inline.
Hi Paolo, so basically this is about IPSec driver implementation to the libvirt ? Exactly...
Ok, right. I don't know much about IPSec itself so some study on the matter would be necessary to provide some better understanding however from what I know the implementation to the Linux kernel directly is possible for the software stack however implementing this in the virtual hardware stack (emulation) is certainly a good thing since you don't need to patch the kernel at all. The only thing I'm not sure about is the technical point of view since I don't know how hard would the implementation be.
I don't think the idea is bad however I'm not working on libvirt too much so you should ask libvirt guys about they opinion.
Since I think implementing this may be a good idea I'm putting DV to the CC list for his opinion on this matter. Thanks for the "support of the cause". ;-)
No problem since I think the idea is good and therefore I support the idea at least :) Have a great time, Michal -- Michal Novotny <minovotn@redhat.com>, RHCE Virtualization Team (xen userspace), Red Hat
On 04/06/2011 09:10 AM, Paolo Smiraglia wrote:
Hi to everyone! First of all, sorry for the thread subject change.
Due to the several issues of the Libvirt implementation of the Trusted Virtual Domains (TVD), I decided to approach the topic in a modular manner.
I think that the first step should be to define the IPSec support or, more in general, the secure tunnel support for Libvirt. I see the implementation divided in two step:
1. define a new driver called 'sectunnel' which describes a generic secure tunnel that will be established using several technologies (for now using only ipsec)
Would this tunnel be necessarily brought up / torn down by libvirt, or could it be (maybe preferable? I don't have an opinion, just asking) that it be part of the host config, populating files in /etc. If the latter, maybe the right thing to do would be to enhance the XML of virInterfaceDefine (as described by netcf) to also configure ipsec tunnels (and openvpn tunnels, and whatever other tunnels) (I mention this both as a possible way to avoid the need for a new libvirt API, as well as to enhance the functionality of virInterface*() and netcf) Currently all traffic from the virtual networks is simply sent to the host's IP stack for forwarding. Are the decisions about which IPSec association to use for traffic based solely on routing, and is there necessarily a single routing domain on a host? Or can the decision be based on ipfilter rules and/or multiple routing domains on a single host (we had experimented with both of these at a former employer, but I don't recall which parts of it were local modifications and which were part of the stock kernel). If all decisions are based on routing, and there is a single routing domain, then the tunnel will necessarily be system-wide on the host, and may just as well be configured as any other interface (the only difference being that interfaces configured with virInterfaceDefine() persist even if libvirt is stopped and never started again, while something defined with virNetworkDefine (and probably a virSecTunnelDefine) may persist beyond stopping libvirt, but would not survive a reboot if libvirt wasn't started. A fine distinction actually, and maybe one that doesn't need to be made in general, but that's a different discussion)
2. modify the existing 'network' driver by adding the possibility to specify the 'sectunnel' that the network have to use in the virtual network definition
Will *all* traffic leaving/entering the virtual network travel via the tunnel, and will there be a single tunnel? If so, this could be given as a part of the <forward> element (similar to the way "dev='eth0'" is used now - it specifies that all traffic entering/leaving must go via eth0). It would look something like this: <forward mode='route' tunnel='sec-net'/> (of course all this really does is to set an ipfilter rule that disallows forwarding traffic to/from any other interface from/to the virtual network). Beyond that, if there will an exact 1:1 correspondence between secure tunnel and virtual network (half of which is implied by the "tunnel='sec-net'"), perhaps the config for the tunnel could just be included in the <network> xml (probably as sub-elements of <forward>) rather than creating yet another API (and also the need to involve virInterface*()/netcf)
As an example, you can see below a possible XML definition of the network which use a secure tunnel and the corresponding 'sectunnel' XML definition:
NETWORK DEFINITION ================== <network> <name>sec-net</name> <uuid>3e3fce45-4f53-4fa7-bb32-11f34168b82b</uuid> <bridge name='virbr0' /> <domain name='example' /> ... <sectunnel name='sec-tun' /> <--(specify the 'sectunnel' to use) </network>
SECTUNNEL DEFINITION ==================== <sectunnel type='ipsec'> <name>sec-tun</name> <uuid>8b7fd1b0-4463-43b7-8b6e-8006344aeb66</uuid>
<!-- Security Association definitions -->
<sa> <secret uuid='...' /> <--(specify the 'secret' which contains the pre-shared key) </sa>
<!-- Security Policy definitions -->
<sp> <src_range address='10.0.0.1' prefixlen='30' port='5000' /> <dst_range address='10.0.0.2' prefixlen='30' port='5000' /> <upperspec protocol='any' />
<policy direction='out' action='ipsec'> <rule protocol='esp' mode='tunnel' level='require'> <src address='192.168.0.1' port='55055' /> <dst address='192.168.0.2' port='55055' /> </rule> </policy> </sp>
<sp> <src_range address='10.0.0.2' prefixlen='30' port='5000' /> <dst_range address='10.0.0.1' prefixlen='30' port='5000' /> <upperspec protocol='any' /> <policy direction='in' action='ipsec'> <rule protocol='esp' mode='tunnel' level='require'> <src address='192.168.0.2' port='55055' /> <dst address='192.168.0.1' port='55055' /> </rule> </policy> </sp> </sectunnel>
As you can see in the 'sectunnel' XML definition, I use a 'secret' element. This element is a Libvirt secret [1] and it stores the pre-shared key used by IPSec to establish the Security Associations (SA). Obviously this feature requires to define a new usage category in the 'secret' driver definition.
Another possible way to establish the SA is to use the X.509 certificates. To this purpose, I think that the certificates already used by Libvirt to setup SSL/TLS remote connections, might be used.
That's all! :-)
What do you think about this possible IPSec implementation?
Thanks in advance for the replies!
Best regards,
PAOLO
LINK LIST --------- [1] http://libvirt.org/formatsecret.html
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Hi to everyone! Sorry for the latency of the response but me and my team we are noticed that the TVD argument can not be treated only with a few lines in some mails. In order to avoid any possible misunderstanding, we decided to produce a little report (just four pages with images) that describes our project. Technical details are not treated in the report. You can download the report by using the link http://dl.dropbox.com/u/824617/tvd_in_libvirt.pdf Our idea is to start the discussion about Libvirt TVD implementation using as starting point the report. As already mentioned in previous mail, we think that the first step for the implementation of the TVD is to make possible the 'tunnel' modeling in Libvirt. Considering the report, what do you think about our tunnel modeling idea? It's right or some changes are needed? Thanks for the patience and (in advance) for the replies... ;-) -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Polytechnic University of Turin Email: paolo.smiraglia@polito.it
On 04/29/2011 12:13 PM, Paolo Smiraglia wrote:
Hi to everyone!
Sorry for the latency of the response but me and my team we are noticed that the TVD argument can not be treated only with a few lines in some mails. In order to avoid any possible misunderstanding, we decided to produce a little report (just four pages with images) that describes our project. Technical details are not treated in the report. You can download the report by using the link
http://dl.dropbox.com/u/824617/tvd_in_libvirt.pdf
Our idea is to start the discussion about Libvirt TVD implementation using as starting point the report.
As already mentioned in previous mail, we think that the first step for the implementation of the TVD is to make possible the 'tunnel' modeling in Libvirt.
Considering the report, what do you think about our tunnel modeling idea? It's right or some changes are needed?
Paolo, Did you see my recent email titled "RFC: disconnecting guest/domain interface config from host config": https://www.redhat.com/archives/libvir-list/2011-April/msg00591.html We both want to expand the usage of <network>, so we'd do well to avoid stepping on each others' toes! :-) I'm wondering how the <sectunnel> element would fit in with network types that were not "bridge". I'm also curious about your work with openvswitch, because one of the potentials I can see as a result of expanding the usage of <network> is that openvswitch could be supported directly by libvirt by defining a new <network type='openvswitch'> (I mention that in one of the followup messages. Also I'm still curious about my questions in my earlier response to you: https://www.redhat.com/archives/libvir-list/2011-April/msg00589.html in particular: 1) does the network on each host always have a <forward ...> element for forwarding local traffic directly out to the public network? or alternately, is it possible to force a network on one host to send all traffic over the L2-over-L3 tunnel to a network on another machine, and from there out to the public network? It seems that, in this case, there would be no default route for the systems on the former network (in the case of no forwarding on a libvirt network, no default route is sent in the dhcp response - maybe that needs to be configurable...) 2) Is there an exact 1:1 correspondence between network and tunnel (or perhaps there may be multiple tunnels for a network, but those tunnels are not used by any other network on the same host)? If so, perhaps your project could be simplified by just putting the tunnel config as a subelement of <network>, rather than referencing it - this way you would avoid the need for the extra APIs to define/undefine/etc sectunnel. 3) Are your tunnels always L2, or do you have provision for setting up L3 tunnels? (Perhaps that could be done by allowing multiple <forward> elements, and having a <forward> that specified a tunnel rather than a physical interface, as well as a list of routes as subelements? This, along with a sectunnel subelement should be enough info to setup a secure L3 tunnel which would be used for the specified routes, right? (BTW, after thinking about it some more, I think I agree that <network> is the right place to implement this, rather than a virInterface (host) based <interface> (although that would also be useful, totally separate from libvirt)). It seems we can gain a lot from each other! I'm hoping to have my expansion of the network config completed by the end of June at latest, but your work may enable/force me to hurry it a bit more than that :-)
On 04/29/2011 01:32 PM, Laine Stump wrote:
On 04/29/2011 12:13 PM, Paolo Smiraglia wrote:
Hi to everyone!
Sorry for the latency of the response but me and my team we are noticed that the TVD argument can not be treated only with a few lines in some mails. In order to avoid any possible misunderstanding, we decided to produce a little report (just four pages with images) that describes our project. Technical details are not treated in the report. You can download the report by using the link
http://dl.dropbox.com/u/824617/tvd_in_libvirt.pdf
Our idea is to start the discussion about Libvirt TVD implementation using as starting point the report.
As already mentioned in previous mail, we think that the first step for the implementation of the TVD is to make possible the 'tunnel' modeling in Libvirt.
Considering the report, what do you think about our tunnel modeling idea? It's right or some changes are needed?
Paolo,
Did you see my recent email titled "RFC: disconnecting guest/domain interface config from host config":
https://www.redhat.com/archives/libvir-list/2011-April/msg00591.html
And more discussion on the same topic that unfortunately got put into a different thread: https://www.redhat.com/archives/libvir-list/2011-April/msg01269.html Please let me know how sectunnel stuff would fit in with that.
We both want to expand the usage of <network>, so we'd do well to avoid stepping on each others' toes! :-)
I'm wondering how the <sectunnel> element would fit in with network types that were not "bridge". I'm also curious about your work with openvswitch, because one of the potentials I can see as a result of expanding the usage of <network> is that openvswitch could be supported directly by libvirt by defining a new <network type='openvswitch'> (I mention that in one of the followup messages.
Also I'm still curious about my questions in my earlier response to you:
https://www.redhat.com/archives/libvir-list/2011-April/msg00589.html
in particular:
1) does the network on each host always have a <forward ...> element for forwarding local traffic directly out to the public network? or alternately, is it possible to force a network on one host to send all traffic over the L2-over-L3 tunnel to a network on another machine, and from there out to the public network? It seems that, in this case, there would be no default route for the systems on the former network (in the case of no forwarding on a libvirt network, no default route is sent in the dhcp response - maybe that needs to be configurable...)
2) Is there an exact 1:1 correspondence between network and tunnel (or perhaps there may be multiple tunnels for a network, but those tunnels are not used by any other network on the same host)? If so, perhaps your project could be simplified by just putting the tunnel config as a subelement of <network>, rather than referencing it - this way you would avoid the need for the extra APIs to define/undefine/etc sectunnel.
3) Are your tunnels always L2, or do you have provision for setting up L3 tunnels? (Perhaps that could be done by allowing multiple <forward> elements, and having a <forward> that specified a tunnel rather than a physical interface, as well as a list of routes as subelements? This, along with a sectunnel subelement should be enough info to setup a secure L3 tunnel which would be used for the specified routes, right?
(BTW, after thinking about it some more, I think I agree that <network> is the right place to implement this, rather than a virInterface (host) based <interface> (although that would also be useful, totally separate from libvirt)).
It seems we can gain a lot from each other! I'm hoping to have my expansion of the network config completed by the end of June at latest, but your work may enable/force me to hurry it a bit more than that :-) -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Paolo,
Did you see my recent email titled "RFC: disconnecting guest/domain interface config from host config":
https://www.redhat.com/archives/libvir-list/2011-April/msg00591.html
We both want to expand the usage of <network>, so we'd do well to avoid stepping on each others' toes! :-)
Referring to the options listed in the link posted above, I agree about the Options 3. Moreover, according to the posted XML network examples, I imagine a network definition like: <network type='tunneled'> <name>red-network</name> <tunnel type='ipsec'> <!-- here all elements to define an ipsec tunnel --> </tunnel> </network> or <network type='tunneled'> <name>red-network</name> <tunnel name='ipsectun0' /> </network> but the second example requires the definition (XML, API, ect) of element <tunnel ...>
I'm wondering how the <sectunnel> element would fit in with network types that were not "bridge". [...]
Sorry, but I don't understand what do you want to say... ;-)
[...] I'm also curious about your work with openvswitch, because one of the potentials I can see as a result of expanding the usage of <network> is that openvswitch could be supported directly by libvirt by defining a new <network type='openvswitch'> (I mention that in one of the followup messages.
I used Open vSwitch to isolate the traffic (by using VLAN tagging) into the tunnel. Moreover, I'm valuating to define a Libvirt hook that will allow the dynamic configuration of Open vSwitch. See you, Paolo -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Polytechnic University of Turin Email: paolo.smiraglia@polito.it
Also I'm still curious about my questions in my earlier response to you:
https://www.redhat.com/archives/libvir-list/2011-April/msg00589.html
in particular:
1) does the network on each host always have a <forward ...> element for forwarding local traffic directly out to the public network? or alternately, is it possible to force a network on one host to send all traffic over the L2-over-L3 tunnel to a network on another machine, and from there out to the public network? It seems that, in this case, there would be no default route for the systems on the former network (in the case of no forwarding on a libvirt network, no default route is sent in the dhcp response - maybe that needs to be configurable...)
"My virtual network" have to be considered as an "inseparable logical entity" which is distributed on several hosts. This means that each virtual network portion defined on each host has a <forward ...> element. AFAIK, if the network definition is different between two hosts the domain migration fails...
2) Is there an exact 1:1 correspondence between network and tunnel (or perhaps there may be multiple tunnels for a network, but those tunnels are not used by any other network on the same host)? If so, perhaps your project could be simplified by just putting the tunnel config as a subelement of <network>, rather than referencing it - this way you would avoid the need for the extra APIs to define/undefine/etc sectunnel.
In "my framework", each host is connected with another one only by using one tunnel in where the different traffic flows are isolated by using mechanisms like VLAN or SELinux labeling.
3) Are your tunnels always L2, or do you have provision for setting up L3 tunnels? (Perhaps that could be done by allowing multiple <forward> elements, and having a <forward> that specified a tunnel rather than a physical interface, as well as a list of routes as subelements? This, along with a sectunnel subelement should be enough info to setup a secure L3 tunnel which would be used for the specified routes, right?
"My tunnels" are always L2-on-L3 because I want to create an L2-adjacency between guests that are running on different hosts which are connected by L3 network.
(BTW, after thinking about it some more, I think I agree that <network> is the right place to implement this, rather than a virInterface (host) based <interface> (although that would also be useful, totally separate from libvirt)).
It seems we can gain a lot from each other! I'm hoping to have my expansion of the network config completed by the end of June at latest, but your work may enable/force me to hurry it a bit more than that :-)
Excellent! :-) :-) -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Polytechnic University of Turin Email: paolo.smiraglia@polito.it
On 04/29/2011 06:13 PM, Paolo Smiraglia wrote:
Hi to everyone!
Sorry for the latency of the response but me and my team we are noticed that the TVD argument can not be treated only with a few lines in some mails. In order to avoid any possible misunderstanding, we decided to produce a little report (just four pages with images) that describes our project. Technical details are not treated in the report. You can download the report by using the link
http://dl.dropbox.com/u/824617/tvd_in_libvirt.pdf
Our idea is to start the discussion about Libvirt TVD implementation using as starting point the report.
As already mentioned in previous mail, we think that the first step for the implementation of the TVD is to make possible the 'tunnel' modeling in Libvirt.
Considering the report, what do you think about our tunnel modeling idea? It's right or some changes are needed?
Thanks for the patience and (in advance) for the replies... ;-)
Hi Paolo, thanks for the document. I read it briefly and the design itself seems good however in the document you mentioned moving the logic from user-space to kernel-space which I'm not sure how would you like to achieve this since libvirt itself is in the user-space stack and not kernel-space. For having some implementation of those things directly in the kernel-space you would require to modify the kernel on the host itself which would be very similar to Xen that requires modified kernel - Xen kernel. This introduces some issues there since if you're not able to make it be merged into the upstream kernel tree then you'll be having the same issues like Xen does. If you implement this as a kernel-module and also if you make the module upstream accepted then you'll be most likely fine however you need to upstream acceptance of the module or provide the source codes for the module somewhere to be recompiled for the kernel the user is having. What exactly would you like to move to the kernel-space ? Thanks, Michal -- Michal Novotny <minovotn@redhat.com>, RHCE Virtualization Team (xen userspace), Red Hat
Hi Paolo, thanks for the document. I read it briefly and the design itself seems good however in the document you mentioned moving the logic from user-space to kernel-space which I'm not sure how would you like to achieve this since libvirt itself is in the user-space stack and not kernel-space. For having some implementation of those things directly in the kernel-space you would require to modify the kernel on the host itself which would be very similar to Xen that requires modified kernel - Xen kernel. This introduces some issues there since if you're not able to make it be merged into the upstream kernel tree then you'll be having the same issues like Xen does. If you implement this as a kernel-module and also if you make the module upstream accepted then you'll be most likely fine however you need to upstream acceptance of the module or provide the source codes for the module somewhere to be recompiled for the kernel the user is having.
What exactly would you like to move to the kernel-space ?
Thanks, Michal
Hi Michal! Due to reduce the implementation time and verify quickly if our project is feasible, we decided to implement the prototype by using the simplest user-space applications (VTun, Open vSwitch). To increase the security, we would like to move in kernel-space all security components. We want to migrate from user to kernel space not by defining new kernel modules or by modifying the existing ones, but by using already defined applications that perform our security requirements in kernel spaces. For instance, we have defined an application which filters all received packets (by analyzing the VLAN tags) before that they are received by the switch. We think that the filtering may be executed by using the SELinux labels. About tunneling, we want to remove VTun from our framework and setup directly the 'gretap' interfaces. Any other questions are welcomed! Paolo -- PAOLO SMIRAGLIA Department of Control and Computer Engineering Polytechnic University of Turin Email: paolo.smiraglia@polito.it
On 05/02/2011 03:12 PM, Paolo Smiraglia wrote:
Hi Michal!
Due to reduce the implementation time and verify quickly if our project is feasible, we decided to implement the prototype by using the simplest user-space applications (VTun, Open vSwitch).
To increase the security, we would like to move in kernel-space all security components. We want to migrate from user to kernel space not by defining new kernel modules or by modifying the existing ones, but by using already defined applications that perform our security requirements in kernel spaces.
For instance, we have defined an application which filters all received packets (by analyzing the VLAN tags) before that they are received by the switch. We think that the filtering may be executed by using the SELinux labels. About tunneling, we want to remove VTun from our framework and setup directly the 'gretap' interfaces.
Any other questions are welcomed!
Paolo
Hi Paolo, thanks for your quick reply. Maybe I can see the point now. If you would like to implement it using the already defined application that performs the security requirements in the kernel-space I guess the application are in the form of kernel module or directly implemented into the kernel so you need to check whether the required feature is present/module loaded to allow the functionality. Is this your aim ? Michal -- Michal Novotny <minovotn@redhat.com>, RHCE Virtualization Team (xen userspace), Red Hat
participants (3)
-
Laine Stump -
Michal Novotny -
Paolo Smiraglia