Re: GSoC'20 Interested Student: Adding support to Jailhouse Hypervisor
Hi Prakhar, On 23.03.20 07:53, PRAKHAR BANSAL wrote:
Hello All,
My name is Prakhar Bansal and I am a graduate student in Computer Engineering at Iowa State University, US. I have experience with Analysing Performance of Applications running inside multiple virtual machines hosted by the libvirt QEMU-KVM through virt-manager.
I am interested in working on the project to develop a Libvirt driver for the Jailhouse hypervisor. I looked into the initial attempt on the Jailhouse driver which seems to be based on the Jailhouse command-line interface. I am currently looking into learning and understanding the kernel APIs for jailhouse hypervisor.
Thanks for your interest!
I followed the below articles mentioned by Valentine Sinitsyn to begin learning about the Jailhouse hypervisor.
https://lwn.net/Articles/578295/ https://lwn.net/Articles/578852/
I have a few questions regarding this project, please let me know if someone can help me out.
Sure, go ahead. Depending on the scope of the question, libvirt might be the better community to ask. Therefore, I'm adding its list to this thread. Jan
Thanks & Regards, Prakhar Bansal
-- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Hi Jan, Thanks for the reply. I looked deeper into the libvirt and Jailhouse source code and found following two things that seem relevant to the project I am interested in. - Libvirt driver interface at [libvirt.git] <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> - Jailhouse tool, which is using the ioctl API of the Jailhouse, available at https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c. With the help of the above two, it looks like, a libvirt driver for the Jailhouse can be implemented. Let me know if I am moving in the right direction so far. I have been looking at the other libvirt driver implementations for hypervisors like HyperV and VMware to understand their implementation and learn from there. Thanks & Regards, Prakhar Bansal Graduate Student' 20, Computer Engineering Iowa State University, Ames, IA On Mon, Mar 23, 2020 at 3:07 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:
Hi Prakhar,
On 23.03.20 07:53, PRAKHAR BANSAL wrote:
Hello All,
My name is Prakhar Bansal and I am a graduate student in Computer Engineering at Iowa State University, US. I have experience with Analysing Performance of Applications running inside multiple virtual machines hosted by the libvirt QEMU-KVM through virt-manager.
I am interested in working on the project to develop a Libvirt driver for the Jailhouse hypervisor. I looked into the initial attempt on the Jailhouse driver which seems to be based on the Jailhouse command-line interface. I am currently looking into learning and understanding the kernel APIs for jailhouse hypervisor.
Thanks for your interest!
I followed the below articles mentioned by Valentine Sinitsyn to begin learning about the Jailhouse hypervisor.
https://lwn.net/Articles/578295/ https://lwn.net/Articles/578852/
I have a few questions regarding this project, please let me know if someone can help me out.
Sure, go ahead. Depending on the scope of the question, libvirt might be the better community to ask. Therefore, I'm adding its list to this thread.
Jan
Thanks & Regards, Prakhar Bansal
-- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Hi All, Gentle Reminder, Please have a look at my email below. It would be great if anyone can help me with this project. Thanks & Regards, Prakhar Bansal Graduate Student' 20, Computer Engineering Iowa State University, Ames, IA On Tue, Mar 24, 2020 at 11:36 PM PRAKHAR BANSAL <prakharbansal0910@gmail.com> wrote:
Hi Jan,
Thanks for the reply. I looked deeper into the libvirt and Jailhouse source code and found following two things that seem relevant to the project I am interested in.
- Libvirt driver interface at [libvirt.git] <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> - Jailhouse tool, which is using the ioctl API of the Jailhouse, available at https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c.
With the help of the above two, it looks like, a libvirt driver for the Jailhouse can be implemented. Let me know if I am moving in the right direction so far.
I have been looking at the other libvirt driver implementations for hypervisors like HyperV and VMware to understand their implementation and learn from there.
Thanks & Regards,
Prakhar Bansal Graduate Student' 20, Computer Engineering Iowa State University, Ames, IA
On Mon, Mar 23, 2020 at 3:07 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:
Hi Prakhar,
On 23.03.20 07:53, PRAKHAR BANSAL wrote:
Hello All,
My name is Prakhar Bansal and I am a graduate student in Computer Engineering at Iowa State University, US. I have experience with Analysing Performance of Applications running inside multiple virtual machines hosted by the libvirt QEMU-KVM through virt-manager.
I am interested in working on the project to develop a Libvirt driver for the Jailhouse hypervisor. I looked into the initial attempt on the Jailhouse driver which seems to be based on the Jailhouse command-line interface. I am currently looking into learning and understanding the kernel APIs for jailhouse hypervisor.
Thanks for your interest!
I followed the below articles mentioned by Valentine Sinitsyn to begin learning about the Jailhouse hypervisor.
https://lwn.net/Articles/578295/ https://lwn.net/Articles/578852/
I have a few questions regarding this project, please let me know if someone can help me out.
Sure, go ahead. Depending on the scope of the question, libvirt might be the better community to ask. Therefore, I'm adding its list to this thread.
Jan
Thanks & Regards, Prakhar Bansal
-- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Hi Prakhar, On 25.03.20 05:36, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the reply. I looked deeper into the libvirt and Jailhouse source code and found following two things that seem relevant to the project I am interested in.
- Libvirt driver interface at [libvirt.git] <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> - Jailhouse tool, which is using the ioctl API of the Jailhouse, available at https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c.
With the help of the above two, it looks like, a libvirt driver for the Jailhouse can be implemented. Let me know if I am moving in the right direction so far.
From the Jailhouse perspective, it is important to not consider the command line tool an interface anymore (like in the first prototype) but build on top of the Linux driver API (ioctls, sysfs). There is already a Python library which started to abstract this interface for Jailhouse-internal use cases. However, I strongly suspect libvirt will rather want a native binding.
I have been looking at the other libvirt driver implementations for hypervisors like HyperV and VMware to understand their implementation and learn from there.
As Jailhouse is a static partitioning hypervisor without abstraction of the underlying hardware, your starting point for the libvirt binding should be a given set of Jailhouse cell configurations describing a complete partitioned system. So rather than instantiating on demand a domain (Jailhouse cell) with, say, a network adapter, the driver should match a user request for a domain against the configuration set and use what is there - or report the mismatch. What it could organize, though, is interconnecting cells that have a (preconfigured) virtual network link to the root cell. Due to this different concept, there will be no 1:1 mapping for commodity hypervisor drivers to the Jailhouse scenario. Still, studying what they do is useful and needed in order to understand what "normally" happens and find a reasonable translation. This is probably the most challenging part of the project. Jan
Hi Jan, Thanks for the reply! I was only considering the command-line tool "code" for reference to the Jailhouse kernel API(ioctl calls) because I didn't find a documentation of the Jailhouse kernel APIs. For the second part as you mentioned that Jailhouse can only create cells with the constraints defined in the root cell configuration. I have a few questions regarding that. 1. Is there a way to know if Jailhouse is enabled on the host and get the root cell configuration(s) from Jailhouse through an API? This can be used while binding the libvirt to the Jailhouse hypervisor. 2. If Jailhouse is not enabled(again can we know this using some API) then, can libvirt enable/disable Jailhouse during the libvirt binding of the Jailhouse driver with a given set of Jailhouse cell configurations describing a complete partitioned system? 3. I was wondering, as you mentioned that libvirt driver should check for mismatch of the cell configuration with the root cell configuration, the question is, isn't that done by Jailhouse itself? If yes, then libvirt can just pass on the cell creation requests to Jailhouse and return the response to the user as it is, rather than driver doing any such mismatch check. -Prakhar On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de> wrote:
Hi Prakhar,
On 25.03.20 05:36, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the reply. I looked deeper into the libvirt and Jailhouse source code and found following two things that seem relevant to the project I am interested in.
- Libvirt driver interface at [libvirt.git] <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h < https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD
- Jailhouse tool, which is using the ioctl API of the Jailhouse, available at https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c.
With the help of the above two, it looks like, a libvirt driver for the Jailhouse can be implemented. Let me know if I am moving in the right direction so far.
From the Jailhouse perspective, it is important to not consider the command line tool an interface anymore (like in the first prototype) but build on top of the Linux driver API (ioctls, sysfs). There is already a Python library which started to abstract this interface for Jailhouse-internal use cases. However, I strongly suspect libvirt will rather want a native binding.
I have been looking at the other libvirt driver implementations for hypervisors like HyperV and VMware to understand their implementation and learn from there.
As Jailhouse is a static partitioning hypervisor without abstraction of the underlying hardware, your starting point for the libvirt binding should be a given set of Jailhouse cell configurations describing a complete partitioned system. So rather than instantiating on demand a domain (Jailhouse cell) with, say, a network adapter, the driver should match a user request for a domain against the configuration set and use what is there - or report the mismatch. What it could organize, though, is interconnecting cells that have a (preconfigured) virtual network link to the root cell.
Due to this different concept, there will be no 1:1 mapping for commodity hypervisor drivers to the Jailhouse scenario. Still, studying what they do is useful and needed in order to understand what "normally" happens and find a reasonable translation. This is probably the most challenging part of the project.
Jan
On 28.03.20 08:47, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the reply!
I was only considering the command-line tool "code" for reference to the Jailhouse kernel API(ioctl calls) because I didn't find a documentation of the Jailhouse kernel APIs.
Right, the IOCTL API is not documented so far. It is currently only used inside the Jailhouse project. This needs to be formalized when there shall be external users like a libvirt driver. That might be a nice small contribution task: Create some Documentation/driver-interfaces.md that describes the IOCTLs along with their parameter structures and that also includes the current sysfs-entries.txt as a section. Then send this as patch here. I'll help out when details are not clear from reading the code.
For the second part as you mentioned that Jailhouse can only create cells with the constraints defined in the root cell configuration. I have a few questions regarding that.
1. Is there a way to know if Jailhouse is enabled on the host and get the root cell configuration(s) from Jailhouse through an API? This can be used while binding the libvirt to the Jailhouse hypervisor.
Look at https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries... for what is reported as runtime information. Full configurations can't be read back at this point. This might be reconsidered in the light of [1], but I wouldn't plat for that yet.
2. If Jailhouse is not enabled(again can we know this using some API) then, can libvirt enable/disable Jailhouse during the libvirt binding of the Jailhouse driver with a given set of Jailhouse cell configurations describing a complete partitioned system?
With the API above and a given configuration set, yes. The config set would have to be provided to the libvirt driver in some to-be-defined way (e.g. /etc/libvirt/jailhouse.conf -> /etc/libvirt/jailhouse/*.cell).
3. I was wondering, as you mentioned that libvirt driver should check for mismatch of the cell configuration with the root cell configuration, the question is, isn't that done by Jailhouse itself? If yes, then libvirt can just pass on the cell creation requests to Jailhouse and return the response to the user as it is, rather than driver doing any such mismatch check.
With matching I'm referring to a libvirt user request like "create a domain with 2 CPUs", while there are no cells left that have more than one CPU. Or "give the domain 1G RAM", and you need to find an available cell with that much memory. Those are simple examples. A request that states "connect the domain to the host network A" implies that a cell has a shared-memory link to, say, the root cell that can be configured to bridge this. But let's keep that for later and start as simple as possible. Jan [1] https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B...
-Prakhar
On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
Hi Prakhar,
On 25.03.20 05:36, PRAKHAR BANSAL wrote: > Hi Jan, > > Thanks for the reply. I looked deeper into the libvirt and Jailhouse > source code and found following two things that seem relevant to the > project I am interested in. > > - Libvirt driver interface at [libvirt.git] > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h > <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> > - Jailhouse tool, which is using the ioctl API of the Jailhouse, > available at > https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c. > > With the help of the above two, it looks like, a libvirt driver for the > Jailhouse can be implemented. Let me know if I am moving in the right > direction so far.
From the Jailhouse perspective, it is important to not consider the command line tool an interface anymore (like in the first prototype) but build on top of the Linux driver API (ioctls, sysfs). There is already a Python library which started to abstract this interface for Jailhouse-internal use cases. However, I strongly suspect libvirt will rather want a native binding.
> > I have been looking at the other libvirt driver implementations for > hypervisors like HyperV and VMware to understand their implementation > and learn from there.
As Jailhouse is a static partitioning hypervisor without abstraction of the underlying hardware, your starting point for the libvirt binding should be a given set of Jailhouse cell configurations describing a complete partitioned system. So rather than instantiating on demand a domain (Jailhouse cell) with, say, a network adapter, the driver should match a user request for a domain against the configuration set and use what is there - or report the mismatch. What it could organize, though, is interconnecting cells that have a (preconfigured) virtual network link to the root cell.
Due to this different concept, there will be no 1:1 mapping for commodity hypervisor drivers to the Jailhouse scenario. Still, studying what they do is useful and needed in order to understand what "normally" happens and find a reasonable translation. This is probably the most challenging part of the project.
Jan
Hi Jan, On Sat, Mar 28, 2020 at 4:12 AM Jan Kiszka <jan.kiszka@web.de> wrote:
On 28.03.20 08:47, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the reply!
I was only considering the command-line tool "code" for reference to the Jailhouse kernel API(ioctl calls) because I didn't find a documentation of the Jailhouse kernel APIs.
Right, the IOCTL API is not documented so far. It is currently only used inside the Jailhouse project. This needs to be formalized when there shall be external users like a libvirt driver.
That might be a nice small contribution task: Create some Documentation/driver-interfaces.md that describes the IOCTLs along with their parameter structures and that also includes the current sysfs-entries.txt as a section. Then send this as patch here. I'll help out when details are not clear from reading the code.
Sure. I will do that.
For the second part as you mentioned that Jailhouse can only create cells with the constraints defined in the root cell configuration. I have a few questions regarding that.
1. Is there a way to know if Jailhouse is enabled on the host and get the root cell configuration(s) from Jailhouse through an API? This can be used while binding the libvirt to the Jailhouse hypervisor.
Look at
https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries... for what is reported as runtime information. Full configurations can't be read back at this point. This might be reconsidered in the light of [1], but I wouldn't plat for that yet.
Ok, sure. I am looking into it.
2. If Jailhouse is not enabled(again can we know this using some API) then, can libvirt enable/disable Jailhouse during the libvirt binding of the Jailhouse driver with a given set of Jailhouse cell configurations describing a complete partitioned system?
With the API above and a given configuration set, yes. The config set would have to be provided to the libvirt driver in some to-be-defined way (e.g. /etc/libvirt/jailhouse.conf -> /etc/libvirt/jailhouse/*.cell).
Cool, got it. Thanks!
3. I was wondering, as you mentioned that libvirt driver should check for mismatch of the cell configuration with the root cell configuration, the question is, isn't that done by Jailhouse itself? If yes, then libvirt can just pass on the cell creation requests to Jailhouse and return the response to the user as it is, rather than driver doing any such mismatch check.
With matching I'm referring to a libvirt user request like "create a domain with 2 CPUs", while there are no cells left that have more than one CPU. Or "give the domain 1G RAM", and you need to find an available cell with that much memory. Those are simple examples. A request that states "connect the domain to the host network A" implies that a cell has a shared-memory link to, say, the root cell that can be configured to bridge this. But let's keep that for later and start as simple as possible.
Do I need to match the libvirt user-requested cell config with only root cells or with all cells present at that time? I wanted to request you for a favor for the proposal as the deadline is approaching. Could I prepare a proposal for this project based on our discussion here and improve it later based on feedback comments after the deadline? I understand that I got late in starting on the project search and selection. Thanks, Prakhar
Jan
[1]
https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B...
-Prakhar
On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
Hi Prakhar,
On 25.03.20 05:36, PRAKHAR BANSAL wrote: > Hi Jan, > > Thanks for the reply. I looked deeper into the libvirt and
Jailhouse
> source code and found following two things that seem relevant to
the
> project I am interested in. > > - Libvirt driver interface at [libvirt.git] > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / driver.h > <
https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD
> - Jailhouse tool, which is using the ioctl API of the Jailhouse, > available at >
https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c.
> > With the help of the above two, it looks like, a libvirt driver for the > Jailhouse can be implemented. Let me know if I am moving in the
right
> direction so far.
From the Jailhouse perspective, it is important to not consider the command line tool an interface anymore (like in the first prototype)
but
build on top of the Linux driver API (ioctls, sysfs). There is
already a
Python library which started to abstract this interface for Jailhouse-internal use cases. However, I strongly suspect libvirt
will
rather want a native binding.
> > I have been looking at the other libvirt driver implementations
for
> hypervisors like HyperV and VMware to understand their
implementation
> and learn from there.
As Jailhouse is a static partitioning hypervisor without abstraction
of
the underlying hardware, your starting point for the libvirt binding should be a given set of Jailhouse cell configurations describing a complete partitioned system. So rather than instantiating on demand a domain (Jailhouse cell) with, say, a network adapter, the driver
should
match a user request for a domain against the configuration set and
use
what is there - or report the mismatch. What it could organize,
though,
is interconnecting cells that have a (preconfigured) virtual network link to the root cell.
Due to this different concept, there will be no 1:1 mapping for commodity hypervisor drivers to the Jailhouse scenario. Still,
studying
what they do is useful and needed in order to understand what
"normally"
happens and find a reasonable translation. This is probably the most challenging part of the project.
Jan
On 30.03.20 10:02, PRAKHAR BANSAL wrote:
Hi Jan,
On Sat, Mar 28, 2020 at 4:12 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
On 28.03.20 08:47, PRAKHAR BANSAL wrote: > Hi Jan, > > Thanks for the reply! > > I was only considering the command-line tool "code" for reference to the > Jailhouse kernel API(ioctl calls) because I didn't find a documentation > of the Jailhouse kernel APIs.
Right, the IOCTL API is not documented so far. It is currently only used inside the Jailhouse project. This needs to be formalized when there shall be external users like a libvirt driver.
That might be a nice small contribution task: Create some Documentation/driver-interfaces.md that describes the IOCTLs along with their parameter structures and that also includes the current sysfs-entries.txt as a section. Then send this as patch here. I'll help out when details are not clear from reading the code.
Sure. I will do that.
> > For the second part as you mentioned that Jailhouse can only create > cells with the constraints defined in the root cell configuration. I > have a few questions regarding that. > > 1. Is there a way to know if Jailhouse is enabled on the host and get > the root cell configuration(s) from Jailhouse through an API? This can > be used while binding the libvirt to the Jailhouse hypervisor.
Look at https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries... for what is reported as runtime information. Full configurations can't be read back at this point. This might be reconsidered in the light of [1], but I wouldn't plat for that yet.
Ok, sure. I am looking into it.
> > 2. If Jailhouse is not enabled(again can we know this using some API) > then, can libvirt enable/disable Jailhouse during the libvirt binding of > the Jailhouse driver with a given set of Jailhouse cell configurations > describing a complete partitioned system?
With the API above and a given configuration set, yes. The config set would have to be provided to the libvirt driver in some to-be-defined way (e.g. /etc/libvirt/jailhouse.conf -> /etc/libvirt/jailhouse/*.cell).
Cool, got it. Thanks!
> > 3. I was wondering, as you mentioned that libvirt driver should check > for mismatch of the cell configuration with the root cell configuration, > the question is, isn't that done by Jailhouse itself? If yes, then > libvirt can just pass on the cell creation requests to Jailhouse and > return the response to the user as it is, rather than driver doing any > such mismatch check.
With matching I'm referring to a libvirt user request like "create a domain with 2 CPUs", while there are no cells left that have more than one CPU. Or "give the domain 1G RAM", and you need to find an available cell with that much memory. Those are simple examples. A request that states "connect the domain to the host network A" implies that a cell has a shared-memory link to, say, the root cell that can be configured to bridge this. But let's keep that for later and start as simple as possible.
Do I need to match the libvirt user-requested cell config with only root cells or with all cells present at that time?
With all non-root cells - the root cell will be occupied already (it runs libvirt e.g.).
I wanted to request you for a favor for the proposal as the deadline is approaching. Could I prepare a proposal for this project based on our discussion here and improve it later based on feedback comments after the deadline? I understand that I got late in starting on the project search and selection.
Sure, please go ahead. Jan
Thanks, Prakhar
Jan
[1] https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B...
> > -Prakhar > > On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>> wrote: > > Hi Prakhar, > > On 25.03.20 05:36, PRAKHAR BANSAL wrote: > > Hi Jan, > > > > Thanks for the reply. I looked deeper into the libvirt and Jailhouse > > source code and found following two things that seem relevant to the > > project I am interested in. > > > > - Libvirt driver interface at [libvirt.git] > > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src > > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / > driver.h > > > <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> > > - Jailhouse tool, which is using the ioctl API of the Jailhouse, > > available at > > https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c. > > > > With the help of the above two, it looks like, a libvirt driver > for the > > Jailhouse can be implemented. Let me know if I am moving in the right > > direction so far. > > From the Jailhouse perspective, it is important to not consider the > command line tool an interface anymore (like in the first prototype) but > build on top of the Linux driver API (ioctls, sysfs). There is already a > Python library which started to abstract this interface for > Jailhouse-internal use cases. However, I strongly suspect libvirt will > rather want a native binding. > > > > > I have been looking at the other libvirt driver implementations for > > hypervisors like HyperV and VMware to understand their implementation > > and learn from there. > > As Jailhouse is a static partitioning hypervisor without abstraction of > the underlying hardware, your starting point for the libvirt binding > should be a given set of Jailhouse cell configurations describing a > complete partitioned system. So rather than instantiating on demand a > domain (Jailhouse cell) with, say, a network adapter, the driver should > match a user request for a domain against the configuration set and use > what is there - or report the mismatch. What it could organize, though, > is interconnecting cells that have a (preconfigured) virtual network > link to the root cell. > > Due to this different concept, there will be no 1:1 mapping for > commodity hypervisor drivers to the Jailhouse scenario. Still, studying > what they do is useful and needed in order to understand what "normally" > happens and find a reasonable translation. This is probably the most > challenging part of the project. > > Jan >
Hi Jan, Thanks for the confirmation to proceed on project proposal. Also, I tried installing Jailhouse on my VM after enabling VT-x/EPT and IOMMU for my VM(Guest OS- Ubuntu 18.04) on VMware fusion hypervisor with MacOS on the host side. However, Jailhouse hardware check was failed because of missing *Preemption timer and Virtualize APIC access*, could you please suggest, if this is hardware limitation? Is there any workaround here? My laptop's processor is Intel quad-core i7. [image: image.png] Also, could you please suggest, if I can talk to you through an IRC or slack channel since it is a bit hard to communicate over email every time. Thanks, Prakhar On Mon, Mar 30, 2020 at 6:15 AM Jan Kiszka <jan.kiszka@web.de> wrote:
On 30.03.20 10:02, PRAKHAR BANSAL wrote:
Hi Jan,
On Sat, Mar 28, 2020 at 4:12 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
On 28.03.20 08:47, PRAKHAR BANSAL wrote: > Hi Jan, > > Thanks for the reply! > > I was only considering the command-line tool "code" for reference to the > Jailhouse kernel API(ioctl calls) because I didn't find a documentation > of the Jailhouse kernel APIs.
Right, the IOCTL API is not documented so far. It is currently only used inside the Jailhouse project. This needs to be formalized when there shall be external users like a libvirt driver.
That might be a nice small contribution task: Create some Documentation/driver-interfaces.md that describes the IOCTLs along with their parameter structures and that also includes the current sysfs-entries.txt as a section. Then send this as patch here. I'll help out when details are not clear from reading the code.
Sure. I will do that.
> > For the second part as you mentioned that Jailhouse can only create > cells with the constraints defined in the root cell configuration. I > have a few questions regarding that. > > 1. Is there a way to know if Jailhouse is enabled on the host and get > the root cell configuration(s) from Jailhouse through an API? This can > be used while binding the libvirt to the Jailhouse hypervisor.
Look at
https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries...
for what is reported as runtime information. Full configurations
can't
be read back at this point. This might be reconsidered in the light
of
[1], but I wouldn't plat for that yet.
Ok, sure. I am looking into it.
> > 2. If Jailhouse is not enabled(again can we know this using some
API)
> then, can libvirt enable/disable Jailhouse during the libvirt binding of > the Jailhouse driver with a given set of Jailhouse cell configurations > describing a complete partitioned system?
With the API above and a given configuration set, yes. The config set would have to be provided to the libvirt driver in some to-be-defined way (e.g. /etc/libvirt/jailhouse.conf ->
/etc/libvirt/jailhouse/*.cell).
Cool, got it. Thanks!
> > 3. I was wondering, as you mentioned that libvirt driver should
check
> for mismatch of the cell configuration with the root cell configuration, > the question is, isn't that done by Jailhouse itself? If yes, then > libvirt can just pass on the cell creation requests to Jailhouse
and
> return the response to the user as it is, rather than driver doing any > such mismatch check.
With matching I'm referring to a libvirt user request like "create a domain with 2 CPUs", while there are no cells left that have more
than
one CPU. Or "give the domain 1G RAM", and you need to find an
available
cell with that much memory. Those are simple examples. A request that states "connect the domain to the host network A" implies that a cell has a shared-memory link to, say, the root cell that can be
configured
to bridge this. But let's keep that for later and start as simple as possible.
Do I need to match the libvirt user-requested cell config with only root cells or with all cells present at that time?
With all non-root cells - the root cell will be occupied already (it runs libvirt e.g.).
I wanted to request you for a favor for the proposal as the deadline is approaching. Could I prepare a proposal for this project based on our discussion here and improve it later based on feedback comments after the deadline? I understand that I got late in starting on the project search and selection.
Sure, please go ahead.
Jan
Thanks, Prakhar
Jan
[1]
https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B...
> > -Prakhar > > On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>> wrote: > > Hi Prakhar, > > On 25.03.20 05:36, PRAKHAR BANSAL wrote: > > Hi Jan, > > > > Thanks for the reply. I looked deeper into the libvirt and Jailhouse > > source code and found following two things that seem relevant to the > > project I am interested in. > > > > - Libvirt driver interface at [libvirt.git] > > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> /
src
> > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / > driver.h > > > <
https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD
> > - Jailhouse tool, which is using the ioctl API of the Jailhouse, > > available at > > https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c. > > > > With the help of the above two, it looks like, a libvirt driver > for the > > Jailhouse can be implemented. Let me know if I am moving in the right > > direction so far. > > From the Jailhouse perspective, it is important to not consider the > command line tool an interface anymore (like in the first prototype) but > build on top of the Linux driver API (ioctls, sysfs). There is already a > Python library which started to abstract this interface for > Jailhouse-internal use cases. However, I strongly suspect libvirt will > rather want a native binding. > > > > > I have been looking at the other libvirt driver implementations for > > hypervisors like HyperV and VMware to understand their implementation > > and learn from there. > > As Jailhouse is a static partitioning hypervisor without abstraction of > the underlying hardware, your starting point for the libvirt binding > should be a given set of Jailhouse cell configurations describing a > complete partitioned system. So rather than instantiating on demand a > domain (Jailhouse cell) with, say, a network adapter, the driver should > match a user request for a domain against the configuration set and use > what is there - or report the mismatch. What it could organize, though, > is interconnecting cells that have a (preconfigured) virtual network > link to the root cell. > > Due to this different concept, there will be no 1:1 mapping
for
> commodity hypervisor drivers to the Jailhouse scenario. Still, studying > what they do is useful and needed in order to understand what "normally" > happens and find a reasonable translation. This is probably the most > challenging part of the project. > > Jan >
On 31.03.20 04:43, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the confirmation to proceed on project proposal.
Also, I tried installing Jailhouse on my VM after enabling VT-x/EPT and IOMMU for my VM(Guest OS- Ubuntu 18.04) on VMware fusion hypervisor with MacOS on the host side. However, Jailhouse hardware check was failed because of missing *Preemption timer and Virtualize APIC access*, could you please suggest, if this is hardware limitation? Is there any workaround here?
You will need a hypervisor that supports both when nesting, but I have no idea if there is one for the Mac. What is known to work is KVM on Linux hosts.
My laptop's processor is Intel quad-core i7.
image.png
Also, could you please suggest, if I can talk to you through an IRC or slack channel since it is a bit hard to communicate over email every time.
I'll be listening on #jailhouse, irc.freenode.net. Jan
Thanks, Prakhar
On Mon, Mar 30, 2020 at 6:15 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
On 30.03.20 10:02, PRAKHAR BANSAL wrote: > Hi Jan, > > On Sat, Mar 28, 2020 at 4:12 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>> wrote: > > On 28.03.20 08:47, PRAKHAR BANSAL wrote: > > Hi Jan, > > > > Thanks for the reply! > > > > I was only considering the command-line tool "code" for reference > to the > > Jailhouse kernel API(ioctl calls) because I didn't find a > documentation > > of the Jailhouse kernel APIs. > > Right, the IOCTL API is not documented so far. It is currently only used > inside the Jailhouse project. This needs to be formalized when there > shall be external users like a libvirt driver. > > That might be a nice small contribution task: Create some > Documentation/driver-interfaces.md that describes the IOCTLs along with > their parameter structures and that also includes the current > sysfs-entries.txt as a section. Then send this as patch here. I'll help > out when details are not clear from reading the code. > > Sure. I will do that. > > > > > For the second part as you mentioned that Jailhouse can only create > > cells with the constraints defined in the root cell configuration. I > > have a few questions regarding that. > > > > 1. Is there a way to know if Jailhouse is enabled on the host and get > > the root cell configuration(s) from Jailhouse through an API? > This can > > be used while binding the libvirt to the Jailhouse hypervisor. > > Look at > https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries... > for what is reported as runtime information. Full configurations can't > be read back at this point. This might be reconsidered in the light of > [1], but I wouldn't plat for that yet. > > > Ok, sure. I am looking into it. > > > > > > 2. If Jailhouse is not enabled(again can we know this using some API) > > then, can libvirt enable/disable Jailhouse during the libvirt > binding of > > the Jailhouse driver with a given set of Jailhouse cell > configurations > > describing a complete partitioned system? > > With the API above and a given configuration set, yes. The config set > would have to be provided to the libvirt driver in some to-be-defined > way (e.g. /etc/libvirt/jailhouse.conf -> /etc/libvirt/jailhouse/*.cell). > > Cool, got it. Thanks! > > > > > 3. I was wondering, as you mentioned that libvirt driver should check > > for mismatch of the cell configuration with the root cell > configuration, > > the question is, isn't that done by Jailhouse itself? If yes, then > > libvirt can just pass on the cell creation requests to Jailhouse and > > return the response to the user as it is, rather than driver > doing any > > such mismatch check. > > With matching I'm referring to a libvirt user request like "create a > domain with 2 CPUs", while there are no cells left that have more than > one CPU. Or "give the domain 1G RAM", and you need to find an available > cell with that much memory. Those are simple examples. A request that > states "connect the domain to the host network A" implies that a cell > has a shared-memory link to, say, the root cell that can be configured > to bridge this. But let's keep that for later and start as simple as > possible. > > > Do I need to match the libvirt user-requested cell config with only root > cells or with all cells present at that time?
With all non-root cells - the root cell will be occupied already (it runs libvirt e.g.).
> > I wanted to request you for a favor for the proposal as the deadline is > approaching. Could I prepare a proposal for this project based on our > discussion here and improve it later based on feedback comments after > the deadline? I understand that I got late in starting on the project > search and selection.
Sure, please go ahead.
Jan
> > Thanks, > Prakhar > > > Jan > > [1] > https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B... > > > > > -Prakhar > > > > On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>> > > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de> <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>>> wrote: > > > > Hi Prakhar, > > > > On 25.03.20 05:36, PRAKHAR BANSAL wrote: > > > Hi Jan, > > > > > > Thanks for the reply. I looked deeper into the libvirt and > Jailhouse > > > source code and found following two things that seem > relevant to the > > > project I am interested in. > > > > > > - Libvirt driver interface at [libvirt.git] > > > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src > > > > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD> / > > driver.h > > > > > > <https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD> > > > - Jailhouse tool, which is using the ioctl API of the > Jailhouse, > > > available at > > > > https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c. > > > > > > With the help of the above two, it looks like, a libvirt > driver > > for the > > > Jailhouse can be implemented. Let me know if I am moving > in the right > > > direction so far. > > > > From the Jailhouse perspective, it is important to not > consider the > > command line tool an interface anymore (like in the first > prototype) but > > build on top of the Linux driver API (ioctls, sysfs). There > is already a > > Python library which started to abstract this interface for > > Jailhouse-internal use cases. However, I strongly suspect > libvirt will > > rather want a native binding. > > > > > > > > I have been looking at the other libvirt driver > implementations for > > > hypervisors like HyperV and VMware to understand their > implementation > > > and learn from there. > > > > As Jailhouse is a static partitioning hypervisor without > abstraction of > > the underlying hardware, your starting point for the libvirt > binding > > should be a given set of Jailhouse cell configurations > describing a > > complete partitioned system. So rather than instantiating on > demand a > > domain (Jailhouse cell) with, say, a network adapter, the > driver should > > match a user request for a domain against the configuration > set and use > > what is there - or report the mismatch. What it could > organize, though, > > is interconnecting cells that have a (preconfigured) virtual > network > > link to the root cell. > > > > Due to this different concept, there will be no 1:1 mapping for > > commodity hypervisor drivers to the Jailhouse scenario. > Still, studying > > what they do is useful and needed in order to understand what > "normally" > > happens and find a reasonable translation. This is probably > the most > > challenging part of the project. > > > > Jan > > >
Hi Jan, I have shared the draft proposal link with libvirt on GSoC's system. Could you please check and provide your feedback, if possible. Thanks, Prakhar On Tue, Mar 31, 2020 at 1:47 AM Jan Kiszka <jan.kiszka@web.de> wrote:
On 31.03.20 04:43, PRAKHAR BANSAL wrote:
Hi Jan,
Thanks for the confirmation to proceed on project proposal.
Also, I tried installing Jailhouse on my VM after enabling VT-x/EPT and IOMMU for my VM(Guest OS- Ubuntu 18.04) on VMware fusion hypervisor with MacOS on the host side. However, Jailhouse hardware check was failed because of missing *Preemption timer and Virtualize APIC access*, could you please suggest, if this is hardware limitation? Is there any workaround here?
You will need a hypervisor that supports both when nesting, but I have no idea if there is one for the Mac. What is known to work is KVM on Linux hosts.
My laptop's processor is Intel quad-core i7.
image.png
Also, could you please suggest, if I can talk to you through an IRC or slack channel since it is a bit hard to communicate over email every time.
I'll be listening on #jailhouse, irc.freenode.net.
Jan
Thanks, Prakhar
On Mon, Mar 30, 2020 at 6:15 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de>> wrote:
On 30.03.20 10:02, PRAKHAR BANSAL wrote: > Hi Jan, > > On Sat, Mar 28, 2020 at 4:12 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>> wrote: > > On 28.03.20 08:47, PRAKHAR BANSAL wrote: > > Hi Jan, > > > > Thanks for the reply! > > > > I was only considering the command-line tool "code" for reference > to the > > Jailhouse kernel API(ioctl calls) because I didn't find a > documentation > > of the Jailhouse kernel APIs. > > Right, the IOCTL API is not documented so far. It is currently only used > inside the Jailhouse project. This needs to be formalized when there > shall be external users like a libvirt driver. > > That might be a nice small contribution task: Create some > Documentation/driver-interfaces.md that describes the IOCTLs along with > their parameter structures and that also includes the current > sysfs-entries.txt as a section. Then send this as patch here. I'll help > out when details are not clear from reading the code. > > Sure. I will do that. > > > > > For the second part as you mentioned that Jailhouse can only create > > cells with the constraints defined in the root cell configuration. I > > have a few questions regarding that. > > > > 1. Is there a way to know if Jailhouse is enabled on the host and get > > the root cell configuration(s) from Jailhouse through an
API?
> This can > > be used while binding the libvirt to the Jailhouse
hypervisor.
> > Look at >
https://github.com/siemens/jailhouse/blob/master/Documentation/sysfs-entries...
> for what is reported as runtime information. Full configurations can't > be read back at this point. This might be reconsidered in the light of > [1], but I wouldn't plat for that yet. > > > Ok, sure. I am looking into it. > > > > > > 2. If Jailhouse is not enabled(again can we know this using some API) > > then, can libvirt enable/disable Jailhouse during the
libvirt
> binding of > > the Jailhouse driver with a given set of Jailhouse cell > configurations > > describing a complete partitioned system? > > With the API above and a given configuration set, yes. The config set > would have to be provided to the libvirt driver in some to-be-defined > way (e.g. /etc/libvirt/jailhouse.conf -> /etc/libvirt/jailhouse/*.cell). > > Cool, got it. Thanks! > > > > > 3. I was wondering, as you mentioned that libvirt driver should check > > for mismatch of the cell configuration with the root cell > configuration, > > the question is, isn't that done by Jailhouse itself? If yes, then > > libvirt can just pass on the cell creation requests to Jailhouse and > > return the response to the user as it is, rather than
driver
> doing any > > such mismatch check. > > With matching I'm referring to a libvirt user request like "create a > domain with 2 CPUs", while there are no cells left that have more than > one CPU. Or "give the domain 1G RAM", and you need to find an available > cell with that much memory. Those are simple examples. A request that > states "connect the domain to the host network A" implies that a cell > has a shared-memory link to, say, the root cell that can be configured > to bridge this. But let's keep that for later and start as simple as > possible. > > > Do I need to match the libvirt user-requested cell config with only root > cells or with all cells present at that time?
With all non-root cells - the root cell will be occupied already (it runs libvirt e.g.).
> > I wanted to request you for a favor for the proposal as the deadline is > approaching. Could I prepare a proposal for this project based on
our
> discussion here and improve it later based on feedback comments
after
> the deadline? I understand that I got late in starting on the
project
> search and selection.
Sure, please go ahead.
Jan
> > Thanks, > Prakhar > > > Jan > > [1] >
https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1QiRhSWZnw%2BkHhJMO-B...
> > > > > -Prakhar > > > > On Thu, Mar 26, 2020 at 1:49 AM Jan Kiszka <jan.kiszka@web.de <mailto:jan.kiszka@web.de> > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>> > > <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de> <mailto:jan.kiszka@web.de <mailto:jan.kiszka@web.de>>>> wrote: > > > > Hi Prakhar, > > > > On 25.03.20 05:36, PRAKHAR BANSAL wrote: > > > Hi Jan, > > > > > > Thanks for the reply. I looked deeper into the libvirt and > Jailhouse > > > source code and found following two things that seem > relevant to the > > > project I am interested in. > > > > > > - Libvirt driver interface at [libvirt.git] > > > <https://libvirt.org/git/?p=libvirt.git;a=tree;hb=HEAD> / src > > > > <https://libvirt.org/git/?p=libvirt.git;a=tree;f=src;hb=HEAD / > > driver.h > > > > > > <
https://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/driver.h;hb=HEAD
> > > - Jailhouse tool, which is using the ioctl API of
the
> Jailhouse, > > > available at > > > >
https://github.com/siemens/jailhouse/blob/master/tools/jailhouse.c.
> > > > > > With the help of the above two, it looks like, a libvirt > driver > > for the > > > Jailhouse can be implemented. Let me know if I am moving > in the right > > > direction so far. > > > > From the Jailhouse perspective, it is important to
not
> consider the > > command line tool an interface anymore (like in the
first
> prototype) but > > build on top of the Linux driver API (ioctls, sysfs). There > is already a > > Python library which started to abstract this interface for > > Jailhouse-internal use cases. However, I strongly
suspect
> libvirt will > > rather want a native binding. > > > > > > > > I have been looking at the other libvirt driver > implementations for > > > hypervisors like HyperV and VMware to understand
their
> implementation > > > and learn from there. > > > > As Jailhouse is a static partitioning hypervisor
without
> abstraction of > > the underlying hardware, your starting point for the libvirt > binding > > should be a given set of Jailhouse cell configurations > describing a > > complete partitioned system. So rather than instantiating on > demand a > > domain (Jailhouse cell) with, say, a network adapter,
the
> driver should > > match a user request for a domain against the configuration > set and use > > what is there - or report the mismatch. What it could > organize, though, > > is interconnecting cells that have a (preconfigured) virtual > network > > link to the root cell. > > > > Due to this different concept, there will be no 1:1 mapping for > > commodity hypervisor drivers to the Jailhouse scenario. > Still, studying > > what they do is useful and needed in order to understand what > "normally" > > happens and find a reasonable translation. This is probably > the most > > challenging part of the project. > > > > Jan > > >
participants (3)
-
Jan Kiszka -
Jan Kiszka -
PRAKHAR BANSAL