[libvirt] security: the qemu agent command "guest-exec" may cause Insider Access
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg: virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output": true } }' {"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}} The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access. So, how to avoid this security problem, what's your suggestion? Thanks! Best Regrads Oscar
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote:
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output": true } }' {"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion?
You can use the "--blacklist" facility of qemu-ga to disable APIs you don't want to support. Or don't run the guest agent at all.
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output":
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote: true } }'
{"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion?
You can use the "--blacklist" facility of qemu-ga to disable APIs you don't want to support. Or don't run the guest agent at all.
This works if the qemu-agent inside the guest is installed by us cloud provider. But if the guest is installed all by the cloud tenant himself, he may not know to add "--blacklist" by default, and doesn't notice that his OS is opposed to host attackers. How to solve this problem? It seems that we have to mitigate the treat on the host side?
On Fri, Aug 25, 2017 at 08:52:16AM +0000, Zhangbo (Oscar) wrote:
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output":
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote: true } }'
{"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion?
You can use the "--blacklist" facility of qemu-ga to disable APIs you don't want to support. Or don't run the guest agent at all.
This works if the qemu-agent inside the guest is installed by us cloud provider. But if the guest is installed all by the cloud tenant himself, he may not know to add "--blacklist" by default, and doesn't notice that his OS is opposed to host attackers. How to solve this problem? It seems that we have to mitigate the treat on the host side?
Compromised host implies all guests to be compromised as well. You cannot (currently) protect from this.
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Fri, Aug 25, 2017 at 08:52:16 +0000, Zhangbo (Oscar) wrote:
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote:
[...]
The Administrator also can use other commands such as "
guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion?
You can use the "--blacklist" facility of qemu-ga to disable APIs you don't want to support. Or don't run the guest agent at all.
This works if the qemu-agent inside the guest is installed by us cloud provider. But if the guest is installed all by the cloud tenant himself, he may not know to add "--blacklist" by default, and doesn't notice that his OS is opposed to host attackers. How to solve this problem? It seems that we have to mitigate the treat on the host side?
That premise is silly. Adding protection from the host on the host side does not make sense, since the host admin can always change it. You can avoid adding the guest agent channel though completely. On the guest, you can e.g. not install the guest agent. If you don't trust the host, don't use it. There's no protection from reading the memory or disk images currently. See [1]. Note that even without the API, root can access all the stuff. [1]: http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMemoryPeek
On Fri, Aug 25, 2017 at 08:52:16 +0000, Zhangbo (Oscar) wrote:
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote:
[...]
The Administrator also can use other commands such as "
guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion?
You can use the "--blacklist" facility of qemu-ga to disable APIs you don't want to support. Or don't run the guest agent at all.
This works if the qemu-agent inside the guest is installed by us cloud provider. But if the guest is installed all by the cloud tenant himself, he may not know to add "--blacklist" by default, and doesn't notice that his OS is opposed to host attackers. How to solve this problem? It seems that we have to mitigate the treat on the host side?
That premise is silly. Adding protection from the host on the host side does not make sense, since the host admin can always change it.
You can avoid adding the guest agent channel though completely. On the guest, you can e.g. not install the guest agent.
If you don't trust the host, don't use it. There's no protection from reading the memory or disk images currently. See [1]. Note that even without the API, root can access all the stuff.
Thank you very much for the detailed reply, any future plan to solve such problem(host has too high authority to access guests' memory things)? What will be the potential mitigation?
[1]: http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMemoryPeek
On Sat, Aug 26, 2017 at 01:05:46 +0000, Zhangbo (Oscar) wrote:
On Fri, Aug 25, 2017 at 08:52:16 +0000, Zhangbo (Oscar) wrote:
On Fri, Aug 25, 2017 at 06:45:18 +0000, Zhangbo (Oscar) wrote:
[...]
If you don't trust the host, don't use it. There's no protection from reading the memory or disk images currently. See [1]. Note that even without the API, root can access all the stuff.
Thank you very much for the detailed reply, any future plan to solve such problem(host has too high authority to access guests' memory things)? What will be the potential mitigation?
The best mitigation is to not allow unauthorized access to the host. In other words: if you don't trust your cloud provider, host your stuff yourself.
On Fri, Aug 25, 2017 at 06:45:18AM +0000, Zhangbo (Oscar) wrote:
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output": true } }' {"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion? Thanks!
What's your setup that this, in particular, is your concern? Do you have everything encrypted by keys that are not reachable for the host administrator? How are those saved? For example, how do you guard against the host administrator killing the domain? Or mounting the disk of the domain, doing whatever they want to with it and starting it back? Or million other things that come to mind. Not trusting the host administrator is kinda (well, precisely) like not trusting root on *NIX machine. Martin P.S.: Maybe more aluminium could help, but I haven't tried yet.
Best Regrads Oscar
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output":
On Fri, Aug 25, 2017 at 06:45:18AM +0000, Zhangbo (Oscar) wrote: true } }'
{"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion? Thanks!
What's your setup that this, in particular, is your concern? Do you have everything encrypted by keys that are not reachable for the host administrator? How are those saved? For example, how do you guard against the host administrator killing the domain? Or mounting the disk
Killing the domain is acceptable somehow at some degree, but leaking sensitive information, for example, trade secret, is horrible. We most concern is to protect that information by (maybe)disabling "guest-exec" guest commands etc. Another concern, as mentioned in another mail, is that, if the cloud tenant himself installed the guest agent, and doesn't know the "adding --blacklist" issue, that guest is in danger.
of the domain, doing whatever they want to with it and starting it back? Or million other things that come to mind. Not trusting the host administrator is kinda (well, precisely) like not trusting root on *NIX machine.
Martin
P.S.: Maybe more aluminium could help, but I haven't tried yet.
Best Regrads Oscar
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Fri, Aug 25, 2017 at 08:59:54AM +0000, Zhangbo (Oscar) wrote:
Hi all: The Host Administrator is capable of running any exec in guests via the qemu-ga command "guest-exec", eg:
virsh qemu-agent-command test_guest '{"execute": "guest-exec", "arguments": {"path": "ifconfig", "arg": [ "eth1", "192.168.0.99" ],"capture-output":
On Fri, Aug 25, 2017 at 06:45:18AM +0000, Zhangbo (Oscar) wrote: true } }'
{"return":{"pid":12425}} virsh qemu-agent-command test_guest '{"execute": "guest-exec-status", "arguments": { "pid": 12425 } }' {"return":{"exitcode":0,"exited":true}}
The example above just change the guests' ip address, the Administrator may also change guests' user password, get sensitive information, etc. which causes Insider Access. The Administrator also can use other commands such as " guest-file-open" that also cause Insider Access.
So, how to avoid this security problem, what's your suggestion? Thanks!
What's your setup that this, in particular, is your concern? Do you have everything encrypted by keys that are not reachable for the host administrator? How are those saved? For example, how do you guard against the host administrator killing the domain? Or mounting the disk
Killing the domain is acceptable somehow at some degree, but leaking sensitive information, for example, trade secret, is horrible. We most concern is to protect that information by (maybe)disabling "guest-exec" guest commands etc.
Host can read all of the guest's memory or mount the image and modify the guest agent. Or even add their own communication program that can do anything.
Another concern, as mentioned in another mail, is that, if the cloud tenant himself installed the guest agent, and doesn't know the "adding --blacklist" issue, that guest is in danger.
of the domain, doing whatever they want to with it and starting it back? Or million other things that come to mind. Not trusting the host administrator is kinda (well, precisely) like not trusting root on *NIX machine.
Martin
P.S.: Maybe more aluminium could help, but I haven't tried yet.
Best Regrads Oscar
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Host can read all of the guest's memory or mount the image and modify the guest agent. Or even add their own communication program that can do anything.
I get your point now! :) Thanks a lot!! Further more, kvm seems not as secure as xen, because xen isolates dom0 and domU well, The administrator on dom0 couldn't access many things belonged to domUs. How to solve such problem in kvm? Any scheme?
On Fri, Aug 25, 2017 at 10:29:03AM +0000, Zhangbo (Oscar) wrote:
Host can read all of the guest's memory or mount the image and modify the guest agent. Or even add their own communication program that can do anything.
I get your point now! :) Thanks a lot!!
Further more, kvm seems not as secure as xen, because xen isolates dom0 and domU well, The administrator on dom0 couldn't access many things belonged to domUs. How to solve such problem in kvm? Any scheme?
I don't know xen much, but maybe AMD SEV or everything-signed-by TPM would help...
Host can read all of the guest's memory or mount the image and modify the guest agent. Or even add their own communication program that can do anything.
I get your point now! :) Thanks a lot!!
Further more, kvm seems not as secure as xen, because xen isolates dom0 and domU well, The administrator on dom0 couldn't access many things belonged to domUs. How to solve such problem in kvm? Any scheme?
I don't know xen much, but maybe AMD SEV or everything-signed-by TPM would help...
Thank you , I'll look further into them.
On 08/25/2017 12:41 PM, Martin Kletzander wrote:
On Fri, Aug 25, 2017 at 10:29:03AM +0000, Zhangbo (Oscar) wrote:
Host can read all of the guest's memory or mount the image and modify the guest agent. Or even add their own communication program that can do anything.
I get your point now! :) Thanks a lot!!
Further more, kvm seems not as secure as xen, because xen isolates dom0 and domU well, The administrator on dom0 couldn't access many things belonged to domUs. How to solve such problem in kvm? Any scheme?
I don't know xen much, but maybe AMD SEV or everything-signed-by TPM would help...
I'm no HW guy, but SEV looks like protection against physical attacks, i.e. a guy working for some government agency walking around your server room with a load of liquid gas. At first Intel's SGX [1] looked promising, but apparently it's flawed. So currently I don't think there's anything we can do. Except not give out root access to everyone. Michal 1: https://en.wikipedia.org/wiki/Software_Guard_Extensions
participants (4)
-
Martin Kletzander -
Michal Privoznik -
Peter Krempa -
Zhangbo (Oscar)