[libvirt] With different libvirt version, lxc-enter-namespace return Different Execution results.
Hi Alex, I noticed you did lots of work about lxc-enter-namespace function. There is one thing really confusing me and I really want to consult with you about it. When I execute lxc-enter-namespace to get disk space info for lxc container under different libvirt version, It return different execution results. Details are as following: Under libvirt of version 1.0.2 with unbuntu host OS: # disk space info of Host ubuntu@lxc-gq:~$ df -hl Filesystem Size Used Avail Use% Mounted on /dev/loop7 20G 16G 3.2G 84% / udev 2.0G 12K 2.0G 1% /dev tmpfs 791M 300K 791M 1% /run none 5.0M 4.0K 5.0M 1% /run/lock none 2.0G 0 2.0G 0% /run/shm tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/nbd5 93M 1.6M 87M 2% /home/ubuntu/test ubuntu@lxc-gq:~$ vir version Compiled against library: libvirt 1.0.2 Using library: libvirt 1.0.2 Using API: LXC 1.0.2 Running hypervisor: LXC 3.8.0 ubuntu@lxc-gq:~$ vir list Id Name State ---------------------------------------------------- 3241 instance-0000004b running # disk space info of instance ubuntu@lxc-gq:~$ vir lxc-enter-namespace 3241 /bin/df -hl Filesystem Size Used Avail Use% Mounted on /dev/nbd1 1.4G 956M 329M 75% / devfs 64K 8.0K 56K 13% /dev tmpfs 64K 0 64K 0% /sys/fs/cgroup none 396M 12M 384M 3% /run none 5.0M 0 5.0M 0% /run/lock none 2.0G 0 2.0G 0% /run/shm Then I logged into the lxc container : ubuntu@lxc-gq:~$ ssh -i id_rsa_lxc ubuntu@10.0.0.4 Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.8.0-25-generic x86_64) * Documentation: https://help.ubuntu.com/ System information disabled due to load higher than 2.0 Get cloud support with Ubuntu Advantage Cloud Guest: http://www.ubuntu.com/business/services/cloud Use Juju to deploy your cloud instances and workloads: https://juju.ubuntu.com/#cloud-precise 0 packages can be updated. 0 updates are security updates. Last login: Thu Aug 8 07:30:04 2013 from 10.0.0.1 To run a command as administrator (user "root"), use "sudo <command>". See "man sudo_root" for details. ubuntu@lxc3:~$ df -hl Filesystem Size Used Avail Use% Mounted on /dev/nbd1 1.4G 956M 328M 75% / devfs 64K 8.0K 56K 13% /dev tmpfs 64K 0 64K 0% /sys/fs/cgroup none 396M 12M 384M 3% /run none 5.0M 0 5.0M 0% /run/lock none 2.0G 0 2.0G 0% /run/shm From above, We can see under libvirt version 1.0.2, through lxc-enter-namespace, disk space info can be obtained and it's the same as info got from lxc inside. But when I did the same thing under libvirt of version 1.1.0 with debian host OS, the execution result changed: #disk info of host hzguanqiang@debian:~$ df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs hzguanqiang@debian:~$ vir version Compiled against library: libvirt 1.1.0 Using library: libvirt 1.1.0 Using API: LXC 1.1.0 Running hypervisor: LXC 3.2.46 hzguanqiang@debian:~$ vir list Id Name State ---------------------------------------------------- 6096 instance-00000037 running 27981 instance-00000038 running 30846 instance-0000001a running hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 /bin/df -hl libvirt: error : argument unsupported: Security model cannot be entered hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 --noseclabel /bin/df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs Then I logged into the lxc container : hzguanqiang@debian:~$ ssh -i id_rsa ubuntu@10.160.70.2 Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.46-openstack-amd64 x86_64) * Documentation: https://help.ubuntu.com/ System information as of Wed Aug 7 10:49:35 UTC 2013 System load: 0.62 Processes: 31 Usage of /: 20.9% of 19.69GB Users logged in: 1 Memory usage: 51% IP address for eth0: 10.160.70.2 Swap usage: 0% Graph this data and manage this system at https://landscape.canonical.com/ Get cloud support with Ubuntu Advantage Cloud Guest: http://www.ubuntu.com/business/services/cloud Use Juju to deploy your cloud instances and workloads: https://juju.ubuntu.com/#cloud-precise Last login: Wed Aug 7 10:49:35 2013 from 10.160.70.3 ubuntu@lxc1:~$ df -hl Filesystem Size Used Avail Use% Mounted on /dev/loop2 20G 11G 8.3G 57% / devfs 64K 8.0K 56K 13% /dev tmpfs 64K 0 64K 0% /sys/fs/cgroup none 4.8G 6.4M 4.8G 1% /run none 5.0M 0 5.0M 0% /run/lock none 24G 0 24G 0% /run/shm From above, We can see under libvirt version 1.1.0, disk space info of lxc container gotten through lxc-enter-namespace are different from info got from lxc inside, but the same as host. Why does this happen? Is there a bug or anything? Thanks -------------------------------------------------------------------------------- ------------------ Best regards! GuanQiang
On Thu, Aug 08, 2013 at 03:56:35PM +0800, hzguanqiang@corp.netease.com wrote:
But when I did the same thing under libvirt of version 1.1.0 with debian host OS, the execution result changed:
#disk info of host hzguanqiang@debian:~$ df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 --noseclabel /bin/df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
What does 'ls /proc/self/ns' show on the host ? Also what is your guest XML configuration for 'intsance-0000000037' ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2013-08-08 16:53 , Daniel P. Berrange wrote: On Thu, Aug 08, 2013 at 03:56:35PM +0800, hzguanqiang@corp.netease.com wrote:
But when I did the same thing under libvirt of version 1.1.0 with debian host OS, the execution result changed:
#disk info of host hzguanqiang@debian:~$ df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 --noseclabel /bin/df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
What does 'ls /proc/self/ns' show on the host ? Also what is your guest XML configuration for 'intsance-0000000037' ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| Hi Daniel, The info you want to see are as following: hzguanqiang@debian:~$ ls /proc/self/ns ipc net uts hzguanqiang@debian:~$ vir dumpxml instance-00000037 <domain type='lxc' id='6096'> <name>instance-00000037</name> <uuid>2b39180a-cec0-4423-83e9-4446a27409ea</uuid> <memory unit='KiB'>10485760</memory> <currentMemory unit='KiB'>10485760</currentMemory> <vcpu placement='static'>8</vcpu> <cputune> <shares>32768</shares> <period>100000</period> <quota>400000</quota> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64'>exe</type> <init>/sbin/init</init> <cmdline>console=tty0 console=ttyS0</cmdline> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/lib/libvirt/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/opt/stack/data/nova/instances/2b39180a-cec0-4423-83e9-4446a27409ea/rootfs'/> <target dir='/'/> </filesystem> <interface type='bridge'> <mac address='fa:16:3e:01:f1:14'/> <source bridge='br100'/> <target dev='veth0'/> <filterref filter='nova-instance-instance-00000037-fa163e01f114'/> </interface> <console type='pty' tty='/dev/pts/2'> <source path='/dev/pts/2'/> <target type='lxc' port='0'/> <alias name='console0'/> </console> </devices> <seclabel type='none'/> </domain> ------------------ Best regards! GuanQiang 16:58:07
On Thu, Aug 08, 2013 at 05:00:38PM +0800, hzguanqiang@corp.netease.com wrote:
On 2013-08-08 16:53 , Daniel P. Berrange wrote: On Thu, Aug 08, 2013 at 03:56:35PM +0800, hzguanqiang@corp.netease.com wrote:
But when I did the same thing under libvirt of version 1.1.0 with debian host OS, the execution result changed:
#disk info of host hzguanqiang@debian:~$ df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 --noseclabel /bin/df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
What does 'ls /proc/self/ns' show on the host ?
Also what is your guest XML configuration for 'intsance-0000000037' ?
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Hi Daniel,
The info you want to see are as following:
hzguanqiang@debian:~$ ls /proc/self/ns ipc net uts
That's your problem. The kernel you are running on is too old to support attaching to the mount namespace. You need to see 'mnt' in the files there, and also 'pid' and 'user' too for completeness. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2013-08-08 17:06 , Daniel P. Berrange wrote: On Thu, Aug 08, 2013 at 05:00:38PM +0800, hzguanqiang@corp.netease.com wrote:
On 2013-08-08 16:53 , Daniel P. Berrange wrote: On Thu, Aug 08, 2013 at 03:56:35PM +0800, hzguanqiang@corp.netease.com wrote:
But when I did the same thing under libvirt of version 1.1.0 with debian host OS, the execution result changed:
#disk info of host hzguanqiang@debian:~$ df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
hzguanqiang@debian:~$ vir lxc-enter-namespace instance-00000037 --noseclabel /bin/df -hl Filesystem Size Used Avail Use% Mounted on rootfs 4.0G 1.6G 2.2G 42% / udev 10M 0 10M 0% /dev tmpfs 4.8G 468K 4.8G 1% /run /dev/disk/by-uuid/12bb656b-4f80-4386-aa07-4bf90a3111b4 4.0G 1.6G 2.2G 42% / tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 9.9G 0 9.9G 0% /run/shm /dev/sda8 428G 112G 313G 27% /home /dev/sda6 7.9G 159M 7.8G 2% /tmp /dev/sda5 16G 2.7G 14G 17% /var cgroup_root 24G 0 24G 0% /sys/fs/cgroup /dev/nbd5 1.4G 851M 466M 65% /home/openstack/stack/data/nova/instances/eed45ccb-3707-4815-9ae4-c7a2f56adb64/rootfs
What does 'ls /proc/self/ns' show on the host ?
Also what is your guest XML configuration for 'intsance-0000000037' ?
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Hi Daniel,
The info you want to see are as following:
hzguanqiang@debian:~$ ls /proc/self/ns ipc net uts
That's your problem. The kernel you are running on is too old to support attaching to the mount namespace. You need to see 'mnt' in the files there, and also 'pid' and 'user' too for completeness. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| Ok, Daniel, I got it. Thanks very much. ------------------ Best regards! GuanQiang 17:15:46
On 08/08/2013 03:16 AM, hzguanqiang@corp.netease.com wrote:
That's your problem. The kernel you are running on is too old to support attaching to the mount namespace. You need to see 'mnt' in the files there, and also 'pid' and 'user' too for completeness.
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| Ok, Daniel, I got it. Thanks very much.
Note that your mailer is non-compliant - most mailers will strip the signature footer after the '-- ' line. As a result of you writing your message after the '-- ' line, your entire reply appeared as the footer of the message (which in my mail view shows up in a different font) - I nearly missed your reply; I also had a very difficult time replying to your message to tell you about your problem, as my mailer tried to strip your reply as part of the footer. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (3)
-
Daniel P. Berrange -
Eric Blake -
hzguanqiang＠corp.netease.com