5:44 p.m.
wow! i was starting to think i would never get past this problem!
[root@PHS-001 ~]# echo $VIRSH_DEFAULT_CONNECT_URI
lxc:///
[root@PHS-001 ~]# virsh create /vps/def/exec/sys/arch-nano.xml
Domain arch-nano created from /vps/def/exec/sys/arch-nano.xml
[root@PHS-001 ~]# virsh console arch-nano
Connected to domain arch-nano
Escape character is ^]
#
[root@PHS-001 ~]# grep cgroup /proc/mounts
none /cgroup cgroup rw,relatime,devices,memory,cpuacct,cpu 0 0
everything seems to be working fine now after disabling "ns" from
cgroup mount; thank you very much Ryota and Daniel for you time and
assistance. i must have sunk 40+ hours into this, but i learned a
tremendous amount, so nothing is lost. :-)
this is somewhat of a digression, and i can submit a new thread if
need be, but my last remaining question is about the <filesystem>
tag/root mounting. i have not found any mention of the <filesystem>
tag in the docs or how to work with it... i want to have libvirt mount
a specified device (a btrfs subvolume) and use it for the root
filesystem of the created LXC container. is there a way to do this?
perhaps <source dev="/dev/sdb" options="subvol....."/>, or do i
need
to use the storage XML? i didnt see a way to use a storage/device
definition as the rootfs.
im eager to get working with python bindings, just trying to see it
all work with XML first.
thanks++
On Thu, Dec 10, 2009 at 6:45 AM, Ryota Ozaki <ozaki.ryota(a)gmail.com> wrote:
On Thu, Dec 10, 2009 at 9:36 PM, Daniel P. Berrange
<berrange(a)redhat.com> wrote:
> On Thu, Dec 10, 2009 at 09:26:39PM +0900, Ryota Ozaki wrote:
>> On Thu, Dec 10, 2009 at 9:03 PM, Daniel P. Berrange <berrange(a)redhat.com>
wrote:
>> > On Thu, Dec 10, 2009 at 02:22:37AM -0600, Tony Risinger wrote:
>> >> i'm trying to get even the simplest busybox container with
libvirt+LXC
>> >> with very limited success. I feel l am missing something supremely
>> >> simple for me to be hung on this for weeks.
>> >>
>> >> i dont see anything interesting in domain log, but getting this error
>> >> from "LIBVIRT_DEBUG=1 libvirtd":
>> >>
>> >> 05:27:56.113: error : lxcDomainGetInfo:462 : internal error Unable to
>> >> get cgroup for arch-nano
>> >> 05:27:56.113: debug : virDomainFree:2004 : domain=0x81d8e68
>> >> 05:27:56.113: debug : virUnrefDomain:422 : unref domain 0x81d8e68
arch-nano 1
>> >> 05:27:56.113: debug : virReleaseDomain:376 : release domain 0x81d8e68
arch-nano
>> >> 05:27:56.113: debug : virReleaseDomain:392 : unref connection 0x81dc0f0
2
>> >> 05:27:56.113: debug : remoteSerializeError:141 : prog=536903814 ver=1
>> >> proc=16 type=1 serial=4, msg=internal error Unable to get cgroup for
>> >> arch-nano
>> >>
>> >> i've been using this root filesystem layout:
>> >>
>> >> [root@PHS-001 arch-nano]# tree
>> >> .
>> >> |-- bin
>> >> | |-- cat -> ../sbin/busybox
>> >> | |-- chdir -> ../sbin/busybox
>> >> | |-- chmod -> ../sbin/busybox
>> >> | |-- ls -> ../sbin/busybox
>> >> | |-- rm -> ../sbin/busybox
>> >> | |-- sh -> ../sbin/busybox
>> >> | `-- vi -> ../sbin/busybox
>> >> |-- dev
>> >> | `-- pts
>> >> |-- etc
>> >> |-- proc
>> >> |-- sbin
>> >> | |-- busybox
>> >> | `-- init -> busybox
>> >> `-- sys
>> >>
>> >> all folders besides /bin and /sbin were created by libvirt. i tried
>> >> using the /sbin/init script previously suggested:
>> >>
>> >> #!/sbin/busybox
>> >> sh
>> >
>> >
>> > Sorry, my suggestion was wrong. I forgot that if you have #!/sbin/busybox
>> > it will attempt to execute the command matching the name of the script.
>> > So it will in fact try to run 'init', rather than 'sh'.
>> >
>> > Just make the libvirt XML point directly to /bin/sh instead and it
>> > should work. I even tested it this time :-)
>>
>> Hem, I still have a problem with ns subsystem enabled. Yes, I can launch
>> a container however the cgroup hierarchy is wrong from libvirtd expecting
>> like:
>>
>> /: libvirtd --daemon
>> /5345: /usr/libexec/libvirt_lxc --name
>>
>> Daniel, could you confirm how about your cgroup hierarchy?
>
> What you do mean by 'ns' subsystem ?
'ns' is one of functions of cgroups like such as devices, memory,
cpu, etc. and it is enabled if you mount cgroup without any options
that Tony is doing.
>
>
> # grep cgroup /proc/mounts
> cgroup /dev/cgroups/cpu cgroup rw,relatime,cpuacct,cpu 0 0
> cgroup /dev/cgroups/memory cgroup rw,relatime,memory 0 0
> cgroup /dev/cgroups/devices cgroup rw,relatime,devices 0 0
Oh, you don't enable 'ns', so yes, things go fine in your environment.
>
> # cat /proc/`pgrep libvirtd`/cgroup
> 32:devices:/sysdefault
> 16:memory:/sysdefault
> 12:cpuacct,cpu:/sysdefault
>
> # cat /proc/`pgrep libvirt_lxc`/cgroup
> 32:devices:/sysdefault/libvirt/lxc/vm1
> 16:memory:/sysdefault/libvirt/lxc/vm1
> 12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1
>
>
> And the process inside the contanier is PID 12309
>
> # cat /proc/12309/cgroup
> 32:devices:/sysdefault/libvirt/lxc/vm1
> 16:memory:/sysdefault/libvirt/lxc/vm1
> 12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1
>
>
> Which all appears to be correct to me
>
> This is on a Fedora 12 host 2.6.31.6-145.fc12.i686.PAE with
>
> CONFIG_UTS_NS=y
> CONFIG_IPC_NS=y
> CONFIG_USER_NS=y
> CONFIG_PID_NS=y
> CONFIG_NET_NS=y
> CONFIG_CGROUP_SCHED=y
> CONFIG_CGROUPS=y
> # CONFIG_CGROUP_DEBUG is not set
> CONFIG_CGROUP_NS=y
This is the function I'm mentioning.
ozaki-r
> CONFIG_CGROUP_FREEZER=y
> CONFIG_CGROUP_DEVICE=y
> CONFIG_CGROUP_CPUACCT=y
> CONFIG_CGROUP_MEM_RES_CTLR=y
> CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y
> CONFIG_NET_CLS_CGROUP=y
>
>
> Regards,
> Daniel
> --
> |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
> |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
>