On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote: > Now we mount /dev as tmpfs and haven't created directory > /dev/shm,so the glibc api such as shm_open/sem_open will > create files under dir /dev.(since /dev is mounted as tmpfs) > > Through these api still useable in container,but this cause > directory /dev looks a little chaos. > > This patch create directory /dev/shm automatically,the files > created by shm_open/sem_open will stay in this directroy. > > Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com&gt; > --- > src/lxc/lxc_container.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c > index 497539c..5150564 100644 > --- a/src/lxc/lxc_container.c > +++ b/src/lxc/lxc_container.c > @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, > "devfs", "/dev", "tmpfs", opts); > goto cleanup; > } > + > + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); > + if (virFileMakePath("/dev/shm") < 0) { > + virReportSystemError(errno, "%s", > + _("Failed to mkdir /dev/shm")); > + goto cleanup; > + } > } > > rc = 0; I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.

On 2013/01/29 07:52, Kamezawa Hiroyuki wrote: > (2013/01/28 19:36), Daniel P. Berrange wrote: >> On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote: >>> Now we mount /dev as tmpfs and haven't created directory >>> /dev/shm,so the glibc api such as shm_open/sem_open will >>> create files under dir /dev.(since /dev is mounted as tmpfs) >>> >>> Through these api still useable in container,but this cause >>> directory /dev looks a little chaos. >>> >>> This patch create directory /dev/shm automatically,the files >>> created by shm_open/sem_open will stay in this directroy. >>> >>> Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com&gt; >>> --- >>> src/lxc/lxc_container.c | 7 +++++++ >>> 1 file changed, 7 insertions(+) >>> >>> diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c >>> index 497539c..5150564 100644 >>> --- a/src/lxc/lxc_container.c >>> +++ b/src/lxc/lxc_container.c >>> @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, >>> "devfs", "/dev", "tmpfs", opts); >>> goto cleanup; >>> } >>> + >>> + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); >>> + if (virFileMakePath("/dev/shm") < 0) { >>> + virReportSystemError(errno, "%s", >>> + _("Failed to mkdir /dev/shm")); >>> + goto cleanup; >>> + } >>> } >>> >>> rc = 0; >> >> I we probably want to mount a separate tmpfs on /dev/shm really, so we can >> do resource limits on /dev and /dev/shm separately. > > I agree. > BTW, if the user wants to tune limit of /dev/shm size, he need to add > > <filesystem type='ram'> > <source usage='XXXXX'/> > <target dir='/dev/shm'/> > </filesystem> > > ? > > How do you think suitable limit for default should be ? > half of memory limit of a container ? > > I think we needn't consider about this problem,The root user of the container should do this job. For libvirt lxc,we only need to limit the memory resource that the container uses.And we needn't setup any xml configuration too. Or maybe I misunderstand what you mean?

On 2013/01/29 12:09, Kamezawa Hiroyuki wrote: > (2013/01/29 11:16), Gao feng wrote: >> On 2013/01/29 07:52, Kamezawa Hiroyuki wrote: >>> (2013/01/28 19:36), Daniel P. Berrange wrote: >>>> On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote: >>>>> Now we mount /dev as tmpfs and haven't created directory >>>>> /dev/shm,so the glibc api such as shm_open/sem_open will >>>>> create files under dir /dev.(since /dev is mounted as tmpfs) >>>>> >>>>> Through these api still useable in container,but this cause >>>>> directory /dev looks a little chaos. >>>>> >>>>> This patch create directory /dev/shm automatically,the files >>>>> created by shm_open/sem_open will stay in this directroy. >>>>> >>>>> Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com&gt; >>>>> --- >>>>> src/lxc/lxc_container.c | 7 +++++++ >>>>> 1 file changed, 7 insertions(+) >>>>> >>>>> diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c >>>>> index 497539c..5150564 100644 >>>>> --- a/src/lxc/lxc_container.c >>>>> +++ b/src/lxc/lxc_container.c >>>>> @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, >>>>> "devfs", "/dev", "tmpfs", opts); >>>>> goto cleanup; >>>>> } >>>>> + >>>>> + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); >>>>> + if (virFileMakePath("/dev/shm") < 0) { >>>>> + virReportSystemError(errno, "%s", >>>>> + _("Failed to mkdir /dev/shm")); >>>>> + goto cleanup; >>>>> + } >>>>> } >>>>> >>>>> rc = 0; >>>> >>>> I we probably want to mount a separate tmpfs on /dev/shm really, so we can >>>> do resource limits on /dev and /dev/shm separately. >>> >>> I agree. >>> BTW, if the user wants to tune limit of /dev/shm size, he need to add >>> >>> <filesystem type='ram'> >>> <source usage='XXXXX'/> >>> <target dir='/dev/shm'/> >>> </filesystem> >>> >>> ? >>> >>> How do you think suitable limit for default should be ? >>> half of memory limit of a container ? >>> >>> >> >> I think we needn't consider about this problem,The root user >> of the container should do this job. >> >> For libvirt lxc,we only need to limit the memory resource that >> the container uses.And we needn't setup any xml configuration too. >> >> Or maybe I misunderstand what you mean? >> > > "How large you can make files on a fs" is different from memcg's limit. > > IIUC, tmpfs's size is detemined as the half of system memory regardless of > the configuration of a domain. So, > > - if a domain is enough big, shm's memory usage will hit tmpfs' limit before > memcg's. > - if a domain is enough small, shm's memory usage will hit memcg's limit before > tmpfs's. > > I think it's complicated and hard to use from viewpoint of application in a > container. > > IMHO, tmpfs's limit size should be same to <memory> tag at default. same or half of memory size of container? it looks like kernel set tmpfs's size to half of physical ram by default.

On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote: > Now we mount /dev as tmpfs and haven't created directory > /dev/shm,so the glibc api such as shm_open/sem_open will > create files under dir /dev.(since /dev is mounted as tmpfs) > > Through these api still useable in container,but this cause > directory /dev looks a little chaos. > > This patch create directory /dev/shm automatically,the files > created by shm_open/sem_open will stay in this directroy. > > Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com&gt; > --- > src/lxc/lxc_container.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c > index 497539c..5150564 100644 > --- a/src/lxc/lxc_container.c > +++ b/src/lxc/lxc_container.c > @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, > "devfs", "/dev", "tmpfs", opts); > goto cleanup; > } > + > + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); > + if (virFileMakePath("/dev/shm") < 0) { > + virReportSystemError(errno, "%s", > + _("Failed to mkdir /dev/shm")); > + goto cleanup; > + } > } > > rc = 0; I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.

I wonder if /dev should be devtmpfs instead of plain tmpfs too.