[libvirt] [PATCH] LXC: create directory /dev/shm automatically
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs) Through these api still useable in container,but this cause directory /dev looks a little chaos. This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy. Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } } rc = 0; -- 1.7.11.7
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately. I wonder if /dev should be devtmpfs instead of plain tmpfs too. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add <filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem> ? How do you think suitable limit for default should be ? half of memory limit of a container ? THanks, -Kame
On 2013/01/29 07:52, Kamezawa Hiroyuki wrote:
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add
<filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem>
?
How do you think suitable limit for default should be ? half of memory limit of a container ?
I think we needn't consider about this problem,The root user of the container should do this job. For libvirt lxc,we only need to limit the memory resource that the container uses.And we needn't setup any xml configuration too. Or maybe I misunderstand what you mean? Thanks
(2013/01/29 11:16), Gao feng wrote:
On 2013/01/29 07:52, Kamezawa Hiroyuki wrote:
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add
<filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem>
?
How do you think suitable limit for default should be ? half of memory limit of a container ?
I think we needn't consider about this problem,The root user of the container should do this job.
For libvirt lxc,we only need to limit the memory resource that the container uses.And we needn't setup any xml configuration too.
Or maybe I misunderstand what you mean?
"How large you can make files on a fs" is different from memcg's limit. IIUC, tmpfs's size is detemined as the half of system memory regardless of the configuration of a domain. So, - if a domain is enough big, shm's memory usage will hit tmpfs' limit before memcg's. - if a domain is enough small, shm's memory usage will hit memcg's limit before tmpfs's. I think it's complicated and hard to use from viewpoint of application in a container. IMHO, tmpfs's limit size should be same to <memory> tag at default. Thanks, -Kame
On 2013/01/29 12:09, Kamezawa Hiroyuki wrote:
(2013/01/29 11:16), Gao feng wrote:
On 2013/01/29 07:52, Kamezawa Hiroyuki wrote:
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add
<filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem>
?
How do you think suitable limit for default should be ? half of memory limit of a container ?
I think we needn't consider about this problem,The root user of the container should do this job.
For libvirt lxc,we only need to limit the memory resource that the container uses.And we needn't setup any xml configuration too.
Or maybe I misunderstand what you mean?
"How large you can make files on a fs" is different from memcg's limit.
IIUC, tmpfs's size is detemined as the half of system memory regardless of the configuration of a domain. So,
- if a domain is enough big, shm's memory usage will hit tmpfs' limit before memcg's. - if a domain is enough small, shm's memory usage will hit memcg's limit before tmpfs's.
I think it's complicated and hard to use from viewpoint of application in a container.
IMHO, tmpfs's limit size should be same to <memory> tag at default.
same or half of memory size of container? it looks like kernel set tmpfs's size to half of physical ram by default. I will look in devtmpfs and the limit of tmpfs. Thanks!
(2013/01/29 16:27), Gao feng wrote:
On 2013/01/29 12:09, Kamezawa Hiroyuki wrote:
(2013/01/29 11:16), Gao feng wrote:
On 2013/01/29 07:52, Kamezawa Hiroyuki wrote:
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add
<filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem>
?
How do you think suitable limit for default should be ? half of memory limit of a container ?
I think we needn't consider about this problem,The root user of the container should do this job.
For libvirt lxc,we only need to limit the memory resource that the container uses.And we needn't setup any xml configuration too.
Or maybe I misunderstand what you mean?
"How large you can make files on a fs" is different from memcg's limit.
IIUC, tmpfs's size is detemined as the half of system memory regardless of the configuration of a domain. So,
- if a domain is enough big, shm's memory usage will hit tmpfs' limit before memcg's. - if a domain is enough small, shm's memory usage will hit memcg's limit before tmpfs's.
I think it's complicated and hard to use from viewpoint of application in a container.
IMHO, tmpfs's limit size should be same to <memory> tag at default.
same or half of memory size of container? it looks like kernel set tmpfs's size to half of physical ram by default.
Yes. And It's a random value from container's point of view. Thanks, -Kame
On Tue, Jan 29, 2013 at 08:52:45AM +0900, Kamezawa Hiroyuki wrote:
(2013/01/28 19:36), Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
I agree. BTW, if the user wants to tune limit of /dev/shm size, he need to add
<filesystem type='ram'> <source usage='XXXXX'/> <target dir='/dev/shm'/> </filesystem>
Oh true, I forgot we added that. That is fine.
How do you think suitable limit for default should be ? half of memory limit of a container ?
I think /dev can just be limited to say 1 MB - it should only ever contain device nodes which have little space requirement. If someone actually wants to be using SHM, then they shuld explicitly configure /dev/shm in the XML with a suitable limit. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 2013/01/28 18:36, Daniel P. Berrange wrote:
On Mon, Jan 28, 2013 at 02:37:11PM +0800, Gao feng wrote:
Now we mount /dev as tmpfs and haven't created directory /dev/shm,so the glibc api such as shm_open/sem_open will create files under dir /dev.(since /dev is mounted as tmpfs)
Through these api still useable in container,but this cause directory /dev looks a little chaos.
This patch create directory /dev/shm automatically,the files created by shm_open/sem_open will stay in this directroy.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 497539c..5150564 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -586,6 +586,13 @@ static int lxcContainerMountBasicFS(bool pivotRoot, "devfs", "/dev", "tmpfs", opts); goto cleanup; } + + VIR_DEBUG("create directory /dev/shm for POSIX shared memory and named semphore"); + if (virFileMakePath("/dev/shm") < 0) { + virReportSystemError(errno, "%s", + _("Failed to mkdir /dev/shm")); + goto cleanup; + } }
rc = 0;
I we probably want to mount a separate tmpfs on /dev/shm really, so we can do resource limits on /dev and /dev/shm separately.
Yes, will update this patch.
I wonder if /dev should be devtmpfs instead of plain tmpfs too.
It seems mounting /dev as devtmpfs will make host's devices leak to container,devtmpfs only has one super-block.it's need kernel to support this feature. I think this is a long term plan.
participants (3)
-
Daniel P. Berrange -
Gao feng -
Kamezawa Hiroyuki