2:28 p.m.
Dear all,
In my system, two containers need to exchange data as quick as possible and the two
containers are hosted by the same physical machine, I wonder if socket is the only method
for communication between containers?
Thank you.
Cheng Wang
6:29 p.m.
On Thu, Feb 27, 2014 at 06:28:46AM +0000, WANG Cheng D wrote:
Dear all,
In my system, two containers need to exchange data as quick as possible
and the two containers are hosted by the same physical machine, I wonder
if socket is the only method for communication between containers?
If you have an area of filesystem that is shared + writable to both
containers, they could create UNIX sockets in that filesystem, rather
than TCP sockets. Also they could create named FIFO pipes.
Another option would be to create a shared memory backed file in shmfs
or hugepagefs and expose it to both containers.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
9:39 p.m.
Dear Daniel,
Thank you for your email.
According to your suggestion, I tried to use the shared memory filesystem for data
exchange between the host machine and the container. The steps are as follows:
1) create an empty file in the host machine (/tmp1/file1).
2) run #chmod 777 /tmp1 and #chmod 777 /tmp1/file1.
3) expose /tmp1 to the container by configuring the container with the following XML
scripts.
<filesystem type="mount" accessmode="passthrough">
<source dir="/tmp1"/>
<target dir="/tmp1"/>
</filesystem>
4) write two programs: program1 writes a string to the shared memory
program2 reads the string from the shared memory
and compile both programs in the host machine.
both of the two programs use ftok() and shmget() to obtain the shmfs id.
5) a. if we run both program1 and program2 in the host machine, the program2 can read the
string that program1 has written.
b. if we run both program1 and program2 in the container, both programs also work well.
c. if we run program1 in the host and run program2 in the container, program2 fails. the
ftok() function can return the expected value, but shmget(...,0,0) fails and prompts
"No such files". It seems that the exposed file cannot be recognized by shmget()
function, although ftok() can work and return the value.
Do you have an idea?
Best wishes.
Cheng Wang
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: 2014年2月27日 18:30
To: WANG Cheng D
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] method for communication between containers
On Thu, Feb 27, 2014 at 06:28:46AM +0000, WANG Cheng D wrote:
Dear all,
In my system, two containers need to exchange data as quick as
possible and the two containers are hosted by the same physical
machine, I wonder if socket is the only method for communication between containers?
If you have an area of filesystem that is shared + writable to both containers, they could
create UNIX sockets in that filesystem, rather than TCP sockets. Also they could create
named FIFO pipes.
Another option would be to create a shared memory backed file in shmfs or hugepagefs and
expose it to both containers.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
9:42 p.m.
On Mon, Mar 03, 2014 at 01:39:03PM +0000, WANG Cheng D wrote:
Dear Daniel,
Thank you for your email.
According to your suggestion, I tried to use the shared memory filesystem for data
exchange between the host machine and the container. The steps are as follows:
1) create an empty file in the host machine (/tmp1/file1).
2) run #chmod 777 /tmp1 and #chmod 777 /tmp1/file1.
3) expose /tmp1 to the container by configuring the container with the following XML
scripts.
<filesystem type="mount" accessmode="passthrough">
<source dir="/tmp1"/>
<target dir="/tmp1"/>
</filesystem>
4) write two programs: program1 writes a string to the shared memory
program2 reads the string from the shared memory
and compile both programs in the host machine.
both of the two programs use ftok() and shmget() to obtain the shmfs id.
5) a. if we run both program1 and program2 in the host machine, the program2 can read the
string that program1 has written.
b. if we run both program1 and program2 in the container, both programs also work
well.
c. if we run program1 in the host and run program2 in the container,
program2 fails. the ftok() function can return the expected value,
but shmget(...,0,0) fails and prompts "No such files". It seems that
the exposed file cannot be recognized by shmget() function, although
ftok() can work and return the value.
Do you have an idea?
Yes, the IPCS namespace causes the container's 'shmget' call to be isolated
from the host, so using shmget & other similar SysV-IPC APIs will fail.
What I was thinking though was that you'd be able to open() the file in
the shared memory fs, and then use mmap() to map its data into RAM in
the container.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:19 p.m.
Dear Daniel,
Your suggestion is right. I followed your method and used mmap(), it works well. Now I can
realize data exchange between containers.
Thank you for your help.
Cheng
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: 2014年3月3日 21:42
To: WANG Cheng D
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] method for communication between containers
On Mon, Mar 03, 2014 at 01:39:03PM +0000, WANG Cheng D wrote:
Dear Daniel,
Thank you for your email.
According to your suggestion, I tried to use the shared memory filesystem for data
exchange between the host machine and the container. The steps are as follows:
1) create an empty file in the host machine (/tmp1/file1).
2) run #chmod 777 /tmp1 and #chmod 777 /tmp1/file1.
3) expose /tmp1 to the container by configuring the container with the following XML
scripts.
<filesystem type="mount" accessmode="passthrough">
<source dir="/tmp1"/>
<target dir="/tmp1"/>
</filesystem>
4) write two programs: program1 writes a string to the shared memory
program2 reads the string from the shared memory
and compile both programs in the host machine.
both of the two programs use ftok() and shmget() to obtain the shmfs id.
5) a. if we run both program1 and program2 in the host machine, the program2 can read the
string that program1 has written.
b. if we run both program1 and program2 in the container, both programs also work
well.
c. if we run program1 in the host and run program2 in the container,
program2 fails. the ftok() function can return the expected value,
but shmget(...,0,0) fails and prompts "No such files". It seems that
the exposed file cannot be recognized by shmget() function, although
ftok() can work and return the value.
Do you have an idea?
Yes, the IPCS namespace causes the container's 'shmget' call to be isolated
from the host, so using shmget & other similar SysV-IPC APIs will fail.
What I was thinking though was that you'd be able to open() the file in the shared
memory fs, and then use mmap() to map its data into RAM in the container.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
1:55 p.m.
Dear Daniel,
As you know, following your suggestion, I can expose shared memory to two containers
hosted by the same machine, thus the two containers can exchange data. Now I have a new
problem: how does the containers avoid concurrent accesses to the shared memory? In a
native system, this can be achieved by using semaphore. But I don't know how the two
containers can see the same semaphore. Do I need to design a mechanism to control the
access to the shared memory?
With my warmest regards,
Cheng
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: 2014年3月3日 21:42
To: WANG Cheng D
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] method for communication between containers
On Mon, Mar 03, 2014 at 01:39:03PM +0000, WANG Cheng D wrote:
Dear Daniel,
Thank you for your email.
According to your suggestion, I tried to use the shared memory filesystem for data
exchange between the host machine and the container. The steps are as follows:
1) create an empty file in the host machine (/tmp1/file1).
2) run #chmod 777 /tmp1 and #chmod 777 /tmp1/file1.
3) expose /tmp1 to the container by configuring the container with the following XML
scripts.
<filesystem type="mount" accessmode="passthrough">
<source dir="/tmp1"/>
<target dir="/tmp1"/>
</filesystem>
4) write two programs: program1 writes a string to the shared memory
program2 reads the string from the shared memory
and compile both programs in the host machine.
both of the two programs use ftok() and shmget() to obtain the shmfs id.
5) a. if we run both program1 and program2 in the host machine, the program2 can read the
string that program1 has written.
b. if we run both program1 and program2 in the container, both programs also work
well.
c. if we run program1 in the host and run program2 in the container,
program2 fails. the ftok() function can return the expected value,
but shmget(...,0,0) fails and prompts "No such files". It seems that
the exposed file cannot be recognized by shmget() function, although
ftok() can work and return the value.
Do you have an idea?
Yes, the IPCS namespace causes the container's 'shmget' call to be isolated
from the host, so using shmget & other similar SysV-IPC APIs will fail.
What I was thinking though was that you'd be able to open() the file in the shared
memory fs, and then use mmap() to map its data into RAM in the container.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
5:07 p.m.
On Wed, Apr 09, 2014 at 05:55:57AM +0000, WANG Cheng D wrote:
Dear Daniel,
As you know, following your suggestion, I can expose shared memory to
two containers hosted by the same machine, thus the two containers can
exchange data. Now I have a new problem: how does the containers avoid
concurrent accesses to the shared memory? In a native system, this can
be achieved by using semaphore. But I don't know how the two containers
can see the same semaphore. Do I need to design a mechanism to control
the access to the shared memory?
Correct, there's no way to share semaphores. Since you hav shared
memory though, can you use a POSIX process shared mutex that is
allocated in that shared memory block.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3861
days inactive
3902
days old
6 comments
2 participants
participants (2)
-
Daniel P. Berrange -
WANG Cheng D