Re: [libvirt] mingw: test-poll pipe part fails
[adding libvirt] On 06/04/2011 12:24 AM, Paolo Bonzini wrote:
On Sat, Jun 4, 2011 at 00:37, Matthias Bolte <matthias.bolte@googlemail.com> wrote:
After testing a while and reading MSDN docs the problem seems to be that MsgWaitForMultipleObjects doesn't work on pipes. It doesn't actually wait but just returns immediately. Digging MSDN and googling about this suggest that there is no simple solution to this.
Yes, Windows pipes are that broken. :(
Using socketpair is a possibly good idea, but I would do it on libvirtd only. I don't know exactly how libvirtd uses this pipe, but perhaps it can be changed to an eventfd-like abstraction that can be used with both Windows and Unix. This is similar to Eric's suggestion, but without the pipe at all. It would also be a libvirtd-specific suggestion.
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal. Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 06/30/2011 07:56 PM, Eric Blake wrote:
Yes, Windows pipes are that broken.:(
Using socketpair is a possibly good idea, but I would do it on libvirtd only. I don't know exactly how libvirtd uses this pipe, but perhaps it can be changed to an eventfd-like abstraction that can be used with both Windows and Unix. This is similar to Eric's suggestion, but without the pipe at all. It would also be a libvirtd-specific suggestion.
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
Perhaps gnulib can provide an eventfd abstraction (or better, a slight variation that only returns 0/1) to be used for pipe-to-self. In Windows it can use an autoreset event, in Linux an eventfd, in Unix a pipe. Paolo
On Thu, Jun 30, 2011 at 11:56:15AM -0600, Eric Blake wrote:
[adding libvirt]
On 06/04/2011 12:24 AM, Paolo Bonzini wrote:
On Sat, Jun 4, 2011 at 00:37, Matthias Bolte <matthias.bolte@googlemail.com> wrote:
After testing a while and reading MSDN docs the problem seems to be that MsgWaitForMultipleObjects doesn't work on pipes. It doesn't actually wait but just returns immediately. Digging MSDN and googling about this suggest that there is no simple solution to this.
Yes, Windows pipes are that broken. :(
Using socketpair is a possibly good idea, but I would do it on libvirtd only. I don't know exactly how libvirtd uses this pipe, but perhaps it can be changed to an eventfd-like abstraction that can be used with both Windows and Unix. This is similar to Eric's suggestion, but without the pipe at all. It would also be a libvirtd-specific suggestion.
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
IIRC, we never resolved this for the last release. I think we should do as you suggest and just use socketpair() on Win32. Given the way in which libvirt uses these capabilities, I don't think the overheads of socketpair() vs pipe() are so onerous that we need worry about an even more fancy Win32 impl or eventfd for linux. Of course if someone wants todo a full job for gnulib meawhile, we won't complain... Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 07/19/2011 09:30 AM, Daniel P. Berrange wrote:
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
IIRC, we never resolved this for the last release. I think we should do as you suggest and just use socketpair() on Win32. Given the way in which libvirt uses these capabilities, I don't think the overheads of socketpair() vs pipe() are so onerous that we need worry about an even more fancy Win32 impl or eventfd for linux. Of course if someone wants todo a full job for gnulib meawhile, we won't complain...
Well, right now, we don't even have socketpair() for Win32. We'd have to wire up some other native APIs for this to work. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Tue, Jul 19, 2011 at 09:31:59AM -0600, Eric Blake wrote:
On 07/19/2011 09:30 AM, Daniel P. Berrange wrote:
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
IIRC, we never resolved this for the last release. I think we should do as you suggest and just use socketpair() on Win32. Given the way in which libvirt uses these capabilities, I don't think the overheads of socketpair() vs pipe() are so onerous that we need worry about an even more fancy Win32 impl or eventfd for linux. Of course if someone wants todo a full job for gnulib meawhile, we won't complain...
Well, right now, we don't even have socketpair() for Win32. We'd have to wire up some other native APIs for this to work.
Oh I read your mail above as indicating we did have socketpair() on Mingw32. A quick look through google, suggests most people go down the route of creating a TCP socket bound to localhost for this purpose. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
I have some code if you want that work. If you could test, I wil lsend you. I have no time to create m4 and proper gnulib integration. It is up to you Bastien On Tue, Jul 19, 2011 at 5:44 PM, Daniel P. Berrange <berrange@redhat.com> wrote:
On Tue, Jul 19, 2011 at 09:31:59AM -0600, Eric Blake wrote:
On 07/19/2011 09:30 AM, Daniel P. Berrange wrote:
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
IIRC, we never resolved this for the last release. I think we should do as you suggest and just use socketpair() on Win32. Given the way in which libvirt uses these capabilities, I don't think the overheads of socketpair() vs pipe() are so onerous that we need worry about an even more fancy Win32 impl or eventfd for linux. Of course if someone wants todo a full job for gnulib meawhile, we won't complain...
Well, right now, we don't even have socketpair() for Win32. We'd have to wire up some other native APIs for this to work.
Oh I read your mail above as indicating we did have socketpair() on Mingw32. A quick look through google, suggests most people go down the route of creating a TCP socket bound to localhost for this purpose.
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
n Tue, Jul 19, 2011 at 6:54 PM, Bastien ROUCARIES <roucaries.bastien@gmail.com> wrote:
I have some code if you want that work. If you could test, I wil lsend you. I have no time to create m4 and proper gnulib integration. It is up to you
Please test, I do not even have compiled it.
Bastien
On Tue, Jul 19, 2011 at 5:44 PM, Daniel P. Berrange <berrange@redhat.com> wrote:
On Tue, Jul 19, 2011 at 09:31:59AM -0600, Eric Blake wrote:
On 07/19/2011 09:30 AM, Daniel P. Berrange wrote:
I'm wondering if the problem here is that libvirt is trying to use the pipe-to-self mechanism as a fundamental event loop idiom. That is, the reason libvirt is calling poll is in order to minimize CPU until something interesting happens, where interesting includes needing to wake up a helper thread to do an action inside locks in response to the receipt of a signal.
Maybe you are on to something, and replacing all uses of pipe() with virPipeToSelf() (which uses pipe() for efficiency on Linux, but socketpair() on mingw), would allow libvirt to continue to use the pipe-to-self idiom while also using fds that can actually be poll'd on mingw.
IIRC, we never resolved this for the last release. I think we should do as you suggest and just use socketpair() on Win32. Given the way in which libvirt uses these capabilities, I don't think the overheads of socketpair() vs pipe() are so onerous that we need worry about an even more fancy Win32 impl or eventfd for linux. Of course if someone wants todo a full job for gnulib meawhile, we won't complain...
Well, right now, we don't even have socketpair() for Win32. We'd have to wire up some other native APIs for this to work.
Oh I read your mail above as indicating we did have socketpair() on Mingw32. A quick look through google, suggests most people go down the route of creating a TCP socket bound to localhost for this purpose.
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (4)
-
Bastien ROUCARIES -
Daniel P. Berrange -
Eric Blake -
Paolo Bonzini