[libvirt] libvirtaio: libvirt-python asyncio event impl

Hello, libvirt-list, I wrote an event implementation wrapping an asyncio event loop [1]. I would like to contribute it back to libvirt-python, to be offered alongside the Default Impl in C. [1] https://github.com/woju/libvirtaio Also it contains a workaround for a memory leak around the custom loop implementation: https://github.com/woju/libvirtaio/blob/8685cfc/libvirtaio.py#L66-L92 How should I submit this? As a patch against libvirt-python.git, appending to libvirt-override.py? -- pozdrawiam / best regards _.-._ Wojtek Porczyk .-^' '^-. Invisible Things Lab |'-.-^-.-'| | | | | I do not fear computers, | '-.-' | I fear lack of them. '-._ : ,-' -- Isaac Asimov `^-^-_>

On Thu, Mar 16, 2017 at 05:30:40PM +0100, Wojtek Porczyk wrote:
Hello, libvirt-list,
I wrote an event implementation wrapping an asyncio event loop [1]. I would like to contribute it back to libvirt-python, to be offered alongside the Default Impl in C.
[1] https://github.com/woju/libvirtaio
Also it contains a workaround for a memory leak around the custom loop implementation: https://github.com/woju/libvirtaio/blob/8685cfc/libvirtaio.py#L66-L92
How should I submit this? As a patch against libvirt-python.git, appending to libvirt-override.py?
IIUC, the asyncio module you're using is new in python 3.4 ? The libvirt python code aims to be compatible with python >= 2.6 at this time, so there's a mis-match there. So if we did include that in the main libvirt python module, we'd have to figure out a suitable approach to avoid trying to import the non-existant module when people do "import libvirt". Or we could just leave it as a standalone python module and document its existence on our python binding page (http://libvirt.org/python.html) Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|

On Thu, Mar 16, 2017 at 04:38:23PM +0000, Daniel P. Berrange wrote:
On Thu, Mar 16, 2017 at 05:30:40PM +0100, Wojtek Porczyk wrote:
Hello, libvirt-list,
I wrote an event implementation wrapping an asyncio event loop [1]. I would like to contribute it back to libvirt-python, to be offered alongside the Default Impl in C.
[1] https://github.com/woju/libvirtaio
Also it contains a workaround for a memory leak around the custom loop implementation: https://github.com/woju/libvirtaio/blob/8685cfc/libvirtaio.py#L66-L92
How should I submit this? As a patch against libvirt-python.git, appending to libvirt-override.py?
IIUC, the asyncio module you're using is new in python 3.4 ? The libvirt python code aims to be compatible with python >= 2.6 at this time, so there's a mis-match there.
So if we did include that in the main libvirt python module, we'd have to figure out a suitable approach to avoid trying to import the non-existant module when people do "import libvirt".
That's right, asyncio (and this module) is compatible with 3.3 (mainly because of "yield from" syntax) and included in standard library from 3.4. We can ship it as separate module (like libvirt vs libvirtmod) and do something like this at the beginning on libvirt-override.py: try: from libvirtaio import LibvirtAsyncIOEventImpl # assuming it won't get renamed in review __all__ += ['LibvirtAsyncIOEventImpl'] except (ImportError, SyntaxError): pass (Just tested on 2.7). People will probably import this unconditionally, since they know, what event loop they want to run. In case of doubt, they always can: import libvirt try: libvirt.LibvirtAsyncIOEventImpl().register() except AttributeError: libvirt.virEventRegisterDefaultImpl() def event_loop(): while True: libvirt.virEventRunDefaultImpl() # ... import threading threading.Thread(target=event_loop).start() execute_ff_callback() can be merged regardless, since it is probably compatible with 2.5 (that's when ctypes arrived).
Or we could just leave it as a standalone python module and document its existence on our python binding page (http://libvirt.org/python.html)
I'd be glad if you merged this module, since 1) it would be more accessible, 2) won't break between releases (currently it is in danger because of the opaque object handling). Since we would be (at least now) the only user of this, I volunteer as tester/patch submitter against this part of code. -- pozdrawiam / best regards _.-._ Wojtek Porczyk .-^' '^-. Invisible Things Lab |'-.-^-.-'| | | | | I do not fear computers, | '-.-' | I fear lack of them. '-._ : ,-' -- Isaac Asimov `^-^-_>

On Thu, Mar 16, 2017 at 06:24:46PM +0100, Wojtek Porczyk wrote:
On Thu, Mar 16, 2017 at 04:38:23PM +0000, Daniel P. Berrange wrote:
On Thu, Mar 16, 2017 at 05:30:40PM +0100, Wojtek Porczyk wrote:
Hello, libvirt-list,
I wrote an event implementation wrapping an asyncio event loop [1]. I would like to contribute it back to libvirt-python, to be offered alongside the Default Impl in C.
[1] https://github.com/woju/libvirtaio
Also it contains a workaround for a memory leak around the custom loop implementation: https://github.com/woju/libvirtaio/blob/8685cfc/libvirtaio.py#L66-L92
How should I submit this? As a patch against libvirt-python.git, appending to libvirt-override.py?
IIUC, the asyncio module you're using is new in python 3.4 ? The libvirt python code aims to be compatible with python >= 2.6 at this time, so there's a mis-match there.
So if we did include that in the main libvirt python module, we'd have to figure out a suitable approach to avoid trying to import the non-existant module when people do "import libvirt".
That's right, asyncio (and this module) is compatible with 3.3 (mainly because of "yield from" syntax) and included in standard library from 3.4. We can ship it as separate module (like libvirt vs libvirtmod) and do something like this at the beginning on libvirt-override.py:
try: from libvirtaio import LibvirtAsyncIOEventImpl # assuming it won't get renamed in review __all__ += ['LibvirtAsyncIOEventImpl'] except (ImportError, SyntaxError): pass
(Just tested on 2.7).
People will probably import this unconditionally, since they know, what event loop they want to run. In case of doubt, they always can:
import libvirt try: libvirt.LibvirtAsyncIOEventImpl().register() except AttributeError: libvirt.virEventRegisterDefaultImpl() def event_loop(): while True: libvirt.virEventRunDefaultImpl() # ... import threading threading.Thread(target=event_loop).start()
execute_ff_callback() can be merged regardless, since it is probably compatible with 2.5 (that's when ctypes arrived).
Or we could just leave it as a standalone python module and document its existence on our python binding page (http://libvirt.org/python.html)
I'd be glad if you merged this module, since 1) it would be more accessible, 2) won't break between releases (currently it is in danger because of the opaque object handling). Since we would be (at least now) the only user of this, I volunteer as tester/patch submitter against this part of code.
Ok, well just go ahead & submit a suitable patch for libvirt-python and we'll take the discussion forward from there. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
participants (2)
-
Daniel P. Berrange
-
Wojtek Porczyk