- Devel - libvirt List Archives

[PATCH] rpc: fix race in waking up client event loop
by Daniel P. Berrangé 18 Dec '23

18 Dec '23

The first thread to issue a client RPC request will own the event loop execution, sitting in the virNetClientIOEventLoop function. It releases the client lock while running: virNetClientUnlock() g_main_loop_run() virNetClientLock() If a second thread arrives with an RPC request, it will queue it for the first thread to process. To inform the first thread that there's a new request it calls g_main_loop_quit() to break it out of the main loop. This works if the first thread is in g_main_loop_run() at that time. There is a small window of opportunity, however, where the first thread has released the client lock, but not yet got into g_main_loop_run(). If that happens, the wakeup from the second thread is lost. This patch deals with that by changing the way the wakeup is performed. Instead of directly calling g_main_loop_quit(), the second thread creates an idle source to run the quit function from within the first thread. This guarantees that the first thread will see the wakeup. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/rpc/virnetclient.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c index 4ab8af68c5..68098b1c8d 100644 --- a/src/rpc/virnetclient.c +++ b/src/rpc/virnetclient.c @@ -1848,6 +1848,15 @@ static void virNetClientIOUpdateCallback(virNetClient *client, } +static gboolean virNetClientIOWakeup(gpointer opaque) +{ + GMainLoop *loop = opaque; + + g_main_loop_quit(loop); + + return G_SOURCE_REMOVE; +} + /* * This function sends a message to remote server and awaits a reply * @@ -1925,7 +1934,9 @@ static int virNetClientIO(virNetClient *client, /* Check to see if another thread is dispatching */ if (client->haveTheBuck) { /* Force other thread to wakeup from poll */ - g_main_loop_quit(client->eventLoop); + GSource *wakeup = g_idle_source_new(); + g_source_set_callback(wakeup, virNetClientIOWakeup, client->eventLoop, NULL); + g_source_attach(wakeup, client->eventCtx); /* If we are non-blocking, detach the thread and keep the call in the * queue. */ -- 2.43.0

4 3

[PATCH 1/3] rpc: mark GSourceFuncs functions in vireventglibwatch.c as static
by Fima Shevrin 18 Dec '23

18 Dec '23

From: "Denis V. Lunev" <den(a)openvz.org> They are not exported from the module and thus should be static. Signed-off-by: Denis V. Lunev <den(a)openvz.org> Signed-off-by: Fima Shevrin <efim.shevrin(a)virtuozzo.com> --- src/util/vireventglibwatch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/vireventglibwatch.c b/src/util/vireventglibwatch.c index b7f3a8786a..b21e505731 100644 --- a/src/util/vireventglibwatch.c +++ b/src/util/vireventglibwatch.c @@ -71,7 +71,7 @@ virEventGLibFDSourceFinalize(GSource *source G_GNUC_UNUSED) } -GSourceFuncs virEventGLibFDSourceFuncs = { +static GSourceFuncs virEventGLibFDSourceFuncs = { .prepare = virEventGLibFDSourcePrepare, .check = virEventGLibFDSourceCheck, .dispatch = virEventGLibFDSourceDispatch, @@ -194,7 +194,7 @@ virEventGLibSocketSourceFinalize(GSource *source) } -GSourceFuncs virEventGLibSocketSourceFuncs = { +static GSourceFuncs virEventGLibSocketSourceFuncs = { .prepare = virEventGLibSocketSourcePrepare, .check = virEventGLibSocketSourceCheck, .dispatch = virEventGLibSocketSourceDispatch, -- 2.34.1

5 12

[PATCH (pushed)] qemuDomainBlockPeek: Fix format checking logic
by Peter Krempa 16 Dec '23

16 Dec '23

Recent refactor which changed the format check to use qemuBlockStorageSourceIsRaw accidentaly inverted the condition. Caught by the CI test suite. Fixes: b600b69f8291226095e0fdb2eb7503baa79063fc Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Pushed as trivial CI fix. src/qemu/qemu_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 9d8be15721..2d75357939 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -10173,7 +10173,7 @@ qemuDomainBlockPeek(virDomainPtr dom, goto cleanup; } - if (qemuBlockStorageSourceIsRaw(disk->src)) { + if (!qemuBlockStorageSourceIsRaw(disk->src)) { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("peeking is only supported for disk with 'raw' format not '%1$s'"), virStorageFileFormatTypeToString(disk->src->format)); -- 2.43.0

1 0

Re: [PATCH 3/3] rpc: Rework rpc notifications in main and side thread
by Daniel P. Berrangé 15 Dec '23

15 Dec '23

On Fri, Dec 15, 2023 at 01:48:09PM +0000, Efim Shevrin wrote: > Hello, > > here are call traces with two threads generated by python script > // ============================== > [root@dandreev-vz-6-0-0-243-master libvirt]# gdb -p 288985 > (gdb) t a a bt > > Thread 2 (Thread 0x7f2112862640 (LWP 288986) "python3"): > #0 0x00007f2121d4296f in poll () at /lib64/libc.so.6 > #1 0x00007f211444650c in g_main_context_iterate.constprop () at /lib64/libglib-2.0.so.0 > #2 0x00007f21143f1483 in g_main_loop_run () at /lib64/libglib-2.0.so.0 > #3 0x00007f211406800b in virNetClientIOEventLoop () at /lib64/libvirt.so.0 > #4 0x00007f2114068a0a in virNetClientIO () at /lib64/libvirt.so.0 > #5 0x00007f21140692c1 in virNetClientSendInternal () at /lib64/libvirt.so.0 > #6 0x00007f211406936d in virNetClientSendWithReply () at /lib64/libvirt.so.0 > #7 0x00007f2114061a1d in virNetClientProgramCall () at /lib64/libvirt.so.0 > #8 0x00007f21140f79ac in callFull () at /lib64/libvirt.so.0 > #9 0x00007f21140f7a2f in call () at /lib64/libvirt.so.0 > #10 0x00007f21140d8435 in remoteDomainCreate () at /lib64/libvirt.so.0 > #11 0x00007f21141dd60c in virDomainCreate () at /lib64/libvirt.so.0 > #12 0x00007f21145c8114 in libvirt_virDomainCreate () at /usr/lib64/python3.9/site-packages/libvirtmod.cpython-39-x86_64-linux-gnu.so Ok, this thread is the primary one responsible for I/O. It is waiting, while not holding any mutex. > Thread 1 (Thread 0x7f21223cf740 (LWP 288985) "python3"): > #0 0x00007f2121c9c39a in __futex_abstimed_wait_common () at /lib64/libc.so.6 > #1 0x00007f2121c9eba0 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libc.so.6 > #2 0x00007f2113f4982a in virCondWait () at /lib64/libvirt.so.0 > #3 0x00007f2113f1fee3 in virObjectWait () at /lib64/libvirt.so.0 > #4 0x00007f211406882a in virNetClientIO () at /lib64/libvirt.so.0 > #5 0x00007f21140692c1 in virNetClientSendInternal () at /lib64/libvirt.so.0 > #6 0x00007f211406936d in virNetClientSendWithReply () at /lib64/libvirt.so.0 > #7 0x00007f2114061a1d in virNetClientProgramCall () at /lib64/libvirt.so.0 > #8 0x00007f21140f79ac in callFull () at /lib64/libvirt.so.0 > #9 0x00007f21140f7a2f in call () at /lib64/libvirt.so.0 > #10 0x00007f21140f24eb in remoteNodeDeviceNumOfCaps () at /lib64/libvirt.so.0 > #11 0x00007f2114207a00 in virNodeDeviceNumOfCaps () at /lib64/libvirt.so.0 This is a second API call that has arrived and has queued its outgoing message, and is waiting either for the other thread to pass ownership of the socket to it, or for the other thread to provide its reply (whichever comes first). This is all totally normal, and working as expected, so I still don't see what the actual problem is ? > // ========================================================================= > > just in case here is python script > > [root@dandreev-vz-6-0-0-243-master ~]# cat a.py > import libvirt > import time > from threading import Thread > > def startVM(connection, vm_name): > try: > # Find the virtual machine by name > print('starting VM') > connection.lookupByName(vm_name).create() > print('done starting VM') > except libvirt.libvirtError as e: > print(f'Libvirt Error: {e}') > > # Replace 'qemu+tcp://10.34.66.13/system' with your actual connection URI > connection_uri = 'qemu+tcp://localhost/system' > connection = libvirt.open(connection_uri) > if connection is None: > print(f'Failed to open connection to {connection_uri}') > exit(1) > > try: > # Replace 'your_vm_name' with the actual name of your virtual machine > # startVM(connection, 'instance-00000002') > thread = Thread(target=lambda: startVM(connection, 'instance-00000002')) > thread.start() > time.sleep(3) > devs = connection.listAllDevices() > for i in range(100000): > for dev in devs: > print('listing caps for dev %i %s' % (i, dev)) > try: > dev.listCaps() > except Exception as e: > print('ERROR: %s' % e) > print('DONE listing caps') > > connection.lookupByName('instance-00000002').destroy() > except: > connection.lookupByName('instance-00000002').destroy() > finally: > thread.join() > # Close the connection outside the function > connection.close() What behaviour are you expecting this demo program to show ? When I run it, the VM starts and the the dev caps listing is then run 3 seconds later (due to the sleep). Removing the sleep, I can see the caps list running in parallel with the 'create'. Basically I see nothing wrong. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

2 1

Re: [PATCH 3/3] rpc: Rework rpc notifications in main and side thread
by Denis V. Lunev 15 Dec '23

15 Dec '23

On 12/15/23 14:48, Efim Shevrin wrote: > Hello, > > here are call traces with two threads generated by python script > // ============================== > [root@dandreev-vz-6-0-0-243-master libvirt]# gdb -p 288985 > (gdb) t a a bt > > Thread 2 (Thread 0x7f2112862640 (LWP 288986) "python3"): > #0 0x00007f2121d4296f in poll () at /lib64/libc.so.6 > #1 0x00007f211444650c in g_main_context_iterate.constprop () at > /lib64/libglib-2.0.so.0 > #2 0x00007f21143f1483 in g_main_loop_run () at /lib64/libglib-2.0.so.0 > #3 0x00007f211406800b in virNetClientIOEventLoop () at > /lib64/libvirt.so.0 > #4 0x00007f2114068a0a in virNetClientIO () at /lib64/libvirt.so.0 > #5 0x00007f21140692c1 in virNetClientSendInternal () at > /lib64/libvirt.so.0 > #6 0x00007f211406936d in virNetClientSendWithReply () at > /lib64/libvirt.so.0 > #7 0x00007f2114061a1d in virNetClientProgramCall () at > /lib64/libvirt.so.0 > #8 0x00007f21140f79ac in callFull () at /lib64/libvirt.so.0 > #9 0x00007f21140f7a2f in call () at /lib64/libvirt.so.0 > #10 0x00007f21140d8435 in remoteDomainCreate () at /lib64/libvirt.so.0 > #11 0x00007f21141dd60c in virDomainCreate () at /lib64/libvirt.so.0 > #12 0x00007f21145c8114 in libvirt_virDomainCreate () at > /usr/lib64/python3.9/site-packages/libvirtmod.cpython-39-x86_64-linux-gnu.so > #13 0x00007f21221268a8 in cfunction_call () at /lib64/libpython3.9.so.1.0 > #14 0x00007f2122118814 in _PyObject_MakeTpCall () at > /lib64/libpython3.9.so.1.0 > #15 0x00007f212211566e in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #16 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #17 0x00007f2122110650 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #18 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #19 0x00007f21221103e8 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #20 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #21 0x00007f21221133d2 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #22 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #23 0x00007f2122110650 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #24 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #25 0x00007f2122110650 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #26 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #27 0x00007f2122125382 in method_vectorcall () at > /lib64/libpython3.9.so.1.0 > #28 0x00007f21221d8c4a in t_bootstrap () at /lib64/libpython3.9.so.1.0 > #29 0x00007f21221d8bf8 in pythread_wrapper () at > /lib64/libpython3.9.so.1.0 > #30 0x00007f2121c9f802 in start_thread () at /lib64/libc.so.6 > #31 0x00007f2121c3f450 in clone3 () at /lib64/libc.so.6 > > Thread 1 (Thread 0x7f21223cf740 (LWP 288985) "python3"): > #0 0x00007f2121c9c39a in __futex_abstimed_wait_common () at > /lib64/libc.so.6 > #1 0x00007f2121c9eba0 in pthread_cond_wait@@GLIBC_2.3.2 () at > /lib64/libc.so.6 > #2 0x00007f2113f4982a in virCondWait () at /lib64/libvirt.so.0 > #3 0x00007f2113f1fee3 in virObjectWait () at /lib64/libvirt.so.0 > #4 0x00007f211406882a in virNetClientIO () at /lib64/libvirt.so.0 > #5 0x00007f21140692c1 in virNetClientSendInternal () at > /lib64/libvirt.so.0 > #6 0x00007f211406936d in virNetClientSendWithReply () at > /lib64/libvirt.so.0 > #7 0x00007f2114061a1d in virNetClientProgramCall () at > /lib64/libvirt.so.0 > #8 0x00007f21140f79ac in callFull () at /lib64/libvirt.so.0 > #9 0x00007f21140f7a2f in call () at /lib64/libvirt.so.0 > #10 0x00007f21140f24eb in remoteNodeDeviceNumOfCaps () at > /lib64/libvirt.so.0 > #11 0x00007f2114207a00 in virNodeDeviceNumOfCaps () at /lib64/libvirt.so.0 > #12 0x00007f21145d8edf in libvirt_virNodeDeviceListCaps.lto_priv.0 () > at > /usr/lib64/python3.9/site-packages/libvirtmod.cpython-39-x86_64-linux-gnu.so > #13 0x00007f21221268a8 in cfunction_call () at /lib64/libpython3.9.so.1.0 > #14 0x00007f2122118814 in _PyObject_MakeTpCall () at > /lib64/libpython3.9.so.1.0 > #15 0x00007f212211566e in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #16 0x00007f212211d013 in function_code_fastcall () at > /lib64/libpython3.9.so.1.0 > #17 0x00007f2122110650 in _PyEval_EvalFrameDefault () at > /lib64/libpython3.9.so.1.0 > #18 0x00007f212210f06d in _PyEval_EvalCode () at > /lib64/libpython3.9.so.1.0 > #19 0x00007f212218c495 in _PyEval_EvalCodeWithName () at > /lib64/libpython3.9.so.1.0 > --Type <RET> for more, q to quit, c to continue without paging-- > #20 0x00007f212218c42d in PyEval_EvalCodeEx () at > /lib64/libpython3.9.so.1.0 > #21 0x00007f212218c3df in PyEval_EvalCode () at /lib64/libpython3.9.so.1.0 > #22 0x00007f21221b7524 in run_eval_code_obj () at > /lib64/libpython3.9.so.1.0 > #23 0x00007f21221b5da6 in run_mod () at /lib64/libpython3.9.so.1.0 > #24 0x00007f212208f0cb in pyrun_file.cold () at /lib64/libpython3.9.so.1.0 > #25 0x00007f21221bb253 in PyRun_SimpleFileExFlags () at > /lib64/libpython3.9.so.1.0 > #26 0x00007f21221b7ee8 in Py_RunMain () at /lib64/libpython3.9.so.1.0 > #27 0x00007f212217f02d in Py_BytesMain () at /lib64/libpython3.9.so.1.0 > #28 0x00007f2121c3feb0 in __libc_start_call_main () at /lib64/libc.so.6 > #29 0x00007f2121c3ff60 in __libc_start_main_impl () at /lib64/libc.so.6 > #30 0x000056013181a095 in _start () > > // > ========================================================================= > > just in case here is python script > > [root@dandreev-vz-6-0-0-243-master ~]# cat a.py > import libvirt > import time > from threading import Thread > > def startVM(connection, vm_name): > try: > # Find the virtual machine by name > print('starting VM') > connection.lookupByName(vm_name).create() > print('done starting VM') > except libvirt.libvirtError as e: > print(f'Libvirt Error: {e}') > > # Replace 'qemu+tcp://10.34.66.13/system' with your actual connection URI > connection_uri = 'qemu+tcp://localhost/system' > connection = libvirt.open(connection_uri) > if connection is None: > print(f'Failed to open connection to {connection_uri}') > exit(1) > > try: > # Replace 'your_vm_name' with the actual name of your virtual machine > # startVM(connection, 'instance-00000002') > thread = Thread(target=lambda: startVM(connection, 'instance-00000002')) > thread.start() > time.sleep(3) > devs = connection.listAllDevices() > for i in range(100000): > for dev in devs: > print('listing caps for dev %i %s' % (i, dev)) > try: > dev.listCaps() > except Exception as e: > print('ERROR: %s' % e) > print('DONE listing caps') > > connection.lookupByName('instance-00000002').destroy() > except: > connection.lookupByName('instance-00000002').destroy() > finally: > thread.join() > # Close the connection outside the function > connection.close() > > // ================================================================= > > We use virtual machine startup as a loaded operation, but migration or > something similar could be used. > > Fima Shevrin > ------------------------------------------------------------------------ > *From:* Daniel P. Berrangé <berrange(a)redhat.com> > *Sent:* Friday, December 15, 2023 19:09 > *To:* Efim Shevrin <efim.shevrin(a)virtuozzo.com> > *Cc:* devel(a)lists.libvirt.org <devel(a)lists.libvirt.org>; > den(a)openvz.org <den(a)openvz.org> > *Subject:* Re: [PATCH 3/3] rpc: Rework rpc notifications in main and > side thread > [You don't often get email from berrange(a)redhat.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > On Fri, Dec 15, 2023 at 02:32:19AM +0800, Fima Shevrin wrote: > > RPC client implementation uses the following paradigm. The critical > > section is organized via virObjectLock(client)/virObjectUnlock(client) > > braces. Though this is potentially problematic as > > main thread: side thread: > > virObjectUnlock(client); > > virObjectLock(client); > > g_main_loop_quit(client->eventLoop); > > virObjectUnlock(client); > > g_main_loop_run(client->eventLoop); > > > > This means in particular that is the main thread is executing very long > > request like VM migration, the wakeup from the side thread could be > > stuck until the main request will be fully completed. > > Can you explain this in more detail, with call traces illustration > for the two threads. You're not saying where the main thread is > doing work with the 'client' lock hold for a long time. Generally > the goal should be for the main thread to only hold the lock for > a short time. Also if the side thread is already holding a reference > on 'client', then potentially we should consider if it is possible > to terminate the event loop without acquiring the mutex, as GMainLoop > protects itself wrt concurrent usage already, provided all threads > hold a reference directly or indirectly. At our opinion the problem here is missed wakeup from the side thread to the main thread. In the original code, i.e. before switching to the main_loop, the problem was not occurred as wakeup event was kept inside the pipe and the poll has been immediately woken up. I have written the same code with pipes but that code is a bit cumbersome due to the Windows compatibility. Thus this application with Prepare seems cleaner. Den

2 2

[PATCH 0/4] virsh/virt-admin: Don't use virErrorRestore
by Peter Krempa 15 Dec '23

15 Dec '23

Client application shouldn't need to reset the error. Peter Krempa (4): virshReconnect: Preserve current URI if not explicitly set in ctl->connname virshCatchDisconnect: Don't probe connection URI vshAdmConnect: Preserve connection URI in ctl->connname if auto-detected vshAdmCatchDisconnect: Don't probe connection URI tools/virsh.c | 13 +++++-------- tools/virt-admin.c | 11 ++++------- 2 files changed, 9 insertions(+), 15 deletions(-) -- 2.43.0

2 4

[PATCH 4/4] vshAdmCatchDisconnect: Don't probe connection URI
by Peter Krempa 15 Dec '23

15 Dec '23

virsh already stores the connection URI in 'ctl->connname', use that instead. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- tools/virt-admin.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/tools/virt-admin.c b/tools/virt-admin.c index 02cf421cee..fa9304c772 100644 --- a/tools/virt-admin.c +++ b/tools/virt-admin.c @@ -72,15 +72,10 @@ vshAdmCatchDisconnect(virAdmConnectPtr conn G_GNUC_UNUSED, { vshControl *ctl = opaque; const char *str = "unknown reason"; - virErrorPtr error; - g_autofree char *uri = NULL; if (reason == VIR_CONNECT_CLOSE_REASON_CLIENT) return; - virErrorPreserveLast(&error); - uri = virAdmConnectGetURI(conn); - switch ((virConnectCloseReason) reason) { case VIR_CONNECT_CLOSE_REASON_ERROR: str = N_("Disconnected from %1$s due to I/O error"); @@ -96,8 +91,7 @@ vshAdmCatchDisconnect(virAdmConnectPtr conn G_GNUC_UNUSED, break; } - vshError(ctl, _(str), NULLSTR(uri)); - virErrorRestore(&error); + vshError(ctl, _(str), NULLSTR(ctl->connname)); } static int -- 2.43.0

1 0

[PATCH] CIM: Include <stdlib.h> in libxkutil for malloc functions
by Florian Weimer 15 Dec '23

15 Dec '23

This avoids implicit function declarations and build failures with future compilers. Not sure where to submit this. Our Fedora package references <http://libvirt.org/CIM/>, but it's 404, and I can't find a CIM component in the Gitlab organization. Thanks, Florian diff --git a/libxkutil/infostore.c b/libxkutil/infostore.c index d2b2f719dc1d3616..060a83b26e170df7 100644 --- a/libxkutil/infostore.c +++ b/libxkutil/infostore.c @@ -26,6 +26,7 @@ #include <sys/stat.h> #include <sys/file.h> #include <string.h> +#include <stdlib.h> #include <libvirt/libvirt.h> #include <libxml/parser.h> diff --git a/libxkutil/xml_parse_test.c b/libxkutil/xml_parse_test.c index 384593d056d07fb2..9af570efc75f7681 100644 --- a/libxkutil/xml_parse_test.c +++ b/libxkutil/xml_parse_test.c @@ -1,5 +1,6 @@ #include <stdio.h> #include <inttypes.h> +#include <stdlib.h> #include <getopt.h>

3 3

Public virErrorPreserveLast()/virErrorRestore()
by John Levon 15 Dec '23

15 Dec '23

These two functions are currently private to libvirt, hence not available to consumers. Would it be possible to make them public? Without them, there's no way to do any libvirt call without stomping on an existing error that you may want to preserve. I have multiple threads sharing a remote connection (to local libvirtd). I need to be able to handle connection drops (e.g. libvirtd restart). Long story short, the only approach I've found that actually works properly is that there's one main conn object, each thread has a virConnectRef() to it, and whenever a thread gets an error, we check in the error callback if !virConnectIsAlive(). If so, we close the thread's conn, and potentially also clean up the main conn. Then when the error gets returned from the relevant library call, we try to re-connect. However, virConnectIsAlive() stomps on the thread's error, meaning we lose the context on a real, non-connection-related error. So I need to be able to save and restore in the error handler. Given how much these are used inside libvirt code already, it feels likely it would be useful in other public cases too. Thoughts? thnaks john

4 13

[PATCH v2 0/3] Rework main and side threads interaction in rpc
by Fima Shevrin 15 Dec '23

15 Dec '23

From: "Fima Shevrin" <efim.shevrin(a)virtuozzo.com> RPC client implementation uses the following paradigm. The critical section is organized via virObjectLock(client)/virObjectUnlock(client) braces. Though this is potentially problematic as main thread: side thread: virObjectUnlock(client); virObjectLock(client); g_main_loop_quit(client->eventLoop); virObjectUnlock(client); g_main_loop_run(client->eventLoop); This means in particular that is the main thread is executing very long request like VM migration, the wakeup from the side thread could be stuck until the main request will be fully completed. Discrubed case is easily reproducible with the simple python scripts doing slow and fast requests in parallel from two different threads. Our idea is to release the lock at the prepare stage and avoid libvirt stuck during the interaction between main and side threads. Changes: Add cover letter and versions From 2d1a5f094101808216970dbf90383ffeadae7fe9 Mon Sep 17 00:00:00 2001 From: "Denis V. Lunev" <den(a)openvz.org> Date: Sat, 25 Nov 2023 14:11:56 +0300 Subject: [PATCH v2 1/3] rpc: mark GSourceFuncs functions in vireventglibwatch.c as static They are not exported from the module and thus should be static. Signed-off-by: Denis V. Lunev <den(a)openvz.org> Signed-off-by: Fima Shevrin <efim.shevrin(a)virtuozzo.com> --- src/util/vireventglibwatch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/vireventglibwatch.c b/src/util/vireventglibwatch.c index b7f3a8786a..b21e505731 100644 --- a/src/util/vireventglibwatch.c +++ b/src/util/vireventglibwatch.c @@ -71,7 +71,7 @@ virEventGLibFDSourceFinalize(GSource *source G_GNUC_UNUSED) } -GSourceFuncs virEventGLibFDSourceFuncs = { +static GSourceFuncs virEventGLibFDSourceFuncs = { .prepare = virEventGLibFDSourcePrepare, .check = virEventGLibFDSourceCheck, .dispatch = virEventGLibFDSourceDispatch, @@ -194,7 +194,7 @@ virEventGLibSocketSourceFinalize(GSource *source) } -GSourceFuncs virEventGLibSocketSourceFuncs = { +static GSourceFuncs virEventGLibSocketSourceFuncs = { .prepare = virEventGLibSocketSourcePrepare, .check = virEventGLibSocketSourceCheck, .dispatch = virEventGLibSocketSourceDispatch, -- 2.34.1

1 2