[PATCH 0/5] qemu: IO error reporting fixes and improvements (part 1?)
This series: - fixed bug in lookup of devices for the IO error which made them useless (2/5) - adds framework to return error messages (3/5) - implements the passthrough of errors (4/5) - logs the I/O errors in the VM log (5/5) Peter Krempa (5): qemuProcessHandleIOError: Refactor to extract 'priv' instead of 'driver' qemuProcessHandleIOError: Fix logic for lookup of disk API: Document more possibilities for @reason field of VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON event qemu: monitor: Plumb in disk IO error message passthrough to VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON qemuProcessHandleIOError: Log IO errors in the VM log file include/libvirt/libvirt-domain.h | 8 +++-- src/qemu/qemu_monitor_json.c | 19 ++++++++++-- src/qemu/qemu_process.c | 51 ++++++++++++++++++-------------- 3 files changed, 51 insertions(+), 27 deletions(-) -- 2.48.1
The VM private data will be used in a sub-sequent patch. To minimize churn, refactor the function before changing the logic. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_process.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6db48b0e7b..34a755a49a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -829,7 +829,7 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, int action, const char *reason) { - virQEMUDriver *driver; + qemuDomainObjPrivate *priv; virObjectEvent *ioErrorEvent = NULL; virObjectEvent *ioErrorEvent2 = NULL; virObjectEvent *lifecycleEvent = NULL; @@ -838,7 +838,7 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, virDomainDiskDef *disk; virObjectLock(vm); - driver = QEMU_DOMAIN_PRIVATE(vm)->driver; + priv = QEMU_DOMAIN_PRIVATE(vm); if (*diskAlias == '\0') diskAlias = NULL; @@ -863,7 +863,6 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, if (action == VIR_DOMAIN_EVENT_IO_ERROR_PAUSE && virDomainObjGetState(vm, NULL) == VIR_DOMAIN_RUNNING) { - qemuDomainObjPrivate *priv = vm->privateData; VIR_WARN("Transitioned guest %s to paused state due to IO error", vm->def->name); if (priv->signalIOError) @@ -875,7 +874,7 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, VIR_DOMAIN_EVENT_SUSPENDED_IOERROR); VIR_FREE(priv->lockState); - if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lockState) < 0) + if (virDomainLockProcessPause(priv->driver->lockManager, vm, &priv->lockState) < 0) VIR_WARN("Unable to release lease on %s", vm->def->name); VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); @@ -883,9 +882,9 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, } virObjectUnlock(vm); - virObjectEventStateQueue(driver->domainEventState, ioErrorEvent); - virObjectEventStateQueue(driver->domainEventState, ioErrorEvent2); - virObjectEventStateQueue(driver->domainEventState, lifecycleEvent); + virObjectEventStateQueue(priv->driver->domainEventState, ioErrorEvent); + virObjectEventStateQueue(priv->driver->domainEventState, ioErrorEvent2); + virObjectEventStateQueue(priv->driver->domainEventState, lifecycleEvent); } -- 2.48.1
On Fri, Jan 24, 2025 at 05:33:02PM +0100, Peter Krempa wrote:
The VM private data will be used in a sub-sequent patch. To minimize churn, refactor the function before changing the logic.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_process.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
There are two bugs in the current disk lookup code in qemuProcessHandleIOError: 1) The QOM name isn't matched qemuProcessFindDomainDiskByAliasOrQOM() We pass NULL as the second argument, but the diskAlias argument can be the QOM path for e.g. virtio disks. This means that the IO error event doesn't actually contain the disk information: event 'io-error' for domain 'cd': () report due to other: Input/output error rather than: event 'io-error' for domain 'cd': /dev/mapper/errdev0 (virtio-disk0) report due to other: Input/output error 2) nodenames are not preferred We now do everything based on node names, which also allow you to pinpoint a image within the backing chain. With the current code the path would always refer to the top image rather than the actual image causing the problem. This patch fixes both issues by re-ordering the lookup to prefer nodenames and selecting the image path based on the node name and also populates the third argument of qemuProcessFindDomainDiskByAliasOrQOM(). Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_process.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 34a755a49a..95d0a40f84 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -833,9 +833,9 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, virObjectEvent *ioErrorEvent = NULL; virObjectEvent *ioErrorEvent2 = NULL; virObjectEvent *lifecycleEvent = NULL; - const char *srcPath; - const char *devAlias; - virDomainDiskDef *disk; + virStorageSource *errsrc = NULL; + const char *srcPath = ""; + const char *devAlias = ""; virObjectLock(vm); priv = QEMU_DOMAIN_PRIVATE(vm); @@ -843,19 +843,21 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, if (*diskAlias == '\0') diskAlias = NULL; - if (diskAlias) - disk = qemuProcessFindDomainDiskByAliasOrQOM(vm, diskAlias, NULL); - else if (nodename) - disk = qemuDomainDiskLookupByNodename(vm->def, NULL, nodename, NULL); - else - disk = NULL; + if (nodename) { + virDomainDiskDef *disk = qemuDomainDiskLookupByNodename(vm->def, priv->backup, nodename, &errsrc); - if (disk) { - srcPath = virDomainDiskGetSource(disk); - devAlias = disk->info.alias; - } else { - srcPath = ""; - devAlias = ""; + if (errsrc->path) + srcPath = errsrc->path; + + if (disk) + devAlias = disk->info.alias; + } else if (diskAlias) { + virDomainDiskDef *disk = qemuProcessFindDomainDiskByAliasOrQOM(vm, diskAlias, diskAlias); + + if (disk) { + srcPath = virDomainDiskGetSource(disk); + devAlias = disk->info.alias; + } } ioErrorEvent = virDomainEventIOErrorNewFromObj(vm, srcPath, devAlias, action); -- 2.48.1
On Fri, Jan 24, 2025 at 17:33:03 +0100, Peter Krempa wrote:
There are two bugs in the current disk lookup code in qemuProcessHandleIOError:
1) The QOM name isn't matched qemuProcessFindDomainDiskByAliasOrQOM()
We pass NULL as the second argument, but the diskAlias argument can be the QOM path for e.g. virtio disks. This means that the IO error event doesn't actually contain the disk information:
I've found the root cause for this one to be in QEMU where starting from qemu-9.2 a 'qom-path' field was introduced for the event in qemu, but it was plumbed in wrong where the 'device' and 'qom-path' fields were exchanged. Based on that I'll slightly modify this patch to also take QOM path if present and plumb it in properly via an explicit path rather than how this patch did it. I'll also post a patch fixing the qemu bug.
The VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON event allows @reason to be reported to the user, but currently we only report 'enospc'. Extend the documentation so that we allow the passthrough of the verbatim error, prefixed by 'other: ' in order to prevent collisions and note that users must not attempt to parse them. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- include/libvirt/libvirt-domain.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2a4b81f4df..e031b23547 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4790,8 +4790,12 @@ typedef void (*virConnectDomainEventIOErrorCallback)(virConnectPtr conn, * If the I/O error is known to be caused by an ENOSPC condition in * the host (where resizing the disk to be larger will allow the guest * to be resumed as if nothing happened), @reason will be "enospc". - * Otherwise, @reason will be "", although future strings may be added - * if determination of other error types becomes possible. + * + * Otherwise, if the hypervisor reported an error @reason will be the verbatim + * error message from hypervisor prefixed by "other: ". Note that this error + * may not be stable and thus is only really usable for human use. In case + * when the hypervisor doesn't report the error @reason will be an empty string + * "". * * Since: 0.8.1 */ -- 2.48.1
On Fri, Jan 24, 2025 at 05:33:04PM +0100, Peter Krempa wrote:
The VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON event allows @reason to be reported to the user, but currently we only report 'enospc'.
Extend the documentation so that we allow the passthrough of the verbatim error, prefixed by 'other: ' in order to prevent collisions and note that users must not attempt to parse them.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- include/libvirt/libvirt-domain.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2a4b81f4df..e031b23547 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4790,8 +4790,12 @@ typedef void (*virConnectDomainEventIOErrorCallback)(virConnectPtr conn, * If the I/O error is known to be caused by an ENOSPC condition in * the host (where resizing the disk to be larger will allow the guest * to be resumed as if nothing happened), @reason will be "enospc". - * Otherwise, @reason will be "", although future strings may be added - * if determination of other error types becomes possible. + * + * Otherwise, if the hypervisor reported an error @reason will be the verbatim + * error message from hypervisor prefixed by "other: ". Note that this error + * may not be stable and thus is only really usable for human use. In case + * when the hypervisor doesn't report the error @reason will be an empty string + * "".
Hmmm, this makes me feel pretty uncomfortable. When set, the 'reason' field has clear long term stable supported semantics, which applications are permitted to match against. Essentially it is an enum field as currently defined. This is now being turned into a free-format human readable string which applications are told not to interpret at all. Effectively we've overloaded the field to have two completely different sets of semantics. I don't think we should do this. If we want a human readable string, it should be distinct from the the enum reason we support already. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Jan 27, 2025 at 10:31:28 +0000, Daniel P. Berrangé wrote:
On Fri, Jan 24, 2025 at 05:33:04PM +0100, Peter Krempa wrote:
The VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON event allows @reason to be reported to the user, but currently we only report 'enospc'.
Extend the documentation so that we allow the passthrough of the verbatim error, prefixed by 'other: ' in order to prevent collisions and note that users must not attempt to parse them.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- include/libvirt/libvirt-domain.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2a4b81f4df..e031b23547 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4790,8 +4790,12 @@ typedef void (*virConnectDomainEventIOErrorCallback)(virConnectPtr conn, * If the I/O error is known to be caused by an ENOSPC condition in * the host (where resizing the disk to be larger will allow the guest * to be resumed as if nothing happened), @reason will be "enospc". - * Otherwise, @reason will be "", although future strings may be added - * if determination of other error types becomes possible. + * + * Otherwise, if the hypervisor reported an error @reason will be the verbatim + * error message from hypervisor prefixed by "other: ". Note that this error + * may not be stable and thus is only really usable for human use. In case + * when the hypervisor doesn't report the error @reason will be an empty string + * "".
Hmmm, this makes me feel pretty uncomfortable.
When set, the 'reason' field has clear long term stable supported semantics, which applications are permitted to match against.
Essentially it is an enum field as currently defined.
This is now being turned into a free-format human readable string which applications are told not to interpret at all.
Effectively we've overloaded the field to have two completely different sets of semantics.
I don't think we should do this.
If we want a human readable string, it should be distinct from the the enum reason we support already.
You're right about this effectively being an 'enum' while not being an enum. A bit of history how we ended here because it was at one point in time passtrhough of arbitrary strings. In certain ancient downstreams this was originally a field which was passed through from qemu verbatim and could have also other values than just 'enospc'. The current state was done after qemu formalized the event upstream, where the only stable field we get is 'nospace' which we've mapped to the exact string ('enospc') the downstream version did especially since it was used by oVirt. No other value did get a stable flag in the event so in turn libvirt didn't ever codify any other value, while still keeping the 'string' field. Now over time this did in fact become an enum with two possible options: "" and "enospc". At this point it felt convenient to use this field as it isn't really an enum, to encode a catch-all/default case for the user-originated string rather than adding yet another libvirt event: 1) Adding event is pain. 2) I'd be a new interface, thus potential users such as Kubevirt would need to use new libvirt instead of using existing interface. 3) It is a string when the CPU looks at it. 4) Adding events is pain! Original addition: 34dcbbb470fb8b93232b8bd709e949f9012a7462 De-facto formalization of enum: e9392e48d4e3b29809da7883b697d5caf3a09680
On Mon, Jan 27, 2025 at 11:55:29AM +0100, Peter Krempa wrote:
On Mon, Jan 27, 2025 at 10:31:28 +0000, Daniel P. Berrangé wrote:
On Fri, Jan 24, 2025 at 05:33:04PM +0100, Peter Krempa wrote:
The VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON event allows @reason to be reported to the user, but currently we only report 'enospc'.
Extend the documentation so that we allow the passthrough of the verbatim error, prefixed by 'other: ' in order to prevent collisions and note that users must not attempt to parse them.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- include/libvirt/libvirt-domain.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2a4b81f4df..e031b23547 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4790,8 +4790,12 @@ typedef void (*virConnectDomainEventIOErrorCallback)(virConnectPtr conn, * If the I/O error is known to be caused by an ENOSPC condition in * the host (where resizing the disk to be larger will allow the guest * to be resumed as if nothing happened), @reason will be "enospc". - * Otherwise, @reason will be "", although future strings may be added - * if determination of other error types becomes possible. + * + * Otherwise, if the hypervisor reported an error @reason will be the verbatim + * error message from hypervisor prefixed by "other: ". Note that this error + * may not be stable and thus is only really usable for human use. In case + * when the hypervisor doesn't report the error @reason will be an empty string + * "".
Hmmm, this makes me feel pretty uncomfortable.
When set, the 'reason' field has clear long term stable supported semantics, which applications are permitted to match against.
Essentially it is an enum field as currently defined.
This is now being turned into a free-format human readable string which applications are told not to interpret at all.
Effectively we've overloaded the field to have two completely different sets of semantics.
I don't think we should do this.
If we want a human readable string, it should be distinct from the the enum reason we support already.
You're right about this effectively being an 'enum' while not being an enum. A bit of history how we ended here because it was at one point in time passtrhough of arbitrary strings.
In certain ancient downstreams this was originally a field which was passed through from qemu verbatim and could have also other values than just 'enospc'. The current state was done after qemu formalized the event upstream, where the only stable field we get is 'nospace' which we've mapped to the exact string ('enospc') the downstream version did especially since it was used by oVirt.
No other value did get a stable flag in the event so in turn libvirt didn't ever codify any other value, while still keeping the 'string' field.
AFAIR, QEMU never codified machine targetted reasons, as there was debate over how much apps ought to know about the root cause. We have enospc, as that was the one thing with a clear use case, where mgmt apps could take specific actions.
Now over time this did in fact become an enum with two possible options: "" and "enospc". At this point it felt convenient to use this field as it isn't really an enum, to encode a catch-all/default case for the user-originated string rather than adding yet another libvirt event:
1) Adding event is pain. 2) I'd be a new interface, thus potential users such as Kubevirt would need to use new libvirt instead of using existing interface.
Apps should not actually be looking at these "new" human targetted "reason" strings though, so I don't think it is actually compelling to expose this in the events. THis is a distinct use case from the existing "reason" which is clearly intended for applications to toggle behaviour from. The only valid thing apps would do with the human targetted reason is to record it as a log message.
3) It is a string when the CPU looks at it. 4) Adding events is pain!
We don't need to add new events IMHO, we can expose it with the existing virDomainGetMessages API which is intended for such human targetted messages. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Pass the generic error message from qemu to the event so that users can possibly use the information. As documented the error is prefixed with "other: " to prevent collisions with already documented values in our API. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_monitor_json.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 9f417d27c6..6de34b03bb 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -695,7 +695,7 @@ qemuMonitorJSONHandleIOError(qemuMonitor *mon, virJSONValue *data) const char *device; const char *nodename; const char *action; - const char *reason = ""; + g_autofree char *reason = NULL; bool nospc = false; int actionID; @@ -713,14 +713,27 @@ qemuMonitorJSONHandleIOError(qemuMonitor *mon, virJSONValue *data) nodename = virJSONValueObjectGetString(data, "node-name"); - if (virJSONValueObjectGetBoolean(data, "nospace", &nospc) == 0 && nospc) - reason = "enospc"; + if (virJSONValueObjectGetBoolean(data, "nospace", &nospc) == 0 && nospc) { + reason = g_strdup("enospc"); + } else { + /* For any unknown error we prefix the qemu-originating error with 'other: ' + * so that applications can use that to filter them from errors we + * define in our API */ + const char *qemureason; + + if ((qemureason = virJSONValueObjectGetString(data, "reason"))) { + reason = g_strdup_printf("other: %s", qemureason); + } + } if ((actionID = qemuMonitorIOErrorActionTypeFromString(action)) < 0) { VIR_WARN("unknown disk io error action '%s'", action); actionID = VIR_DOMAIN_EVENT_IO_ERROR_NONE; } + if (!reason) + reason = g_strdup(""); + qemuMonitorEmitIOError(mon, device, nodename, actionID, reason); } -- 2.48.1
On Fri, Jan 24, 2025 at 05:33:05PM +0100, Peter Krempa wrote:
Pass the generic error message from qemu to the event so that users can possibly use the information. As documented the error is prefixed with "other: " to prevent collisions with already documented values in our API.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_monitor_json.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 9f417d27c6..6de34b03bb 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -695,7 +695,7 @@ qemuMonitorJSONHandleIOError(qemuMonitor *mon, virJSONValue *data) const char *device; const char *nodename; const char *action; - const char *reason = ""; + g_autofree char *reason = NULL; bool nospc = false; int actionID;
@@ -713,14 +713,27 @@ qemuMonitorJSONHandleIOError(qemuMonitor *mon, virJSONValue *data)
nodename = virJSONValueObjectGetString(data, "node-name");
- if (virJSONValueObjectGetBoolean(data, "nospace", &nospc) == 0 && nospc) - reason = "enospc"; + if (virJSONValueObjectGetBoolean(data, "nospace", &nospc) == 0 && nospc) { + reason = g_strdup("enospc"); + } else { + /* For any unknown error we prefix the qemu-originating error with 'other: ' + * so that applications can use that to filter them from errors we + * define in our API */ + const char *qemureason; + + if ((qemureason = virJSONValueObjectGetString(data, "reason"))) { + reason = g_strdup_printf("other: %s", qemureason); + } + }
if ((actionID = qemuMonitorIOErrorActionTypeFromString(action)) < 0) { VIR_WARN("unknown disk io error action '%s'", action); actionID = VIR_DOMAIN_EVENT_IO_ERROR_NONE; }
+ if (!reason) + reason = g_strdup(""); + qemuMonitorEmitIOError(mon, device, nodename, actionID, reason);
Per the last patch comments, we should expose the QMP "reason" as a separate field from our current libvirt "reason" enum. This would prevent us exposing it in the libvirt API for the IOError event, but we can still at least record it in the log which apps/users will collect for bug reporting purposes. This isn't a huge loss, as we're already saying that even if we did expose it in the API, apps shouldn't be looking at it, and we still have the option to expose it in virDomainGetMessages later if desired. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Add a log entry to the VM log file for every time we receive an IO error event from qemu. The log entry is as follows: 2025-01-24 16:03:28.928+0000: IO error device='virtio-disk0' node-name='libvirt-1-storage' reason='other: Input/output error' Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_process.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 95d0a40f84..e813c11f7e 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -836,6 +836,7 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, virStorageSource *errsrc = NULL; const char *srcPath = ""; const char *devAlias = ""; + g_autofree char *timestamp = NULL; virObjectLock(vm); priv = QEMU_DOMAIN_PRIVATE(vm); @@ -884,6 +885,11 @@ qemuProcessHandleIOError(qemuMonitor *mon G_GNUC_UNUSED, } virObjectUnlock(vm); + if ((timestamp = virTimeStringNow()) != NULL) { + qemuDomainLogAppendMessage(priv->driver, vm, "%s: IO error device='%s' node-name='%s' reason='%s'\n", + timestamp, NULLSTR(devAlias), NULLSTR(nodename), NULLSTR(reason)); + } + virObjectEventStateQueue(priv->driver->domainEventState, ioErrorEvent); virObjectEventStateQueue(priv->driver->domainEventState, ioErrorEvent2); virObjectEventStateQueue(priv->driver->domainEventState, lifecycleEvent); -- 2.48.1
participants (2)
-
Daniel P. Berrangé -
Peter Krempa