[libvirt] [PATCH 0/2] use qemu's dump-guest-meory when vm uses host device
Currently, we use migrate to dump guest's memory. There is one restriction in migrate command: the device's status should be stored in qemu because the device's status should be passed to target machine. If we passthrough a host device to guest, the device's status is stored in the real device. So migrate command will fail. We usually use dump when guest is panicked. So there is no need to store device's status in the vmcore. qemu will have a new monitor command dump-guest-memory to dump guest memory, but it doesn't support async now(it will support later when the common async API is implemented). So I use dump-guest-memory only when the guest uses host device in this patchset. Wen Congyang (2): qemu: implement qemu's dump-guest-memory qemu: try to use qemu's dump-guest-meory when vm uses host device src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 42 ++++++++++++++++++++- src/qemu/qemu_monitor.c | 32 ++++++++++++++++ src/qemu/qemu_monitor.h | 13 +++++++ src/qemu/qemu_monitor_json.c | 42 +++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 7 ++++ src/qemu/qemu_monitor_text.c | 83 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_text.h | 7 ++++ 9 files changed, 226 insertions(+), 2 deletions(-)
This patch introduces the API to use qemu's new monitor command dump-guest-memory --- src/qemu/qemu_monitor.c | 32 ++++++++++++++++ src/qemu/qemu_monitor.h | 13 +++++++ src/qemu/qemu_monitor_json.c | 42 +++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 7 ++++ src/qemu/qemu_monitor_text.c | 83 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_text.h | 7 ++++ 6 files changed, 184 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index e1a8d4c..69e2502 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2018,6 +2018,38 @@ int qemuMonitorMigrateCancel(qemuMonitorPtr mon) return ret; } +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorDumpToFd(qemuMonitorPtr mon, + unsigned int flags, + int fd, + unsigned long long begin, + unsigned long long length) +{ + int ret; + VIR_DEBUG("mon=%p fd=%d flags=%x begin=%llx length=%llx", + mon, fd, flags, begin, length); + + if (!mon) { + qemuReportError(VIR_ERR_INVALID_ARG, "%s", + _("monitor must not be NULL")); + return -1; + } + + if (qemuMonitorSendFileHandle(mon, "dump", fd) < 0) + return -1; + + if (mon->json) + ret = qemuMonitorJSONDump(mon, flags, "fd:dump", begin, length); + else + ret = qemuMonitorTextDump(mon, flags, "fd:dump", begin, length); + + if (ret < 0) { + if (qemuMonitorCloseFileHandle(mon, "dump") < 0) + VIR_WARN("failed to close dumping handle"); + } + + return ret; +} int qemuMonitorGraphicsRelocate(qemuMonitorPtr mon, int type, diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index b480966..315cb9e 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -379,6 +379,19 @@ int qemuMonitorMigrateToUnix(qemuMonitorPtr mon, int qemuMonitorMigrateCancel(qemuMonitorPtr mon); +typedef enum { + QEMU_MONITOR_DUMP_HAVE_FILTER = 1 << 0, + QEMU_MONITOR_DUMP_PAGING = 1 << 1, + QEMU_MONITOR_DUMP_FLAGS_LAST +} QEMU_MONITOR_DUMP; + +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorDumpToFd(qemuMonitorPtr mon, + unsigned int flags, + int fd, + unsigned long long begin, + unsigned long long length); + int qemuMonitorGraphicsRelocate(qemuMonitorPtr mon, int type, const char *hostname, diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index eeeb6a6..57bc788 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2413,6 +2413,48 @@ int qemuMonitorJSONMigrateCancel(qemuMonitorPtr mon) return ret; } +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorJSONDump(qemuMonitorPtr mon, + unsigned int flags, + const char *protocol, + unsigned long long begin, + unsigned long long length) +{ + int ret; + virJSONValuePtr cmd = NULL; + virJSONValuePtr reply = NULL; + + if (flags & QEMU_MONITOR_DUMP_HAVE_FILTER) + cmd = qemuMonitorJSONMakeCommand("dump-guest-memory", + "b:paging", flags & QEMU_MONITOR_DUMP_PAGING ? 1 : 0, + "s:protocol", protocol, + "U:begin", begin, + "U:length", length, + NULL); + else + cmd = qemuMonitorJSONMakeCommand("dump-guest-memory", + "b:paging", flags & QEMU_MONITOR_DUMP_PAGING ? 1 : 0, + "s:protocol", protocol, + NULL); + if (!cmd) + return -1; + + ret = qemuMonitorJSONCommand(mon, cmd, &reply); + + if (ret == 0) { + if (qemuMonitorJSONHasError(reply, "CommandNotFound")) { + ret = -2; + goto cleanup; + } + + ret = qemuMonitorJSONCheckError(cmd, reply); + } + +cleanup: + virJSONValueFree(cmd); + virJSONValueFree(reply); + return ret; +} int qemuMonitorJSONGraphicsRelocate(qemuMonitorPtr mon, int type, diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index a0f67aa..a8b70d4 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -136,6 +136,13 @@ int qemuMonitorJSONMigrate(qemuMonitorPtr mon, int qemuMonitorJSONMigrateCancel(qemuMonitorPtr mon); +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorJSONDump(qemuMonitorPtr mon, + unsigned int flags, + const char *protocol, + unsigned long long begin, + unsigned long long length); + int qemuMonitorJSONGraphicsRelocate(qemuMonitorPtr mon, int type, const char *hostname, diff --git a/src/qemu/qemu_monitor_text.c b/src/qemu/qemu_monitor_text.c index 30a0416..035d200 100644 --- a/src/qemu/qemu_monitor_text.c +++ b/src/qemu/qemu_monitor_text.c @@ -1745,6 +1745,89 @@ int qemuMonitorTextMigrateCancel(qemuMonitorPtr mon) return 0; } +static int qemuMonitorTextCheckDumpingReply(const char *info) +{ + /* QERR_FD_NOT_FOUND */ + if (strstr(info, "File descriptor named")) + return -1; + + /* QERR_OPEN_FILE_FAILED */ + if (strstr(info, "Could not open")) + return -1; + + /* QERR_INVALID_PARAMETER */ + if (strstr(info, "Invalid parameter")) + return -1; + + /* QERR_IO_ERROR */ + if (strstr(info, "An IO error has occurred")) + return -1; + + /* QERR_PIPE_OR_SOCKET_FD */ + if (strstr(info, "lseek() failed: the fd is associated with a pipe, socket")) + return -1; + + return 0; +} + +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorTextDump(qemuMonitorPtr mon, + unsigned int flags, + const char *protocol, + unsigned long long begin, + unsigned long long length) +{ + char *cmd = NULL; + char *info = NULL; + int ret = -1; + char *safeprotocol = qemuMonitorEscapeArg(protocol); + + if (!safeprotocol) { + virReportOOMError(); + return -1; + } + + if (flags & QEMU_MONITOR_DUMP_HAVE_FILTER) { + ret = virAsprintf(&cmd, "dump-guest-memory %s \"%s\" %llu %llu", + flags & QEMU_MONITOR_DUMP_PAGING ? "-p" : "", + safeprotocol, begin, length); + } else { + ret = virAsprintf(&cmd, "dump-guest-memory %s \"%s\"", + flags & QEMU_MONITOR_DUMP_PAGING ? "-p" : "", + safeprotocol); + } + if (ret < 0) { + virReportOOMError(); + goto cleanup; + } + + if (qemuMonitorHMPCommand(mon, cmd, &info) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("unable to start dumping to %s"), protocol); + goto cleanup; + } + + /* If the command isn't supported then qemu prints: + * unknown command: dump" */ + if (strstr(info, "unknown command:")) { + ret = -2; + goto cleanup; + } + + if (qemuMonitorTextCheckDumpingReply(info) < 0) { + qemuReportError(VIR_ERR_OPERATION_FAILED, + _("dumping to '%s' failed: %s"), protocol, info); + goto cleanup; + } + + ret = 0; + +cleanup: + VIR_FREE(safeprotocol); + VIR_FREE(info); + VIR_FREE(cmd); + return ret; +} int qemuMonitorTextGraphicsRelocate(qemuMonitorPtr mon, int type, diff --git a/src/qemu/qemu_monitor_text.h b/src/qemu/qemu_monitor_text.h index 4525864..ca330ac 100644 --- a/src/qemu/qemu_monitor_text.h +++ b/src/qemu/qemu_monitor_text.h @@ -128,6 +128,13 @@ int qemuMonitorTextMigrate(qemuMonitorPtr mon, int qemuMonitorTextMigrateCancel(qemuMonitorPtr mon); +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +int qemuMonitorTextDump(qemuMonitorPtr mon, + unsigned int flags, + const char *protocol, + unsigned long long begin, + unsigned long long length); + int qemuMonitorTextGraphicsRelocate(qemuMonitorPtr mon, int type, const char *hostname, -- 1.7.1
--- src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 69d9e6e..e3a668a 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -158,6 +158,7 @@ qemuDomainObjResetAsyncJob(qemuDomainObjPrivatePtr priv) job->phase = 0; job->mask = DEFAULT_JOB_MASK; job->start = 0; + job->qemu_dump = false; memset(&job->info, 0, sizeof(job->info)); } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index adccfed..f1ab0e6 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -97,6 +97,7 @@ struct qemuDomainJobObj { int phase; /* Job phase (mainly for migrations) */ unsigned long long mask; /* Jobs allowed during async job */ unsigned long long start; /* When the async job started */ + bool qemu_dump; /* use qemu dump to do dump */ virDomainJobInfo info; /* Async job progress data */ }; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d503deb..0186d60 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2922,6 +2922,31 @@ cleanup: return ret; } +/* Return 0 on success, -1 on failure, or -2 if not supported. */ +static int qemuDumpToFd(struct qemud_driver *driver, virDomainObjPtr vm, + int fd, enum qemuDomainAsyncJob asyncJob) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + int ret = -1; + + if (!qemuCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_QEMU_FD)) + return -2; + + if (virSecurityManagerSetImageFDLabel(driver->securityManager, vm->def, + fd) < 0) + return -1; + + priv->job.qemu_dump = true; + + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) + return -1; + + ret = qemuMonitorDumpToFd(priv->mon, 0, fd, 0, 0); + qemuDomainObjExitMonitorWithDriver(driver, vm); + + return ret; +} + static int doCoreDump(struct qemud_driver *driver, virDomainObjPtr vm, @@ -2953,6 +2978,19 @@ doCoreDump(struct qemud_driver *driver, NULL, NULL)) < 0) goto cleanup; + if (vm->def->nhostdevs > 0) { + /* + * If the guest uses host devices, migrate command will fail. So we + * should use dump command. + * + * qemu dump does not support fd that is associated with a pipe, + * socket, or FIFO. So we should call qemuDumpTpFd() before calling + * virFileWrapperFdNew(). */ + ret = qemuDumpToFd(driver, vm, fd, QEMU_ASYNC_JOB_DUMP); + if (ret != -2) + goto cleanup; + } + if (!(wrapperFd = virFileWrapperFdNew(&fd, path, flags))) goto cleanup; @@ -9298,7 +9336,7 @@ static int qemuDomainGetJobInfo(virDomainPtr dom, priv = vm->privateData; if (virDomainObjIsActive(vm)) { - if (priv->job.asyncJob) { + if (priv->job.asyncJob && !priv->job.qemu_dump) { memcpy(info, &priv->job.info, sizeof(*info)); /* Refresh elapsed time again just to ensure it @@ -9356,7 +9394,7 @@ static int qemuDomainAbortJob(virDomainPtr dom) { priv = vm->privateData; - if (!priv->job.asyncJob) { + if (!priv->job.asyncJob || priv->job.qemu_dump) { qemuReportError(VIR_ERR_OPERATION_INVALID, "%s", _("no job is active on the domain")); goto endjob; -- 1.7.1
On 04/01/2012 10:01 PM, Wen Congyang wrote:
Currently, we use migrate to dump guest's memory. There is one restriction in migrate command: the device's status should be stored in qemu because the device's status should be passed to target machine.
If we passthrough a host device to guest, the device's status is stored in the real device. So migrate command will fail.
We usually use dump when guest is panicked. So there is no need to store device's status in the vmcore.
qemu will have a new monitor command dump-guest-memory to dump guest memory, but it doesn't support async now(it will support later when the common async API is implemented).
I've seen conversation on this patch on qemu-devel, but is it actually committed yet, or still pending there? This is late enough that it should wait until after 0.9.11 to actually be applied, so I haven't quite reviewed it yet.
So I use dump-guest-memory only when the guest uses host device in this patchset.
Wen Congyang (2): qemu: implement qemu's dump-guest-memory qemu: try to use qemu's dump-guest-meory when vm uses host device
src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 42 ++++++++++++++++++++- src/qemu/qemu_monitor.c | 32 ++++++++++++++++ src/qemu/qemu_monitor.h | 13 +++++++ src/qemu/qemu_monitor_json.c | 42 +++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 7 ++++ src/qemu/qemu_monitor_text.c | 83 ++++++++++++++++++++++++++++++++++++++++++
Why are we bothering with a text implementation? We know that the feature is only present if you have qemu 1.1 or later (assuming that qemu-devel did apply your series adding the monitor command), and therefore we know that we have QMP, so we should only implement this for JSON and not bother with the text monitor. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
At 04/02/2012 10:50 PM, Eric Blake Wrote:
On 04/01/2012 10:01 PM, Wen Congyang wrote:
Currently, we use migrate to dump guest's memory. There is one restriction in migrate command: the device's status should be stored in qemu because the device's status should be passed to target machine.
If we passthrough a host device to guest, the device's status is stored in the real device. So migrate command will fail.
We usually use dump when guest is panicked. So there is no need to store device's status in the vmcore.
qemu will have a new monitor command dump-guest-memory to dump guest memory, but it doesn't support async now(it will support later when the common async API is implemented).
I've seen conversation on this patch on qemu-devel, but is it actually committed yet, or still pending there? This is late enough that it should wait until after 0.9.11 to actually be applied, so I haven't quite reviewed it yet.
The patch on qemu-devel is still pending there. QMP maitainer Luiz has acked it, but he waits an ack from Jan and/or Anthony.
So I use dump-guest-memory only when the guest uses host device in this patchset.
Wen Congyang (2): qemu: implement qemu's dump-guest-memory qemu: try to use qemu's dump-guest-meory when vm uses host device
src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 42 ++++++++++++++++++++- src/qemu/qemu_monitor.c | 32 ++++++++++++++++ src/qemu/qemu_monitor.h | 13 +++++++ src/qemu/qemu_monitor_json.c | 42 +++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 7 ++++ src/qemu/qemu_monitor_text.c | 83 ++++++++++++++++++++++++++++++++++++++++++
Why are we bothering with a text implementation? We know that the feature is only present if you have qemu 1.1 or later (assuming that qemu-devel did apply your series adding the monitor command), and therefore we know that we have QMP, so we should only implement this for JSON and not bother with the text monitor.
In most cases, the libvirt is built with json. But if the libvirt is built without json, we will use text monitor.
On 04/02/2012 06:23 PM, Wen Congyang wrote:
Why are we bothering with a text implementation? We know that the feature is only present if you have qemu 1.1 or later (assuming that qemu-devel did apply your series adding the monitor command), and therefore we know that we have QMP, so we should only implement this for JSON and not bother with the text monitor.
In most cases, the libvirt is built with json. But if the libvirt is built without json, we will use text monitor.
If libvirt is built without json, it will die a horrible death on any qemu newer than 0.15, thanks to commit 6e769eba. You can safely assume that a qemu 1.1 feature implies that libvirt was compiled with json support. The _only_ situation that requires the text monitor is when building libvirt for RHEL 5 where qemu is still stuck at 0.10 and where JSON libraries do not exist. I suspect no one will be backporting the new dump-guest-monitor command to RHEL 5. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
At 04/03/2012 08:46 AM, Eric Blake Wrote:
On 04/02/2012 06:23 PM, Wen Congyang wrote:
Why are we bothering with a text implementation? We know that the feature is only present if you have qemu 1.1 or later (assuming that qemu-devel did apply your series adding the monitor command), and therefore we know that we have QMP, so we should only implement this for JSON and not bother with the text monitor.
In most cases, the libvirt is built with json. But if the libvirt is built without json, we will use text monitor.
If libvirt is built without json, it will die a horrible death on any qemu newer than 0.15, thanks to commit 6e769eba. You can safely assume that a qemu 1.1 feature implies that libvirt was compiled with json support. The _only_ situation that requires the text monitor is when building libvirt for RHEL 5 where qemu is still stuck at 0.10 and where JSON libraries do not exist. I suspect no one will be backporting the new dump-guest-monitor command to RHEL 5.
I see this commit now. So I agree with removing text monitor support now. Thanks for point it out.
participants (2)
-
Eric Blake -
Wen Congyang