[libvirt] converting save/dump output into physical memory image
A lot of people in the security community, myself included, are interested in memory forensics these days. Virtualization is a natural fit with memory forensics because it allows one to get access to a guest's memory without having to introduce any extra software into the guest or otherwise interfere with it. Incident responders are particularly interested in getting memory dumps from systems they're investigating. Virsh has "save" and "dump" commands for storing the state of a guest to a file on disk, but memory of KVM guests doesn't get saved in the "standard" input format for memory forensics tools, which is a raw physical memory image. (This is what you'd get via the classical "dd /dev/mem" approach or the contemporary equivalent using the crash driver; and VMware Server and Workstation produce .vmem files, which are such raw physical memory images, when a guest is paused or snapshotted.) In order to analyze the memory of Libvirt/KVM guests with my Linux memory forensics software, Second Look, I've created a tool for converting Libvirt-QEMU-save files (output of virsh save command) or QEMU-savevm files (output of virsh dump command) to raw physical memory images. I've got a basic working capability, though I'm still tracking down some problems with a guest allocated 8GB RAM--not all the memory seems to be present in the save or dump file. And I haven't tested very extensively yet, version support is limited to what I myself am currently running, etc. I'd like to know if this is a capability that others are interested in. Is this something that would be of interest to the Libvirt project if I were to contribute the code, or to the KVM project, or do you think it best exists as a separate project? I've also got a proof-of-concept tool for converting hibernate images to raw physical memory images. Perhaps a collection of tools for converting various memory dump formats would be a good project. Anyone else interested in this kind of stuff? As an author of commercial memory forensics software I've got a vested interest in availability of good memory acquisition capabilities. But there are a number of people working on FOSS Linux memory analysis tools, too... Andrew
On 05/05/2011 11:56 AM, Andrew Tappert wrote:
A lot of people in the security community, myself included, are interested in memory forensics these days. Virtualization is a natural fit with memory forensics because it allows one to get access to a guest's memory without having to introduce any extra software into the guest or otherwise interfere with it. Incident responders are particularly interested in getting memory dumps from systems they're investigating.
Virsh has "save" and "dump" commands for storing the state of a guest to a file on disk, but memory of KVM guests doesn't get saved in the "standard" input format for memory forensics tools, which is a raw physical memory image. (This is what you'd get via the classical "dd /dev/mem" approach or the contemporary equivalent using the crash driver; and VMware Server and Workstation produce .vmem files, which are such raw physical memory images, when a guest is paused or snapshotted.)
Libvirt also has the virDomainMemoryPeek API; right now, it is not exposed by virsh, but we could add a command-line-interface for it if that proves useful. Does that API fit your needs any better than converting a qemu dump image back into raw memory? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 05/05/2011 02:33 PM, Eric Blake wrote:
On 05/05/2011 11:56 AM, Andrew Tappert wrote:
Virsh has "save" and "dump" commands for storing the state of a guest to a file on disk, but memory of KVM guests doesn't get saved in the "standard" input format for memory forensics tools, which is a raw physical memory image. (This is what you'd get via the classical "dd /dev/mem" approach or the contemporary equivalent using the crash driver; and VMware Server and Workstation produce .vmem files, which are such raw physical memory images, when a guest is paused or snapshotted.)
Libvirt also has the virDomainMemoryPeek API; right now, it is not exposed by virsh, but we could add a command-line-interface for it if that proves useful. Does that API fit your needs any better than converting a qemu dump image back into raw memory?
Before starting to write the LibvirtQemuSave/QemuSavevm conversion tool I did explore the possibility of writing a Libvirt memory dump program using that API. But given the way it writes to a temporary file it did not seem like it would be feasible/efficient for full memory dumps. I just signed up to your mailing list today, but I saw in your last post a mention of qemu raw monitor passthrough commands. Maybe directly invoking qemu's pmemsave is the way to go. The nice thing about the qemu savevm format, though, is that it does contain cpu state, which is actually quite relevant to "memory" forensics... it's not all about getting a raw physical memory image, actually. Andrew
On Thu, May 05, 2011 at 01:56:10PM -0400, Andrew Tappert wrote:
A lot of people in the security community, myself included, are interested in memory forensics these days. Virtualization is a natural fit with memory forensics because it allows one to get access to a guest's memory without having to introduce any extra software into the guest or otherwise interfere with it. Incident responders are particularly interested in getting memory dumps from systems they're investigating.
Virsh has "save" and "dump" commands for storing the state of a guest to a file on disk, but memory of KVM guests doesn't get saved in the "standard" input format for memory forensics tools, which is a raw physical memory image. (This is what you'd get via the classical "dd /dev/mem" approach or the contemporary equivalent using the crash driver; and VMware Server and Workstation produce .vmem files, which are such raw physical memory images, when a guest is paused or snapshotted.)
In order to analyze the memory of Libvirt/KVM guests with my Linux memory forensics software, Second Look, I've created a tool for converting Libvirt-QEMU-save files (output of virsh save command) or QEMU-savevm files (output of virsh dump command) to raw physical memory images.
I've got a basic working capability, though I'm still tracking down some problems with a guest allocated 8GB RAM--not all the memory seems to be present in the save or dump file. And I haven't tested very extensively yet, version support is limited to what I myself am currently running, etc.
KVM only allocates memory on demand, when the guest touches the page, so the data will almost certainly be sparse. FWIW, the 'crash' tool has been taught how to understand the output of 'virsh dump' for Linux guests. I agree, it could be nice to have a raw memory dump though, instead of the KVM save format, at least as an option. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Andrew Tappert -
Daniel P. Berrange -
Eric Blake