[libvirt-users] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet?
Hello everybody, This is a cross post to libvirt-users, libguestfs and ceph-users. I came back from FOSDEM 2016 and this was my 7th year or so and seen the awesome development around visualization going on and want to thank everybody for there contributions. I seen presentations from oVirt, OpenStack and quite a few great Redhat people, just like the last previous years. I personally been supporting Linux-HA (pacemaker) DRBD/iSCSI/KVM platform for years now and last year I started supporting Ceph RBD clusters with KVM hosts. But I keep hitting some “pains” and I been wondering why they have not been solved and how I can best request help around this? Tools that don't seem to fully work with Ceph RBD yet: - virsh snapshot-create --quiesce --disk-only $quest - virt-filesystems ${array_volume[@]} - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$disk - virt-install also doesn't have Ceph RBD and I currently use virsh edit to add the RBD storage disks. My request is to get these tools working as well with Ceph RBD and maybe not only integrate oVirt or OpenStack alike systems. I use several types of back-up strategies depending on the size on the data storage use. I am not sure how oVirt and alike create secure encrypted incremental off-site back-ups of large data sets but I use(d) combinations of rdiff-backup, duplicity and full guest dumps with dd, rbd export, xz. Are these issues known and important enough, roadmap, bug reports? Should I start working for Redhat as I don't have the resources myself, but I can help with some money towards a crowed fund or bounty. Maybe my software is outdated? virsh 1.2.9 ceph version 0.80.10 libguestfs-tools 1:1.28.1-1 3.16.7-ckt20-1+deb8u2 My current workarounds for full storage exports: virsh domfsfreeze $quest sleep 2 virsh domblklist $quest rbd snap create --snap snapshot $blkdevice virsh domfsthaw $quest rbd export $blkdevice@snapshot - | xz -1 | ssh -p 222 $user@$server "dd of=/$location/$blkdevice$snapshot-$daystamp.dd.disk.gz" rbd snap rm $blkdevice@snapshot Kind regards, Jelle de Jong irc: #tuxcrafter
I can only contribute on one point: I only get the external snapshots working on Qemu 2.1. Not on a more recent version of Qemu... ________________________________________ Van: libvirt-users-bounces@redhat.com <libvirt-users-bounces@redhat.com> namens Jelle de Jong <jelledejong@powercraft.nl> Verzonden: maandag 1 februari 2016 22:52 Aan: libvirt-users@redhat.com; libguestfs@redhat.com; ceph-users@lists.ceph.com Onderwerp: [libvirt-users] virsh, virt-filesystems, guestmount, virt-install not working well with ceph rbd yet? Hello everybody, This is a cross post to libvirt-users, libguestfs and ceph-users. I came back from FOSDEM 2016 and this was my 7th year or so and seen the awesome development around visualization going on and want to thank everybody for there contributions. I seen presentations from oVirt, OpenStack and quite a few great Redhat people, just like the last previous years. I personally been supporting Linux-HA (pacemaker) DRBD/iSCSI/KVM platform for years now and last year I started supporting Ceph RBD clusters with KVM hosts. But I keep hitting some “pains” and I been wondering why they have not been solved and how I can best request help around this? Tools that don't seem to fully work with Ceph RBD yet: - virsh snapshot-create --quiesce --disk-only $quest - virt-filesystems ${array_volume[@]} - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$disk - virt-install also doesn't have Ceph RBD and I currently use virsh edit to add the RBD storage disks. My request is to get these tools working as well with Ceph RBD and maybe not only integrate oVirt or OpenStack alike systems. I use several types of back-up strategies depending on the size on the data storage use. I am not sure how oVirt and alike create secure encrypted incremental off-site back-ups of large data sets but I use(d) combinations of rdiff-backup, duplicity and full guest dumps with dd, rbd export, xz. Are these issues known and important enough, roadmap, bug reports? Should I start working for Redhat as I don't have the resources myself, but I can help with some money towards a crowed fund or bounty. Maybe my software is outdated? virsh 1.2.9 ceph version 0.80.10 libguestfs-tools 1:1.28.1-1 3.16.7-ckt20-1+deb8u2 My current workarounds for full storage exports: virsh domfsfreeze $quest sleep 2 virsh domblklist $quest rbd snap create --snap snapshot $blkdevice virsh domfsthaw $quest rbd export $blkdevice@snapshot - | xz -1 | ssh -p 222 $user@$server "dd of=/$location/$blkdevice$snapshot-$daystamp.dd.disk.gz" rbd snap rm $blkdevice@snapshot Kind regards, Jelle de Jong irc: #tuxcrafter _______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On Mon, Feb 01, 2016 at 10:52:59PM +0100, Jelle de Jong wrote: [...] Upstream libguestfs supports Ceph and has done for many years. It was disabled in RHEL up to RHEL <= 7.1, and then enabled in RHEL 7.2.
- virt-filesystems ${array_volume[@]} - guestmount --ro ${array_volume[@]} -m $mount /mnt/$name-$disk
You'll need to provide the precise commands that you're running. You need to use a URL with the -a option, as described here: http://libguestfs.org/guestfish.1.html#adding-remote-storage
Maybe my software is outdated? libguestfs-tools 1:1.28.1-1
What precise version? RHEL 7.1 and 7.2 both had 1.28.1-1, but with different release numbers. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
participants (3)
-
Dominique Ramaekers -
Jelle de Jong -
Richard W.M. Jones