[libvirt] need some help/asdive
Background: I am a newbie to Linux Virtualization. However, I have been using Linux for a long time. After a short time with Slackware, I started with Red Hat Linux 2 and went on to 2.1, 3.0.3, 4, 4.2, 5, 5.2, ..., 8, 9. And then on to Fedora Core 1, 2, ... until I installed Fedora 11 about a month ago. I have also run VMware starting when it first appeared. Initially I used VMware to run Windows. These days I use it to run up to from 25 to 30 guests (between 1 and 5 at the same time) [yes, I have lots of system memory]. Besides a couple of Windows guests, I have some guests supporting both 32 bit (i386) and 64 bit (x86_64) architectures of older releases as well as alternatives such as CentOS -- I use these to build RPMs. Then there are the guests supporting other operating systems such as Solaris and *BSD. Finally, I have a set of test guest that I use for testing stuff which could destroy the operating system or which require a lot of re-booting. All of this in 25 to 30 guests. With Fedora 11, I started by trying VMware-server. Server 1.0.9 (which I used on Fedora 9) looks like much more work to get running ... more or less "a bridge too far". So, I tried server 2.0.2 ... but the client interface changed from a separate GUI application to a browser based interface: 1) this impacts my usage of the firefox browser; 2) it is a buggy implementation; and 3) it is a crappy interface which I do not like. So, I (finally) noticed that Linux in general and Fedora specifically has packages supporting running guests virtually ... so, lets change my "problem sets" and give this a try ... especially when I noticed that it could use the hardware support in my new AMD Phenom II 940 processor. Before getting into my specific questions/problem, let me describe how my systems are set up. First of all, I have a basic rule: "don't screw with a working system." Therefore, on each of my systems I have at least two root partitions defined -- one for the current (working) system and one for the next (new) system [these days it is actually a /boot partition with root being on a striped logical volume]. I also have a small (think minimal install) partition with Fedora to serve as a boot selector (using chainloader), as something to to define striped logical volumes, and to be there for "emergency" maintenance. Guys, this works for me and always doing fresh installs saves on other headaches. Implicit in the above is that I put all of my data and my /home into separate partitions or logical volumes. One of these are my VMware guest systems. Now VMware puts all of the files (configuration, disk image, etc.) associated a guest under a separate directory. Fedora virtualization does not do this and separation is something I need so that I can easily move to a new version of the operating system such as when Fedora 12 becomes available in November. Questions/problems: My first question: it appears to me that a guest is defined by two things -- a configuration file in /etc/libvirt/qemu and one or more disk images in /var/lib/libvirt/images ... is this it or are there other files I need to consider? My second question concerns the location of the disk images: what is the "best practical way" to move these out of the root partition? I have come up with these different approaches which could/may work: 1. change the "path" value in /etc/libvirt/storage/default.xml to point to a directory in a different partition. 2. Use a /etc/fstab entry to "bind mount" a directory in a separate partition onto /var/lib/libvirt/images 3. Just use an /etc/fstab entry to mount a partition on /var/lib/libvirt/images or, perhaps, on /var/lib/libvirt. [This could be done during system installation] OK, any comments on these?? I can live with changing some configuration files, I can live with copying guest configuration files between systems, I CANNOT live with the disk image files being put into the root partition. Before jumping "whole hog" into using Fedora Virtualization in place of VMware, I need to be able to live with the result. Figure 30 guest virtuals with 10GB of virtual disk each, that is 300GB. I have that space but not for every root partition on my system! Comment: It sure would be nice if Fedora Virtualization put all files related to a guest under a single directory the way VMware does. Comment: It sure would be nice if there was some system-wide configuration parameter that defined where virtualization files went. Right now, as a newbie, I am using the virt-manage GUI interface. I am studying the command-line programs to see what they offer me. So far, I like what I see in Fedora Virtualization but I need to address how things will be practically managed sooner rather than later. Any help appreciated. Gene
On Sun, Jun 28, 2009 at 11:07:23AM -0400, Gene Czarcinski wrote:
Questions/problems:
My first question: it appears to me that a guest is defined by two things -- a configuration file in /etc/libvirt/qemu and one or more disk images in /var/lib/libvirt/images ... is this it or are there other files I need to consider?
For most people, this is sufficient, just the guest config and the disk image(s). You can have > 1 disk image, and can share disk image between guests if you want to get advanced.
My second question concerns the location of the disk images: what is the "best practical way" to move these out of the root partition? I have come up with these different approaches which could/may work:
1. change the "path" value in /etc/libvirt/storage/default.xml to point to a directory in a different partition.
2. Use a /etc/fstab entry to "bind mount" a directory in a separate partition onto /var/lib/libvirt/images
3. Just use an /etc/fstab entry to mount a partition on /var/lib/libvirt/images or, perhaps, on /var/lib/libvirt. [This could be done during system installation]
OK, any comments on these??
There's no real 'right' answer here, but I'd recommend 1. really. Historically (ie <= Fedora 10), we setup /var/lib/libvirt/images by default since that matches the 'official' location defined by the SELinux policy and libvirt did not ever touch SELinux labels. In Fedora 11, we introduced sVirt, and libvirt now explicitly labels disk images to allow SELinux access. Thus, it doesn't really matter where you keep them anymore, within reaon. The only thing that will trip you up, is if there's something in the SELinux that prevents the guest traversing the parent directories. Worst case, you can use 'semanage' to define new locations in the policy very easily and this works for all versions of Fedora. So there's not really any compelling reason to bind mount, or mount partitions at /var/lib/libvirt//images. Just update the storage pool config to point to whereever you desir. You can create multiple storage pools too, if you wish to keep images in several different locations. The out of the box config you see, is just a starting point for your own usage, not a hard restriction.
Comment: It sure would be nice if Fedora Virtualization put all files related to a guest under a single directory the way VMware does.
We can't assume people are using file based disks in their guests, not assume a single location. So having config files in same location as the guest images is not an appropriate model to use by default.
Comment: It sure would be nice if there was some system-wide configuration parameter that defined where virtualization files went.
We could possibly allow for customization of the config file location of each libvirt driver if someone wanted to submit patches todo so... Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Sunday 28 June 2009 11:35:55 Daniel P. Berrange wrote:
My second question concerns the location of the disk images: what is the "best practical way" to move these out of the root partition? I have come up with these different approaches which could/may work:
1. change the "path" value in /etc/libvirt/storage/default.xml to point to a directory in a different partition.
2. Use a /etc/fstab entry to "bind mount" a directory in a separate partition onto /var/lib/libvirt/images
3. Just use an /etc/fstab entry to mount a partition on /var/lib/libvirt/images or, perhaps, on /var/lib/libvirt. [This could be done during system installation]
OK, any comments on these??
There's no real 'right' answer here, but I'd recommend 1. really.
There might not be a right answer but 1 is definitely a wrong answer. Changing /etc/libvirt/storage/default.xml to point to a directory in a separate partition with lost of free space was easy. However, when I tried to create a new guest using virt-manager and the creation "wizard", for the disk image, it still showed the free space available in the root partition (8GB) rather than that of the other partition (120GB), So I resorted to 2 which appears to work OK. My current thinking is to use option 3 ultimately. That way, I can get things configured when I install a new system. If you believe that option 1 should work then there appears to be a bug. Please clarify. If there is a bug, I will bugzilla it. Gene
On Tue, Jun 30, 2009 at 04:47:31AM -0400, Gene Czarcinski wrote:
On Sunday 28 June 2009 11:35:55 Daniel P. Berrange wrote:
My second question concerns the location of the disk images: what is the "best practical way" to move these out of the root partition? I have come up with these different approaches which could/may work:
1. change the "path" value in /etc/libvirt/storage/default.xml to point to a directory in a different partition.
2. Use a /etc/fstab entry to "bind mount" a directory in a separate partition onto /var/lib/libvirt/images
3. Just use an /etc/fstab entry to mount a partition on /var/lib/libvirt/images or, perhaps, on /var/lib/libvirt. [This could be done during system installation]
OK, any comments on these??
There's no real 'right' answer here, but I'd recommend 1. really.
There might not be a right answer but 1 is definitely a wrong answer.
No it isn't.
Changing /etc/libvirt/storage/default.xml to point to a directory in a separate partition with lost of free space was easy. However, when I tried to create a new guest using virt-manager and the creation "wizard", for the disk image, it still showed the free space available in the root partition (8GB) rather than that of the other partition (120GB),
Sounds like you didn't reload the configuration after editing it. If you edit the file directly you need to send SIGHUP to the daemon. It is better to use the 'virsh pool-edit' command which ensures it is immediately reloaded. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Tuesday 30 June 2009 06:19:46 Daniel P. Berrange wrote:
On Tue, Jun 30, 2009 at 04:47:31AM -0400, Gene Czarcinski wrote:
On Sunday 28 June 2009 11:35:55 Daniel P. Berrange wrote:
My second question concerns the location of the disk images: what is the "best practical way" to move these out of the root partition? I have come up with these different approaches which could/may work:
1. change the "path" value in /etc/libvirt/storage/default.xml to point to a directory in a different partition.
2. Use a /etc/fstab entry to "bind mount" a directory in a separate partition onto /var/lib/libvirt/images
3. Just use an /etc/fstab entry to mount a partition on /var/lib/libvirt/images or, perhaps, on /var/lib/libvirt. [This could be done during system installation]
OK, any comments on these??
There's no real 'right' answer here, but I'd recommend 1. really.
There might not be a right answer but 1 is definitely a wrong answer.
No it isn't.
Changing /etc/libvirt/storage/default.xml to point to a directory in a separate partition with lost of free space was easy. However, when I tried to create a new guest using virt-manager and the creation "wizard", for the disk image, it still showed the free space available in the root partition (8GB) rather than that of the other partition (120GB),
Sounds like you didn't reload the configuration after editing it. If you edit the file directly you need to send SIGHUP to the daemon. It is better to use the 'virsh pool-edit' command which ensures it is immediately reloaded.
So we are all working for the same base, I am running Fedora 11 with all updates applied as of the date of this message: libvirt-0.6.2-12.fc11.x86_64 selinux-policy-3.6.12-57.fc11.noarch (from updates-testing) kernel-2.6.29.5-191.fc11.x86_64 qemu-common-0.10.5-3.fc11.x86_64 qemu-kvm-0.10.5-3.fc11.x86_64 The processor is an AMD Phenom II 940 with 8 GB ram. OK, I removed my bind-mount (which works) and edited /etc/libvirt/storage/default.xml so that "path" points to the directory in a separate "extra" partition from root. Root currently has about 8GB free space and the "extra" partition has 120GB free space. The file context for the directory on the "extra" partition has been setup with semanage. I am not sure just what daemon you are talking about doing a SIGHUP to but I did: "/etc/init.d/libvirtd restart" When I try to create a new guest, the free space for the disk image is 8GB (same as root). OK, reboot the host! [this should certainly force the configuration to be re- read] I try creating a new guest again but, again, the free space for the disk image is 8GB (still the same as root). Now I change the "path" in /etc/libvirt/storage/default.xml back to /var/lib/libvirt/images", add the bind-mount entry to /etc/fstab to mount the "extra" partition on /var/lib/libvirt/images, reboot ... now when I try to create a new guest, the amount of free space for the disk image shows as 120GB. My guess is that "/var/lib/libvirt/images" is hardcoded somewhere in the code. What am I missing? * * * * * BTW, the "virsh pool-edit" may be a great command but when I try it, it says it requires the "<pool>" option. Also, when I do "virsh -h", a large number of commands are listed none of which I can find any documentation on through google (let alone on any man page). * * * * * Gene
participants (2)
-
Daniel P. Berrange -
Gene Czarcinski