[libvirt] <soapbox>Thoughts on Open Source Hypervisor management

Over the the last few years I've setup and experimented with a large number of vm technologies. Everything from x86 based stuff like VMware, kvm, qemu, xen, to more exotic things like hercules, and z/VM. So far the solution I've found to be the most baked is VMware ESX with Virtual Center. It's pretty sweet to set up a DRS cluster and have VM's load balance across available hardware. However, It's super expensive and is in bed with windows way to much. i.e. Virtual Center and the Virtual Infrastructure Client only run on windows don't have LDAP auth and only use MSQL. I've also been working implementing a provisioning solution for our developers. We do automated installation of hundreds of machines of which many are underutilized. Many are newer machines with some pretty beefy specs (8 cores and at least 8 gigs of ram). I've done a lot of thinking in a vacuum about what would be ideal. There's a lot out there I don't know so I hope you give me some grace as I talk. So I want to be able to pull these machines into a cluster of hypervisors; a VM fabric aggregated via software. Or thought of in another way a p2p vm cluster. In some ways I think oVirt is the closest and in someways I think simply extending libvirt would be better and sometimes I think it should be a new project. So I thought I'd post this and see what people think. Since I'm still unsure as to where the proper place to abstract this is, I'll use the working package name of genet for the rest of the discussion. virtualization solutions for more than local vm's is one of management. They're great for one machine but as soon as you get to more than one machine there is a lot of human administrative overhead. oVirt seems like it could be really good, but in an existing environment it requires a lot of substantial changes and requires significant modification to work with an existing provisioning infrastructure. So as I imagined a silver bullet I used biology as a metaphor and used started thinking with the idea of "every piece contains the image of the whole." So the here's the 10,000 ft overview of an idea. 1. cluster administration is done from the command line 2. cluster administration can be performed from any node. 3. a new node can join a cluster on a local subnet with one command. 4. local storage resources are presented to the cluster so there is no need to have predefined NAS/SAN/iSCSI 5. cluster will load balance vm instances from node to node. 6. a node shouldn't need more than one nic but adding additional nic's provides failover and load balancing. I'm kind of ignoring storage at this point, but I think it's the biggest pieces of the puzzle as long as migration needs network/san storage. The ideal scenario would be something really simple like if I have SAN connectivity use that else I use some sort of cluster file system which adds in local storage and doesn't require reformating, but that might be wistful thinking. Use cases 1. looking for clusters on the same subnet via avahi # [yum|apt-get] install genet # genet list clusters forsythia hazelnut sumac # rhizome info sumac Password: sumac info: 8 nodes 64 Cores 128 G Ram 1.5 T storage 50% Cluster CPU utilized 75% Memory utilized 80% Storage utilized 2. a machine joins an existing cluster # genet join sumac Password: localhost has successfully joined sumac # Help output #genet -h usage: genet [options] command genet is a simple command line interface for creating and maintaining a cluster of hypervisors. Commands: create cluster [cluster name] create vm [vm name] (probably integrates with virt-install) evacuate [hostname] - migrate all vm instances off of [hostname] (defaults to localhost) list clusters [hostname] - list clusters on local subnet or via hostname on another network/subnet list vm [cluster] - list all vm instances in a cluster defaults to cluster local node is in. console [vmname] - virt-viewer to vmname So this all begs the the statement, "show me the code" and I think that perfectly valid. I may write some code, but I don't want to duplicate effort and I want to find out what other people are thinking. I welcome your thoughts. </soapbox> Regards, -- Nathan Charles