Well, this is because of a peculiarity with Xen PV domains.  In Xen PV guests, you specify a "maxmem" and a "memory" parameter in the configuration file.  The "maxmem" parameter is presented to the guest as the end of the e820 map, hence the end of real memory as far as the guest is concerned (you can see that in the output of dmesg from the guest).  When the balloon driver in the guest loads, it will "allocate" (maxmem - memory), so that free inside the guest looks like it only has 1GB.  Later on, you can balloon back up, which means that the balloon driver "releases" memory back to the domain (but never above the maxmem parameter, since that's what's in the e820 map for the guest). I would say for ESX driver, you probably want to follow the Qemu model; it's the model that KVM and even Xen FV guests follow, so seems to be more common. I hope that helps somewhat.

Matthias Bolte wrote: > Hello, > > I just took a look at the driver functions for SetMaxMemory and > SetMemory, as they are not implemented yet for the ESX driver and > Daniel Veillard was a bit surprised that they are missing, as he > expects them to be simple to implement. The problem is that I'm not > sure how the memory model of ESX maps to SetMaxMemory vs. SetMemory. > > An ESX virtual machine has a defined memory size. That's the size > reported to the guest OS as the available "physical" memory size. > Beside this ESX allows the user to control how the hypervisor > satisfies this "physical" memory size. You can define a reservation > and an upper limit. The hypervisor will at least use the reserved > amount of real physical memory to satisfy the "physical" memory size, > but will not use more than the upper limit. > > How does this map to SetMaxMemory and SetMemory? My first assumption > was, SetMaxMemory defines the "physical" memory size and SetMemory > defines the upper limit of real physical memory to satisfy the > "physical" memory size. This assumption seems to be in sync with the > QEMU driver from just looking at the code. But with Xen it seems to be > different. If I call SetMaxMemory and SetMemory with 2GB than free > inside the domain reports 2GB total memory. After I call SetMemory > with 1GB, free reports 1GB of total memory. > > I'm confused. So, what is the intended semantic for SetMaxMemory and SetMemory? Well, this is because of a peculiarity with Xen PV domains. In Xen PV guests, you specify a "maxmem" and a "memory" parameter in the configuration file. The "maxmem" parameter is presented to the guest as the end of the e820 map, hence the end of real memory as far as the guest is concerned (you can see that in the output of dmesg from the guest). When the balloon driver in the guest loads, it will "allocate" (maxmem - memory), so that free inside the guest looks like it only has 1GB. Later on, you can balloon back up, which means that the balloon driver "releases" memory back to the domain (but never above the maxmem parameter, since that's what's in the e820 map for the guest). I would say for ESX driver, you probably want to follow the Qemu model; it's the model that KVM and even Xen FV guests follow, so seems to be more common.

2009/5/22 Daniel P. Berrange <berrange(a)redhat.com&gt;: > Actually QEMU, KVM, Xen PV, Xen FV all  follow the same model, providing > you have the balloon driver available in the guest. 'maxmem', confusingly > calling <memory> in the XML sets the maximum possible memory  for the > guest, as exposed in e820 maps. When the guest runs this maximum can be > reduced setting 'memory', confusingly called <currentMemory> in the XML, > to a lower value. The hosts talks to the balloon driver in the guest and > asks it to release memory. This isn't a guarenteed lower limit, since it > relies on guest cooperation, but at least the guest is aware of what > the host it telling it todo. > > Depending on bugs in the guest ballloon driver, the 'free' command may or > may not, update the 'total memory' setting in the guest. > > What Matthias is talking about wrt  VMWare ESX is about how the hypervisor > satisfies the memory allocation for the guest, eg how much real RAM ist > guarantees, with the rest of guest RAM susceptible to swapping. This is > more of a tuning parameter, and does not map onto the libvirt memory/maxmem > settings. Well, ESX has support for ballooning, too. If the balloon driver is installed inside the guest, then ESX does the same as you described for QEMU etc. If you set the memory value (the limit in ESX terms) below the max-memory value, ESX lets the balloon driver allocate memory in the guest to "steal" it from the guest. But ESX can do this even without the balloon driver by swapping, as you described in the third paragraph. So IMHO this maps somewhat onto the libvirt memory/max-memory semantics, if the balloon driver is installed. See also http://www.vmware.com/pdf/esx3_memory.pdf page 3-4, "Memory Balloon Driver" and "Swapping".