[Libvir] can't start windows guest domains
Hi. I'm running F7 x86_64 with kernel 2.6.20-2931.fc7xen and libvirt-0.3.2-1.fc7. Since i updated to the latest versions i can't start any windows domains anymore. I thought undefine and reinstall the domains could resolve the problem, but i get the exact same error message. There is also a F7 x86_64 guest domain runnning fine on the same host. Why i can't start any windows domains anymore? [thomas@srv1 ~]$ sudo virt-install -d -n sbs -r 512 --vcpus=1 -f /dev/VolGroup01/sbs -w bridge:virbr0 --vnc -k DE -v --arch=i386 --os-type=windows --os-variant=win2k3 -c /var/www/html/install/SBS2003R2DE_DVD.iso Mon, 10 Sep 2007 20:38:28 DEBUG Path is block file: Assuming Block disk type. Disk /dev/VolGroup01/sbs is already in use by another guest! Do you really want to use the disk (yes or no)? y Starting install... libvir: Xen Daemon error : GET operation failed: libvir: Xen Daemon error : GET operation failed: Mon, 10 Sep 2007 20:38:31 DEBUG Creating guest from '<domain type='xen'> <name>sbs</name> <currentMemory>524288</currentMemory> <memory>524288</memory> <uuid>5f73764f-897c-401c-e9e5-17cb21333334</uuid> <os> <type arch='i386'>hvm</type> <loader>/usr/lib/xen/boot/hvmloader</loader> <boot dev='cdrom'/> </os> <features> <apic/><acpi/><pae/> </features> <on_poweroff>destroy</on_poweroff> <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <vcpu>1</vcpu> <devices> <emulator>/usr/lib64/xen/bin/qemu-dm</emulator> <console device='pty'/> <disk type='block' device='disk'> <source dev='/dev/VolGroup01/sbs'/> <target dev='hda'/> </disk> <disk type='file' device='cdrom'> <source file='/var/www/html/install/SBS2003R2DE_DVD.iso'/> <target dev='hdc'/> <readonly/> </disk> <interface type='bridge'> <source bridge='virbr0'/> <mac address='00:16:3e:33:eb:7b'/> </interface> <graphics type='vnc' port='-1' keymap='DE' /> </devices> </domain> ' libvir: Xen Daemon error : POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')") virDomainCreateLinux() failed POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')") Domain installation may not have been successful. If it was, you can restart your domain by running 'virsh start sbs'; otherwise, please restart your installation. Mon, 10 Sep 2007 20:38:31 ERROR virDomainCreateLinux() failed POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')") Traceback (most recent call last): File "/usr/bin/virt-install", line 474, in <module> main() File "/usr/bin/virt-install", line 438, in main dom = guest.start_install(conscb,progresscb) File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 710, in start_install return self._do_install(consolecb, meter) File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 727, in _do_install self.domain = self.conn.createLinux(install_xml, 0) File "/usr/lib64/python2.5/site-packages/libvirt.py", line 563, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: virDomainCreateLinux() failed POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')") [thomas@srv1 ~]$ free total used free shared buffers cached Mem: 6154240 1312276 4841964 0 144660 600632 -/+ buffers/cache: 566984 5587256 Swap: 8323064 0 8323064 kind regards, Thomas
Thomas Antony wrote:
Mon, 10 Sep 2007 20:38:31 ERROR virDomainCreateLinux() failed POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')")
I think I've seen this one, and if so then it has been fixed but the problem may not have made it out to Fedora yet (it was a bug in Xen itself). It was caused by Domain-0 not being able to balloon down its memory usage by enough to accomodate the new domain. The workaround is to reduce the amount of memory allocated to Domain-0 manually first, eg: virsh setmaxmem 0 128000 but you'll have to modify '128000' to be some amount of memory in kilobytes which is the current Domain-0 allocation, less what is needed for your new domain, less a bit more for good luck... Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Richard W.M. Jones schrieb:
Thomas Antony wrote:
Mon, 10 Sep 2007 20:38:31 ERROR virDomainCreateLinux() failed POST operation failed: (xend.err "Error creating domain: (12, 'Cannot allocate memory')")
I think I've seen this one, and if so then it has been fixed but the problem may not have made it out to Fedora yet (it was a bug in Xen itself).
It was caused by Domain-0 not being able to balloon down its memory usage by enough to accomodate the new domain. The workaround is to reduce the amount of memory allocated to Domain-0 manually first, eg:
virsh setmaxmem 0 128000
but you'll have to modify '128000' to be some amount of memory in kilobytes which is the current Domain-0 allocation, less what is needed for your new domain, less a bit more for good luck...
Rich.
Hi Richard, I tried "virsh setmaxmem 0 512000" and then the system hung and i had to reset it. After the system is started again i can start the windows guest domains now, but the state is blocked. Why is the state blocked and not running? Which tools can show me why the state is blocked? I'm running F7 on a Dell PE2950 with 8GB RAM. It would be nice if you could explain how much RAM i should allocate to Dom0 and what are the minimum requirements for Dom0. Thank you in advance, Thomas
Thomas Antony wrote:
I tried "virsh setmaxmem 0 512000" and then the system hung and i had to
Oh that's nasty. Were there any logs produced at the time the system hung? Basically that's a bug in Xen anyway -- all that 'virsh setmaxmem' does is to make a Xen hypercall to balloon down Domain-0, so all the hard stuff happens in Xen.
reset it. After the system is started again i can start the windows guest domains now, but the state is blocked. Why is the state blocked and not running? Which tools can show me why the state is blocked?
'Blocked' is the normal state (unless you mean 'Paused'). Domains are only 'Running' when they are using CPU. Can you ping the domains (if they have a network connection)? Or view their console?
I'm running F7 on a Dell PE2950 with 8GB RAM. It would be nice if you could explain how much RAM i should allocate to Dom0 and what are the minimum requirements for Dom0.
I would expect that allocating 512 MB to Dom0 as you did should be sufficient (unless it's running a huge database or something like that). The problem, though, isn't likely to be the allocation, but the fact that you are using 'virsh setmaxmem' to balloon it down from something larger to 512 MB. In any case, I'm not sure what the problem is - unless you can find something in a logfile, it's hard to tell. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
'Blocked' is the normal state (unless you mean 'Paused'). Domains are only 'Running' when they are using CPU. Doesn't a guest domain (or any OS) always use CPU, at least 0,01% or something similiar? For example i'm running a Windows SBS 2003 as Domain Controller with Sharepoint Services and Exchange Server and virsh list tells me the domain is blocked.
Can you ping the domains (if they have a network connection)? Or view their console?
I can access the domains with vnc. Since i undefined and redefined all domains i can't ping the guest from the host and when i am connected with vnc i can't ping anything. The windows guest does have a nic and has an ip in the same network as the the host. It should be a bridge setup. <interface type='bridge'> <source bridge='virbr0'/> <target dev='vif6.0'/> <mac address='00:16:3e:33:eb:7b'/> <script path='vif-bridge'/> </interface> ifconfig tells me there is a peth0, tap, virbr0 and vif6.0 interface.
I'm running F7 on a Dell PE2950 with 8GB RAM. It would be nice if you could explain how much RAM i should allocate to Dom0 and what are the minimum requirements for Dom0.
I would expect that allocating 512 MB to Dom0 as you did should be sufficient (unless it's running a huge database or something like that).
The problem, though, isn't likely to be the allocation, but the fact that you are using 'virsh setmaxmem' to balloon it down from something larger to 512 MB. In any case, I'm not sure what the problem is - unless you can find something in a logfile, it's hard to tell.
After i allocated 512MB RAM to Dom0 the kernel printed many out of memory messages like this and began to kill services. Sep 11 08:54:04 srv1 kernel: mysqld invoked oom-killer: gfp_mask=0x200d2, order=0, oomkilladj=0 Sep 11 08:54:04 srv1 kernel: Sep 11 08:54:05 srv1 kernel: Call Trace: Sep 11 08:54:05 srv1 kernel: [<ffffffff802b17e6>] out_of_memory+0x6d/0x1ce Sep 11 08:54:05 srv1 kernel: [<ffffffff8020ef31>] __alloc_pages+0x23e/0x2c7 Sep 11 08:54:05 srv1 kernel: [<ffffffff802329cb>] read_swap_cache_async+0x42/0xd1 Sep 11 08:54:05 srv1 kernel: [<ffffffff802b515b>] swapin_readahead+0x4e/0x77 Sep 11 08:54:05 srv1 kernel: [<ffffffff8020984a>] __handle_mm_fault+0xcad/0x1182 Sep 11 08:54:05 srv1 kernel: [<ffffffff80262de5>] do_page_fault+0xf23/0x128d Sep 11 08:54:05 srv1 kernel: [<ffffffff802c6d22>] core_sys_select+0x1bb/0x264 Sep 11 08:54:05 srv1 kernel: [<ffffffff80260827>] error_exit+0x0/0x79 Sep 11 08:54:05 srv1 kernel: [<ffffffff80270798>] xen_send_IPI_mask+0x0/0xd2 Sep 11 08:54:05 srv1 kernel: [<ffffffff8025cd87>] copy_user_generic_string+0x17/0x40 Sep 11 08:54:05 srv1 kernel: [<ffffffff80216884>] sys_select+0x11c/0x17c Sep 11 08:54:11 srv1 kernel: [<ffffffff8025b2ff>] tracesys+0xb2/0xb7 Sep 11 08:54:12 srv1 kernel: Sep 11 08:54:12 srv1 kernel: Mem-info: Sep 11 08:54:12 srv1 kernel: DMA per-cpu: Sep 11 08:54:12 srv1 kernel: CPU 0: Hot: hi: 186, btch: 31 usd: 157 Cold: hi: 62, btch: 15 usd: 48 Sep 11 08:54:13 srv1 kernel: CPU 1: Hot: hi: 186, btch: 31 usd: 117 Cold: hi: 62, btch: 15 usd: 11 Sep 11 08:54:13 srv1 kernel: CPU 2: Hot: hi: 186, btch: 31 usd: 30 Cold: hi: 62, btch: 15 usd: 48 Sep 11 08:54:13 srv1 kernel: CPU 3: Hot: hi: 186, btch: 31 usd: 171 Cold: hi: 62, btch: 15 usd: 47 Sep 11 08:54:13 srv1 kernel: Active:72 inactive:83 dirty:0 writeback:127 unstable:0 free:2834 slab:5953 mapped:76 pagetables:2898 Sep 11 08:54:14 srv1 kernel: DMA free:11336kB min:11324kB low:14152kB high:16984kB active:288kB inactive:368kB present:8019468kB pages_scanned:100 all_unrecl aimable? no Sep 11 08:54:14 srv1 kernel: lowmem_reserve[]: 0 0 0 Sep 11 08:54:14 srv1 kernel: DMA: 14*4kB 4*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 11336kB Sep 11 08:54:14 srv1 kernel: Swap cache: add 50937, delete 50841, find 216/359, race 0+0 Sep 11 08:54:14 srv1 kernel: Free swap = 8123812kB Sep 11 08:54:14 srv1 kernel: Total swap = 8323064kB Sep 11 08:54:14 srv1 kernel: Free swap: 8123812kB Sep 11 08:54:14 srv1 kernel: dsm_sa_datamgr3 invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Sep 11 08:54:14 srv1 kernel: Sep 11 08:54:21 srv1 kernel: Call Trace: ... ... ... Thomas
On Tue, Sep 11, 2007 at 06:51:29PM +0200, Thomas Antony wrote:
I'm running F7 on a Dell PE2950 with 8GB RAM. It would be nice if you could explain how much RAM i should allocate to Dom0 and what are the minimum requirements for Dom0.
I would expect that allocating 512 MB to Dom0 as you did should be sufficient (unless it's running a huge database or something like that).
The problem, though, isn't likely to be the allocation, but the fact that you are using 'virsh setmaxmem' to balloon it down from something larger to 512 MB. In any case, I'm not sure what the problem is - unless you can find something in a logfile, it's hard to tell.
After i allocated 512MB RAM to Dom0 the kernel printed many out of memory messages like this and began to kill services.
Well pretty clearly then you are running so many processes in Dom0 that you need more than just 512 MB for Dom0. This isn't a bug in libvirt, you merely reduced the RAM available to below that which is needed for the set of applications you are running. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
participants (3)
-
Daniel P. Berrange -
Richard W.M. Jones -
Thomas Antony