Re: [Libvir] Re; virDomainBlockStats
Ok, question;
Does TAP:AIO work on networked filesystems .. my testing says not.
(I've tried on local filesystems and seem to have it partially working)
XEN without a good underlying cluster filesystem is to say the least "limited".
If you can't use AIO in this sort of environment, it also becomes limited .. and if
libvirt requires this, it means libvirt can't be used on large-scale roll-outs on
(certain?) network filesystems.
(I'm using gluster)
Interestingly (looking at the Xen list re; performance problems) I'm getting >
65Mb/sec on my DomU's .vs. 70Mb/sec on my host nodes Dom0 ... and it's proving to
be reliable atm ..
Be interesting to know why Xen only documents "file" given the critical nature /
apparent flaws in the driver.
fyi; I run root fs's off DomU's but store data on separately mounted shared
filesystems .. (more efficient when it comes to fail-over / file-system self-heal in the
event of a filesystem node outage)
Is there "no" way of forcing a loopback interface to flush itself?
Gareth.
----- Original Message -----
step 3.: "Daniel P. Berrange" <berrange(a)redhat.com>
To: "Gareth Bult" <gareth(a)encryptec.net>
Cc: "libvir-list" <libvir-list(a)redhat.com>
Sent: 27 January 2008 02:14:26 o'clock (GMT) Europe/London
Subject: Re: [Libvir] Re; virDomainBlockStats
On Sat, Jan 26, 2008 at 11:29:54PM +0000, Gareth Bult wrote:
Is there any documentation to the effect that "file" is
bad??
The official XEN documentation lists "file" as the standard and makes no
mention of "aio".
The upstream documentation leaves alot to be desired. First of all I
should say, there is a difference between file: with PV vs file: with
HVM. file: with HVM has no problem, because all IO is handled by QEMU.
It is only file: with PV guests that is flawed. This is because it
uses loopback devices to acess the file. Loop devices cache data writes
in memory for an undefined amount of time, even if the guest requests
a sycn to disk. The result is that if the host OS crashes your guest
disks can loss huge amounts of data that they thought were already synced
to disk. Not even a journalling FS helps, because there's no write ordering
guarentees.
tap:aio: addresses all these issues by doing Direct IO + Async I/O
to allow you to have multiple outstanding I/O operations, to avoid the
host OS buffer cache, and to provide firm guarentee that the data is on
disk when the guest asks for it to be.
On Sat, Jan 19, 2008 at 05:18:58PM +0000, Gareth Bult wrote:
> Mmm,
>
> Interesting ..
>
> First off, xentop doesn't display block device stats for tap:aio based systems
and it does for file.
> Second, tap:aio generated kernel Ooops's when you shutdown a DomU.
>
> Not exactly what I'd call mainstream (!)
That must be a flaw in Ubuntu's kernels. tap:aio works flawlessly
in Fedora / RHEL and is the only supported option, because file:
has catastrophic data loss issues during host crashes.
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|