On Fri, Mar 07, 2014 at 10:04:48PM +0400, Roman Bogorodskiy wrote:
Daniel P. Berrange wrote:
On Fri, Mar 07, 2014 at 07:49:04PM +0400, Roman Bogorodskiy wrote:
Daniel P. Berrange wrote:
On Fri, Mar 07, 2014 at 06:15:24PM +0400, Roman Bogorodskiy wrote:
Daniel P. Berrange wrote:
All those mangled messages are being printed out by libvirt client code, and it is totally expected that log messages libvirt prints are mangled in this way for the console.
What's more important is what data coming from the guest looks like.
All those spaces come from the stream as well. For example, I've added the following debugging to daemonStreamHandleRead() into daemon/stream.c:
732 ret = virStreamRecv(stream->st, buffer, bufferLen); 733 VIR_ERROR("buffer = [%s]", buffer);
And it prints stuff like:
2014-03-07 13:29:34.060+0000: 34489787392: error : daemonStreamHandleRead:733 : buffer = [ File "/usr/bin/cloud raise DataSourceNotFoundException(msg) ] 2014-03-07 13:29:34.060+0000: 34489787392: error : daemonStreamHandleRead:733 : buffer = [ ] 2014-03-07 13:29:34.061+0000: 34489787392: error : daemonStreamHandleRead:733 : buffer = [ ] 2014-03-07 13:29:34.061+0000: 34489787392: error : daemonStreamHandleRead:733 : buffer = [ ]
The debug logs are *always* going to be strange when you have the terminal in raw mode. If you're debugging the console support you really must not try to use stderr for debug logs - you'll never get sane results. Set libvirt to log to a file or syslog instead.
That's strange because libvirtd and virsh use different terminals for logging, but I've disabled stderr logging to be on the safe side.
Anyway, I've conducted an experiment like you suggested:
1. run 'virsh' on linux to freebsd libvirtd, output is mangled:
http://people.freebsd.org/~novel/misc/ttyweirdness2.png
2. run 'virsh' on freebsd to linux libvirtd, output is good:
http://people.freebsd.org/~novel/misc/ttyweirdness3.png
Guest uses the same image and the same domain xml.
Also, I removed all my custom debug stuff from virsh except mutex lock (as it doesn't work otherwise). So the problem seem to be on the libvirtd side...
Agreed, it is either libvirtd or bhyve that's the source of the mangling then.
Very unlikely that it's bhyve, as I'm testing with qemu (plan to get to bhyve when I'll be able to get it fixed for qemu).
Oh, I didn't realize you were testing this with QEMU on BSD! One further idea I just had - in QEMU code util/qemu-openpty.c we call cfmakeraw()/tcsetattr on the PTY slave file descriptor. On Linux at least this affects both ends of the device, but I wonder if there's perhaps a need to call this on the master file descriptor as well as or instead of on the slave file descriptor ? Or perhaps the client (in this case libvirtd) itself has to call cfmakeraw/tcsetattr since although its opening the same slave, it is a different file descriptor for the slave
And considering that 'cu' works properly, it's most likely the daemon. I'll continue looking in this direction. Thanks for the pointer, that helped to narrow down the problem indeed.
True, but interesting to strace 'cu' and libvirtd to see what differences they have if any when opening the slave FD.
Roman Bogorodskiy
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|