On 09/22/2010 03:07 AM, Daniel P. Berrange wrote:

On Tue, Sep 21, 2010 at 04:37:26PM +1000, Justin Clift wrote:

...

Hi all,

This is for anyone using the Homebrew package management system on Mac OS X. :)

A first working (but still experimental) libvirt formula is online:

http://github.com/justinclift/libvirt

If this is intended to be rnu with proper release tarballs, then you should change autogen.sh to ./configure, so you avoid needing automake/autoconf/etc on OS-X.

Very good point. When 0.8.5 is out, we should be able to do this. As long as we get the minor compilation nits fixed upstream first, which is being worked on.

Also, does libvirtd not actually work ? If QEMU itself builds on OS-X, then in theory it ought to be possible to build libvirtd and manage it.

So far, the configure script dies when it detects there's no device-mapper/devel package installed. Might be able to work around this by disabling options on the configure line, but didn't try it. Getting the daemon working would be useful. We'd be able to manage (at least) VirtualBox instances running on OS X that way.

Do you need a depends_on for libxml2& portablexdr too ?

Not for libxml2, as the version that comes with OS X works. (tried it both ways). Didn't think to look for portablexdr, but will check that out. :)

What's the situation with Python on OS-X ? Any liklihood of python bindings working ?

Python comes with OS X "out of the box". Running version 10.6.4 of Mac OS X (latest release) here, and it includes Python 2.6.4. So, I think we'll be in good shape. :)

If you add curl + xmlrpc-c then the ESX driver ought to work. If there's a virtualbox port to OS-X that driver should work too. I'd be interest to see what configure prints as the summary for features it enabled...

By default, pretty much everything is set to "no" from the configure output by default. (./configure --without-libvirt) Didn't put any effort into getting things running other than the basic client layer, but can try things out without too much hassle. :) Regards and best wishes, Justin Clift

On 09/22/2010 10:52 PM, Daniel Veillard wrote: <SNIP>

...

If you have time to test it and report success/failure/(etc), please do.

I would suggest to put the experimental binaries that you managed to build on libvirt.org ~ftp/libvirt/osx/ , you should have write access to that directory and this may help getting more people to try it out even if limited. Just put a README or something along the binaries.

Thanks. Will be looking at proper binary packages for it in a bit, so that will be appropriate then. :) The Homebrew packaging system is possibly a "bit different" from others. It's a system that uses git as their main versioning tool, pulling from a central git repository on github: http://github.com/mxcl/homebrew The master branch there is used for the mainline stuff, and other branches are used for various other things. People are encouraged to fork that on github, develop their own package script(s), then submit them. If accepted, they get pulled back into the main git repository for everyone's use. So, we're on track with this. :) The github.com URL I gave before is a properly created fork (using the github tools), and should be of direct use to a Homebrew user. They can use that URL, and Homebrew will work with it. Or so the theory goes. :>

Hi Justin, On Thu, Oct 7, 2010 at 05:34, Justin Clift <jclift@redhat.com> wrote:

On 09/22/2010 11:56 PM, Justin Clift wrote: <snip>

As a follow up, our libvirt formula has been pulled into the upstream repository, and is now available on OSX to everyone using Homebrew:

 http://mxcl.github.com/homebrew/

I thought I'd give it a try, to see how far I get on Leopard (10.5). ranlib: file: .libs/libvirt_test.a(libvirt_util_la-bridge.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_util_la-macvtap.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_util_la-stats_linux.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_driver_la-driver.o) has no symbols ranlib: file: .libs/libvirt_test.a(close-hook.o) has no symbols ranlib .libs/libvirt_test.a ranlib: file: .libs/libvirt_test.a(libvirt_util_la-bridge.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_util_la-macvtap.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_util_la-stats_linux.o) has no symbols ranlib: file: .libs/libvirt_test.a(libvirt_driver_la-driver.o) has no symbols ranlib: file: .libs/libvirt_test.a(close-hook.o) has no symbols rm -fr .libs/libvirt_test.lax creating libvirt_test.la (cd .libs && rm -f libvirt_test.la && ln -s ../libvirt_test.la libvirt_test.la) make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Exit status: 2 http://github.com/mxcl/homebrew/blob/master/Library/Formula/libvirt.rb#L34 ==> Environment HOMEBREW_VERSION: 0.7 HEAD: b8490eda4bd66260521fad17140f1fa550f62c55 HOMEBREW_PREFIX: /usr/local HOMEBREW_CELLAR: /usr/local/Cellar HOMEBREW_REPOSITORY: /usr/local HOMEBREW_LIBRARY_PATH: /usr/local/Library/Homebrew Hardware: dual-core 64-bit penryn OS X: 10.5.8 Kernel Architecture: i386 Ruby: 1.8.6-369 /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby Xcode: 3.1.4 GCC-4.0: build 5493 GCC-4.2: build 5577 LLVM: N/A MacPorts or Fink? false X11 installed? true ==> Build Flags CC: /usr/bin/cc => /usr/bin/gcc-4.0 CXX: /usr/bin/c++ => /usr/bin/c++-4.0 LD: /usr/bin/cc => /usr/bin/gcc-4.0 CFLAGS: -O3 -march=nocona -mfpmath=sse -w -pipe CXXFLAGS: -O3 -march=nocona -mfpmath=sse -w -pipe MAKEFLAGS: -j2 Error: Failure while executing: make Kind regards, Ruben

On Fri, Oct 22, 2010 at 08:54, Justin Clift <jclift@redhat.com> wrote:

On 10/22/2010 05:18 PM, Ruben Kerkhof wrote: <snip>

...

I thought I'd give it a try, to see how far I get on Leopard (10.5).

Hey Ruben, thanks for the attempt. :)

Would you be ok to try a slightly updated version of the libvirt formula?  If so, you just need to update the libvirt.rb formula file, to use this url and md5:

 url 'http://justinclift.fedorapeople.org/libvirt_experimental/libvirt-0.8.4-7.tar...'  md5 '2b8948e336070c94c5278ccd36495709'

It uses a somewhat newer source snapshot for libvirt than the previous one, plus Mitchell Hashimoto has been hacking away at it to further increase it's robustness on OSX.

No guarantees, but I think it's worth a shot, just in case. :)

?

Ah, great, I'll give it a shot when I get home in a few hours! Thanks, Ruben

On Fri, Oct 22, 2010 at 18:19, Mitchell Hashimoto <mitchell.hashimoto@gmail.com> wrote:

On Thu, Oct 21, 2010 at 11:18 PM, Ruben Kerkhof <ruben@rubenkerkhof.com> wrote:

...

I thought I'd give it a try, to see how far I get on Leopard (10.5).

From what I know following this list, I don't think anyone has ever tried to compile for Leopard (10.5) and has focussed exclusively on Snow Leopard (10.6), so there very well may be errors during compilation/linking.

I'll wait for you to try the new formula but if that errors as well then I may have to spin up  a leopard VM to play around see if I can figure out what is going on there.

Mitchell

It's getting a bit further, the logs are at http://fpaste.org/DC3R/ It seems to have some problems with libxml2. I've also tried with libxml2-2.7.7 installed by brew, but that didn't help Readline has always been a problem on Tiger/Leopard. I remember something about it being libedit in readline-compatibility mode. Thanks for your help! Ruben

On Sat, Oct 23, 2010 at 10:55:16AM +1100, Justin Clift wrote:

On 10/23/2010 06:06 AM, Ruben Kerkhof wrote: <snip>

...

It's getting a bit further, the logs are at http://fpaste.org/DC3R/

It seems to have some problems with libxml2.

Daniel, any idea what would cause this libxml2 failure?

GEN virt-pki-validate GEN virt-xml-validate.1 GEN virt-pki-validate.1 GEN virsh.1 CCLD virsh Undefined symbols: "_xmlSaveToBuffer", referenced from: _cmdCPUBaseline in virsh-virsh.o "_rl_completion_matches", referenced from: _vshReadlineCompletion in virsh-virsh.o _vshReadlineCompletion in virsh-virsh.o ld: symbol(s) not found collect2: ld returned 1 exit status make[3]: *** [virsh] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2

Ruben, the readline one can probably be worked around in the short term. I've just submitted a patch to libvirt, allowing the use of readline can be disabled. (easiest way forward)

We'll need to figure out the libxml2 problem though. Probably not going to be today, more likely sometime next week. (with luck)

The only thing strange about this entry point in libxml2 is that it got dropped by mistake in 2.6.11 and readded in 2.6.23 so if you have a version of libxml2 in that range that may raise that problem history of all symbols since 2.4.30 available from http://xmlsoft.org/symbols.xml Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

On 10/23/2010 06:06 AM, Ruben Kerkhof wrote: <snip>

It seems to have some problems with libxml2. I've also tried with libxml2-2.7.7 installed by brew, but that didn't help Readline has always been a problem on Tiger/Leopard. I remember something about it being libedit in readline-compatibility mode.

It also looks like there's a specific "readline" formula in brew too. Would you be ok to add a dependency for it to your libvirt formula, and try again? It'll just mean adding this line to your libvirt.rb formula file: depends_on "readline" (just add it after the existing 'depends_on "gnutls"' line) My understanding (which might need adjustment), is that the Homebrew readline build is normally hidden from things (keg_only). This can be modified for a specific formula, by adding that depends_on line. So, that's also worth a shot I reckon, to see if it stops those readline specific errors from turning up in the libvirt compile run. You ok to try it out? Regards and best wishes, Justin Clift

On 10/24/2010 05:26 AM, Ruben Kerkhof wrote: <snip>

Works great, thanks!

Hey, that's good news. :) That means we just need to figure the libxml2 errors out, then we can tweak the libvirt formula so it works on Leopard as well. :)

On 10/24/2010 06:49 AM, Ruben Kerkhof wrote: <snip>

That depends_on line in the formula also works for libxml2 :-)

Hey, good work. :) Looks like I need to tweak the libvirt formula next. ;> As a thought, what sort of connection URL did you use with virsh in your example, and what sort of remote box were you connecting to? Asking because I've been having troubles with qemu+ssh:// connections to remote RHEL 6 and Fedora boxes, due to what I think is an interaction with PolicyKit on the remote boxes. Kind of thinking it won't happen for remote Ubuntu boxes though as they don't have PolicyKit set up in the same way. The more data I have around this, the better, just in case it turns out to be something different. :)

On Sat, Oct 23, 2010 at 22:13, Justin Clift <jclift@redhat.com> wrote:

On 10/24/2010 07:04 AM, Justin Clift wrote:

...

On 10/24/2010 06:49 AM, Ruben Kerkhof wrote: <snip>

...

That depends_on line in the formula also works for libxml2 :-)

Hey, good work. :)

Looks like I need to tweak the libvirt formula next. ;>

Ruben, would you be ok to test the following code snippet for me, in the libvirt formula?

   if MACOS_VERSION < 10.6      # Needed on Leopard, but not Snow Leopard      depends_on "readline"      depends_on "libxml2"    end

If so, it should go after the other depends_on lines, like this:

   depends_on "gawk"    depends_on "gnutls"

   if MACOS_VERSION < 10.6      # Needed on Leopard, but not Snow Leopard      depends_on "readline"      depends_on "libxml2"    end

If that works, then we'll be able to get that into the libvirt formula pronto. :)

Regards and best wishes,

Justin Clift

Yes, this works perfectly. Thanks, Ruben

On 10/24/2010 12:05 PM, Justin Clift wrote:

On 10/24/2010 09:28 AM, Ruben Kerkhof wrote: <snip>

...

Yes, this works perfectly.

Thanks. Submitted that addition to the Homebrew guys, which hopefully they'll get done soon. Adam V is normally pretty responsive. :)

Done. It's now in the main Homebrew libvirt formula. Thanks Ruben. :)

On 10/24/2010 09:33 AM, Ruben Kerkhof wrote: <snip>

None at all, actually. I just started libvirtd on my local mac on which I also have VirtualBox installed. Speaking of which, it would be nice to have a launchctl file for libvirtd. I might be able to come up with something...

Please do. It'd be nice to have that part working "out of the box" for people as well. :) <snip>

Have you tried connecting with gnutls? Curious if that works.

Yep, it definitely does. "qemu+tls://" works fine, and "qemu+tcp://" does too. Tried them after discovering the "qemu+ssh://" problem a while back, when looking to get a better understanding of works and what didn't. If you're using TLS for stuff already (sounds like you are), then it all "just works" already. :)

On 10/25/2010 12:18 PM, Ruben Kerkhof wrote:

On Sun, Oct 24, 2010 at 02:46, Justin Clift <jclift@redhat.com> wrote:

...

On 10/24/2010 09:33 AM, Ruben Kerkhof wrote: <snip>

...

None at all, actually. I just started libvirtd on my local mac on which I also have VirtualBox installed. Speaking of which, it would be nice to have a launchctl file for libvirtd. I might be able to come up with something...

Please do. It'd be nice to have that part working "out of the box" for people as well. :)

For that to work, I'd like to run libvirtd as my own user, so I can add the launchtl file to my own Library directory.

I'm curious, can you successfully run libvirtd as your own user (no sudo)?

03:10:17.562: error : qemudListenUnix:582 : Failed to bind socket to '@/Users/ruben/.libvirt/libvirt-sock': No such file or directory

Actually, that looks familiar. I think I tried the same thing, but was ok running it as root instead after getting the same error. I didn't look into it any more though. ;)

Stepping through the code now, I see 2 (possible) issues:

First: qemudInitPaths doesn't seem to create the ~/.libvirt directory Second: in qemudListenUnix, this piece of code:

addr.sun_family = AF_UNIX; if (virStrcpyStatic(addr.sun_path, path) == NULL) { VIR_ERROR(_("Path %s too long for unix socket"), path); goto cleanup; } if (addr.sun_path[0] == '@') addr.sun_path[0] = '\0';

So the first byte of the sun_path is '\0', something that Leopard doesn't seem to like. Breaking into gdb and setting the path manually to "/Users/ruben/.libvirt/libvirt-sock" seems to work.

Interesting. We can definitely pull together a temporary OSX workaround patch for the moment (purely in the Homebrew formula). But it would be better to have a proper fix in libvirt instead. How good is your C coding? :) Regards and best wishes, Justin Clift

On Mon, Oct 25, 2010 at 10:48, Daniel P. Berrange <berrange@redhat.com> wrote:

NACK, this results in 'path' being a zer-length string, so no data is copied in the next virStrcpyStatic line. The original code is correctly creating a socket in the abstract namespace, ie one which does not appear in the filesystem

Daniel

Ah, I didn't knew that, sorry. Am I right in assuming that an abstract namespace is a linux-only feature? The unix manpage on my Mac doesn't describe it, neither does FreeBSD (http://www.freebsd.org/cgi/man.cgi?query=unix&apropos=0&sektion=0&manpath=FreeBSD+8.1-RELEASE&format=html). Thanks, Ruben

On 10/25/2010 07:20 PM, Ruben Kerkhof wrote: <snip>

I haven't been able to bootstrap a build from libvirt git yet, mainly gettext issues.

What are the gettext errors that happen for you? Thinking you'll need to install a newer gettext that the OSX provided one (as Homebrew does), and then make sure the headers/libraries for it are found first. Same as you've done for the readline libraries. Something like: ******************** ./configure \ CPPFLAGS='-I/usr/local/Cellar/gettext/0.17/include -I/usr/local/Cellar/libxml2/2.7.7/include -I/usr/local/Cellar/readline/6.1/include' \ LDFLAGS='-L/usr/local/Cellar/gettext/0.17/lib -L/usr/local/Cellar/libxml2/2.7.7/lib -L/usr/local/Cellar/readline/6.1/lib' ******************** (That should actually work, as long as you have gettext 0.17 installed.) :) Regards and best wishes, Justin Clift