Thanks Cole, unfortunately I'm a n00b in this area so your response just raised many more questions for me :/ * what are the config file names for libxl, libvirt and virt-manager * where are they located in the respective source trees * are the build flags defined in those config files? * libxl is part of xen 4.5.0 distro (xen/tools/libxl after untarring). But no config file is in xen/tools/libxl directory * I do see CFLAGS definitions in the various makefiles within each directory. Are those the build flags you are referring to? here's an example from libxl/Makefile: CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \ -Wno-declaration-after-statement -Wformat-nonliteral CFLAGS += -I. -fPIC * by distro I assume you mean binary package, and that I can install virt-manager binary package on top of libxl/libvirt/xen that I built and installed locally from source. if I have compiled libxl/libvirt/xen with virt-manager compile flags. Does that summarize what you are recommending? If so then how do I know virt-manager flags? That was a binary. Do I have to download virt-manager source to find those?

* libxl was part of xen source tarball, and therefore has a source tree position within xen (xen/tools/libxl as mentioned before). Should I place libvirt and virt-manager in some specific source tree spot relative to xen? Should libvirt and virt-manager be aware of libxl source location? Is there any makefile engineering involved?

Trying to work with your answer but I have a serious background deficit to deal with. For this reason I love technical 'cookbooks' that provide all missing details :)

On Mon, Mar 2, 2015 at 12:12 PM, Cole Robinson <crobinso(a)redhat.com <mailto:crobinso@redhat.com>> wrote: On 03/02/2015 10:55 AM, Wayne Mills wrote: > Hi, > > I built and installed Xen 4.5.0 from source, on top of Ubuntu 14.04.2, using > "make world" and "make install" targets. I then installed latest virt-manager > from pre-built packages. After bringing up virt-manager I attempt to connect > to Xen hypervisor and got: > > unable to connect to libvirt > Failed to connect socket to /var/run/libvirt/libvirt-sock' No such file > or directory > > I then noticed libvirt-bin isn't running. If I try to issue 'service > libvirt-bin start' it gives me back a process number, but it apparently dies > quickly because the service still shows as down. Three log files are touched > during my start attempt: > > -rw-r--r-- 1 root root 39757 Feb 27 06:00 /var/log/xenstored-access.log > -rw-r--r-- 1 root root 44908 Feb 27 06:00 > /var/log/libvirt/libxl/libxl-driver.log > -rw------- 1 root root 46396 Feb 27 06:00 /var/log/libvirt/libvirtd.log > > * xenstored-access.log has 22 new entries, grouped into pairs that increment > an "Axx" identifier and go from A63 to A73. Here is the A63 logs: > > [20150227T11:00:29.478Z] A63 newconn > [20150227T11:00:29.479Z] A63 endconn > > * libxl-driver.log also has 11 log groupings that are just the same set of > logs repeated 11 times. Here is the first group: > > xc: detail: sysctl operation failed -- need to rebuild the user-space > tool set? > libxl: error: libxl.c:4320:libxl_get_physinfo: getting physinfo: > Permission denied > xc: debug: hypercall buffer: total allocations:7 total releases:7 > xc: debug: hypercall buffer: current allocations:0 maximum allocations:1 > xc: debug: hypercall buffer: cache current size:1 > xc: debug: hypercall buffer: cache hits:6 misses:1 toobig:0 > > * libvirtd.log also has 11 log groupings, that are just the same set of logs > repeated 11 times. Here is that group: > > 2015-02-27 11:00:29.479+0000: 4842: info : libvirt version: 1.2.2 > 2015-02-27 11:00:29.479+0000: 4842: error : libxlDriverConfigNew:1131 : > Unable to configure libxl's memory management parameters > 2015-02-27 11:00:29.479+0000: 4842: error : virStateInitialize:749 : > Initialization of LIBXL state driver failed: Unknown problem > 2015-02-27 11:00:29.479+0000: 4842: error : daemonRunStateInit:920 : > Driver state initialization failed > > When I google for libxl_get_physinfo I see other reported errors during "xl > info" and other "xl" commands. I tried "xl info" and that works for me and > does not alter the log files: > > root@<server>:~# xl info > host : server > release : 3.13.0-46-generic > version : #75-Ubuntu SMP Tue Feb 10 15:24:04 UTC 2015 > machine : x86_64 > nr_cpus : 8 > max_cpu_id : 7 > nr_nodes : 2 > cores_per_socket : 4 > threads_per_core : 1 > cpu_mhz : 2400 > hw_caps : > bfebfbff:2c100800:00000000:00003f00:17bee3ff:00000000:00000001:00000000 > virt_caps : hvm hvm_directio > total_memory : 98168 > free_memory : 128 > sharing_freed_memory : 0 > sharing_used_memory : 0 > outstanding_claims : 0 > free_cpus : 0 > xen_major : 4 > xen_minor : 5 > xen_extra : .0 > xen_version : 4.5.0 > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xffff800000000000 > xen_changeset : Mon Jan 12 11:30:05 2015 -0500 git:a8ac229 > xen_commandline : placeholder > cc_compiler : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2 > cc_compile_by : root > cc_compile_domain : cisco.com <http://cisco.com> <http://cisco.com> > cc_compile_date : Thu Feb 19 07:26:00 EST 2015 > xend_config_format : 4 > root@<server>:~# > > After consulting with xen-users mailer, I was given this response by "Ian": > > "Did you install libvirt from source or from packages? If the latter then you > may have a disconnect between the packaged version and your source-built Xen. > You'll probably need to rebuild libvirt against your Xen libraries." > > What are the virt-related steps to take to test out Ian's idea? I have root > access to one machine that serves both as a host to VM's as well as a build > environment for source builds such as this. I already built xen 4.5.0 and > installed it. Do I now need to separately download and build libvirt, then > download and build virt-manager? Is it important to place the virt* code > within the xen source tree, or do something else so that the virt* code > compiles "against" an appropriate xen environment? > > In summary, I would like some guidance on how to build libvirt, and if needed, > virt-manager, source against a specific Xen version. > Distro packages of virt-manager should work fine, once you get libvirt building. I recommend you download the ubuntu .deb config files for libxl, and rebuild+install libxl with the exact same build flags your distro packages use. They've already figured out the necessary flags to make things work together with the other distro packages. Then do the same with libvirt. - Cole

I downloaded and untarred the deb package and grepped for 'configure', but none of the hits were configure commands. Is there a better way to find it?

On Mon, Mar 2, 2015 at 3:08 PM, Cole Robinson <crobinso(a)redhat.com <mailto:crobinso@redhat.com>> wrote: On 03/02/2015 01:39 PM, Wayne Mills wrote: > Thanks Cole, unfortunately I'm a n00b in this area so your response just > raised many more questions for me :/ > > * what are the config file names for libxl, libvirt and virt-manager > * where are they located in the respective source trees > * are the build flags defined in those config files? > * libxl is part of xen 4.5.0 distro (xen/tools/libxl after untarring). But no > config file is in xen/tools/libxl directory > * I do see CFLAGS definitions in the various makefiles within each directory. > Are those the build flags you are referring to? here's an example from > libxl/Makefile: > > CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \ > -Wno-declaration-after-statement -Wformat-nonliteral > CFLAGS += -I. -fPIC > > * by distro I assume you mean binary package, and that I can install > virt-manager binary package on top of libxl/libvirt/xen that I built and > installed locally from source. if I have compiled libxl/libvirt/xen with > virt-manager compile flags. Does that summarize what you are recommending? > If so then how do I know virt-manager flags? That was a binary. Do I have to > download virt-manager source to find those? > I meant the files that are used to generate your distro's binary packages. The bits that determine how software is compiled before wrapping it up in a distribution package. For example, I googled 'libvirt ubuntu package' and got to this: https://launchpad.net/ubuntu/+source/libvirt I clicked the top link for 1.2.12-0ubuntu7: https://launchpad.net/ubuntu/+source/libvirt/1.2.12-0ubuntu7 On this page is a link to the .deb packaging files: https://launchpad.net/ubuntu/+archive/primary/+files/libvirt_1.2.12-0ubun... Somewhere in there ./configure command line that generates the build for ubuntu's libvirt package. You'll want to pull that out for libvirt and for whatever provides libxl on ubuntu. At least that's the process I follow on Fedora when I want to compile some random software package from source. > * libxl was part of xen source tarball, and therefore has a source tree > position within xen (xen/tools/libxl as mentioned before). Should I place > libvirt and virt-manager in some specific source tree spot relative to xen? > Should libvirt and virt-manager be aware of libxl source location? Is there > any makefile engineering involved? > If you copy the distribution ./configure lines, you shouldn't have to deal with specifying any explicit paths. > Trying to work with your answer but I have a serious background deficit to > deal with. For this reason I love technical 'cookbooks' that provide all > missing details :) > I don't know of any explicit instructions for what you want, since these types of things vary a lot distro to distro and between software. Building multiple dependent packages from source is often a non-trivial task. And if you are 'make install'ing various different build configurations and things don't 'just work', it can be very difficult to debug things if they go wrong. - Cole > On Mon, Mar 2, 2015 at 12:12 PM, Cole Robinson <crobinso(a)redhat.com <mailto:crobinso@redhat.com> > <mailto:crobinso@redhat.com <mailto:crobinso@redhat.com>>> wrote: > > On 03/02/2015 10:55 AM, Wayne Mills wrote: > > Hi, > > > > I built and installed Xen 4.5.0 from source, on top of Ubuntu 14.04.2, using > > "make world" and "make install" targets. I then installed latest > virt-manager > > from pre-built packages. After bringing up virt-manager I attempt to > connect > > to Xen hypervisor and got: > > > > unable to connect to libvirt > > Failed to connect socket to /var/run/libvirt/libvirt-sock' No such file > > or directory > > > > I then noticed libvirt-bin isn't running. If I try to issue 'service > > libvirt-bin start' it gives me back a process number, but it apparently dies > > quickly because the service still shows as down. Three log files are > touched > > during my start attempt: > > > > -rw-r--r-- 1 root root 39757 Feb 27 06:00 > /var/log/xenstored-access.log > > -rw-r--r-- 1 root root 44908 Feb 27 06:00 > > /var/log/libvirt/libxl/libxl-driver.log > > -rw------- 1 root root 46396 Feb 27 06:00 > /var/log/libvirt/libvirtd.log > > > > * xenstored-access.log has 22 new entries, grouped into pairs that increment > > an "Axx" identifier and go from A63 to A73. Here is the A63 logs: > > > > [20150227T11:00:29.478Z] A63 newconn > > [20150227T11:00:29.479Z] A63 endconn > > > > * libxl-driver.log also has 11 log groupings that are just the same set of > > logs repeated 11 times. Here is the first group: > > > > xc: detail: sysctl operation failed -- need to rebuild the user-space > > tool set? > > libxl: error: libxl.c:4320:libxl_get_physinfo: getting physinfo: > > Permission denied > > xc: debug: hypercall buffer: total allocations:7 total releases:7 > > xc: debug: hypercall buffer: current allocations:0 maximum > allocations:1 > > xc: debug: hypercall buffer: cache current size:1 > > xc: debug: hypercall buffer: cache hits:6 misses:1 toobig:0 > > > > * libvirtd.log also has 11 log groupings, that are just the same set of logs > > repeated 11 times. Here is that group: > > > > 2015-02-27 11:00:29.479+0000: 4842: info : libvirt version: 1.2.2 > > 2015-02-27 11:00:29.479+0000: 4842: error : libxlDriverConfigNew:1131 : > > Unable to configure libxl's memory management parameters > > 2015-02-27 11:00:29.479+0000: 4842: error : virStateInitialize:749 : > > Initialization of LIBXL state driver failed: Unknown problem > > 2015-02-27 11:00:29.479+0000: 4842: error : daemonRunStateInit:920 : > > Driver state initialization failed > > > > When I google for libxl_get_physinfo I see other reported errors during "xl > > info" and other "xl" commands. I tried "xl info" and that works for > me and > > does not alter the log files: > > > > root@<server>:~# xl info > > host : server > > release : 3.13.0-46-generic > > version : #75-Ubuntu SMP Tue Feb 10 15:24:04 UTC 2015 > > machine : x86_64 > > nr_cpus : 8 > > max_cpu_id : 7 > > nr_nodes : 2 > > cores_per_socket : 4 > > threads_per_core : 1 > > cpu_mhz : 2400 > > hw_caps : > > bfebfbff:2c100800:00000000:00003f00:17bee3ff:00000000:00000001:00000000 > > virt_caps : hvm hvm_directio > > total_memory : 98168 > > free_memory : 128 > > sharing_freed_memory : 0 > > sharing_used_memory : 0 > > outstanding_claims : 0 > > free_cpus : 0 > > xen_major : 4 > > xen_minor : 5 > > xen_extra : .0 > > xen_version : 4.5.0 > > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > > hvm-3.0-x86_32p hvm-3.0-x86_64 > > xen_scheduler : credit > > xen_pagesize : 4096 > > platform_params : virt_start=0xffff800000000000 > > xen_changeset : Mon Jan 12 11:30:05 2015 -0500 git:a8ac229 > > xen_commandline : placeholder > > cc_compiler : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2 > > cc_compile_by : root > > cc_compile_domain : cisco.com <http://cisco.com> <http://cisco.com> <http://cisco.com> > > cc_compile_date : Thu Feb 19 07:26:00 EST 2015 > > xend_config_format : 4 > > root@<server>:~# > > > > After consulting with xen-users mailer, I was given this response by "Ian": > > > > "Did you install libvirt from source or from packages? If the latter then you > > may have a disconnect between the packaged version and your source-built Xen. > > You'll probably need to rebuild libvirt against your Xen libraries." > > > > What are the virt-related steps to take to test out Ian's idea? I have root > > access to one machine that serves both as a host to VM's as well as a build > > environment for source builds such as this. I already built xen 4.5.0 and > > installed it. Do I now need to separately download and build libvirt, then > > download and build virt-manager? Is it important to place the virt* code > > within the xen source tree, or do something else so that the virt* code > > compiles "against" an appropriate xen environment? > > > > In summary, I would like some guidance on how to build libvirt, and if needed, > > virt-manager, source against a specific Xen version. > > > > Distro packages of virt-manager should work fine, once you get libvirt > building. > > I recommend you download the ubuntu .deb config files for libxl, and > rebuild+install libxl with the exact same build flags your distro packages > use. They've already figured out the necessary flags to make things work > together with the other distro packages. Then do the same with libvirt. > > - Cole > >

On Tue, Mar 03, 2015 at 02:31:27PM -0500, Wayne Mills wrote: > I downloaded and untarred the deb package and grepped for 'configure', but > none of the hits were configure commands. Is there a better way to find it? Assuming this is still based on the Debian package look for DEB_CONFIGURE_EXTRA_ARGS in debian/rules. Cheers, -- Guido

I'm have a lot of difficulty implementing the actions suggested so far by those who've chimed in about this issue. I've resolved now to do the following: * make clean and make world on xen source, and capture build logs * look up libxl build logs and see what compiler flags were used * repeat above for libvirt source and virt-manager source * report back on the compiler flag differences I'm seeing, if any. I think that's probably the best I can hope to accomplish without getting further lost in the weeds. If I can do this then maybe it will inform further steps. Wayne On Wed, Mar 4, 2015 at 7:17 AM, Wayne Mills <wmills.on.gm(a)gmail.com&gt; wrote: > Thanks Guido, unfortunately no luck: > > root@<server>:~/debian# grep DEB_CONFIGURE_EXTRA_ARGS * > grep: apparmor: Is a directory > grep: patches: Is a directory > grep: source: Is a directory > root@<server>:~/debian# > root@<server>:~/debian# > root@<server>:~/debian# > root@<server>:~/debian# grep DEB_CONFIGURE_EXTRA_ARGS */* > root@<server>:~/debian# > root@<server>:~/debian# > root@<server>:~/debian# grep DEB_CONFIGURE_EXTRA_ARGS */*/* > grep: */*/*: No such file or directory > root@<server>:~/debian# > > > On Wed, Mar 4, 2015 at 3:01 AM, Guido Günther <agx(a)sigxcpu.org&gt; wrote: > >> On Tue, Mar 03, 2015 at 02:31:27PM -0500, Wayne Mills wrote: >> > I downloaded and untarred the deb package and grepped for 'configure', >> but >> > none of the hits were configure commands. Is there a better way to >> find it? >> >> Assuming this is still based on the Debian package look for >> DEB_CONFIGURE_EXTRA_ARGS in debian/rules. >> >> Cheers, >> -- Guido >> > >