2:13 p.m.
Hello,
I am trying to test some changes made to libvirt. I tried compiling and
installing, following the available documentation, with:
ninja -C build clean
meson build --prefix=$HOME/usr
ninja -C build -Dsystem=true
sudo ninja -C build install
After doing this, I try to run virt-install and get the following error on
the active libvirtd daemon:
Failed to connect socket to '/var/local/run/libvirt/virtqemud-sock': No
such file or directory
Indeed, that file does not exist:
$ ls /var/local/run/libvirt/
common hostdevmgr lockd lxc network nwfilter nwfilter-binding
secrets storage
virt-install was working fine before started changing libvirt's source code.
I'm working with Ubuntu 22.04 LTS, virsh v8.7.0.
I would appreciate any directions on how to fix this/successfully install
libvirt.
Thanks in advance.
Carlos
3:34 p.m.
On Tue, Aug 30, 2022 at 14:13:36 -0500, Carlos Bilbao wrote:
Hello,
I am trying to test some changes made to libvirt. I tried compiling and
installing, following the available documentation, with:
ninja -C build clean
meson build --prefix=$HOME/usr
ninja -C build -Dsystem=true
note that the '-Dsystem=true' argument is supposed to be used with
'meson' as it sets up configure time options. This way it probably did
nothing.
sudo ninja -C build install
After doing this, I try to run virt-install and get the following error on
the active libvirtd daemon:
Failed to connect socket to '/var/local/run/libvirt/virtqemud-sock': No
such file or directory
Indeed, that file does not exist:
$ ls /var/local/run/libvirt/
common hostdevmgr lockd lxc network nwfilter nwfilter-binding secrets
storage
So the directory really looks like libvirtd/virtqemud or any other of
the libvirt daemons never ran.
How did you start the daemons, did they log something?
virt-install was working fine before started changing libvirt's
source code.
I'm working with Ubuntu 22.04 LTS, virsh v8.7.0.
Generally the most straightforward way is to build distribution packages
from the tree and install them directly in your system because then you
avoid issues such as possibly having two libvirtd instances running and
such.
12:50 p.m.
On 8/30/22 15:34, Peter Krempa wrote:
On Tue, Aug 30, 2022 at 14:13:36 -0500, Carlos Bilbao wrote:
> Hello,
>
> I am trying to test some changes made to libvirt. I tried compiling and
> installing, following the available documentation, with:
>
> ninja -C build clean
> meson build --prefix=$HOME/usr
> ninja -C build -Dsystem=true
note that the '-Dsystem=true' argument is supposed to be used with
'meson' as it sets up configure time options. This way it probably did
nothing.
Noted, thanks!
> sudo ninja -C build install
>
> After doing this, I try to run virt-install and get the following error on
> the active libvirtd daemon:
>
> Failed to connect socket to '/var/local/run/libvirt/virtqemud-sock': No
> such file or directory
>
> Indeed, that file does not exist:
>
> $ ls /var/local/run/libvirt/
> common hostdevmgr lockd lxc network nwfilter nwfilter-binding secrets
> storage
So the directory really looks like libvirtd/virtqemud or any other of
the libvirt daemons never ran.
How did you start the daemons, did they log something?
So, what I did this time was:
$ ninja -C build clean
$ meson build --reconfigure --prefix=$HOME/usr -Dsystem=true
$ ninja -C build
$ sudo ninja -C build install
$ systemctl stop libvirtd
$ systemctl start libvirtd
but I still get:
$ virt-install --name ubuntu-sev --boot
loader=/usr/share/OVMF/OVMF_CODE.fd,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/OVMF/OVMF_VARS.fd,loader_secure=no
--vcpus 8 --memory 4096 --memtune hard_limit=16777216 --disk
pool=default,device=disk,size=32,format=raw --controller
type=scsi,model=virtio-scsi --network bridge=virbr0,model=virtio
--controller type=virtio-serial --machine q35 --cpu host-passthrough
--cdrom /var/lib/libvirt/images/ubuntu.iso --osinfo detect=on,require=on
--launchSecurity sev,policy=0x00 --graphics none --tpm none
ERROR Failed to connect socket to
'/var/local/run/libvirt/virtqemud-sock': No such file or directory
and the daemon shows the same error:
$ systemctl status libvirtd
○ libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
enabled; vendor preset: enabled)
Active: inactive (dead) since Tue 2022-09-06 17:43:53 UTC; 1min
57s ago
TriggeredBy: ● libvirtd-admin.socket
● libvirtd-ro.socket
● libvirtd.socket
Docs: man:libvirtd(8)
https://libvirt.org
Process: 2272757 ExecStart=/usr/local/sbin/libvirtd $LIBVIRTD_ARGS
(code=exited, status=0/SUCCESS)
Main PID: 2272757 (code=exited, status=0/SUCCESS)
Tasks: 2 (limit: 32768)
Memory: 60.5M
CPU: 49ms
CGroup: /system.slice/libvirtd.service
├─2760 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─2761 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
Sep 06 17:41:04 host systemd[1]: Starting Virtualization daemon...
Sep 06 17:41:04 host systemd[1]: Started Virtualization daemon.
Sep 06 17:41:53 host libvirtd[2272757]: libvirt version: 8.7.0
Sep 06 17:41:53 host libvirtd[2272757]: hostname: host
Sep 06 17:41:53 host libvirtd[2272757]: Failed to connect socket to
'/var/local/run/libvirt/virtqemud-sock': No such file or directory
Sep 06 17:41:53 host libvirtd[2272757]: End of file while reading data:
Input/output error
> virt-install was working fine before started changing libvirt's source code.
> I'm working with Ubuntu 22.04 LTS, virsh v8.7.0.
Generally the most straightforward way is to build distribution packages
from the tree and install them directly in your system because then you
avoid issues such as possibly having two libvirtd instances running and
such.
Does libvirt have any script or tool to ease such process?
Thanks in advance,
Carlos
10:28 a.m.
On 9/6/22 12:50, Carlos Bilbao wrote:
On 8/30/22 15:34, Peter Krempa wrote:
> On Tue, Aug 30, 2022 at 14:13:36 -0500, Carlos Bilbao wrote:
>> Hello,
>>
>> I am trying to test some changes made to libvirt. I tried compiling and
>> installing, following the available documentation, with:
>>
>> ninja -C build clean
>> meson build --prefix=$HOME/usr
>> ninja -C build -Dsystem=true
> note that the '-Dsystem=true' argument is supposed to be used with
> 'meson' as it sets up configure time options. This way it probably did
> nothing.
Noted, thanks!
>> sudo ninja -C build install
>>
>> After doing this, I try to run virt-install and get the following
>> error on
>> the active libvirtd daemon:
>>
>> Failed to connect socket to '/var/local/run/libvirt/virtqemud-sock': No
>> such file or directory
>>
>> Indeed, that file does not exist:
>>
>> $ ls /var/local/run/libvirt/
>> common hostdevmgr lockd lxc network nwfilter nwfilter-binding
>> secrets
>> storage
> So the directory really looks like libvirtd/virtqemud or any other of
> the libvirt daemons never ran.
>
> How did you start the daemons, did they log something?
So, what I did this time was:
$ ninja -C build clean
$ meson build --reconfigure --prefix=$HOME/usr -Dsystem=true
$ ninja -C build
$ sudo ninja -C build install
$ systemctl stop libvirtd
$ systemctl start libvirtd
but I still get:
$ virt-install --name ubuntu-sev --boot
loader=/usr/share/OVMF/OVMF_CODE.fd,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/OVMF/OVMF_VARS.fd,loader_secure=no
--vcpus 8 --memory 4096 --memtune hard_limit=16777216 --disk
pool=default,device=disk,size=32,format=raw --controller
type=scsi,model=virtio-scsi --network bridge=virbr0,model=virtio
--controller type=virtio-serial --machine q35 --cpu host-passthrough
--cdrom /var/lib/libvirt/images/ubuntu.iso --osinfo
detect=on,require=on --launchSecurity sev,policy=0x00 --graphics none
--tpm none
ERROR Failed to connect socket to
'/var/local/run/libvirt/virtqemud-sock': No such file or directory
and the daemon shows the same error:
$ systemctl status libvirtd
○ libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
enabled; vendor preset: enabled)
Active: inactive (dead) since Tue 2022-09-06 17:43:53 UTC; 1min
57s ago
TriggeredBy: ● libvirtd-admin.socket
● libvirtd-ro.socket
● libvirtd.socket
Docs: man:libvirtd(8)
https://libvirt.org
Process: 2272757 ExecStart=/usr/local/sbin/libvirtd $LIBVIRTD_ARGS
(code=exited, status=0/SUCCESS)
Main PID: 2272757 (code=exited, status=0/SUCCESS)
Tasks: 2 (limit: 32768)
Memory: 60.5M
CPU: 49ms
CGroup: /system.slice/libvirtd.service
├─2760 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─2761 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
Sep 06 17:41:04 host systemd[1]: Starting Virtualization daemon...
Sep 06 17:41:04 host systemd[1]: Started Virtualization daemon.
Sep 06 17:41:53 host libvirtd[2272757]: libvirt version: 8.7.0
Sep 06 17:41:53 host libvirtd[2272757]: hostname: host
Sep 06 17:41:53 host libvirtd[2272757]: Failed to connect socket to
'/var/local/run/libvirt/virtqemud-sock': No such file or directory
Sep 06 17:41:53 host libvirtd[2272757]: End of file while reading
data: Input/output error
>
>> virt-install was working fine before started changing libvirt's
>> source code.
>> I'm working with Ubuntu 22.04 LTS, virsh v8.7.0.
> Generally the most straightforward way is to build distribution packages
> from the tree and install them directly in your system because then you
> avoid issues such as possibly having two libvirtd instances running and
> such.
Does libvirt have any script or tool to ease such process?
Answering to myself here, because I have made some progress but still have
to fix some blockings. Decided to use debhelper to build Debian packages
and test libvirt. Did:
$ sudo apt-get install debhelper dh-make
$ sudo dh_make --createorig -p libvirt-snp_1
$ dh_auto_configure --buildsystem=meson
$ dpkg-buildpackage -rfakeroot -us -uc -b
Notice that the option -b for dpkg-buildpackage is to avoid the Debian
package bureaucracy. But still, some checks fail for the package (see below
). I don't think this errors have anything to do with the code I modified,
but I will be happy to share a patch for reproduction.
Summary of Failures:
76/255 libvirt / domaincapstest FAIL 1.65s exit status 1
97/255 libvirt / vircgrouptest FAIL 1.53s exit status 1
105/255 libvirt / virhostcputest FAIL 1.50s exit status 1
106/255 libvirt / virhostdevtest FAIL 1.50s exit status 1
107/255 libvirt / viridentitytest FAIL 1.50s exit status 1
108/255 libvirt / viriscsitest FAIL 1.49s exit status 1
113/255 libvirt / virnetdevtest FAIL 1.47s exit status 1
116/255 libvirt / virpcitest FAIL 1.46s exit status 1
128/255 libvirt / fchosttest FAIL 1.37s exit status 1
130/255 libvirt / vircaps2xmltest FAIL 1.36s exit status 1
151/255 libvirt / virnetserverclienttest FAIL 1.28s exit
status 1
169/255 libvirt:syntax-check /
TAB_in_indentation FAIL 0.88s exit status 2
239/255 libvirt:syntax-check /
prohibit_obj_free_apis_in_virsh FAIL 0.39s exit status 2
249/255 libvirt:syntax-check /
trailing_blank FAIL 0.28s exit status 2
Ok: 236
Expected Fail: 0
Fail: 14
Unexpected Pass: 0
Skipped: 5
Timeout: 0
dh_auto_test: error: cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8
MESON_TESTTHREADS=192 meson test returned exit code 14
make: *** [debian/rules:18: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit
status 2
Thanks!
Carlos
Thanks in advance,
Carlos
11:19 a.m.
On Wed, Sep 07, 2022 at 10:28:17AM -0500, Carlos Bilbao wrote:
On 9/6/22 12:50, Carlos Bilbao wrote:
> So, what I did this time was:
>
> $ ninja -C build clean
I would recommend nuking the build/ directory completely if you're
going to reconfigure, just to be on the safe side.
> $ meson build --reconfigure --prefix=$HOME/usr -Dsystem=true
Passing --prefix along with -Dsystem=true doesn't really make sense.
-Dsystem=true tells the build system to set various paths to sane
defaults for a system-wide installation, and the value passed to
--prefix is used to construct some of those paths. So don't use both
at the same time - stick to -Dsystem=true only.
As an aside, setting --prefix to a directory under $HOME is unlikely
to result in a working configuration anyway.
> $ ninja -C build
> $ sudo ninja -C build install
> $ systemctl stop libvirtd
> $ systemctl start libvirtd
Before restarting libvirtd, since you've just installed new unit
files as part of the steps above, you should run
$ sudo systemctl daemon-reload
> $ systemctl status libvirtd
> ○ libvirtd.service - Virtualization daemon
> Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
> enabled; vendor preset: enabled)
Note how the unit file...
> Active: inactive (dead) since Tue 2022-09-06 17:43:53 UTC;
1min 57s
> ago
> TriggeredBy: ● libvirtd-admin.socket
> ● libvirtd-ro.socket
> ● libvirtd.socket
> Docs: man:libvirtd(8)
> https://libvirt.org
> Process: 2272757 ExecStart=/usr/local/sbin/libvirtd $LIBVIRTD_ARGS
... the libvirtd binary...
> (code=exited, status=0/SUCCESS)
> Main PID: 2272757 (code=exited, status=0/SUCCESS)
> Tasks: 2 (limit: 32768)
> Memory: 60.5M
> CPU: 49ms
> CGroup: /system.slice/libvirtd.service
> ├─2760 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
> └─2761 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
>
> Sep 06 17:41:04 host systemd[1]: Starting Virtualization daemon...
> Sep 06 17:41:04 host systemd[1]: Started Virtualization daemon.
> Sep 06 17:41:53 host libvirtd[2272757]: libvirt version: 8.7.0
> Sep 06 17:41:53 host libvirtd[2272757]: hostname: host
> Sep 06 17:41:53 host libvirtd[2272757]: Failed to connect socket to
> '/var/local/run/libvirt/virtqemud-sock': No such file or directory
... and the socket are all under /{usr,var}/local. This is not what
you want, and using -Dsystem=true should have not resulted in such a
setup.
Make sure you run 'systemd daemon-reload' after installing unit
files, as I've mentioned above, and also look around for leftovers
from a previous installation of libvirt that might be interfering
with the current one.
Regardless of all that, the client is trying to connect to virtqemud
but you seem to have the monolithic libvirtd daemon running. I can't
remember whether that's expected, but it's looks like a red flag to
me. I think the client might have gotten confused about whether split
daemons are in use.
Please check out https://libvirt.org/daemons.html and ensure you're
in one of the two supported deployment scenarios and not a weird mix
of the two.
> > Generally the most straightforward way is to build
distribution packages
> > from the tree and install them directly in your system because then you
> > avoid issues such as possibly having two libvirtd instances running and
> > such.
>
> Does libvirt have any script or tool to ease such process?
Answering to myself here, because I have made some progress but still have
to fix some blockings. Decided to use debhelper to build Debian packages
and test libvirt. Did:
$ sudo apt-get install debhelper dh-make
$ sudo dh_make --createorig -p libvirt-snp_1
$ dh_auto_configure --buildsystem=meson
$ dpkg-buildpackage -rfakeroot -us -uc -b
Notice that the option -b for dpkg-buildpackage is to avoid the Debian
package bureaucracy. But still, some checks fail for the package (see below
). I don't think this errors have anything to do with the code I modified,
but I will be happy to share a patch for reproduction.
If you want to build a Debian package, I recommend using the one
that's already in the archive as a starting point instead of using
dh_make to create one from scratch. It's currently a couple of
versions behind upstream, but should be pretty solid otherwise.
dh_make is intended to create a very basic skeleton for a package,
and you're expected to make significant changes to the files it
produces before being able to build a working package.
--
Andrea Bolognani / Red Hat / Virtualization
12:02 p.m.
On 9/7/22 11:19, Andrea Bolognani wrote:
On Wed, Sep 07, 2022 at 10:28:17AM -0500, Carlos Bilbao wrote:
> On 9/6/22 12:50, Carlos Bilbao wrote:
>> So, what I did this time was:
>>
>> $ ninja -C build clean
I would recommend nuking the build/ directory completely if you're
going to reconfigure, just to be on the safe side.
>> $ meson build --reconfigure --prefix=$HOME/usr -Dsystem=true
Passing --prefix along with -Dsystem=true doesn't really make sense.
-Dsystem=true tells the build system to set various paths to sane
defaults for a system-wide installation, and the value passed to
--prefix is used to construct some of those paths. So don't use both
at the same time - stick to -Dsystem=true only.
As an aside, setting --prefix to a directory under $HOME is unlikely
to result in a working configuration anyway.
>> $ ninja -C build
>> $ sudo ninja -C build install
>> $ systemctl stop libvirtd
>> $ systemctl start libvirtd
Before restarting libvirtd, since you've just installed new unit
files as part of the steps above, you should run
$ sudo systemctl daemon-reload
>> $ systemctl status libvirtd
>> ○ libvirtd.service - Virtualization daemon
>> Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
>> enabled; vendor preset: enabled)
Note how the unit file...
>> Active: inactive (dead) since Tue 2022-09-06 17:43:53 UTC;
1min 57s
>> ago
>> TriggeredBy: ● libvirtd-admin.socket
>> ● libvirtd-ro.socket
>> ● libvirtd.socket
>> Docs: man:libvirtd(8)
>>
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
>> Process: 2272757 ExecStart=/usr/local/sbin/libvirtd
$LIBVIRTD_ARGS
... the libvirtd binary...
>> (code=exited, status=0/SUCCESS)
>> Main PID: 2272757 (code=exited, status=0/SUCCESS)
>> Tasks: 2 (limit: 32768)
>> Memory: 60.5M
>> CPU: 49ms
>> CGroup: /system.slice/libvirtd.service
>> ├─2760 /usr/sbin/dnsmasq
>> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
>> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
>> └─2761 /usr/sbin/dnsmasq
>> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
>> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
>>
>> Sep 06 17:41:04 host systemd[1]: Starting Virtualization daemon...
>> Sep 06 17:41:04 host systemd[1]: Started Virtualization daemon.
>> Sep 06 17:41:53 host libvirtd[2272757]: libvirt version: 8.7.0
>> Sep 06 17:41:53 host libvirtd[2272757]: hostname: host
>> Sep 06 17:41:53 host libvirtd[2272757]: Failed to connect socket to
>> '/var/local/run/libvirt/virtqemud-sock': No such file or directory
... and the socket are all under /{usr,var}/local. This is not what
you want, and using -Dsystem=true should have not resulted in such a
setup.
Make sure you run 'systemd daemon-reload' after installing unit
files, as I've mentioned above, and also look around for leftovers
from a previous installation of libvirt that might be interfering
with the current one.
Thanks so much for your tips Andrea, very helpful stuff. I have tried again
with the proper options for meson. The libvirt daemon still fails to load,
but this time there seems to be other issue -I think it's related with what
you point out below regarding a mix of daemon modes (I'll discuss that
below). The current status is 'masked':
# systemctl start libvirtd
Failed to start libvirtd.service: Unit libvirtd.service is masked.
# systemctl status libvirtd
○ libvirtd.service
Loaded: masked (Reason: Unit libvirtd.service is masked.)
Active: inactive (dead) since Tue 2022-09-06 17:43:53 UTC; 22h ago
Main PID: 2272757 (code=exited, status=0/SUCCESS)
Tasks: 2 (limit: 153604)
Memory: 53.3M
CPU: 292ms
CGroup: /system.slice/libvirtd.service
├─2760 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─2761 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
Sep 07 15:22:51 host dnsmasq-dhcp[2760]: DHCPREQUEST(virbr0) <ip> <mac>
Sep 07 15:22:51 host dnsmasq-dhcp[2760]: DHCPACK(virbr0) <ip> <mac>
guest-machine
Sep 07 15:22:51 host dnsmasq-script[2760]: libvirt_leaseshelper:
/usr/local/lib/x86_64-linux-gnu/libvirt.so.0: version
`LIBVIRT_PRIVATE_8.0.0' not found (required by li>
Sep 07 15:22:51 host dnsmasq[2760]: script process exited with status 1
Sep 07 15:50:11 host dnsmasq-dhcp[2760]: DHCPREQUEST(virbr0) <ip> <mac>
Sep 07 15:50:11 host dnsmasq-dhcp[2760]: DHCPACK(virbr0) <ip> <mac>
guest-machine
Sep 07 15:50:11 host dnsmasq[2760]: failed to execute
/usr/lib/libvirt/libvirt_leaseshelper: No such file or directory
Sep 07 16:16:28 host dnsmasq-dhcp[2760]: DHCPREQUEST(virbr0) <ip> <mac>
Sep 07 16:16:28 host dnsmasq-dhcp[2760]: DHCPACK(virbr0) <ip> <mac>
guest-machine
Sep 07 16:16:28 host dnsmasq[2760]: failed to execute
/usr/lib/libvirt/libvirt_leaseshelper: No such file or directory
Regardless of all that, the client is trying to connect to virtqemud
but you seem to have the monolithic libvirtd daemon running. I can't
remember whether that's expected, but it's looks like a red flag to
me. I think the client might have gotten confused about whether split
daemons are in use.
Please check out
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
and ensure you're
in one of the two supported deployment scenarios and not a weird mix
of the two.
Yes, indeed everything points out to a weird mix. For example, I can find
stuff related to the daemon virtsecretd:
# ls /etc/libvirt/ | grep virtsecretd
virtsecretd.conf
However, it is definitely not running:
# systemctl | grep virt
sys-devices-virtual-block-dm\x2d0.device loaded active plugged
/sys/devices/virtual/block/dm-0
sys-devices-virtual-block-loop0.device loaded active plugged
/sys/devices/virtual/block/loop0
sys-devices-virtual-block-loop1.device loaded active plugged
/sys/devices/virtual/block/loop1
sys-devices-virtual-block-loop2.device loaded active plugged
/sys/devices/virtual/block/loop2
sys-devices-virtual-block-loop3.device loaded active plugged
/sys/devices/virtual/block/loop3
sys-devices-virtual-block-loop4.device loaded active plugged
/sys/devices/virtual/block/loop4
sys-devices-virtual-block-loop5.device loaded active plugged
/sys/devices/virtual/block/loop5
sys-devices-virtual-misc-rfkill.device loaded active plugged
/sys/devices/virtual/misc/rfkill
sys-devices-virtual-net-virbr0.device loaded active plugged
/sys/devices/virtual/net/virbr0
sys-devices-virtual-net-vnet15.device loaded active plugged
/sys/devices/virtual/net/vnet15
sys-devices-virtual-tty-ttyprintk.device loaded active plugged
/sys/devices/virtual/tty/ttyprintk
● libvirt-guests.service masked failed failed libvirt-guests.service
virt-guest-shutdown.target loaded active active Libvirt guests shutdown
# ps -aux | grep "[v]irtsecret"
#
I'm not sure what is the best way to go around this. Probably, delete
absolutely everything that has to do with libvirt and then proceed to
install again from the tree. Although it would be nice to understand why
this happened on the first place. Maybe there is a configuration option in
the build/install process where I should be specifying monolithic mode?
>>> Generally the most straightforward way is to build distribution
packages
>>> from the tree and install them directly in your system
because
then you
>>> avoid issues such as possibly having two libvirtd
instances
running and
>>> such.
>>
>> Does libvirt have any script or tool to ease such process?
>
> Answering to myself here, because I have made some progress but
still have
> to fix some blockings. Decided to use debhelper to build Debian
packages
> and test libvirt. Did:
>
> $ sudo apt-get install debhelper dh-make
> $ sudo dh_make --createorig -p libvirt-snp_1
> $ dh_auto_configure --buildsystem=meson
> $ dpkg-buildpackage -rfakeroot -us -uc -b
>
> Notice that the option -b for dpkg-buildpackage is to avoid the Debian
> package bureaucracy. But still, some checks fail for the package
(see below
> ). I don't think this errors have anything to do with the
code I
modified,
> but I will be happy to share a patch for reproduction.
If you want to build a Debian package, I recommend using the one
that's already in the archive as a starting point instead of using
dh_make to create one from scratch. It's currently a couple of
versions behind upstream, but should be pretty solid otherwise.
Not sure what location you are referring to. Is this on a separate repo in
the GitLab? Also, I need to be able to test changes in the source code of
libvirt (e.g. src/qemu/qemu_capabilities.c). How would I go around that
with the prepared Debian package? Apologies if this is obvious, I have no
experience with dpkg-buildpackage.
dh_make is intended to create a very basic skeleton for a package,
and you're expected to make significant changes to the files it
produces before being able to build a working package.
Thanks again for your assistance, Andrea, greatly appreciated.
Carlos
11:07 a.m.
Hello, made some progress and I think I see what the issue may be now...
and it looks like virt-install's fault.
> Regardless of all that, the client is trying to connect to
virtqemud
> but you seem to have the monolithic libvirtd daemon running. I can't
> remember whether that's expected, but it's looks like a red flag to
> me. I think the client might have gotten confused about whether split
> daemons are in use.
>
> Please check out
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
and ensure you're
> in one of the two supported deployment scenarios and not a weird
mix
> of the two.
So, I started again from scratch. Deleted absolutely everything that had
to do with libvirt. I also had to unmask a bunch of service-related stuff
that were on masked state:
for ANNOYING_SERVICE in libvirtd virtlogd.socket virtlogd.service \
virtlockd.socket virtlockd.service virtlogd-admin.socket
virtlockd-admin.socket
do
systemctl unmask $ANNOYING_SERVICE
done
So, after doing this, building and installing the tree, reloading the
daemon, and starting all services and sockets again, I finally get an
active libvirtd daemon:
# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-09-08 15:51:39 UTC; 1min
30s ago
Docs: man:libvirtd(8)
https://libvirt.org
Main PID: 2429457 (libvirtd)
Tasks: 20 (limit: 32768)
Memory: 55.6M
CPU: 28ms
CGroup: /system.slice/libvirtd.service
├─ 2760 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
├─ 2761 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─2429457 /usr/local/sbin/libvirtd --timeout 120
Sep 08 15:51:39 host systemd[1]: Starting Virtualization daemon...
Sep 08 15:51:39 host systemd[1]: Started Virtualization daemon.
#
Now, this is when things get interesting... The next step is testing
virt-install. Since I removed it, I proceed to install again:
# sudo apt install -y virtinst
(...)
Unpacking virtinst (1:4.0.0-1) ...
Setting up libvirt-glib-1.0-data (4.0.0-2) ...
Setting up libvirt0:amd64 (8.0.0-1ubuntu7.1) ...
Setting up libvirt-glib-1.0-0:amd64 (4.0.0-2) ...
Setting up python3-libvirt (8.0.0-1build1) ...
Setting up libvirt-clients (8.0.0-1ubuntu7.1) ...
Setting up libvirt-daemon-driver-qemu (8.0.0-1ubuntu7.1) ...
Setting up virt-viewer (7.0-2build2) ...
Setting up libvirt-daemon (8.0.0-1ubuntu7.1) ...
Setting up virtinst (1:4.0.0-1) ...
As you can see, virtinst also installs the libvirt daemon. I didn't think
this would be an issue because the libvirtd daemon running is still my
version. However, when I try to do virt-install I get:
# virt-install --connect qemu:///system --name ubuntu-sev \
--boot
loader=/usr/share/OVMF/OVMF_CODE.fd,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/OVMF/OVMF_VARS.fd,loader_secure=no
\
--vcpus 8 --memory 4096 --memtune hard_limit=16777216 \
--disk pool=default,device=disk,size=32,format=raw \
--controller type=scsi,model=virtio-scsi --network
bridge=virbr0,model=virtio \
--controller type=virtio-serial --machine q35 --cpu host-passthrough \
--cdrom /var/lib/libvirt/images/ubuntu.iso --osinfo detect=on,require=on \
--launchSecurity sev,policy=0x00 --graphics none --tpm none
ERROR Failed to connect socket to '/var/run/libvirt/virtqemud-sock':
No such file or directory
#
So, back to square one. However, notice that virtqemud is one of the modular
daemons. So, I believe that what happens is that virt-install assumes
libvirt is using the new daemon mode. Is this something that can be
configured? I don't see that option in the man pages of virt-install.
>
> If you want to build a Debian package, I recommend using the one
> that's already in the archive as a starting point instead of using
> dh_make to create one from scratch. It's currently a couple of
> versions behind upstream, but should be pretty solid otherwise.
>
Not sure what location you are referring to. Is this on a separate
repo in
the GitLab? Also, I need to be able to test changes in the source
code of
libvirt (e.g. src/qemu/qemu_capabilities.c). How would I go around that
with the prepared Debian package? Apologies if this is obvious, I have no
experience with dpkg-buildpackage.
> dh_make is intended to create a very basic skeleton for a package,
> and you're expected to make significant changes to the files it
> produces before being able to build a working package.
>
Thanks,
Carlos
11:12 a.m.
On Thu, Sep 08, 2022 at 11:07:30 -0500, Carlos Bilbao wrote:
Hello, made some progress and I think I see what the issue may be
now...
and it looks like virt-install's fault.
> > Regardless of all that, the client is trying to connect to virtqemud
> > but you seem to have the monolithic libvirtd daemon running. I can't
> > remember whether that's expected, but it's looks like a red flag to
> > me. I think the client might have gotten confused about whether split
> > daemons are in use.
> >
> > Please check out
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
and ensure you're
> > in one of the two supported deployment scenarios and not a weird mix
> > of the two.
>
So, I started again from scratch. Deleted absolutely everything that had
to do with libvirt. I also had to unmask a bunch of service-related stuff
that were on masked state:
for ANNOYING_SERVICE in libvirtd virtlogd.socket virtlogd.service \
virtlockd.socket virtlockd.service virtlogd-admin.socket
virtlockd-admin.socket
do
systemctl unmask $ANNOYING_SERVICE
done
So, after doing this, building and installing the tree, reloading the
daemon, and starting all services and sockets again, I finally get an
active libvirtd daemon:
# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-09-08 15:51:39 UTC; 1min 30s
ago
Docs: man:libvirtd(8)
https://libvirt.org
Main PID: 2429457 (libvirtd)
Tasks: 20 (limit: 32768)
Memory: 55.6M
CPU: 28ms
CGroup: /system.slice/libvirtd.service
├─ 2760 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
├─ 2761 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─2429457 /usr/local/sbin/libvirtd --timeout 120
Sep 08 15:51:39 host systemd[1]: Starting Virtualization daemon...
Sep 08 15:51:39 host systemd[1]: Started Virtualization daemon.
#
Now, this is when things get interesting... The next step is testing
virt-install. Since I removed it, I proceed to install again:
# sudo apt install -y virtinst
(...)
Unpacking virtinst (1:4.0.0-1) ...
Setting up libvirt-glib-1.0-data (4.0.0-2) ...
Setting up libvirt0:amd64 (8.0.0-1ubuntu7.1) ...
Setting up libvirt-glib-1.0-0:amd64 (4.0.0-2) ...
Setting up python3-libvirt (8.0.0-1build1) ...
Setting up libvirt-clients (8.0.0-1ubuntu7.1) ...
Setting up libvirt-daemon-driver-qemu (8.0.0-1ubuntu7.1) ...
Setting up virt-viewer (7.0-2build2) ...
Setting up libvirt-daemon (8.0.0-1ubuntu7.1) ...
Setting up virtinst (1:4.0.0-1) ...
As you can see, virtinst also installs the libvirt daemon. I didn't think
this would be an issue because the libvirtd daemon running is still my
version. However, when I try to do virt-install I get:
# virt-install --connect qemu:///system --name ubuntu-sev \
--boot
loader=/usr/share/OVMF/OVMF_CODE.fd,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/OVMF/OVMF_VARS.fd,loader_secure=no
\
--vcpus 8 --memory 4096 --memtune hard_limit=16777216 \
--disk pool=default,device=disk,size=32,format=raw \
--controller type=scsi,model=virtio-scsi --network
bridge=virbr0,model=virtio \
--controller type=virtio-serial --machine q35 --cpu host-passthrough \
--cdrom /var/lib/libvirt/images/ubuntu.iso --osinfo detect=on,require=on \
--launchSecurity sev,policy=0x00 --graphics none --tpm none
ERROR Failed to connect socket to '/var/run/libvirt/virtqemud-sock': No
such file or directory
#
So, back to square one. However, notice that virtqemud is one of the modular
daemons. So, I believe that what happens is that virt-install assumes
libvirt is using the new daemon mode. Is this something that can be
configured? I don't see that option in the man pages of virt-install.
You seem to have explicitly started 'libvirtd' service. If that is the
case the error message above is actually a bit cryptic, but in fact
means that the 'qemu' driver was not compiled.
This is because by default it's marked as optional and compiled only if
you have all dependencies installed.
You need to pass '-Ddriver_qemu=enabled' to the meson configure step to
force-enable it. Meson will report what's missing.
3:45 p.m.
On 9/8/22 11:12, Peter Krempa wrote:
On Thu, Sep 08, 2022 at 11:07:30 -0500, Carlos Bilbao wrote:
> Hello, made some progress and I think I see what the issue may be now...
> and it looks like virt-install's fault.
>
>>> Regardless of all that, the client is trying to connect to virtqemud
>>> but you seem to have the monolithic libvirtd daemon running. I can't
>>> remember whether that's expected, but it's looks like a red flag to
>>> me. I think the client might have gotten confused about whether split
>>> daemons are in use.
>>>
>>> Please check out
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
> and ensure you're
>>> in one of the two supported deployment scenarios and not a weird mix
>>> of the two.
>>
>
> So, I started again from scratch. Deleted absolutely everything that had
> to do with libvirt. I also had to unmask a bunch of service-related
stuff
> that were on masked state:
>
> for ANNOYING_SERVICE in libvirtd virtlogd.socket virtlogd.service \
> virtlockd.socket virtlockd.service virtlogd-admin.socket
> virtlockd-admin.socket
> do
> systemctl unmask $ANNOYING_SERVICE
> done
>
> So, after doing this, building and installing the tree, reloading the
> daemon, and starting all services and sockets again, I finally get an
> active libvirtd daemon:
>
> # systemctl status libvirtd
> ● libvirtd.service - Virtualization daemon
> Loaded: loaded (/usr/local/lib/systemd/system/libvirtd.service;
> enabled; vendor preset: enabled)
> Active: active (running) since Thu 2022-09-08 15:51:39 UTC;
1min 30s
> ago
> Docs: man:libvirtd(8)
>
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvirt...
> Main PID: 2429457 (libvirtd)
> Tasks: 20 (limit: 32768)
> Memory: 55.6M
> CPU: 28ms
> CGroup: /system.slice/libvirtd.service
> ├─ 2760 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
> ├─ 2761 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
> └─2429457 /usr/local/sbin/libvirtd --timeout 120
>
> Sep 08 15:51:39 host systemd[1]: Starting Virtualization daemon...
> Sep 08 15:51:39 host systemd[1]: Started Virtualization daemon.
> #
>
> Now, this is when things get interesting... The next step is testing
> virt-install. Since I removed it, I proceed to install again:
>
> # sudo apt install -y virtinst
> (...)
> Unpacking virtinst (1:4.0.0-1) ...
> Setting up libvirt-glib-1.0-data (4.0.0-2) ...
> Setting up libvirt0:amd64 (8.0.0-1ubuntu7.1) ...
> Setting up libvirt-glib-1.0-0:amd64 (4.0.0-2) ...
> Setting up python3-libvirt (8.0.0-1build1) ...
> Setting up libvirt-clients (8.0.0-1ubuntu7.1) ...
> Setting up libvirt-daemon-driver-qemu (8.0.0-1ubuntu7.1) ...
> Setting up virt-viewer (7.0-2build2) ...
> Setting up libvirt-daemon (8.0.0-1ubuntu7.1) ...
> Setting up virtinst (1:4.0.0-1) ...
>
> As you can see, virtinst also installs the libvirt daemon. I didn't
think
> this would be an issue because the libvirtd daemon running is
still my
> version. However, when I try to do virt-install I get:
>
> # virt-install --connect qemu:///system --name ubuntu-sev \
> --boot
loader=/usr/share/OVMF/OVMF_CODE.fd,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/OVMF/OVMF_VARS.fd,loader_secure=no
> \
> --vcpus 8 --memory 4096 --memtune hard_limit=16777216 \
> --disk pool=default,device=disk,size=32,format=raw \
> --controller type=scsi,model=virtio-scsi --network
> bridge=virbr0,model=virtio \
> --controller type=virtio-serial --machine q35 --cpu host-passthrough \
> --cdrom /var/lib/libvirt/images/ubuntu.iso --osinfo
detect=on,require=on \
> --launchSecurity sev,policy=0x00 --graphics none --tpm none
> ERROR Failed to connect socket to
'/var/run/libvirt/virtqemud-sock': No
> such file or directory
> #
>
> So, back to square one. However, notice that virtqemud is one of the
modular
> daemons. So, I believe that what happens is that virt-install
assumes
> libvirt is using the new daemon mode. Is this something that can be
> configured? I don't see that option in the man pages of virt-install.
You seem to have explicitly started 'libvirtd' service. If that is the
case the error message above is actually a bit cryptic, but in fact
means that the 'qemu' driver was not compiled.
This is because by default it's marked as optional and compiled only if
you have all dependencies installed.
You need to pass '-Ddriver_qemu=enabled' to the meson configure step to
force-enable it. Meson will report what's missing.
Yep, passing -Ddriver_qemu=enabled compiled all the Qemu stuff. Then,
after starting the new libvirtd daemon, I was able to start
virtqemud.socket,
which in turn fixed this error when using virt-install.
Thanks,
Carlos
798
days inactive
807
days old
8 comments
3 participants
participants (3)
-
Andrea Bolognani -
Carlos Bilbao -
Peter Krempa