5:32 p.m.
Where are the configuration files for libvirt and/or virsh actually stored? It seems that
most of them are in /etc/libvirt but identical copies are also found in /var/lib/libvirt.
In my case I am having issues with the network/default.xml file being different. Is there
a reason why the config seems to be in two places at once? And which one is the
'correct' one?
-David Mitchell
-----------------------------------------------------------------
| David Mitchell (mitchell(a)ucar.edu) Network Engineer IV |
| Tel: (303) 497-1845 National Center for |
| FAX: (303) 497-1818 Atmospheric Research |
-----------------------------------------------------------------
8:56 a.m.
On 12/07/2012 04:32 PM, David Mitchell wrote:
Where are the configuration files for libvirt and/or virsh actually
stored?
That's an implementation detail, and might change in the future. You
shouldn't need to care where they are stored, and instead access the
data via the API (such as virDomainGetXMLDesc and virDomainDefine) or
API wrappers (such as virsh dumpxml/define). That said:
It seems that most of them are in /etc/libvirt
That would be the persistent definitions (virsh dumpxml --inactive)
but identical copies are also found in /var/lib/libvirt.
Those are not identical, but are the runtime definitions. It is
possible to make changes to either one in isolation (hotplugs to an
in-use object affect /var, changes that affect only the next start of an
object affect /etc, and it is possible to affect both files at once).
In my case I am having issues with the network/default.xml file being
different. Is there a reason why the config seems to be in two places at once?
Yes - because all libvirt objects (domains, networks, etc.) have both an
active state and a persistent state, and the two can be different.
And which one is the 'correct' one?
Correct for what? Both are used by libvirt, but you should never edit
either. It sounds like you instead need to do:
# edit the persistent definition, which will touch /etc
virsh net-edit default
# stop the running instance with the old definition from /var
virsh net-destroy default
# restart, which will repopulate /var with your changes
virsh net-start default
all without you ever directly touching files in /etc or /var.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
11:36 a.m.
On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
On 12/07/2012 04:32 PM, David Mitchell wrote:
> Where are the configuration files for libvirt and/or virsh actually stored?
That's an implementation detail, and might change in the future. You
shouldn't need to care where they are stored, and instead access the
data via the API (such as virDomainGetXMLDesc and virDomainDefine) or
API wrappers (such as virsh dumpxml/define). That said:
In my case I'm trying to sync the configuration files between two physical servers. To
that end I have a drbd partition which I mirror. I have /etc/libvirt softlinked into that
partition. I could easily link /var/lib/libvirt into that partition as well but without
understanding what it's for I'm not sure of all the ramifications.
For clean stops and starts I do plan on using the API to save or destroy the running
domains, etc and that should keep the /var/lib/ and /etc/configurations consistent if I
understand their purpose. I'm a little worried about what will happen during unclean
stops though.
> It seems that most of them are in /etc/libvirt
That would be the persistent definitions (virsh dumpxml --inactive)
> but identical copies are also found in /var/lib/libvirt.
Those are not identical, but are the runtime definitions. It is
possible to make changes to either one in isolation (hotplugs to an
in-use object affect /var, changes that affect only the next start of an
object affect /etc, and it is possible to affect both files at once).
> In my case I am having issues with the network/default.xml file being different. Is
there a reason why the config seems to be in two places at once?
Yes - because all libvirt objects (domains, networks, etc.) have both an
active state and a persistent state, and the two can be different.
> And which one is the 'correct' one?
Correct for what? Both are used by libvirt, but you should never edit
either. It sounds like you instead need to do:
# edit the persistent definition, which will touch /etc
virsh net-edit default
# stop the running instance with the old definition from /var
virsh net-destroy default
# restart, which will repopulate /var with your changes
virsh net-start default
I will give that a try. The problem I'm having is that the /var configuration is being
used even after a reboot instead of the /etc/ configuration.
all without you ever directly touching files in /etc or /var.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
-----------------------------------------------------------------
| David Mitchell (mitchell(a)ucar.edu) Network Engineer IV |
| Tel: (303) 497-1845 National Center for |
| FAX: (303) 497-1818 Atmospheric Research |
-----------------------------------------------------------------
11:56 p.m.
On 12/08/2012 10:36 AM, David Mitchell wrote:
On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
> On 12/07/2012 04:32 PM, David Mitchell wrote:
>> Where are the configuration files for libvirt and/or virsh actually stored?
>
> That's an implementation detail, and might change in the future. You
> shouldn't need to care where they are stored, and instead access the
> data via the API (such as virDomainGetXMLDesc and virDomainDefine) or
> API wrappers (such as virsh dumpxml/define). That said:
In my case I'm trying to sync the configuration files between two physical servers.
To that end I have a drbd partition which I mirror. I have /etc/libvirt softlinked into
that partition. I could easily link /var/lib/libvirt into that partition as well but
without understanding what it's for I'm not sure of all the ramifications.
Don't. Mirroring the contents of /etc/libvirt or /var/libvirt is going
behind libvirt's back, and risks data inconsistencies if the libvirtd on
one machine sees data in an incomplete state as written by a libvirtd on
another machine.
The _only_ two safe ways to share configuration between multiple
machines is to share the output of 'virsh dumpxml' and then to 'virsh
define' it on the destination; or to use only transient objects in the
first place (VDSM does the latter). (Well, you don't have to use virsh
- you can use any language binding that eventually boils down to the
same underlying C API that virsh is using).
For clean stops and starts I do plan on using the API to save or
destroy the running domains, etc and that should keep the /var/lib/ and
/etc/configurations consistent if I understand their purpose. I'm a little worried
about what will happen during unclean stops though.
_Always_ use the API, rather than attempting to read implementation
detail files behind libvirt's back.
> # edit the persistent definition, which will touch /etc
> virsh net-edit default
> # stop the running instance with the old definition from /var
> virsh net-destroy default
> # restart, which will repopulate /var with your changes
> virsh net-start default
I will give that a try. The problem I'm having is that the /var configuration is
being used even after a reboot instead of the /etc/ configuration.
You'd have to provide more details as to why you think the runtime
configuration is surviving a reboot; are you sure you are properly using
the API to set the persistent definition?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
9:28 a.m.
On Dec 8, 2012, at 10:56 PM, Eric Blake wrote:
On 12/08/2012 10:36 AM, David Mitchell wrote:
>
> On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
>
>> On 12/07/2012 04:32 PM, David Mitchell wrote:
>>> Where are the configuration files for libvirt and/or virsh actually stored?
>>
>> That's an implementation detail, and might change in the future. You
>> shouldn't need to care where they are stored, and instead access the
>> data via the API (such as virDomainGetXMLDesc and virDomainDefine) or
>> API wrappers (such as virsh dumpxml/define). That said:
>
> In my case I'm trying to sync the configuration files between two physical
servers. To that end I have a drbd partition which I mirror. I have /etc/libvirt
softlinked into that partition. I could easily link /var/lib/libvirt into that partition
as well but without understanding what it's for I'm not sure of all the
ramifications.
>
Don't. Mirroring the contents of /etc/libvirt or /var/libvirt is going
behind libvirt's back, and risks data inconsistencies if the libvirtd on
one machine sees data in an incomplete state as written by a libvirtd on
another machine.
Being a DRBD partition the mirror would only ever be visible to one physical server at a
time. My expectation is that libvirt won't be running at all on the backup server at
all unless libvirt has been stopped on the primary.
The _only_ two safe ways to share configuration between multiple
machines is to share the output of 'virsh dumpxml' and then to 'virsh
define' it on the destination; or to use only transient objects in the
first place (VDSM does the latter). (Well, you don't have to use virsh
- you can use any language binding that eventually boils down to the
same underlying C API that virsh is using).
I can certainly work on getting all the management stuff done this way but it seems
worrisome to me to say that the config files can't be safely copied from one server to
another. Is this really the case even if I stop the libvirt service first? Does this imply
that if I want to move my installation to new hardware that I can't just copy the
config files? Or am I reading too much into this?
I think I can probably manage to have some kind of script which can help ensure that my
servers are in sync using only the libvirt API and ignoring the config files. I think I am
still a little unclear on exactly how transient transient is. In my mind only the
persistent and defined objects would actually have their configuration saved in a file and
persist across reboots. It's surprising to me that so-called transient objects are
also saved in config files and can persist across reboots.
-David Mitchell
> For clean stops and starts I do plan on using the API to save or destroy the running
domains, etc and that should keep the /var/lib/ and /etc/configurations consistent if I
understand their purpose. I'm a little worried about what will happen during unclean
stops though.
_Always_ use the API, rather than attempting to read implementation
detail files behind libvirt's back.
>> # edit the persistent definition, which will touch /etc
>> virsh net-edit default
>> # stop the running instance with the old definition from /var
>> virsh net-destroy default
>> # restart, which will repopulate /var with your changes
>> virsh net-start default
>
> I will give that a try. The problem I'm having is that the /var configuration is
being used even after a reboot instead of the /etc/ configuration.
You'd have to provide more details as to why you think the runtime
configuration is surviving a reboot; are you sure you are properly using
the API to set the persistent definition?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
-----------------------------------------------------------------
| David Mitchell (mitchell(a)ucar.edu) Network Engineer IV |
| Tel: (303) 497-1845 National Center for |
| FAX: (303) 497-1818 Atmospheric Research |
-----------------------------------------------------------------
10:09 a.m.
On Dec 10, 2012, at 8:28 AM, David Mitchell wrote:
I think I can probably manage to have some kind of script which can help ensure that my
servers are in sync using only the libvirt API and ignoring the config files. I think I am
still a little unclear on exactly how transient transient is. In my mind only the
persistent and defined objects would actually have their configuration saved in a file and
persist across reboots. It's surprising to me that so-called transient objects are
also saved in config files and can persist across reboots.
I should probably add that part of the confusion to me seems to be related to the fact
that I can't find a way to have virsh tell me if something is persistent or transient,
at least with respect to a network object. There are specific flags for domains but not
for other object types.
-David Mitchell
-David Mitchell
-----------------------------------------------------------------
| David Mitchell (mitchell(a)ucar.edu) Network Engineer IV |
| Tel: (303) 497-1845 National Center for |
| FAX: (303) 497-1818 Atmospheric Research |
-----------------------------------------------------------------
11:21 a.m.
On 12/10/2012 09:09 AM, David Mitchell wrote:
On Dec 10, 2012, at 8:28 AM, David Mitchell wrote:
>
>
> I think I can probably manage to have some kind of script which can help ensure that
my servers are in sync using only the libvirt API and ignoring the config files. I think I
am still a little unclear on exactly how transient transient is. In my mind only the
persistent and defined objects would actually have their configuration saved in a file and
persist across reboots. It's surprising to me that so-called transient objects are
also saved in config files and can persist across reboots.
Transient data is saved so that it can survive libvirtd restarts; but it
should not survive host restarts. Contents in /var/libvirt should not
survive a reboot.
I should probably add that part of the confusion to me seems to be related to the fact
that I can't find a way to have virsh tell me if something is persistent or transient,
at least with respect to a network object. There are specific flags for domains but not
for other object types.
[Your mailer doesn't wrap long lines, which makes it a bit harder to read]
# dump the persistent definition:
virsh net-dumpxml $net --inactive
# dump the transient definition, if the network is running:
virsh net-dumpxml $net
# determine which networks are persistent vs. transient:
virsh net-list --all
This design mirrors the same things done for domains.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
4398
days inactive
4401
days old
6 comments
2 participants
participants (2)
-
David Mitchell -
Eric Blake