[libvirt-users] Configuration file question
Where are the configuration files for libvirt and/or virsh actually stored? It seems that most of them are in /etc/libvirt but identical copies are also found in /var/lib/libvirt. In my case I am having issues with the network/default.xml file being different. Is there a reason why the config seems to be in two places at once? And which one is the 'correct' one? -David Mitchell ----------------------------------------------------------------- | David Mitchell (mitchell@ucar.edu) Network Engineer IV | | Tel: (303) 497-1845 National Center for | | FAX: (303) 497-1818 Atmospheric Research | -----------------------------------------------------------------
On 12/07/2012 04:32 PM, David Mitchell wrote:
Where are the configuration files for libvirt and/or virsh actually stored?
That's an implementation detail, and might change in the future. You shouldn't need to care where they are stored, and instead access the data via the API (such as virDomainGetXMLDesc and virDomainDefine) or API wrappers (such as virsh dumpxml/define). That said:
It seems that most of them are in /etc/libvirt
That would be the persistent definitions (virsh dumpxml --inactive)
but identical copies are also found in /var/lib/libvirt.
Those are not identical, but are the runtime definitions. It is possible to make changes to either one in isolation (hotplugs to an in-use object affect /var, changes that affect only the next start of an object affect /etc, and it is possible to affect both files at once).
In my case I am having issues with the network/default.xml file being different. Is there a reason why the config seems to be in two places at once?
Yes - because all libvirt objects (domains, networks, etc.) have both an active state and a persistent state, and the two can be different.
And which one is the 'correct' one?
Correct for what? Both are used by libvirt, but you should never edit either. It sounds like you instead need to do: # edit the persistent definition, which will touch /etc virsh net-edit default # stop the running instance with the old definition from /var virsh net-destroy default # restart, which will repopulate /var with your changes virsh net-start default all without you ever directly touching files in /etc or /var. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
On 12/07/2012 04:32 PM, David Mitchell wrote:
Where are the configuration files for libvirt and/or virsh actually stored?
That's an implementation detail, and might change in the future. You shouldn't need to care where they are stored, and instead access the data via the API (such as virDomainGetXMLDesc and virDomainDefine) or API wrappers (such as virsh dumpxml/define). That said:
In my case I'm trying to sync the configuration files between two physical servers. To that end I have a drbd partition which I mirror. I have /etc/libvirt softlinked into that partition. I could easily link /var/lib/libvirt into that partition as well but without understanding what it's for I'm not sure of all the ramifications. For clean stops and starts I do plan on using the API to save or destroy the running domains, etc and that should keep the /var/lib/ and /etc/configurations consistent if I understand their purpose. I'm a little worried about what will happen during unclean stops though.
It seems that most of them are in /etc/libvirt
That would be the persistent definitions (virsh dumpxml --inactive)
but identical copies are also found in /var/lib/libvirt.
Those are not identical, but are the runtime definitions. It is possible to make changes to either one in isolation (hotplugs to an in-use object affect /var, changes that affect only the next start of an object affect /etc, and it is possible to affect both files at once).
In my case I am having issues with the network/default.xml file being different. Is there a reason why the config seems to be in two places at once?
Yes - because all libvirt objects (domains, networks, etc.) have both an active state and a persistent state, and the two can be different.
And which one is the 'correct' one?
Correct for what? Both are used by libvirt, but you should never edit either. It sounds like you instead need to do:
# edit the persistent definition, which will touch /etc virsh net-edit default # stop the running instance with the old definition from /var virsh net-destroy default # restart, which will repopulate /var with your changes virsh net-start default
I will give that a try. The problem I'm having is that the /var configuration is being used even after a reboot instead of the /etc/ configuration.
all without you ever directly touching files in /etc or /var.
-- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
----------------------------------------------------------------- | David Mitchell (mitchell@ucar.edu) Network Engineer IV | | Tel: (303) 497-1845 National Center for | | FAX: (303) 497-1818 Atmospheric Research | -----------------------------------------------------------------
On 12/08/2012 10:36 AM, David Mitchell wrote:
On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
On 12/07/2012 04:32 PM, David Mitchell wrote:
Where are the configuration files for libvirt and/or virsh actually stored?
That's an implementation detail, and might change in the future. You shouldn't need to care where they are stored, and instead access the data via the API (such as virDomainGetXMLDesc and virDomainDefine) or API wrappers (such as virsh dumpxml/define). That said:
In my case I'm trying to sync the configuration files between two physical servers. To that end I have a drbd partition which I mirror. I have /etc/libvirt softlinked into that partition. I could easily link /var/lib/libvirt into that partition as well but without understanding what it's for I'm not sure of all the ramifications.
Don't. Mirroring the contents of /etc/libvirt or /var/libvirt is going behind libvirt's back, and risks data inconsistencies if the libvirtd on one machine sees data in an incomplete state as written by a libvirtd on another machine. The _only_ two safe ways to share configuration between multiple machines is to share the output of 'virsh dumpxml' and then to 'virsh define' it on the destination; or to use only transient objects in the first place (VDSM does the latter). (Well, you don't have to use virsh - you can use any language binding that eventually boils down to the same underlying C API that virsh is using).
For clean stops and starts I do plan on using the API to save or destroy the running domains, etc and that should keep the /var/lib/ and /etc/configurations consistent if I understand their purpose. I'm a little worried about what will happen during unclean stops though.
_Always_ use the API, rather than attempting to read implementation detail files behind libvirt's back.
# edit the persistent definition, which will touch /etc virsh net-edit default # stop the running instance with the old definition from /var virsh net-destroy default # restart, which will repopulate /var with your changes virsh net-start default
I will give that a try. The problem I'm having is that the /var configuration is being used even after a reboot instead of the /etc/ configuration.
You'd have to provide more details as to why you think the runtime configuration is surviving a reboot; are you sure you are properly using the API to set the persistent definition? -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Dec 8, 2012, at 10:56 PM, Eric Blake wrote:
On 12/08/2012 10:36 AM, David Mitchell wrote:
On Dec 8, 2012, at 7:56 AM, Eric Blake wrote:
On 12/07/2012 04:32 PM, David Mitchell wrote:
Where are the configuration files for libvirt and/or virsh actually stored?
That's an implementation detail, and might change in the future. You shouldn't need to care where they are stored, and instead access the data via the API (such as virDomainGetXMLDesc and virDomainDefine) or API wrappers (such as virsh dumpxml/define). That said:
In my case I'm trying to sync the configuration files between two physical servers. To that end I have a drbd partition which I mirror. I have /etc/libvirt softlinked into that partition. I could easily link /var/lib/libvirt into that partition as well but without understanding what it's for I'm not sure of all the ramifications.
Don't. Mirroring the contents of /etc/libvirt or /var/libvirt is going behind libvirt's back, and risks data inconsistencies if the libvirtd on one machine sees data in an incomplete state as written by a libvirtd on another machine.
Being a DRBD partition the mirror would only ever be visible to one physical server at a time. My expectation is that libvirt won't be running at all on the backup server at all unless libvirt has been stopped on the primary.
The _only_ two safe ways to share configuration between multiple machines is to share the output of 'virsh dumpxml' and then to 'virsh define' it on the destination; or to use only transient objects in the first place (VDSM does the latter). (Well, you don't have to use virsh - you can use any language binding that eventually boils down to the same underlying C API that virsh is using).
I can certainly work on getting all the management stuff done this way but it seems worrisome to me to say that the config files can't be safely copied from one server to another. Is this really the case even if I stop the libvirt service first? Does this imply that if I want to move my installation to new hardware that I can't just copy the config files? Or am I reading too much into this? I think I can probably manage to have some kind of script which can help ensure that my servers are in sync using only the libvirt API and ignoring the config files. I think I am still a little unclear on exactly how transient transient is. In my mind only the persistent and defined objects would actually have their configuration saved in a file and persist across reboots. It's surprising to me that so-called transient objects are also saved in config files and can persist across reboots. -David Mitchell
For clean stops and starts I do plan on using the API to save or destroy the running domains, etc and that should keep the /var/lib/ and /etc/configurations consistent if I understand their purpose. I'm a little worried about what will happen during unclean stops though.
_Always_ use the API, rather than attempting to read implementation detail files behind libvirt's back.
# edit the persistent definition, which will touch /etc virsh net-edit default # stop the running instance with the old definition from /var virsh net-destroy default # restart, which will repopulate /var with your changes virsh net-start default
I will give that a try. The problem I'm having is that the /var configuration is being used even after a reboot instead of the /etc/ configuration.
You'd have to provide more details as to why you think the runtime configuration is surviving a reboot; are you sure you are properly using the API to set the persistent definition?
-- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
----------------------------------------------------------------- | David Mitchell (mitchell@ucar.edu) Network Engineer IV | | Tel: (303) 497-1845 National Center for | | FAX: (303) 497-1818 Atmospheric Research | -----------------------------------------------------------------
On Dec 10, 2012, at 8:28 AM, David Mitchell wrote:
I think I can probably manage to have some kind of script which can help ensure that my servers are in sync using only the libvirt API and ignoring the config files. I think I am still a little unclear on exactly how transient transient is. In my mind only the persistent and defined objects would actually have their configuration saved in a file and persist across reboots. It's surprising to me that so-called transient objects are also saved in config files and can persist across reboots.
I should probably add that part of the confusion to me seems to be related to the fact that I can't find a way to have virsh tell me if something is persistent or transient, at least with respect to a network object. There are specific flags for domains but not for other object types. -David Mitchell
-David Mitchell
----------------------------------------------------------------- | David Mitchell (mitchell@ucar.edu) Network Engineer IV | | Tel: (303) 497-1845 National Center for | | FAX: (303) 497-1818 Atmospheric Research | -----------------------------------------------------------------
On 12/10/2012 09:09 AM, David Mitchell wrote:
On Dec 10, 2012, at 8:28 AM, David Mitchell wrote:
I think I can probably manage to have some kind of script which can help ensure that my servers are in sync using only the libvirt API and ignoring the config files. I think I am still a little unclear on exactly how transient transient is. In my mind only the persistent and defined objects would actually have their configuration saved in a file and persist across reboots. It's surprising to me that so-called transient objects are also saved in config files and can persist across reboots.
Transient data is saved so that it can survive libvirtd restarts; but it should not survive host restarts. Contents in /var/libvirt should not survive a reboot.
I should probably add that part of the confusion to me seems to be related to the fact that I can't find a way to have virsh tell me if something is persistent or transient, at least with respect to a network object. There are specific flags for domains but not for other object types.
[Your mailer doesn't wrap long lines, which makes it a bit harder to read] # dump the persistent definition: virsh net-dumpxml $net --inactive # dump the transient definition, if the network is running: virsh net-dumpxml $net # determine which networks are persistent vs. transient: virsh net-list --all This design mirrors the same things done for domains. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (2)
-
David Mitchell -
Eric Blake