[libvirt] PATCH: Add a augeas lens for libvirtd.conf
Augeas is a awesome config file manipulation tool. libvirtd has a config file. libvirtd meet augeas; augeas meet libvirt. Now instead of telling people 'edit /etc/libvirt/libvirtd.conf and change listen_tls to 1, and auth_tls to sasl' we can say run # augtool <<EOF set /files/etc/libvirt/libvirtd.conf/listen_tls 1 set /files/etc/libvirt/libvirtd.conf/auth_tls sasl save EOF THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself. libvirt.spec.in | 2 qemud/Makefile.am | 8 qemud/libvirtd.aug | 64 ++++++ qemud/test_libvirtd.aug | 484 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 558 insertions(+) Daniel Index: qemud/Makefile.am =================================================================== RCS file: /data/cvs/libvirt/qemud/Makefile.am,v retrieving revision 1.51 diff -u -p -r1.51 Makefile.am --- qemud/Makefile.am 20 Aug 2008 20:48:35 -0000 1.51 +++ qemud/Makefile.am 26 Aug 2008 20:03:48 -0000 @@ -24,6 +24,8 @@ EXTRA_DIST = \ libvirtd.policy \ libvirtd.sasl \ libvirtd.sysconf \ + libvirtd.aug \ + test_libvirtd.aug \ $(AVAHI_SOURCES) \ $(DAEMON_SOURCES) @@ -56,6 +58,12 @@ sbin_PROGRAMS = libvirtd confdir = $(sysconfdir)/libvirt/ conf_DATA = libvirtd.conf +augeasdir = $(datadir)/augeas/lenses +augeas_DATA = libvirtd.aug + +augeastestsdir = $(datadir)/augeas/lenses/tests +augeastests_DATA = test_libvirtd.aug + libvirtd_SOURCES = $(DAEMON_SOURCES) #-D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 -D_POSIX_C_SOURCE=199506L Index: qemud/libvirtd.aug =================================================================== RCS file: qemud/libvirtd.aug diff -N qemud/libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,64 @@ +(* /etc/libvirt/libvirtd.conf *) + +module Libvirtd = + autoload xfm + + let eol = del /[ \t]*\n/ "\n" + let value_sep = del /[ \t]*=[ \t]*/ " = " + let prespace = del /[ \t]*/ "" + + let array_sep = del /,[ \t\n]*/ ", " + let array_start = del /\[[ \t\n]*/ "[ " + let array_end = del /\]/ " ]" + + let str_val = del /\"/ "\"" . store /[^\"]*/ . del /\"/ "\"" + let bool_val = store /0|1/ + let str_array_element = [ str_val ] . del /[ \t\n]*/ "" + let str_array_val = array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end + + let str_entry (kw:string) = [ prespace . key kw . value_sep . str_val . eol ] + let bool_entry (kw:string) = [ prespace . key kw . value_sep . bool_val . eol ] + let str_array_entry (kw:string) = [ prespace . key kw . value_sep . str_array_val . eol ] + + let network_entry = bool_entry "listen_tls" + | bool_entry "listen_tcp" + | str_entry "tls_port" + | str_entry "tcp_port" + | str_entry "listen_addr" + | bool_entry "mdns_adv" + | str_entry "mdns_name" + + let sock_acl_entry = str_entry "unix_sock_group" + | str_entry "unix_sock_ro_perms" + | str_entry "unix_sock_rw_perms" + + let authentication_entry = str_entry "auth_unix_ro" + | str_entry "auth_unix_rw" + | str_entry "auth_tcp" + | str_entry "auth_tls" + + let certificate_entry = str_entry "key_file" + | str_entry "cert_file" + | str_entry "ca_file" + | str_entry "crl_file" + + let authorization_entry = bool_entry "tls_no_verify_certificate" + | str_array_entry "tls_allowed_dn_list" + | str_array_entry "sasl_allowed_username_list" + + let entry = network_entry + | sock_acl_entry + | authentication_entry + | certificate_entry + | authorization_entry + + let comment = [ label "comment" . del /#[ \t]*/ "# " . store /([^ \t\n][^\n]*)?/ . del /\n/ "\n" ] + let empty = [ label "empty" . del /[ \t]*\n/ "" ] + + let lns = ( entry | comment | empty ) + + + let filter = incl "/etc/libvirt/libvirtd.conf" + . Util.stdexcl + + let xfm = transform lns filter + Index: qemud/test_libvirtd.aug =================================================================== RCS file: qemud/test_libvirtd.aug diff -N qemud/test_libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/test_libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,484 @@ +module Test_libvirtd = + let conf1 = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 +" + + let conf = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 + +# Listen for unencrypted TCP connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# Using the TCP socket requires SASL authentication by default. Only +# SASL mechanisms which support data encryption are allowed. This is +# DIGEST_MD5 and GSSAPI (Kerberos5) +# +# This is disabled by default, uncomment this to enable it. +listen_tcp = 1 + + + +# Override the port for accepting secure TLS connections +# This can be a port number, or service name +# +tls_port = \"16514\" + +# Override the port for accepting insecure TCP connections +# This can be a port number, or service name +# +tcp_port = \"16509\" + + +# Override the default configuration which binds to all network +# interfaces. This can be a numeric IPv4/6 address, or hostname +# +listen_addr = \"192.168.0.1\" + + +# Flag toggling mDNS advertizement of the libvirt service. +# +# Alternatively can disable for all services on a host by +# stopping the Avahi daemon +# +# This is enabled by default, uncomment this to disable it +mdns_adv = 0 + +# Override the default mDNS advertizement name. This must be +# unique on the immediate broadcast network. +# +# The default is \"Virtualization Host HOSTNAME\", where HOSTNAME +# is subsituted for the short hostname of the machine (without domain) +# +mdns_name = \"Virtualization Host Joe Demo\" + + +################################################################# +# +# UNIX socket access controls +# + +# Set the UNIX domain socket group ownership. This can be used to +# allow a 'trusted' set of users access to management capabilities +# without becoming root. +# +# This is restricted to 'root' by default. +unix_sock_group = \"libvirt\" + +# Set the UNIX socket permissions for the R/O socket. This is used +# for monitoring VM status only +# +# Default allows any user. If setting group ownership may want to +# restrict this to: +unix_sock_ro_perms = \"0777\" + +# Set the UNIX socket permissions for the R/W socket. This is used +# for full management of VMs +# +# Default allows only root. If PolicyKit is enabled on the socket, +# the default will change to allow everyone (eg, 0777) +# +# If not using PolicyKit and setting group ownership for access +# control then you may want to relax this to: +unix_sock_rw_perms = \"0770\" + + + +################################################################# +# +# Authentication. +# +# - none: do not perform auth checks. If you can connect to the +# socket you are allowed. This is suitable if there are +# restrictions on connecting to the socket (eg, UNIX +# socket permissions), or if there is a lower layer in +# the network providing auth (eg, TLS/x509 certificates) +# +# - sasl: use SASL infrastructure. The actual auth scheme is then +# controlled from /etc/sasl2/libvirt.conf. For the TCP +# socket only GSSAPI & DIGEST-MD5 mechanisms will be used. +# For non-TCP or TLS sockets, any scheme is allowed. +# +# - polkit: use PolicyKit to authenticate. This is only suitable +# for use on the UNIX sockets. The default policy will +# require a user to supply their own password to gain +# full read/write access (aka sudo like), while anyone +# is allowed read/only access. +# +# Set an authentication scheme for UNIX read-only sockets +# By default socket permissions allow anyone to connect +# +# To restrict monitoring of domains you may wish to enable +# an authentication mechanism here +auth_unix_ro = \"none\" + +# Set an authentication scheme for UNIX read-write sockets +# By default socket permissions only allow root. If PolicyKit +# support was compiled into libvirt, the default will be to +# use 'polkit' auth. +# +# If the unix_sock_rw_perms are changed you may wish to enable +# an authentication mechanism here +auth_unix_rw = \"none\" + +# Change the authentication scheme for TCP sockets. +# +# If you don't enable SASL, then all TCP traffic is cleartext. +# Don't do this outside of a dev/test scenario. For real world +# use, always enable SASL and use the GSSAPI or DIGEST-MD5 +# mechanism in /etc/sasl2/libvirt.conf +auth_tcp = \"sasl\" + +# Change the authentication scheme for TLS sockets. +# +# TLS sockets already have encryption provided by the TLS +# layer, and limited authentication is done by certificates +# +# It is possible to make use of any SASL authentication +# mechanism as well, by using 'sasl' for this option +auth_tls = \"none\" + + + +################################################################# +# +# TLS x509 certificate configuration +# + + +# Override the default server key file path +# +key_file = \"/etc/pki/libvirt/private/serverkey.pem\" + +# Override the default server certificate file path +# +cert_file = \"/etc/pki/libvirt/servercert.pem\" + +# Override the default CA certificate path +# +ca_file = \"/etc/pki/CA/cacert.pem\" + +# Specify a certificate revocation list. +# +# Defaults to not using a CRL, uncomment to enable it +crl_file = \"/etc/pki/CA/crl.pem\" + + + +################################################################# +# +# Authorization controls +# + + +# Flag to disable verification of client certificates +# +# Client certificate verification is the primary authentication mechanism. +# Any client which does not present a certificate signed by the CA +# will be rejected. +# +# Default is to always verify. Uncommenting this will disable +# verification - make sure an IP whitelist is set +tls_no_verify_certificate = 1 + + +# A whitelist of allowed x509 Distinguished Names +# This list may contain wildcards such as +# +# \"C=GB,ST=London,L=London,O=Red Hat,CN=*\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no DN's are checked + tls_allowed_dn_list = [\"DN1\", \"DN2\"] + + +# A whitelist of allowed SASL usernames. The format for usernames +# depends on the SASL authentication mechanism. Kerberos usernames +# look like username@REALM +# +# This list may contain wildcards such as +# +# \"*@EXAMPLE.COM\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no Username's are checked +sasl_allowed_username_list = [ + \"joe@EXAMPLE.COM\", + \"fred@EXAMPLE.COM\" +] +" + + test Libvirtd.lns get conf = + { "comment" = "Master libvirt daemon configuration file" } + { "comment" = "" } + { "comment" = "For further information consult http://libvirt.org/format.html" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Network connectivity controls" } + { "comment" = "" } + { "empty" } + { "comment" = "Flag listening for secure TLS connections on the public TCP/IP port." } + { "comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "comment" = "have any effect." } + { "comment" = "" } + { "comment" = "It is necessary to setup a CA and issue server certificates before" } + { "comment" = "using this capability." } + { "comment" = "" } + { "comment" = "This is enabled by default, uncomment this to disable it" } + { "listen_tls" = "0" } + { "empty" } + { "comment" = "Listen for unencrypted TCP connections on the public TCP/IP port." } + { "comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "comment" = "have any effect." } + { "comment" = "" } + { "comment" = "Using the TCP socket requires SASL authentication by default. Only" } + { "comment" = "SASL mechanisms which support data encryption are allowed. This is" } + { "comment" = "DIGEST_MD5 and GSSAPI (Kerberos5)" } + { "comment" = "" } + { "comment" = "This is disabled by default, uncomment this to enable it." } + { "listen_tcp" = "1" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "Override the port for accepting secure TLS connections" } + { "comment" = "This can be a port number, or service name" } + { "comment" = "" } + { "tls_port" = "16514" } + { "empty" } + { "comment" = "Override the port for accepting insecure TCP connections" } + { "comment" = "This can be a port number, or service name" } + { "comment" = "" } + { "tcp_port" = "16509" } + { "empty" } + { "empty" } + { "comment" = "Override the default configuration which binds to all network" } + { "comment" = "interfaces. This can be a numeric IPv4/6 address, or hostname" } + { "comment" = "" } + { "listen_addr" = "192.168.0.1" } + { "empty" } + { "empty" } + { "comment" = "Flag toggling mDNS advertizement of the libvirt service." } + { "comment" = "" } + { "comment" = "Alternatively can disable for all services on a host by" } + { "comment" = "stopping the Avahi daemon" } + { "comment" = "" } + { "comment" = "This is enabled by default, uncomment this to disable it" } + { "mdns_adv" = "0" } + { "empty" } + { "comment" = "Override the default mDNS advertizement name. This must be" } + { "comment" = "unique on the immediate broadcast network." } + { "comment" = "" } + { "comment" = "The default is \"Virtualization Host HOSTNAME\", where HOSTNAME" } + { "comment" = "is subsituted for the short hostname of the machine (without domain)" } + { "comment" = "" } + { "mdns_name" = "Virtualization Host Joe Demo" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "UNIX socket access controls" } + { "comment" = "" } + { "empty" } + { "comment" = "Set the UNIX domain socket group ownership. This can be used to" } + { "comment" = "allow a 'trusted' set of users access to management capabilities" } + { "comment" = "without becoming root." } + { "comment" = "" } + { "comment" = "This is restricted to 'root' by default." } + { "unix_sock_group" = "libvirt" } + { "empty" } + { "comment" = "Set the UNIX socket permissions for the R/O socket. This is used" } + { "comment" = "for monitoring VM status only" } + { "comment" = "" } + { "comment" = "Default allows any user. If setting group ownership may want to" } + { "comment" = "restrict this to:" } + { "unix_sock_ro_perms" = "0777" } + { "empty" } + { "comment" = "Set the UNIX socket permissions for the R/W socket. This is used" } + { "comment" = "for full management of VMs" } + { "comment" = "" } + { "comment" = "Default allows only root. If PolicyKit is enabled on the socket," } + { "comment" = "the default will change to allow everyone (eg, 0777)" } + { "comment" = "" } + { "comment" = "If not using PolicyKit and setting group ownership for access" } + { "comment" = "control then you may want to relax this to:" } + { "unix_sock_rw_perms" = "0770" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Authentication." } + { "comment" = "" } + { "comment" = "- none: do not perform auth checks. If you can connect to the" } + { "comment" = "socket you are allowed. This is suitable if there are" } + { "comment" = "restrictions on connecting to the socket (eg, UNIX" } + { "comment" = "socket permissions), or if there is a lower layer in" } + { "comment" = "the network providing auth (eg, TLS/x509 certificates)" } + { "comment" = "" } + { "comment" = "- sasl: use SASL infrastructure. The actual auth scheme is then" } + { "comment" = "controlled from /etc/sasl2/libvirt.conf. For the TCP" } + { "comment" = "socket only GSSAPI & DIGEST-MD5 mechanisms will be used." } + { "comment" = "For non-TCP or TLS sockets, any scheme is allowed." } + { "comment" = "" } + { "comment" = "- polkit: use PolicyKit to authenticate. This is only suitable" } + { "comment" = "for use on the UNIX sockets. The default policy will" } + { "comment" = "require a user to supply their own password to gain" } + { "comment" = "full read/write access (aka sudo like), while anyone" } + { "comment" = "is allowed read/only access." } + { "comment" = "" } + { "comment" = "Set an authentication scheme for UNIX read-only sockets" } + { "comment" = "By default socket permissions allow anyone to connect" } + { "comment" = "" } + { "comment" = "To restrict monitoring of domains you may wish to enable" } + { "comment" = "an authentication mechanism here" } + { "auth_unix_ro" = "none" } + { "empty" } + { "comment" = "Set an authentication scheme for UNIX read-write sockets" } + { "comment" = "By default socket permissions only allow root. If PolicyKit" } + { "comment" = "support was compiled into libvirt, the default will be to" } + { "comment" = "use 'polkit' auth." } + { "comment" = "" } + { "comment" = "If the unix_sock_rw_perms are changed you may wish to enable" } + { "comment" = "an authentication mechanism here" } + { "auth_unix_rw" = "none" } + { "empty" } + { "comment" = "Change the authentication scheme for TCP sockets." } + { "comment" = "" } + { "comment" = "If you don't enable SASL, then all TCP traffic is cleartext." } + { "comment" = "Don't do this outside of a dev/test scenario. For real world" } + { "comment" = "use, always enable SASL and use the GSSAPI or DIGEST-MD5" } + { "comment" = "mechanism in /etc/sasl2/libvirt.conf" } + { "auth_tcp" = "sasl" } + { "empty" } + { "comment" = "Change the authentication scheme for TLS sockets." } + { "comment" = "" } + { "comment" = "TLS sockets already have encryption provided by the TLS" } + { "comment" = "layer, and limited authentication is done by certificates" } + { "comment" = "" } + { "comment" = "It is possible to make use of any SASL authentication" } + { "comment" = "mechanism as well, by using 'sasl' for this option" } + { "auth_tls" = "none" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "TLS x509 certificate configuration" } + { "comment" = "" } + { "empty" } + { "empty" } + { "comment" = "Override the default server key file path" } + { "comment" = "" } + { "key_file" = "/etc/pki/libvirt/private/serverkey.pem" } + { "empty" } + { "comment" = "Override the default server certificate file path" } + { "comment" = "" } + { "cert_file" = "/etc/pki/libvirt/servercert.pem" } + { "empty" } + { "comment" = "Override the default CA certificate path" } + { "comment" = "" } + { "ca_file" = "/etc/pki/CA/cacert.pem" } + { "empty" } + { "comment" = "Specify a certificate revocation list." } + { "comment" = "" } + { "comment" = "Defaults to not using a CRL, uncomment to enable it" } + { "crl_file" = "/etc/pki/CA/crl.pem" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Authorization controls" } + { "comment" = "" } + { "empty" } + { "empty" } + { "comment" = "Flag to disable verification of client certificates" } + { "comment" = "" } + { "comment" = "Client certificate verification is the primary authentication mechanism." } + { "comment" = "Any client which does not present a certificate signed by the CA" } + { "comment" = "will be rejected." } + { "comment" = "" } + { "comment" = "Default is to always verify. Uncommenting this will disable" } + { "comment" = "verification - make sure an IP whitelist is set" } + { "tls_no_verify_certificate" = "1" } + { "empty" } + { "empty" } + { "comment" = "A whitelist of allowed x509 Distinguished Names" } + { "comment" = "This list may contain wildcards such as" } + { "comment" = "" } + { "comment" = "\"C=GB,ST=London,L=London,O=Red Hat,CN=*\"" } + { "comment" = "" } + { "comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "comment" = "" } + { "comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "comment" = "entirely rather than using empty list to disable these checks" } + { "comment" = "" } + { "comment" = "By default, no DN's are checked" } + { "tls_allowed_dn_list" + { = "DN1"} + { = "DN2"} + } + { "empty" } + { "empty" } + { "comment" = "A whitelist of allowed SASL usernames. The format for usernames" } + { "comment" = "depends on the SASL authentication mechanism. Kerberos usernames" } + { "comment" = "look like username@REALM" } + { "comment" = "" } + { "comment" = "This list may contain wildcards such as" } + { "comment" = "" } + { "comment" = "\"*@EXAMPLE.COM\"" } + { "comment" = "" } + { "comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "comment" = "" } + { "comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "comment" = "entirely rather than using empty list to disable these checks" } + { "comment" = "" } + { "comment" = "By default, no Username's are checked" } + { "sasl_allowed_username_list" + { = "joe@EXAMPLE.COM" } + { = "fred@EXAMPLE.COM" } + } Index: libvirt.spec.in =================================================================== RCS file: /data/cvs/libvirt/libvirt.spec.in,v retrieving revision 1.91 diff -u -p -r1.91 libvirt.spec.in --- libvirt.spec.in 21 Aug 2008 09:28:54 -0000 1.91 +++ libvirt.spec.in 26 Aug 2008 20:04:24 -0000 @@ -252,6 +252,8 @@ fi %dir %{_localstatedir}/lib/libvirt/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/images/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/boot/ +%{_datadir}/augeas/lenses/libvirtd.aug +%{_datadir}/augeas/lenses/tests/test_libvirtd.aug %if %{with_polkit} %{_datadir}/PolicyKit/policy/org.libvirt.unix.policy %endif -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Hi Daniel, On Tue, Aug 26, 2008 at 10:05 PM, Daniel P. Berrange <berrange@redhat.com>wrote:
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
Very nice idea. This might have to be thought about for the future since so far we're only adding lenses to the Augeas repository.
=================================================================== RCS file: qemud/libvirtd.aug diff -N qemud/libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,64 @@ +(* /etc/libvirt/libvirtd.conf *) + +module Libvirtd = + autoload xfm + + let eol = del /[ \t]*\n/ "\n"
This is also Util.eol
+ let value_sep = del /[ \t]*=[ \t]*/ " = " + let prespace = del /[ \t]*/ "" +
And this is Util.indent
+ let array_sep = del /,[ \t\n]*/ ", " + let array_start = del /\[[ \t\n]*/ "[ " + let array_end = del /\]/ " ]" + + let str_val = del /\"/ "\"" . store /[^\"]*/ . del /\"/ "\"" + let bool_val = store /0|1/ + let str_array_element = [ str_val ] . del /[ \t\n]*/ "" + let str_array_val = array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end + + let str_entry (kw:string) = [ prespace . key kw . value_sep . str_val . eol ] + let bool_entry (kw:string) = [ prespace . key kw . value_sep . bool_val . eol ] + let str_array_entry (kw:string) = [ prespace . key kw . value_sep . str_array_val . eol ] + + let network_entry = bool_entry "listen_tls" + | bool_entry "listen_tcp" + | str_entry "tls_port" + | str_entry "tcp_port" + | str_entry "listen_addr" + | bool_entry "mdns_adv" + | str_entry "mdns_name" +
While I can see why it is useful to gather these entries logically, it's not very optimised for the parser. Eventually, all this will be compiled into a huge regexp, so it's more efficient to regroup entries by type and feed the functions with regexps instead of strings, like let str_entry (kw:regexp) = [ prespace . key kw . value_sep . str_val . eol ] let bool_entry (kw:regexp) = [ prespace . key kw . value_sep . bool_val . eol ] let str_array_entry (kw:regexp) = [ prespace . key kw . value_sep . str_array_val . eol ] let str_keys = "tls_port" | "tcp_port" | "listen_addr" | "mdns_name" | etc. let bool_keys = "listen_tls" | "listen_tcp" | "mdns_adv" | etc. let str_entries = str_entry str_keys let bool_entries = bool_entry bool_keys You get the point, each function is only evaluated once, with a single regexp (every string ets turned into a regexp, and so will str_keys and bool_keys). The less you will call the *_entry functions, the faster your lens will be.
+ let sock_acl_entry = str_entry "unix_sock_group" + | str_entry "unix_sock_ro_perms" + | str_entry "unix_sock_rw_perms" + + let authentication_entry = str_entry "auth_unix_ro" + | str_entry "auth_unix_rw" + | str_entry "auth_tcp" + | str_entry "auth_tls" + + let certificate_entry = str_entry "key_file" + | str_entry "cert_file" + | str_entry "ca_file" + | str_entry "crl_file" + + let authorization_entry = bool_entry "tls_no_verify_certificate" + | str_array_entry "tls_allowed_dn_list" + | str_array_entry "sasl_allowed_username_list" + + let entry = network_entry + | sock_acl_entry + | authentication_entry + | certificate_entry + | authorization_entry +
So here you would have something like let entry = str_entries | bool_entries | etc. gather entries by parser type. + let comment = [ label "comment" . del /#[ \t]*/ "# " . store /([^
\t\n][^\n]*)?/ . del /\n/ "\n" ]
Some time ago, we thought that it would be good to map comments in "#comment" nodes to avoid confusion with possible "comment" nodes, especially when the lens allows a wide range of labels. Util.comment would fit your need here, and the new version of it would provide a "#comment" node. + let empty = [ label "empty" . del /[ \t]*\n/ "" ] Util.empty provides a definition of empty lines, too. Do you really need to map them with a label? I thought about doing that some time ago, but mapping them in "#empty" for the same reason that we map comments in "#comment". The useful application to this is to allow users to control empty lines in conffile from the augeas API.
+ + let lns = ( entry | comment | empty ) +
Is an empty file not valid?
+ + let filter = incl "/etc/libvirt/libvirtd.conf" + . Util.stdexcl + + let xfm = transform lns filter + Index: qemud/test_libvirtd.aug =================================================================== RCS file: qemud/test_libvirtd.aug diff -N qemud/test_libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/test_libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,484 @@ +module Test_libvirtd = + let conf1 = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 +" + + let conf = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 + +# Listen for unencrypted TCP connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# Using the TCP socket requires SASL authentication by default. Only +# SASL mechanisms which support data encryption are allowed. This is +# DIGEST_MD5 and GSSAPI (Kerberos5) +# +# This is disabled by default, uncomment this to enable it. +listen_tcp = 1 + + + +# Override the port for accepting secure TLS connections +# This can be a port number, or service name +# +tls_port = \"16514\" + +# Override the port for accepting insecure TCP connections +# This can be a port number, or service name +# +tcp_port = \"16509\" + + +# Override the default configuration which binds to all network +# interfaces. This can be a numeric IPv4/6 address, or hostname +# +listen_addr = \"192.168.0.1\" + + +# Flag toggling mDNS advertizement of the libvirt service. +# +# Alternatively can disable for all services on a host by +# stopping the Avahi daemon +# +# This is enabled by default, uncomment this to disable it +mdns_adv = 0 + +# Override the default mDNS advertizement name. This must be +# unique on the immediate broadcast network. +# +# The default is \"Virtualization Host HOSTNAME\", where HOSTNAME +# is subsituted for the short hostname of the machine (without domain) +# +mdns_name = \"Virtualization Host Joe Demo\" + + +################################################################# +# +# UNIX socket access controls +# + +# Set the UNIX domain socket group ownership. This can be used to +# allow a 'trusted' set of users access to management capabilities +# without becoming root. +# +# This is restricted to 'root' by default. +unix_sock_group = \"libvirt\" + +# Set the UNIX socket permissions for the R/O socket. This is used +# for monitoring VM status only +# +# Default allows any user. If setting group ownership may want to +# restrict this to: +unix_sock_ro_perms = \"0777\" + +# Set the UNIX socket permissions for the R/W socket. This is used +# for full management of VMs +# +# Default allows only root. If PolicyKit is enabled on the socket, +# the default will change to allow everyone (eg, 0777) +# +# If not using PolicyKit and setting group ownership for access +# control then you may want to relax this to: +unix_sock_rw_perms = \"0770\" + + + +################################################################# +# +# Authentication. +# +# - none: do not perform auth checks. If you can connect to the +# socket you are allowed. This is suitable if there are +# restrictions on connecting to the socket (eg, UNIX +# socket permissions), or if there is a lower layer in +# the network providing auth (eg, TLS/x509 certificates) +# +# - sasl: use SASL infrastructure. The actual auth scheme is then +# controlled from /etc/sasl2/libvirt.conf. For the TCP +# socket only GSSAPI & DIGEST-MD5 mechanisms will be used. +# For non-TCP or TLS sockets, any scheme is allowed. +# +# - polkit: use PolicyKit to authenticate. This is only suitable +# for use on the UNIX sockets. The default policy will +# require a user to supply their own password to gain +# full read/write access (aka sudo like), while anyone +# is allowed read/only access. +# +# Set an authentication scheme for UNIX read-only sockets +# By default socket permissions allow anyone to connect +# +# To restrict monitoring of domains you may wish to enable +# an authentication mechanism here +auth_unix_ro = \"none\" + +# Set an authentication scheme for UNIX read-write sockets +# By default socket permissions only allow root. If PolicyKit +# support was compiled into libvirt, the default will be to +# use 'polkit' auth. +# +# If the unix_sock_rw_perms are changed you may wish to enable +# an authentication mechanism here +auth_unix_rw = \"none\" + +# Change the authentication scheme for TCP sockets. +# +# If you don't enable SASL, then all TCP traffic is cleartext. +# Don't do this outside of a dev/test scenario. For real world +# use, always enable SASL and use the GSSAPI or DIGEST-MD5 +# mechanism in /etc/sasl2/libvirt.conf +auth_tcp = \"sasl\" + +# Change the authentication scheme for TLS sockets. +# +# TLS sockets already have encryption provided by the TLS +# layer, and limited authentication is done by certificates +# +# It is possible to make use of any SASL authentication +# mechanism as well, by using 'sasl' for this option +auth_tls = \"none\" + + + +################################################################# +# +# TLS x509 certificate configuration +# + + +# Override the default server key file path +# +key_file = \"/etc/pki/libvirt/private/serverkey.pem\" + +# Override the default server certificate file path +# +cert_file = \"/etc/pki/libvirt/servercert.pem\" + +# Override the default CA certificate path +# +ca_file = \"/etc/pki/CA/cacert.pem\" + +# Specify a certificate revocation list. +# +# Defaults to not using a CRL, uncomment to enable it +crl_file = \"/etc/pki/CA/crl.pem\" + + + +################################################################# +# +# Authorization controls +# + + +# Flag to disable verification of client certificates +# +# Client certificate verification is the primary authentication mechanism. +# Any client which does not present a certificate signed by the CA +# will be rejected. +# +# Default is to always verify. Uncommenting this will disable +# verification - make sure an IP whitelist is set +tls_no_verify_certificate = 1 + + +# A whitelist of allowed x509 Distinguished Names +# This list may contain wildcards such as +# +# \"C=GB,ST=London,L=London,O=Red Hat,CN=*\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no DN's are checked + tls_allowed_dn_list = [\"DN1\", \"DN2\"] + + +# A whitelist of allowed SASL usernames. The format for usernames +# depends on the SASL authentication mechanism. Kerberos usernames +# look like username@REALM +# +# This list may contain wildcards such as +# +# \"*@EXAMPLE.COM\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no Username's are checked +sasl_allowed_username_list = [ + \"joe@EXAMPLE.COM\", + \"fred@EXAMPLE.COM\" +] +" + + test Libvirtd.lns get conf = + { "comment" = "Master libvirt daemon configuration file" } + { "comment" = "" } + { "comment" = "For further information consult http://libvirt.org/format.html" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Network connectivity controls" } + { "comment" = "" } + { "empty" } + { "comment" = "Flag listening for secure TLS connections on the public TCP/IP port." } + { "comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "comment" = "have any effect." } + { "comment" = "" } + { "comment" = "It is necessary to setup a CA and issue server certificates before" } + { "comment" = "using this capability." } + { "comment" = "" } + { "comment" = "This is enabled by default, uncomment this to disable it" } + { "listen_tls" = "0" } + { "empty" } + { "comment" = "Listen for unencrypted TCP connections on the public TCP/IP port." } + { "comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "comment" = "have any effect." } + { "comment" = "" } + { "comment" = "Using the TCP socket requires SASL authentication by default. Only" } + { "comment" = "SASL mechanisms which support data encryption are allowed. This is" } + { "comment" = "DIGEST_MD5 and GSSAPI (Kerberos5)" } + { "comment" = "" } + { "comment" = "This is disabled by default, uncomment this to enable it." } + { "listen_tcp" = "1" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "Override the port for accepting secure TLS connections" } + { "comment" = "This can be a port number, or service name" } + { "comment" = "" } + { "tls_port" = "16514" } + { "empty" } + { "comment" = "Override the port for accepting insecure TCP connections" } + { "comment" = "This can be a port number, or service name" } + { "comment" = "" } + { "tcp_port" = "16509" } + { "empty" } + { "empty" } + { "comment" = "Override the default configuration which binds to all network" } + { "comment" = "interfaces. This can be a numeric IPv4/6 address, or hostname" } + { "comment" = "" } + { "listen_addr" = "192.168.0.1" } + { "empty" } + { "empty" } + { "comment" = "Flag toggling mDNS advertizement of the libvirt service." } + { "comment" = "" } + { "comment" = "Alternatively can disable for all services on a host by" } + { "comment" = "stopping the Avahi daemon" } + { "comment" = "" } + { "comment" = "This is enabled by default, uncomment this to disable it" } + { "mdns_adv" = "0" } + { "empty" } + { "comment" = "Override the default mDNS advertizement name. This must be" } + { "comment" = "unique on the immediate broadcast network." } + { "comment" = "" } + { "comment" = "The default is \"Virtualization Host HOSTNAME\", where HOSTNAME" } + { "comment" = "is subsituted for the short hostname of the machine (without domain)" } + { "comment" = "" } + { "mdns_name" = "Virtualization Host Joe Demo" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "UNIX socket access controls" } + { "comment" = "" } + { "empty" } + { "comment" = "Set the UNIX domain socket group ownership. This can be used to" } + { "comment" = "allow a 'trusted' set of users access to management capabilities" } + { "comment" = "without becoming root." } + { "comment" = "" } + { "comment" = "This is restricted to 'root' by default." } + { "unix_sock_group" = "libvirt" } + { "empty" } + { "comment" = "Set the UNIX socket permissions for the R/O socket. This is used" } + { "comment" = "for monitoring VM status only" } + { "comment" = "" } + { "comment" = "Default allows any user. If setting group ownership may want to" } + { "comment" = "restrict this to:" } + { "unix_sock_ro_perms" = "0777" } + { "empty" } + { "comment" = "Set the UNIX socket permissions for the R/W socket. This is used" } + { "comment" = "for full management of VMs" } + { "comment" = "" } + { "comment" = "Default allows only root. If PolicyKit is enabled on the socket," } + { "comment" = "the default will change to allow everyone (eg, 0777)" } + { "comment" = "" } + { "comment" = "If not using PolicyKit and setting group ownership for access" } + { "comment" = "control then you may want to relax this to:" } + { "unix_sock_rw_perms" = "0770" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Authentication." } + { "comment" = "" } + { "comment" = "- none: do not perform auth checks. If you can connect to the" } + { "comment" = "socket you are allowed. This is suitable if there are" } + { "comment" = "restrictions on connecting to the socket (eg, UNIX" } + { "comment" = "socket permissions), or if there is a lower layer in" } + { "comment" = "the network providing auth (eg, TLS/x509 certificates)" } + { "comment" = "" } + { "comment" = "- sasl: use SASL infrastructure. The actual auth scheme is then" } + { "comment" = "controlled from /etc/sasl2/libvirt.conf. For the TCP" } + { "comment" = "socket only GSSAPI & DIGEST-MD5 mechanisms will be used." } + { "comment" = "For non-TCP or TLS sockets, any scheme is allowed." } + { "comment" = "" } + { "comment" = "- polkit: use PolicyKit to authenticate. This is only suitable" } + { "comment" = "for use on the UNIX sockets. The default policy will" } + { "comment" = "require a user to supply their own password to gain" } + { "comment" = "full read/write access (aka sudo like), while anyone" } + { "comment" = "is allowed read/only access." } + { "comment" = "" } + { "comment" = "Set an authentication scheme for UNIX read-only sockets" } + { "comment" = "By default socket permissions allow anyone to connect" } + { "comment" = "" } + { "comment" = "To restrict monitoring of domains you may wish to enable" } + { "comment" = "an authentication mechanism here" } + { "auth_unix_ro" = "none" } + { "empty" } + { "comment" = "Set an authentication scheme for UNIX read-write sockets" } + { "comment" = "By default socket permissions only allow root. If PolicyKit" } + { "comment" = "support was compiled into libvirt, the default will be to" } + { "comment" = "use 'polkit' auth." } + { "comment" = "" } + { "comment" = "If the unix_sock_rw_perms are changed you may wish to enable" } + { "comment" = "an authentication mechanism here" } + { "auth_unix_rw" = "none" } + { "empty" } + { "comment" = "Change the authentication scheme for TCP sockets." } + { "comment" = "" } + { "comment" = "If you don't enable SASL, then all TCP traffic is cleartext." } + { "comment" = "Don't do this outside of a dev/test scenario. For real world" } + { "comment" = "use, always enable SASL and use the GSSAPI or DIGEST-MD5" } + { "comment" = "mechanism in /etc/sasl2/libvirt.conf" } + { "auth_tcp" = "sasl" } + { "empty" } + { "comment" = "Change the authentication scheme for TLS sockets." } + { "comment" = "" } + { "comment" = "TLS sockets already have encryption provided by the TLS" } + { "comment" = "layer, and limited authentication is done by certificates" } + { "comment" = "" } + { "comment" = "It is possible to make use of any SASL authentication" } + { "comment" = "mechanism as well, by using 'sasl' for this option" } + { "auth_tls" = "none" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "TLS x509 certificate configuration" } + { "comment" = "" } + { "empty" } + { "empty" } + { "comment" = "Override the default server key file path" } + { "comment" = "" } + { "key_file" = "/etc/pki/libvirt/private/serverkey.pem" } + { "empty" } + { "comment" = "Override the default server certificate file path" } + { "comment" = "" } + { "cert_file" = "/etc/pki/libvirt/servercert.pem" } + { "empty" } + { "comment" = "Override the default CA certificate path" } + { "comment" = "" } + { "ca_file" = "/etc/pki/CA/cacert.pem" } + { "empty" } + { "comment" = "Specify a certificate revocation list." } + { "comment" = "" } + { "comment" = "Defaults to not using a CRL, uncomment to enable it" } + { "crl_file" = "/etc/pki/CA/crl.pem" } + { "empty" } + { "empty" } + { "empty" } + { "comment" = "################################################################" } + { "comment" = "" } + { "comment" = "Authorization controls" } + { "comment" = "" } + { "empty" } + { "empty" } + { "comment" = "Flag to disable verification of client certificates" } + { "comment" = "" } + { "comment" = "Client certificate verification is the primary authentication mechanism." } + { "comment" = "Any client which does not present a certificate signed by the CA" } + { "comment" = "will be rejected." } + { "comment" = "" } + { "comment" = "Default is to always verify. Uncommenting this will disable" } + { "comment" = "verification - make sure an IP whitelist is set" } + { "tls_no_verify_certificate" = "1" } + { "empty" } + { "empty" } + { "comment" = "A whitelist of allowed x509 Distinguished Names" } + { "comment" = "This list may contain wildcards such as" } + { "comment" = "" } + { "comment" = "\"C=GB,ST=London,L=London,O=Red Hat,CN=*\"" } + { "comment" = "" } + { "comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "comment" = "" } + { "comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "comment" = "entirely rather than using empty list to disable these checks" } + { "comment" = "" } + { "comment" = "By default, no DN's are checked" } + { "tls_allowed_dn_list" + { = "DN1"} + { = "DN2"} + } + { "empty" } + { "empty" } + { "comment" = "A whitelist of allowed SASL usernames. The format for usernames" } + { "comment" = "depends on the SASL authentication mechanism. Kerberos usernames" } + { "comment" = "look like username@REALM" } + { "comment" = "" } + { "comment" = "This list may contain wildcards such as" } + { "comment" = "" } + { "comment" = "\"*@EXAMPLE.COM\"" } + { "comment" = "" } + { "comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "comment" = "" } + { "comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "comment" = "entirely rather than using empty list to disable these checks" } + { "comment" = "" } + { "comment" = "By default, no Username's are checked" } + { "sasl_allowed_username_list" + { = "joe@EXAMPLE.COM" } + { = "fred@EXAMPLE.COM" } + } Index: libvirt.spec.in =================================================================== RCS file: /data/cvs/libvirt/libvirt.spec.in,v retrieving revision 1.91 diff -u -p -r1.91 libvirt.spec.in --- libvirt.spec.in 21 Aug 2008 09:28:54 -0000 1.91 +++ libvirt.spec.in 26 Aug 2008 20:04:24 -0000 @@ -252,6 +252,8 @@ fi %dir %{_localstatedir}/lib/libvirt/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/images/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/boot/ +%{_datadir}/augeas/lenses/libvirtd.aug +%{_datadir}/augeas/lenses/tests/test_libvirtd.aug %if %{with_polkit} %{_datadir}/PolicyKit/policy/org.libvirt.unix.policy %endif
-- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/:| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/<http://search.cpan.org/%7Edanberr/>:| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
_______________________________________________ augeas-devel mailing list augeas-devel@redhat.com https://www.redhat.com/mailman/listinfo/augeas-devel
On Tue, Aug 26, 2008 at 10:37:02PM +0200, Rapha?l Pinson wrote:
On Tue, Aug 26, 2008 at 10:05 PM, Daniel P. Berrange <berrange@redhat.com>wrote:
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
Very nice idea. This might have to be thought about for the future since so far we're only adding lenses to the Augeas repository.
My motivation is that we add more config file entries in each release, so its more sustainable to distribute lens with the corresponding upstream tar.gz, than in augeas. IMHO its only worth putting stuff in augeas if upstream is not willing to accept the lens.
+ let str_entry (kw:string) = [ prespace . key kw . value_sep . str_val . eol ] + let bool_entry (kw:string) = [ prespace . key kw . value_sep . bool_val . eol ] + let str_array_entry (kw:string) = [ prespace . key kw . value_sep . str_array_val . eol ] + + let network_entry = bool_entry "listen_tls" + | bool_entry "listen_tcp" + | str_entry "tls_port" + | str_entry "tcp_port" + | str_entry "listen_addr" + | bool_entry "mdns_adv" + | str_entry "mdns_name" +
While I can see why it is useful to gather these entries logically, it's not very optimised for the parser. Eventually, all this will be compiled into a huge regexp, so it's more efficient to regroup entries by type and feed the functions with regexps instead of strings, like
Is performance really that much of a problem ? The libvirt config file only has 20 or so different settings, and while we'll add more I can't see it getting very much larger. Changing these settings is also not something that would be done on a frequent basis / performance critical path. I find it more readable to group them by functional area unless there is a serious real world performance issue.
+ let comment = [ label "comment" . del /#[ \t]*/ "# " . store /([^
\t\n][^\n]*)?/ . del /\n/ "\n" ]
Some time ago, we thought that it would be good to map comments in "#comment" nodes to avoid confusion with possible "comment" nodes, especially when the lens allows a wide range of labels. Util.comment would fit your need here, and the new version of it would provide a "#comment" node.
Thanks, will try that.
+ let empty = [ label "empty" . del /[ \t]*\n/ "" ]
Util.empty provides a definition of empty lines, too. Do you really need to map them with a label? I thought about doing that some time ago, but mapping them in "#empty" for the same reason that we map comments in "#comment". The useful application to this is to allow users to control empty lines in conffile from the augeas API.
Will switch to Util.empty too.
+ + let lns = ( entry | comment | empty ) +
Is an empty file not valid?
It is - typo left over from testing. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Tue, 2008-08-26 at 22:01 +0100, Daniel P. Berrange wrote:
While I can see why it is useful to gather these entries logically, it's not very optimised for the parser. Eventually, all this will be compiled into a huge regexp, so it's more efficient to regroup entries by type and feed the functions with regexps instead of strings, like
Is performance really that much of a problem ? The libvirt config file only has 20 or so different settings, and while we'll add more I can't see it getting very much larger. Changing these settings is also not something that would be done on a frequent basis / performance critical path. I find it more readable to group them by functional area unless there is a serious real world performance issue.
We've seen some cases where it made a _dramatic_ difference. But if it's fast enough now, I wouldn't worry about it - just keep that in mind when either running augparse or augtool on it becomes unbearably slow. David
On Tue, Aug 26, 2008 at 09:46:22PM +0000, David Lutterkort wrote:
On Tue, 2008-08-26 at 22:01 +0100, Daniel P. Berrange wrote:
While I can see why it is useful to gather these entries logically, it's not very optimised for the parser. Eventually, all this will be compiled into a huge regexp, so it's more efficient to regroup entries by type and feed the functions with regexps instead of strings, like
Is performance really that much of a problem ? The libvirt config file only has 20 or so different settings, and while we'll add more I can't see it getting very much larger. Changing these settings is also not something that would be done on a frequent basis / performance critical path. I find it more readable to group them by functional area unless there is a serious real world performance issue.
We've seen some cases where it made a _dramatic_ difference. But if it's fast enough now, I wouldn't worry about it - just keep that in mind when either running augparse or augtool on it becomes unbearably slow.
Out of interest, do you ever hit the 16 bit limit in the size of compiled regexps? The "bytecode" used to represent compiled GNU regexps has (or perhaps had) a 16 bit limit in the jump offsets, which I hit a few years back. Had to switch to using a flex-compiled pattern matcher in the end. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones Read my OCaml programming blog: http://camltastic.blogspot.com/ Fedora now supports 64 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
Dan pointed out that you have your own DFA implementation. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Wed, 2008-08-27 at 12:15 +0100, Richard W.M. Jones wrote:
Dan pointed out that you have your own DFA implementation.
Actually, libf does everything but matching - it does all the regexp calculations needed for the typechecker; for matching, I use GNU regex. David
On Wed, 2008-08-27 at 12:03 +0100, Richard W.M. Jones wrote:
Out of interest, do you ever hit the 16 bit limit in the size of compiled regexps? The "bytecode" used to represent compiled GNU regexps has (or perhaps had) a 16 bit limit in the jump offsets, which I hit a few years back. Had to switch to using a flex-compiled pattern matcher in the end.
No, I haven't run into that. Was that with the 'old' GNU regex implementation (before ~ 2002) or with the current one ? David
On Wed, Aug 27, 2008 at 09:20:18PM +0000, David Lutterkort wrote:
On Wed, 2008-08-27 at 12:03 +0100, Richard W.M. Jones wrote:
Out of interest, do you ever hit the 16 bit limit in the size of compiled regexps? The "bytecode" used to represent compiled GNU regexps has (or perhaps had) a 16 bit limit in the jump offsets, which I hit a few years back. Had to switch to using a flex-compiled pattern matcher in the end.
No, I haven't run into that. Was that with the 'old' GNU regex implementation (before ~ 2002) or with the current one ?
Yes, looking at glibc it seems the 2^16 limit has been removed. There is still an error for it: regex.h: REG_ESIZE, /* Compiled pattern bigger than 2^16 bytes. */ but nothing in the code appears to generate this error. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Tue, 2008-08-26 at 22:37 +0200, Raphaël Pinson wrote:
Hi Daniel,
On Tue, Aug 26, 2008 at 10:05 PM, Daniel P. Berrange <berrange@redhat.com> wrote: THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
Very nice idea. This might have to be thought about for the future since so far we're only adding lenses to the Augeas repository.
Long term, it was always my plan to have upstream ship their lenses - it gets around a whole host of worries you need to have otherwise that the lenses that come with Augeas really do match every version of upstream's config file format.
=================================================================== RCS file: qemud/libvirtd.aug diff -N qemud/libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,64 @@ +(* /etc/libvirt/libvirtd.conf *) + +module Libvirtd = + autoload xfm + + let eol = del /[ \t]*\n/ "\n"
This is also Util.eol
In general, I would agree that you should use definitions from Util etc. But since we are not really that good yet at defining which of the modules that come with Augeas are fixed (let alone having tests for all that), and since these are all relatively small things, it's safer to just leave the Libvirtd module standalone. David
On Tue, 2008-08-26 at 21:05 +0100, Daniel P. Berrange wrote:
Now instead of telling people
'edit /etc/libvirt/libvirtd.conf and change listen_tls to 1, and auth_tls to sasl'
we can say run
# augtool <<EOF set /files/etc/libvirt/libvirtd.conf/listen_tls 1 set /files/etc/libvirt/libvirtd.conf/auth_tls sasl save EOF
Very nice.
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
libvirt.spec.in | 2 qemud/Makefile.am | 8 qemud/libvirtd.aug | 64 ++++++ qemud/test_libvirtd.aug | 484 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 558 insertions(+)
Daniel
Index: qemud/Makefile.am =================================================================== RCS file: /data/cvs/libvirt/qemud/Makefile.am,v retrieving revision 1.51 diff -u -p -r1.51 Makefile.am --- qemud/Makefile.am 20 Aug 2008 20:48:35 -0000 1.51 +++ qemud/Makefile.am 26 Aug 2008 20:03:48 -0000 @@ -24,6 +24,8 @@ EXTRA_DIST = \ libvirtd.policy \ libvirtd.sasl \ libvirtd.sysconf \ + libvirtd.aug \ + test_libvirtd.aug \ $(AVAHI_SOURCES) \ $(DAEMON_SOURCES)
@@ -56,6 +58,12 @@ sbin_PROGRAMS = libvirtd confdir = $(sysconfdir)/libvirt/ conf_DATA = libvirtd.conf
+augeasdir = $(datadir)/augeas/lenses +augeas_DATA = libvirtd.aug + +augeastestsdir = $(datadir)/augeas/lenses/tests +augeastests_DATA = test_libvirtd.aug + libvirtd_SOURCES = $(DAEMON_SOURCES)
You might also add a test that runs augparse -I . test_libvirtd.aug during 'make check'
#-D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 -D_POSIX_C_SOURCE=199506L Index: qemud/libvirtd.aug =================================================================== RCS file: qemud/libvirtd.aug diff -N qemud/libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,64 @@ +(* /etc/libvirt/libvirtd.conf *) + +module Libvirtd = + autoload xfm + + let eol = del /[ \t]*\n/ "\n" + let value_sep = del /[ \t]*=[ \t]*/ " = " + let prespace = del /[ \t]*/ "" + + let array_sep = del /,[ \t\n]*/ ", " + let array_start = del /\[[ \t\n]*/ "[ " + let array_end = del /\]/ " ]"
Augeas should throw an error here, but doesn't ;) The default value you give as the second argument of del really should match the first (regexp) argument. Also, Augeas automatically promotes strings to regexps as needed, so you can say let array_end = del "]" "]"
+ let str_val = del /\"/ "\"" . store /[^\"]*/ . del /\"/ "\"" + let bool_val = store /0|1/ + let str_array_element = [ str_val ] . del /[ \t\n]*/ "" + let str_array_val = array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end
You should really have some sort of label on each array element/str val (either the same for all of them using 'label' or just consecutive numbers using 'seq') - without that, you won't be able to get to individual entries through the public API.
+ let comment = [ label "comment" . del /#[ \t]*/ "# " . store /([^ \t\n][^\n]*)?/ . del /\n/ "\n" ]
We've been trying to label all comments as '#comment'; it's not an issue for libvirt, but with some other file formats, using 'comment' leads to conflicts with actual entries. Would be good to stick to that convention.
+ let empty = [ label "empty" . del /[ \t]*\n/ "" ]
Do you really care that empty entries show up in the tree ? If you don't want to see them, you can remove the 'label' from the above, and possibly also the '[ .. ]'. And the missing check for 'del' strikes again - that needs to be 'del /[ \t]*\n/ "\n"' - I really need to fix that.
Index: qemud/test_libvirtd.aug =================================================================== RCS file: qemud/test_libvirtd.aug diff -N qemud/test_libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/test_libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,484 @@ +module Test_libvirtd = + let conf1 = "# Master libvirt daemon configuration file
I've been thinking that we need to have some function that reads a file and returns a string so we don't have to clutter tests with these long strings. But for now, that's what it is :(
+ { "tls_allowed_dn_list" + { = "DN1"} + { = "DN2"} + }
This happens because str_array_element produces tree nodes without labels. Users won't have a way to change e.g. the 'DN1' value to 'myDN' since you can't address a node without a label through the public API. All in all, very nice, and I am really glad that upstream is shipping a lens :) David
On Tue, Aug 26, 2008 at 09:40:41PM +0000, David Lutterkort wrote:
On Tue, 2008-08-26 at 21:05 +0100, Daniel P. Berrange wrote:
diff -u -p -r1.51 Makefile.am --- qemud/Makefile.am 20 Aug 2008 20:48:35 -0000 1.51 +++ qemud/Makefile.am 26 Aug 2008 20:03:48 -0000 @@ -24,6 +24,8 @@ EXTRA_DIST = \ libvirtd.policy \ libvirtd.sasl \ libvirtd.sysconf \ + libvirtd.aug \ + test_libvirtd.aug \ $(AVAHI_SOURCES) \ $(DAEMON_SOURCES)
@@ -56,6 +58,12 @@ sbin_PROGRAMS = libvirtd confdir = $(sysconfdir)/libvirt/ conf_DATA = libvirtd.conf
+augeasdir = $(datadir)/augeas/lenses +augeas_DATA = libvirtd.aug + +augeastestsdir = $(datadir)/augeas/lenses/tests +augeastests_DATA = test_libvirtd.aug + libvirtd_SOURCES = $(DAEMON_SOURCES)
You might also add a test that runs augparse -I . test_libvirtd.aug during 'make check'
Yep, I've added a check-local rule for that in this new patch.
+ let array_sep = del /,[ \t\n]*/ ", " + let array_start = del /\[[ \t\n]*/ "[ " + let array_end = del /\]/ " ]"
Augeas should throw an error here, but doesn't ;) The default value you give as the second argument of del really should match the first (regexp) argument.
Ahh, nice catch.
+ let str_val = del /\"/ "\"" . store /[^\"]*/ . del /\"/ "\"" + let bool_val = store /0|1/ + let str_array_element = [ str_val ] . del /[ \t\n]*/ "" + let str_array_val = array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end
You should really have some sort of label on each array element/str val (either the same for all of them using 'label' or just consecutive numbers using 'seq') - without that, you won't be able to get to individual entries through the public API.
I've changed that to which looks to give gives each element a unique ID let str_array_element = [ seq "el" . str_val ] . del /[ \t\n]*/ "" let str_array_val = counter "el" . array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end
+ let comment = [ label "comment" . del /#[ \t]*/ "# " . store /([^ \t\n][^\n]*)?/ . del /\n/ "\n" ]
We've been trying to label all comments as '#comment'; it's not an issue for libvirt, but with some other file formats, using 'comment' leads to conflicts with actual entries. Would be good to stick to that convention.
Ok, I was just copying one of the existing lens on my system. I've changed it to #comment now
+ let empty = [ label "empty" . del /[ \t]*\n/ "" ]
Do you really care that empty entries show up in the tree ? If you don't want to see them, you can remove the 'label' from the above, and possibly also the '[ .. ]'.
Ok, this gets fun. I don't particularly want the label or tree node. I can remove the label OK, but that gives anonymous tree nods. If I remove the [...], then I end up with test -x /usr/bin/augparse && /usr/bin/augparse -I . test_libvirtd.aug ./libvirtd.aug:63.3-.43:Failed to compile lns ./libvirtd.aug:63.13-.43:exception: ambiguous tree iteration 'ca_file/' can be split into '|=|ca_file/' and 'ca_file/|=|' Iterated lens: ./libvirtd.aug:63.15-.39 test_libvirtd.aug:253.8-.20:Could not load module Libvirtd for Libvirtd.lns test_libvirtd.aug:253.8-.20:Undefined variable Libvirtd.lns test_libvirtd.aug: error: Loading failed And I'm utterly stuck on what this means / how to fix it. I didn't expect removing the square brackets to change the DFA.
diff -N qemud/test_libvirtd.aug --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ qemud/test_libvirtd.aug 26 Aug 2008 20:03:48 -0000 @@ -0,0 +1,484 @@ +module Test_libvirtd = + let conf1 = "# Master libvirt daemon configuration file
I've been thinking that we need to have some function that reads a file and returns a string so we don't have to clutter tests with these long strings. But for now, that's what it is :(
Yes, that would be very nice - and/or supporting use of <<HERE documents for strings, so I don't have to escape " throughout the embedded config file
+ { "tls_allowed_dn_list" + { = "DN1"} + { = "DN2"} + }
This happens because str_array_element produces tree nodes without labels. Users won't have a way to change e.g. the 'DN1' value to 'myDN' since you can't address a node without a label through the public API.
This now has integer keys. One other useful improvement would be for the test suite to only print out sections of the tree which differ. In debugging this it printed out a 400 line 'Expected' tree structure, and then a 400 line 'Actual' tree structure and wanted me to play "Where's Waldo" to find the typo. It would be good to trim the leading and trailing nodes which are the same
All in all, very nice, and I am really glad that upstream is shipping a lens :)
Just discovered one minor issue. The Fedora RPM guidelines require that an RPM either own a directory, or require an RPM that owns it, and two RPMs aren't allowed to own the same directory. I don't want/need to depend on Augeas directly, but equally I need someone to own the /usr/share/augeas/lens directory. Wonder if that directory should be put into the 'filesystem' RPM ? Daniel diff -r 387c16703a90 configure.in --- a/configure.in Wed Aug 27 11:32:54 2008 +0100 +++ b/configure.in Wed Aug 27 11:35:41 2008 +0100 @@ -100,7 +100,7 @@ AC_PATH_PROG([XMLLINT], [xmllint], [/usr/bin/xmllint]) AC_PATH_PROG([XMLCATALOG], [xmlcatalog], [/usr/bin/xmlcatalog]) AC_PATH_PROG([XSLTPROC], [xsltproc], [/usr/bin/xsltproc]) - +AC_PATH_PROG([AUGPARSE], [augparse], [/usr/bin/augparse]) AC_PROG_MKDIR_P dnl External programs that we can use if they are available. diff -r 387c16703a90 libvirt.spec.in --- a/libvirt.spec.in Wed Aug 27 11:32:54 2008 +0100 +++ b/libvirt.spec.in Wed Aug 27 11:35:41 2008 +0100 @@ -252,6 +252,8 @@ %dir %{_localstatedir}/lib/libvirt/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/images/ %dir %attr(0700, root, root) %{_localstatedir}/lib/libvirt/boot/ +%{_datadir}/augeas/lenses/libvirtd.aug +%{_datadir}/augeas/lenses/tests/test_libvirtd.aug %if %{with_polkit} %{_datadir}/PolicyKit/policy/org.libvirt.unix.policy %endif diff -r 387c16703a90 qemud/Makefile.am --- a/qemud/Makefile.am Wed Aug 27 11:32:54 2008 +0100 +++ b/qemud/Makefile.am Wed Aug 27 11:35:41 2008 +0100 @@ -24,6 +24,8 @@ libvirtd.policy \ libvirtd.sasl \ libvirtd.sysconf \ + libvirtd.aug \ + test_libvirtd.aug \ $(AVAHI_SOURCES) \ $(DAEMON_SOURCES) @@ -55,6 +57,12 @@ confdir = $(sysconfdir)/libvirt/ conf_DATA = libvirtd.conf + +augeasdir = $(datadir)/augeas/lenses +augeas_DATA = libvirtd.aug + +augeastestsdir = $(datadir)/augeas/lenses/tests +augeastests_DATA = test_libvirtd.aug libvirtd_SOURCES = $(DAEMON_SOURCES) @@ -172,6 +180,9 @@ chmod a+x $@-t mv $@-t $@ +check-local: + test -x $(AUGPARSE) && $(AUGPARSE) -I $(srcdir) test_libvirtd.aug + else install-init: diff -r 387c16703a90 qemud/libvirtd.aug --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/qemud/libvirtd.aug Wed Aug 27 11:35:41 2008 +0100 @@ -0,0 +1,69 @@ +(* /etc/libvirt/libvirtd.conf *) + +module Libvirtd = + autoload xfm + + let eol = del /[ \t]*\n/ "\n" + let value_sep = del /[ \t]*=[ \t]*/ " = " + let indent = del /[ \t]*/ "" + + let array_sep = del /,[ \t\n]*/ ", " + let array_start = del /\[[ \t\n]*/ "[ " + let array_end = del /\]/ "]" + + let str_val = del /\"/ "\"" . store /[^\"]*/ . del /\"/ "\"" + let bool_val = store /0|1/ + let str_array_element = [ seq "el" . str_val ] . del /[ \t\n]*/ "" + let str_array_val = counter "el" . array_start . ( str_array_element . ( array_sep . str_array_element ) * ) ? . array_end + + let str_entry (kw:string) = [ key kw . value_sep . str_val ] + let bool_entry (kw:string) = [ key kw . value_sep . bool_val ] + let str_array_entry (kw:string) = [ key kw . value_sep . str_array_val ] + + + (* Config entry grouped by function - same order as example config *) + let network_entry = bool_entry "listen_tls" + | bool_entry "listen_tcp" + | str_entry "tls_port" + | str_entry "tcp_port" + | str_entry "listen_addr" + | bool_entry "mdns_adv" + | str_entry "mdns_name" + + let sock_acl_entry = str_entry "unix_sock_group" + | str_entry "unix_sock_ro_perms" + | str_entry "unix_sock_rw_perms" + + let authentication_entry = str_entry "auth_unix_ro" + | str_entry "auth_unix_rw" + | str_entry "auth_tcp" + | str_entry "auth_tls" + + let certificate_entry = str_entry "key_file" + | str_entry "cert_file" + | str_entry "ca_file" + | str_entry "crl_file" + + let authorization_entry = bool_entry "tls_no_verify_certificate" + | str_array_entry "tls_allowed_dn_list" + | str_array_entry "sasl_allowed_username_list" + + + (* Each enty in the config is one of the following three ... *) + let entry = network_entry + | sock_acl_entry + | authentication_entry + | certificate_entry + | authorization_entry + let comment = [ label "#comment" . del /#[ \t]*/ "# " . store /([^ \t\n][^\n]*)?/ . del /\n/ "\n" ] + let empty = [ label "#empty" . eol ] + + let record = indent . entry . eol + + let lns = ( record | comment | empty ) * + + let filter = incl "/etc/libvirt/libvirtd.conf" + . Util.stdexcl + + let xfm = transform lns filter + diff -r 387c16703a90 qemud/test_libvirtd.aug --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/qemud/test_libvirtd.aug Wed Aug 27 11:35:41 2008 +0100 @@ -0,0 +1,484 @@ +module Test_libvirtd = + let conf1 = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 +" + + let conf = "# Master libvirt daemon configuration file +# +# For further information consult http://libvirt.org/format.html + + +################################################################# +# +# Network connectivity controls +# + +# Flag listening for secure TLS connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# It is necessary to setup a CA and issue server certificates before +# using this capability. +# +# This is enabled by default, uncomment this to disable it +listen_tls = 0 + +# Listen for unencrypted TCP connections on the public TCP/IP port. +# NB, must pass the --listen flag to the libvirtd process for this to +# have any effect. +# +# Using the TCP socket requires SASL authentication by default. Only +# SASL mechanisms which support data encryption are allowed. This is +# DIGEST_MD5 and GSSAPI (Kerberos5) +# +# This is disabled by default, uncomment this to enable it. +listen_tcp = 1 + + + +# Override the port for accepting secure TLS connections +# This can be a port number, or service name +# +tls_port = \"16514\" + +# Override the port for accepting insecure TCP connections +# This can be a port number, or service name +# +tcp_port = \"16509\" + + +# Override the default configuration which binds to all network +# interfaces. This can be a numeric IPv4/6 address, or hostname +# +listen_addr = \"192.168.0.1\" + + +# Flag toggling mDNS advertizement of the libvirt service. +# +# Alternatively can disable for all services on a host by +# stopping the Avahi daemon +# +# This is enabled by default, uncomment this to disable it +mdns_adv = 0 + +# Override the default mDNS advertizement name. This must be +# unique on the immediate broadcast network. +# +# The default is \"Virtualization Host HOSTNAME\", where HOSTNAME +# is subsituted for the short hostname of the machine (without domain) +# +mdns_name = \"Virtualization Host Joe Demo\" + + +################################################################# +# +# UNIX socket access controls +# + +# Set the UNIX domain socket group ownership. This can be used to +# allow a 'trusted' set of users access to management capabilities +# without becoming root. +# +# This is restricted to 'root' by default. +unix_sock_group = \"libvirt\" + +# Set the UNIX socket permissions for the R/O socket. This is used +# for monitoring VM status only +# +# Default allows any user. If setting group ownership may want to +# restrict this to: +unix_sock_ro_perms = \"0777\" + +# Set the UNIX socket permissions for the R/W socket. This is used +# for full management of VMs +# +# Default allows only root. If PolicyKit is enabled on the socket, +# the default will change to allow everyone (eg, 0777) +# +# If not using PolicyKit and setting group ownership for access +# control then you may want to relax this to: +unix_sock_rw_perms = \"0770\" + + + +################################################################# +# +# Authentication. +# +# - none: do not perform auth checks. If you can connect to the +# socket you are allowed. This is suitable if there are +# restrictions on connecting to the socket (eg, UNIX +# socket permissions), or if there is a lower layer in +# the network providing auth (eg, TLS/x509 certificates) +# +# - sasl: use SASL infrastructure. The actual auth scheme is then +# controlled from /etc/sasl2/libvirt.conf. For the TCP +# socket only GSSAPI & DIGEST-MD5 mechanisms will be used. +# For non-TCP or TLS sockets, any scheme is allowed. +# +# - polkit: use PolicyKit to authenticate. This is only suitable +# for use on the UNIX sockets. The default policy will +# require a user to supply their own password to gain +# full read/write access (aka sudo like), while anyone +# is allowed read/only access. +# +# Set an authentication scheme for UNIX read-only sockets +# By default socket permissions allow anyone to connect +# +# To restrict monitoring of domains you may wish to enable +# an authentication mechanism here +auth_unix_ro = \"none\" + +# Set an authentication scheme for UNIX read-write sockets +# By default socket permissions only allow root. If PolicyKit +# support was compiled into libvirt, the default will be to +# use 'polkit' auth. +# +# If the unix_sock_rw_perms are changed you may wish to enable +# an authentication mechanism here +auth_unix_rw = \"none\" + +# Change the authentication scheme for TCP sockets. +# +# If you don't enable SASL, then all TCP traffic is cleartext. +# Don't do this outside of a dev/test scenario. For real world +# use, always enable SASL and use the GSSAPI or DIGEST-MD5 +# mechanism in /etc/sasl2/libvirt.conf +auth_tcp = \"sasl\" + +# Change the authentication scheme for TLS sockets. +# +# TLS sockets already have encryption provided by the TLS +# layer, and limited authentication is done by certificates +# +# It is possible to make use of any SASL authentication +# mechanism as well, by using 'sasl' for this option +auth_tls = \"none\" + + + +################################################################# +# +# TLS x509 certificate configuration +# + + +# Override the default server key file path +# +key_file = \"/etc/pki/libvirt/private/serverkey.pem\" + +# Override the default server certificate file path +# +cert_file = \"/etc/pki/libvirt/servercert.pem\" + +# Override the default CA certificate path +# +ca_file = \"/etc/pki/CA/cacert.pem\" + +# Specify a certificate revocation list. +# +# Defaults to not using a CRL, uncomment to enable it +crl_file = \"/etc/pki/CA/crl.pem\" + + + +################################################################# +# +# Authorization controls +# + + +# Flag to disable verification of client certificates +# +# Client certificate verification is the primary authentication mechanism. +# Any client which does not present a certificate signed by the CA +# will be rejected. +# +# Default is to always verify. Uncommenting this will disable +# verification - make sure an IP whitelist is set +tls_no_verify_certificate = 1 + + +# A whitelist of allowed x509 Distinguished Names +# This list may contain wildcards such as +# +# \"C=GB,ST=London,L=London,O=Red Hat,CN=*\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no DN's are checked + tls_allowed_dn_list = [\"DN1\", \"DN2\"] + + +# A whitelist of allowed SASL usernames. The format for usernames +# depends on the SASL authentication mechanism. Kerberos usernames +# look like username@REALM +# +# This list may contain wildcards such as +# +# \"*@EXAMPLE.COM\" +# +# See the POSIX fnmatch function for the format of the wildcards. +# +# NB If this is an empty list, no client can connect, so comment out +# entirely rather than using empty list to disable these checks +# +# By default, no Username's are checked +sasl_allowed_username_list = [ + \"joe@EXAMPLE.COM\", + \"fred@EXAMPLE.COM\" +] +" + + test Libvirtd.lns get conf = + { "#comment" = "Master libvirt daemon configuration file" } + { "#comment" = "" } + { "#comment" = "For further information consult http://libvirt.org/format.html" } + { "#empty" } + { "#empty" } + { "#comment" = "################################################################" } + { "#comment" = "" } + { "#comment" = "Network connectivity controls" } + { "#comment" = "" } + { "#empty" } + { "#comment" = "Flag listening for secure TLS connections on the public TCP/IP port." } + { "#comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "#comment" = "have any effect." } + { "#comment" = "" } + { "#comment" = "It is necessary to setup a CA and issue server certificates before" } + { "#comment" = "using this capability." } + { "#comment" = "" } + { "#comment" = "This is enabled by default, uncomment this to disable it" } + { "listen_tls" = "0" } + { "#empty" } + { "#comment" = "Listen for unencrypted TCP connections on the public TCP/IP port." } + { "#comment" = "NB, must pass the --listen flag to the libvirtd process for this to" } + { "#comment" = "have any effect." } + { "#comment" = "" } + { "#comment" = "Using the TCP socket requires SASL authentication by default. Only" } + { "#comment" = "SASL mechanisms which support data encryption are allowed. This is" } + { "#comment" = "DIGEST_MD5 and GSSAPI (Kerberos5)" } + { "#comment" = "" } + { "#comment" = "This is disabled by default, uncomment this to enable it." } + { "listen_tcp" = "1" } + { "#empty" } + { "#empty" } + { "#empty" } + { "#comment" = "Override the port for accepting secure TLS connections" } + { "#comment" = "This can be a port number, or service name" } + { "#comment" = "" } + { "tls_port" = "16514" } + { "#empty" } + { "#comment" = "Override the port for accepting insecure TCP connections" } + { "#comment" = "This can be a port number, or service name" } + { "#comment" = "" } + { "tcp_port" = "16509" } + { "#empty" } + { "#empty" } + { "#comment" = "Override the default configuration which binds to all network" } + { "#comment" = "interfaces. This can be a numeric IPv4/6 address, or hostname" } + { "#comment" = "" } + { "listen_addr" = "192.168.0.1" } + { "#empty" } + { "#empty" } + { "#comment" = "Flag toggling mDNS advertizement of the libvirt service." } + { "#comment" = "" } + { "#comment" = "Alternatively can disable for all services on a host by" } + { "#comment" = "stopping the Avahi daemon" } + { "#comment" = "" } + { "#comment" = "This is enabled by default, uncomment this to disable it" } + { "mdns_adv" = "0" } + { "#empty" } + { "#comment" = "Override the default mDNS advertizement name. This must be" } + { "#comment" = "unique on the immediate broadcast network." } + { "#comment" = "" } + { "#comment" = "The default is \"Virtualization Host HOSTNAME\", where HOSTNAME" } + { "#comment" = "is subsituted for the short hostname of the machine (without domain)" } + { "#comment" = "" } + { "mdns_name" = "Virtualization Host Joe Demo" } + { "#empty" } + { "#empty" } + { "#comment" = "################################################################" } + { "#comment" = "" } + { "#comment" = "UNIX socket access controls" } + { "#comment" = "" } + { "#empty" } + { "#comment" = "Set the UNIX domain socket group ownership. This can be used to" } + { "#comment" = "allow a 'trusted' set of users access to management capabilities" } + { "#comment" = "without becoming root." } + { "#comment" = "" } + { "#comment" = "This is restricted to 'root' by default." } + { "unix_sock_group" = "libvirt" } + { "#empty" } + { "#comment" = "Set the UNIX socket permissions for the R/O socket. This is used" } + { "#comment" = "for monitoring VM status only" } + { "#comment" = "" } + { "#comment" = "Default allows any user. If setting group ownership may want to" } + { "#comment" = "restrict this to:" } + { "unix_sock_ro_perms" = "0777" } + { "#empty" } + { "#comment" = "Set the UNIX socket permissions for the R/W socket. This is used" } + { "#comment" = "for full management of VMs" } + { "#comment" = "" } + { "#comment" = "Default allows only root. If PolicyKit is enabled on the socket," } + { "#comment" = "the default will change to allow everyone (eg, 0777)" } + { "#comment" = "" } + { "#comment" = "If not using PolicyKit and setting group ownership for access" } + { "#comment" = "control then you may want to relax this to:" } + { "unix_sock_rw_perms" = "0770" } + { "#empty" } + { "#empty" } + { "#empty" } + { "#comment" = "################################################################" } + { "#comment" = "" } + { "#comment" = "Authentication." } + { "#comment" = "" } + { "#comment" = "- none: do not perform auth checks. If you can connect to the" } + { "#comment" = "socket you are allowed. This is suitable if there are" } + { "#comment" = "restrictions on connecting to the socket (eg, UNIX" } + { "#comment" = "socket permissions), or if there is a lower layer in" } + { "#comment" = "the network providing auth (eg, TLS/x509 certificates)" } + { "#comment" = "" } + { "#comment" = "- sasl: use SASL infrastructure. The actual auth scheme is then" } + { "#comment" = "controlled from /etc/sasl2/libvirt.conf. For the TCP" } + { "#comment" = "socket only GSSAPI & DIGEST-MD5 mechanisms will be used." } + { "#comment" = "For non-TCP or TLS sockets, any scheme is allowed." } + { "#comment" = "" } + { "#comment" = "- polkit: use PolicyKit to authenticate. This is only suitable" } + { "#comment" = "for use on the UNIX sockets. The default policy will" } + { "#comment" = "require a user to supply their own password to gain" } + { "#comment" = "full read/write access (aka sudo like), while anyone" } + { "#comment" = "is allowed read/only access." } + { "#comment" = "" } + { "#comment" = "Set an authentication scheme for UNIX read-only sockets" } + { "#comment" = "By default socket permissions allow anyone to connect" } + { "#comment" = "" } + { "#comment" = "To restrict monitoring of domains you may wish to enable" } + { "#comment" = "an authentication mechanism here" } + { "auth_unix_ro" = "none" } + { "#empty" } + { "#comment" = "Set an authentication scheme for UNIX read-write sockets" } + { "#comment" = "By default socket permissions only allow root. If PolicyKit" } + { "#comment" = "support was compiled into libvirt, the default will be to" } + { "#comment" = "use 'polkit' auth." } + { "#comment" = "" } + { "#comment" = "If the unix_sock_rw_perms are changed you may wish to enable" } + { "#comment" = "an authentication mechanism here" } + { "auth_unix_rw" = "none" } + { "#empty" } + { "#comment" = "Change the authentication scheme for TCP sockets." } + { "#comment" = "" } + { "#comment" = "If you don't enable SASL, then all TCP traffic is cleartext." } + { "#comment" = "Don't do this outside of a dev/test scenario. For real world" } + { "#comment" = "use, always enable SASL and use the GSSAPI or DIGEST-MD5" } + { "#comment" = "mechanism in /etc/sasl2/libvirt.conf" } + { "auth_tcp" = "sasl" } + { "#empty" } + { "#comment" = "Change the authentication scheme for TLS sockets." } + { "#comment" = "" } + { "#comment" = "TLS sockets already have encryption provided by the TLS" } + { "#comment" = "layer, and limited authentication is done by certificates" } + { "#comment" = "" } + { "#comment" = "It is possible to make use of any SASL authentication" } + { "#comment" = "mechanism as well, by using 'sasl' for this option" } + { "auth_tls" = "none" } + { "#empty" } + { "#empty" } + { "#empty" } + { "#comment" = "################################################################" } + { "#comment" = "" } + { "#comment" = "TLS x509 certificate configuration" } + { "#comment" = "" } + { "#empty" } + { "#empty" } + { "#comment" = "Override the default server key file path" } + { "#comment" = "" } + { "key_file" = "/etc/pki/libvirt/private/serverkey.pem" } + { "#empty" } + { "#comment" = "Override the default server certificate file path" } + { "#comment" = "" } + { "cert_file" = "/etc/pki/libvirt/servercert.pem" } + { "#empty" } + { "#comment" = "Override the default CA certificate path" } + { "#comment" = "" } + { "ca_file" = "/etc/pki/CA/cacert.pem" } + { "#empty" } + { "#comment" = "Specify a certificate revocation list." } + { "#comment" = "" } + { "#comment" = "Defaults to not using a CRL, uncomment to enable it" } + { "crl_file" = "/etc/pki/CA/crl.pem" } + { "#empty" } + { "#empty" } + { "#empty" } + { "#comment" = "################################################################" } + { "#comment" = "" } + { "#comment" = "Authorization controls" } + { "#comment" = "" } + { "#empty" } + { "#empty" } + { "#comment" = "Flag to disable verification of client certificates" } + { "#comment" = "" } + { "#comment" = "Client certificate verification is the primary authentication mechanism." } + { "#comment" = "Any client which does not present a certificate signed by the CA" } + { "#comment" = "will be rejected." } + { "#comment" = "" } + { "#comment" = "Default is to always verify. Uncommenting this will disable" } + { "#comment" = "verification - make sure an IP whitelist is set" } + { "tls_no_verify_certificate" = "1" } + { "#empty" } + { "#empty" } + { "#comment" = "A whitelist of allowed x509 Distinguished Names" } + { "#comment" = "This list may contain wildcards such as" } + { "#comment" = "" } + { "#comment" = "\"C=GB,ST=London,L=London,O=Red Hat,CN=*\"" } + { "#comment" = "" } + { "#comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "#comment" = "" } + { "#comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "#comment" = "entirely rather than using empty list to disable these checks" } + { "#comment" = "" } + { "#comment" = "By default, no DN's are checked" } + { "tls_allowed_dn_list" + { "1" = "DN1"} + { "2" = "DN2"} + } + { "#empty" } + { "#empty" } + { "#comment" = "A whitelist of allowed SASL usernames. The format for usernames" } + { "#comment" = "depends on the SASL authentication mechanism. Kerberos usernames" } + { "#comment" = "look like username@REALM" } + { "#comment" = "" } + { "#comment" = "This list may contain wildcards such as" } + { "#comment" = "" } + { "#comment" = "\"*@EXAMPLE.COM\"" } + { "#comment" = "" } + { "#comment" = "See the POSIX fnmatch function for the format of the wildcards." } + { "#comment" = "" } + { "#comment" = "NB If this is an empty list, no client can connect, so comment out" } + { "#comment" = "entirely rather than using empty list to disable these checks" } + { "#comment" = "" } + { "#comment" = "By default, no Username's are checked" } + { "sasl_allowed_username_list" + { "1" = "joe@EXAMPLE.COM" } + { "2" = "fred@EXAMPLE.COM" } + } -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, 2008-08-27 at 12:31 +0100, Daniel P. Berrange wrote:
On Tue, Aug 26, 2008 at 09:40:41PM +0000, David Lutterkort wrote:
On Tue, 2008-08-26 at 21:05 +0100, Daniel P. Berrange wrote:
+ let empty = [ label "empty" . del /[ \t]*\n/ "" ]
Do you really care that empty entries show up in the tree ? If you don't want to see them, you can remove the 'label' from the above, and possibly also the '[ .. ]'.
Ok, this gets fun. I don't particularly want the label or tree node. I can remove the label OK, but that gives anonymous tree nods.
Thinking about this some more, anonymous (or labelled) nodes are probably your best bet here, because:
If I remove the [...], then I end up with
test -x /usr/bin/augparse && /usr/bin/augparse -I . test_libvirtd.aug ./libvirtd.aug:63.3-.43:Failed to compile lns ./libvirtd.aug:63.13-.43:exception: ambiguous tree iteration 'ca_file/' can be split into '|=|ca_file/'
and 'ca_file/|=|'
Iterated lens: ./libvirtd.aug:63.15-.39
test_libvirtd.aug:253.8-.20:Could not load module Libvirtd for Libvirtd.lns test_libvirtd.aug:253.8-.20:Undefined variable Libvirtd.lns test_libvirtd.aug: error: Loading failed
And I'm utterly stuck on what this means / how to fix it. I didn't expect removing the square brackets to change the DFA.
The error message is admittedly obtuse (though I am not sure how to improve it, suggestions welcome) What Augeas is telling you (or trying to, atleast) is that when it goes to write a tree back into the file (that's why it mentions 'tree iteration') it has a choice, and doesn't know what to do in the lens on line 63, columns 3 - 43, which is the toplevel 'lns' lens: let lns = ( record | comment | empty ) * The '|=|' mark in the error message shows where Augeas thinks it could split a tree with a single node 'ca_file': either into (no node, cafile/) or (cafile/, no node) .. what makes this error message doubly heinous is that there's no good notation for 'no node' - it's the empty string in the error message. The issue is that when it sees a tree with a single node 'ca_file' it doesn't know whether to process that as a single 'record' or as a 'record . empty' or 'empty . record', since all of these match the tree with a single 'ca_file' entry, simply because the 'empty' lens now leaves no traces in the tree that it had been used. When you define 'empty' as '[ eol ]', Augeas knows how to process that tree since every time 'empty' should be used, there must be a node in the tree with a NULL label, so a single tree node 'ca_file' can only be processed by using 'record', and not 'record . empty' or similar. As I said, the simplest way around this is to define 'empty' as '[ eol ]' - you could also try to change things so that empty lines are pulled into record or comment entries, but that's probably more trouble than it's worth.
I've been thinking that we need to have some function that reads a file and returns a string so we don't have to clutter tests with these long strings. But for now, that's what it is :(
Yes, that would be very nice - and/or supporting use of <<HERE documents for strings, so I don't have to escape " throughout the embedded config file
One more note on this: to get started, pulling in a whole config file and checking that that gets processed in a reasonable way is deifnitely the best thing to do. Over time, as you make changes to the file format and/or fix bugs in the lens, you're probably better off writing small tests that check for one particular thing (the test_shellvars.aug in Augeas hg is a good example for this) I put a ticket into Trac for this[1]
One other useful improvement would be for the test suite to only print out sections of the tree which differ. In debugging this it printed out a 400 line 'Expected' tree structure, and then a 400 line 'Actual' tree structure and wanted me to play "Where's Waldo" to find the typo.
It would be good to trim the leading and trailing nodes which are the same
Yeah, I've run into this before, too. Yet another ticket[2]
All in all, very nice, and I am really glad that upstream is shipping a lens :)
Just discovered one minor issue. The Fedora RPM guidelines require that an RPM either own a directory, or require an RPM that owns it, and two RPMs aren't allowed to own the same directory. I don't want/need to depend on Augeas directly, but equally I need someone to own the /usr/share/augeas/lens directory. Wonder if that directory should be put into the 'filesystem' RPM ?
Yeah, that's the only feasible solution. I'll need to figure out what the process for that is. David [1] http://fedorahosted.org/augeas/ticket/10 [2] http://fedorahosted.org/augeas/ticket/11
On Tue, Aug 26, 2008 at 09:05:37PM +0100, Daniel P. Berrange wrote:
Augeas is a awesome config file manipulation tool. libvirtd has a config file. libvirtd meet augeas; augeas meet libvirt.
Now instead of telling people
'edit /etc/libvirt/libvirtd.conf and change listen_tls to 1, and auth_tls to sasl'
we can say run
# augtool <<EOF set /files/etc/libvirt/libvirtd.conf/listen_tls 1 set /files/etc/libvirt/libvirtd.conf/auth_tls sasl save EOF
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
libvirt.spec.in | 2 qemud/Makefile.am | 8 qemud/libvirtd.aug | 64 ++++++ qemud/test_libvirtd.aug | 484 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 558 insertions(+)
+1, to the general concept, and the code apart from the lens itself which I didn't look at too closely. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones Read my OCaml programming blog: http://camltastic.blogspot.com/ Fedora now supports 64 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
On Wed, Aug 27, 2008 at 12:04:17PM +0100, Richard W.M. Jones wrote:
On Tue, Aug 26, 2008 at 09:05:37PM +0100, Daniel P. Berrange wrote:
Augeas is a awesome config file manipulation tool. libvirtd has a config file. libvirtd meet augeas; augeas meet libvirt.
Now instead of telling people
'edit /etc/libvirt/libvirtd.conf and change listen_tls to 1, and auth_tls to sasl'
we can say run
# augtool <<EOF set /files/etc/libvirt/libvirtd.conf/listen_tls 1 set /files/etc/libvirt/libvirtd.conf/auth_tls sasl save EOF
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
libvirt.spec.in | 2 qemud/Makefile.am | 8 qemud/libvirtd.aug | 64 ++++++ qemud/test_libvirtd.aug | 484 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 558 insertions(+)
+1, to the general concept, and the code apart from the lens itself which I didn't look at too closely.
Same thing, I like the idea and if David agrees with the .aug files and changes then I guess this should be commited, Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, Sep 03, 2008 at 10:40:45PM +0200, Daniel Veillard wrote:
On Wed, Aug 27, 2008 at 12:04:17PM +0100, Richard W.M. Jones wrote:
On Tue, Aug 26, 2008 at 09:05:37PM +0100, Daniel P. Berrange wrote:
Augeas is a awesome config file manipulation tool. libvirtd has a config file. libvirtd meet augeas; augeas meet libvirt.
Now instead of telling people
'edit /etc/libvirt/libvirtd.conf and change listen_tls to 1, and auth_tls to sasl'
we can say run
# augtool <<EOF set /files/etc/libvirt/libvirtd.conf/listen_tls 1 set /files/etc/libvirt/libvirtd.conf/auth_tls sasl save EOF
THis patch is intended to be committed to libvirt, so the config file rules are distributed alongside libvirt. I'm CC'ing augeas-devel for feedback on the lens itself.
libvirt.spec.in | 2 qemud/Makefile.am | 8 qemud/libvirtd.aug | 64 ++++++ qemud/test_libvirtd.aug | 484 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 558 insertions(+)
+1, to the general concept, and the code apart from the lens itself which I didn't look at too closely.
Same thing, I like the idea and if David agrees with the .aug files and changes then I guess this should be commited,
Thanks, I've committed this now. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (5)
-
Daniel P. Berrange -
Daniel Veillard -
David Lutterkort -
Raphaël Pinson -
Richard W.M. Jones