[RFCv3 04/25] docs: Add xmlgen.rst to explain how to use it

Signed-off-by: Shi Lei <shi_lei(a)massclouds.com&gt; --- docs/meson.build | 1 + docs/xmlgen.rst | 684 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 685 insertions(+) create mode 100644 docs/xmlgen.rst diff --git a/docs/meson.build b/docs/meson.build index f550629d..a8a58815 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -124,6 +124,7 @@ docs_rst_files = [ 'programming-languages', 'styleguide', 'submitting-patches', + 'xmlgen', ] # list of web targets to build for docs/web rule diff --git a/docs/xmlgen.rst b/docs/xmlgen.rst new file mode 100644 index 00000000..caea1f99 --- /dev/null +++ b/docs/xmlgen.rst @@ -0,0 +1,684 @@ +====== +xmlgen +====== + +In libvirt, developers usually have to implement and maintain parse/format functions. This tool xmlgen aims to generate most of these functions automatically and help relieve developers' burden. + +A Quick Start +============= + +Take virNetworkDNSDef for example, which is in 'src/conf/network_conf.h'. + +In the past, we have to manually implement virNetworkDNSDefParseXML and virNetworkDNSFormatBuf. +And now, we can just take several steps to have xmlgen generate these functions and apply them into libvirt project. + +Step1. Add directives on the struct's declaration. +-------------------------------------------------- + +Directives for xmlgen are used to help direct the generating process. As below: + + :: + + typedef struct _virNetworkDNSDef virNetworkDNSDef; + struct _virNetworkDNSDef { /* genparse, genformat */ + virTristateBool enable; /* xmlattr */ + virTristateBool forwardPlainNames; /* xmlattr */ + size_t nfwds; + virNetworkDNSForwarder *forwarders; /* xmlelem, array:nfwds */ + size_t ntxts; + virNetworkDNSTxtDef *txts; /* xmlelem, array */ + ... ... + }; + +On the line of struct's declaration, we set two directives **genparse** and **genformat**, which direct xmlgen to generate parse/format functions for this struct respectively. In this example, these functions include virNetworkDNSDefParseXML, virNetworkDNSDefFormatBuf and some auxilliary functions. Other directives are for members. They direct xmlgen to generate code blocks in the parse/format functions. Directive **xmlattr** indicates that the member matches an xml attribute, and **xmlelem** is for an xml element. Additional directive **array** indicates that the member matches an array of xml node. + +Step2. Preview generated functions. +----------------------------------- + +By the below command line: + + :: + + # ./scripts/xmlgen/go list + +Got a list of structs detected by xmlgen, including *virNetworkDNSDef*. +Then we execute the command line as below: + + :: + + # ./scripts/xmlgen/go show virNetworkDNSDef + +All the generated functions related to virNetworkDNSDef are displayed, then we ought to preview them before really use them into libvirt project. +There is a special part **[Tips]** except the generated functions and the declaration of hooks. + +[**Tips**] provides instructions about how to apply these generated functions into project and how to enable hooks. [**Tips**] will be used in step3. + +Also, we got the declaration of hooks. In step4, we will implement a hook according to it. + +Step3. Enable hooks and include generated functions. +---------------------------------------------------- + +According to [**Tips**] that we got in step2: + + :: + + [Tips] + + /* Put these lines at the bottom of "conf/network_conf.h" */ + /* Makesure "network_conf.h" to be appended into conf_xmlgen_input in src/conf/meson.build */ + + /* Define macro to enable hook or redefine check when necessary */ + /* #define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK */ + /* #define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK_SET_ARGS */ + /* #define ENABLE_VIR_NETWORK_DNSDEF_FORMAT_HOOK */ + + /* #define RESET_VIR_NETWORK_DNSDEF_CHECK */ + + /* Makesure below is the bottom line! */ + #include "network_conf.generated.h" + +Uncomment macros and enable hooks when necessary. In this example, we only need to define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK to enable the post-hook for parse-function. + +Include the header-file to apply the generated functions. +In this example, we just include "network_conf.generated.h" into "conf/network_conf.h" and modify "src/conf/meson.build" as instructed. + +Step4. Implement hooks when necessary. +-------------------------------------- + +In original implementation of virNetworkDNSDefParseXML, there's a piece of error-checking code as below: + + :: + + if (def->enable == VIR_TRISTATE_BOOL_NO && (nfwds || nhosts || nsrvs || ntxts)) { + virReportError(VIR_ERR_XML_ERROR, + _("Extra data in disabled network '%s'"), + networkName); + } + +Since this piece of code can't be generated, we need a hook to do the check. +In step2, we have gotten the declaration of virNetworkDNSDefParseHook, as below: + + :: + + #ifdef ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK + + int + virNetworkDNSDefParseHook(xmlNodePtr node, + virNetworkDNSDef *def, + const char *instname, + void *parent, + void *opaque, + const char *enableStr, + const char *forwardPlainNamesStr, + int nForwarderNodes, + int nTxtNodes, + int nSrvNodes, + int nHostNodes); + + #endif + +The arguments provide all necessary information for us to do some extra things. It's easy for us to transfer the error-checking code into this hook. +Then we put the implementation into a file, which should include the declaration of the hook. In this example, we use 'conf/network_conf.c'. + +Directives +========== + +Directives help direct xmlgen to generate parse/format functions. They are in the form of comment, so they are only valid for xmlgen and ignored by **C** compilers. Directives work on either a struct or a member, so they must appear in the same line of struct's or member's declaration. + +On the other hand, some directives are basic, which can be used on their own; and the others are addional, which must be accompanied with baisc ones. + +genparse +-------- + +(only for struct; basic) + +Generate parse-function for the struct. At the meanwhile, clear-function is also generated automatically, which is used for cleaning all the members when parse-function fails. + +E.g. set **genparse** on virNetworkDNSTxtDef, as below: + + :: + + typedef struct _virNetworkDNSTxtDef virNetworkDNSTxtDef; + struct _virNetworkDNSTxtDef { /* genparse */ + char *name; /* xmlattr, required */ + char *value; /* xmlattr */ + }; + +Then we'll get the implementation of virNetworkDNSTxtDefParseXML (parse-function) and virNetworkDNSTxtDefClear (clear-function). + +We can enable a pre-hook and a post-hook in the parse function when necessary. The usage of hooks will be explained in detail in the following chapter **Hooks**. + +Note: Specify **xmlattr** or **xmlelem** for at least one member, or we'll get an empty parse-function. + +genformat[:separate] +-------------------- + +(only for struct; basic) + +Generate format-function for the struct. At the meanwhile, check-function is also generated automatically, which is used to determine whether an instance of this type is empty. + +E.g. set **genformat** on virNetworkDHCPRangeDef, as below: + + :: + + typedef struct _virNetworkDHCPRangeDef virNetworkDHCPRangeDef; + struct _virNetworkDHCPRangeDef { /* genformat */ + struct { + virSocketAddr start; /* xmlattr, required */ + virSocketAddr end; /* xmlattr, required */ + } addr; + + virNetworkDHCPLeaseTimeDef *lease; /* xmlelem */ + }; + +Then we'll get the implementation of virNetworkDHCPRangeDefFormatBuf (format-function) and virNetworkDHCPRangeDefCheck (check-function). + +By default, the format-function outputs xml for all attributes and all elements. + +But in some special cases, we need two separate format-functions: one for formatting attributes and another for formatting elements. For that, we should specify **genformat:separate**. + +E.g., for virDomainGraphicsAuthDef + + :: + + struct _virDomainGraphicsAuthDef { /* genformat:separate */ + char *passwd; /* xmlattr */ + /* Whether there is an expiry time set */ + bool expires; /* specify:validTo */ + /* seconds since epoch */ + time_t validTo; /* xmlattr:passwdValidTo */ + /* action if connected */ + virDomainGraphicsAuthConnectedType connected; /* xmlattr */ + }; + +Then we'll get virDomainGraphicsAuthDefFormatAttr and virDomainGraphicsAuthDefFormatElem, which are for formatting attributes and elements respectively. In the meanwhile, virDomainGraphicsAuthDefCheckAttr and virDomainGraphicsAuthDefCheckElem are generated to check whether all attributes/elements are all empty. + +We can enable a pre-hook for format-function when necessary. Also, the check-function can be redefined if it's not suitable. The usage of hook and check-function will be explained in detail in the following chapter **Hooks**. + +Note: Specify **xmlattr** or **xmlelem** for at least one member, or we'll get an empty format-function. + +xmlattr[:[parentname/]thename] +------------------------------ + +(only for member; basic) + +Indicate that the member matches an xml attribute.There're 3 cases: + +1) By default, use the member's name as attribute's name to generate parse/format code block. + +2) Specify the attribute's name by **thename** when necessary. + +3) Use the form of **xmlattr:parentname/thename** to indicate that this member matches an attribute of an child-element. + +E.g.: + + :: + + struct _virNetworkIPDef { /* genparse, genformat */ + ... ... + char *family; /* xmlattr */ + virTristateBool localPTR; /* xmlattr:localPtr */ + char *tftproot; /* xmlattr:tftp/root */ + ... ... + }; + +This example demonstrates all those three usages: + +The member **family** has the same name with its corresponding xml attribute. As in: <ip family='..' />. + +But for **localPTR**, the name of its corresponding attribute is "localPtr", as in: <ip localPtr='..' />. So we have to specify it explicitly. + +And for **tftproot**, in fact it matches the attribute *root* of the child-element *tftp*, as in: <ip><tftp root='..' /></ip> in xml. So we use the most complicated form 'xmlattr:tftp/root'. + +xmlelem[:thename] +----------------- + +(only for member; basic) + +Indicate that the member matches an xml element. + +By default, use the member's name as element's name to generate parse/format code block. + +Specify the element's name by **thename** when necessary. + +E.g.: + + :: + + struct _virNetworkForwardNatDef { /* genparse, genformat */ + virSocketAddrRange addr; /* xmlelem:address */ + virPortRange port; /* xmlelem */ + ... ... + }; + +For the member **addr**, it matches xml element <address ... /> in <nat>; +and **port** matches <port ... /> in <nat>. + +In special cases, the corresponding element is a *text-node*, like +<name>...</name>. An additional directive **xmltext** should be appended to +handle this situation. + +xmltext +----------------- + +(only for member; additional) + +Indicate that this member matches *text node*. It must be with **xmlelem**. + +E.g.: + + :: + + typedef struct _virNetworkDef virNetworkDef; + struct _virNetworkDef { /* genparse, genformat */ + virUUID uuid; /* xmlelem, xmltext */ + ... ... + }; + +The member **uuid** matches <uuid>...</uuid>. + + +array[:countername] +------------------- + +(only for member; additional) + +Indicate that this member matches an array of xml node. It must be with **xmlattr** or **xmlelem**. + +Each array member is accompanied with a counter member in the same struct, which name should be in the form of: 'n' + member-name. + +If the counter member's name doesn't follow the common rule, we should specify it. As in: + + :: + + struct _virNetworkDNSDef { /* genparse, genformat */ + ... ... + size_t nfwds; + virNetworkDNSForwarder *forwarders; /* xmlelem, array:nfwds */ + size_t ntxts; + virNetworkDNSTxtDef *txts; /* xmlelem, array */ + ... ... + }; + +For member **forwarders**, its counter member **nfwds** doesn't follow the common name rule, so we have to specify it explicitly. + +Another note: for array member, use the **singular** form of its name as its corresponding xml node's name. +As the member **txts**, it matches a group of element *<txt ... />*. + +required +-------- + +(only for member; additional) + +Indicate that the corresponding xml node of this member must exist and have valid value. It must be with **xmlattr** or **xmlelem**. E.g.: + + :: + + struct _virNetworkDNSTxtDef { /* genparse, genformat */ + char *name; /* xmlattr, required */ + ... ... + }; + +In parse-function, there will be a piece of code to check the existence of corresponding attribute, or it will report an error. Just like: + + :: + + def->name = virXMLPropString(node, "name"); + if (def->name == NULL) { + virReportError(VIR_ERR_XML_ERROR, + _("Missing '%s' setting in '%s'"), + "name", instname); + goto error; + } + +specify:thename +--------------- + +(only for member; basic) + +Indicate that this member specify the existence of another member named with **thename**. E.g.: + + :: + + struct _virNetworkDef { /* genparse, genformat */ + ... ... + virMacAddr mac; /* xmlattr:mac/address */ + bool mac_specified; /* specify:mac */ + }; + +The member **mac_specified** specify the existence of another member **mac**. + +In parse-function, **mac_specified** will be automatically set when **mac** exists. + +In format-function, whether to format **mac** depends on whether **mac_specified** is *TRUE*. + +skipparse +--------- + +(only for member; additional) + +Ignore this member in parse-function. E.g.: + + :: + + struct _virNetworkDef { /* genparse, genformat */ + int connections; /* xmlattr, skipparse */ + ... ... + }; + +The member **connections** won't appear in the generated parse-function, but it still appears in the format-function. + +xmlgroup +-------- + +(only for member which type is a struct; basic) + +This member matches an group of xml nodes rather than an xml element. E.g.: + + :: + + struct _virDomainGraphicsVNCDef { /* genparse */ + ... ... + virDomainGraphicsAuthDef auth; /* xmlgroup */ + }; + +For the member **auth**, we can't find an corresponding element in xml. +In fact, it matches an group of attributes, including *passwd*, *passwdValidTo*, and *connected*. +Then, parse/format code block will skip over this level and traverse its children directly. + +xmlswitch:thename +----------------- + +(only for member which type is a union; basic) + +Indicate that this member matches an xml choice and generates *switch* statement to parse/format xml. + +The union member should have a relative enum member which is specified by **thename**. E.g., + + :: + + /* Implementation of virDomainGraphicsType */ + VIR_ENUM_IMPL(virDomainGraphics, + VIR_DOMAIN_GRAPHICS_TYPE_LAST, + "sdl", + "vnc", + "rdp", + "desktop", + "spice", + "egl-headless", + ); + + struct _virDomainGraphicsDef { /* genparse, genformat */ + virDomainGraphicsType type; /* xmlattr */ + ... ... + union { + virDomainGraphicsSDLDef sdl; /* xmlgroup */ + virDomainGraphicsVNCDef vnc; /* xmlgroup */ + virDomainGraphicsRDPDef rdp; /* xmlgroup */ + virDomainGraphicsDesktopDef desktop; /* xmlgroup */ + virDomainGraphicsSpiceDef spice; /* xmlgroup */ + virDomainGraphicsEGLHeadlessDef egl_headless; /* xmlgroup */ + } data; /* xmlswitch:type */ + }; + +For the union member **data**, the enum member **type** is its relative counterpart. + +Note: Each child of **data** has the same name with each child of **type** in order. + +Automatic Features +================== + +Other than directives, xmlgen can automatically detect some features. + +Default item of enum +-------------------- + +For all enums in libvirt, there're two cases: + +- Most of them have an extra default item which value is *ZERO*. + It usually represents that the absence of the corresponding attribute in xml. E.g. + + :: + + typedef enum { + VIR_NETWORK_BRIDGE_MAC_TABLE_MANAGER_DEFAULT = 0, + VIR_NETWORK_BRIDGE_MAC_TABLE_MANAGER_KERNEL, + VIR_NETWORK_BRIDGE_MAC_TABLE_MANAGER_LIBVIRT, + VIR_NETWORK_BRIDGE_MAC_TABLE_MANAGER_LAST, + } virNetworkBridgeMACTableManagerType; + +In generated parse-function, it's illegal that virNetworkBridgeMACTableManagerTypeFromString returns ZERO. + +- The others have no this kind of default item. All items have their actual meanings. E.g., + + :: + + typedef enum { + VIR_NETWORK_DHCP_LEASETIME_UNIT_SECONDS = 0, + VIR_NETWORK_DHCP_LEASETIME_UNIT_MINUTES, + VIR_NETWORK_DHCP_LEASETIME_UNIT_HOURS, + VIR_NETWORK_DHCP_LEASETIME_UNIT_LAST, + } virNetworkDHCPLeaseTimeUnitType; + +In generated parse-function, the return-value of virNetworkDHCPLeaseTimeUnitTypeFromString can be ZERO, +which indicates the first item VIR_NETWORK_DHCP_LEASETIME_UNIT_SECONDS. + +The tool xmlgen can distinguish them and generate proper parsing/formatting code by checking whether the first item ends with '_DEFAULT', '_NONE' or '_ABSENT'. + +Namespace +--------- + +For some top structs, such as virNetworkDef, virDomainDef, etc., there're some code blocks about **namespace**. + +The tool xmlgen generates extra code block to deal with **namespace** in parse/format/clear function if it finds that a struct has a member named '**namespaceData**'. + +Pointer +------- + +Some members' type is a pointer. The tool xmlgen determines it by checking "*" or "Ptr" so that it can generate proper code. + +Hooks +===== + +Generated parse/format functions have some hooks which provide flexibility. +By default, hooks are disabled, so we should implement and enable them when necessary. + +Post-hook for parse-function +---------------------------- + +This hook is used to hold error-checking code and even to set/change any member's value in the post process. E.g.: + +:: + + int + virNetworkDNSDefParseXML(xmlNodePtr node, + virNetworkDNSDef *def, + const char *instname G_GNUC_UNUSED, + void *parent G_GNUC_UNUSED, + void *opaque G_GNUC_UNUSED) + { + /* Parsing block for each member */ + ... ... + + #ifdef ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK + if (virNetworkDNSDefParseHook(node, def, instname, parent, opaque, enableStr, forwardPlainNamesStr, nForwarderNodes, nTxtNodes, nSrvNodes, nHostNodes) < 0) + goto error; + #endif + + return 0; + + error: + ... ... + return -1; + } + +The hook virNetworkDNSDefParseHook is after all members' parsing blocks. +Now we can take 3 steps to enable this hook to hold some error-checking code. + +step1. Look through helpful information. +........................................ + +Execute below command line: + + :: + + # ./scripts/xmlgen/go show virNetworkDNSDef -kp + +Then we got the declaration of virNetworkDNSDefParseHook and tips. +The declaration will be used in step3. And the tips is as below: + + :: + + [Tips] + + /* Put these lines at the bottom of "conf/network_conf.h" */ + /* Makesure "network_conf.h" to be appended into conf_xmlgen_input in src/conf/meson.build */ + + /* Define macro to enable hook or redefine check when necessary */ + /* #define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK */ + /* #define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK_SET_ARGS */ + + /* Makesure below is the bottom line! */ + #include "network_conf.generated.h" + +step2. Enable hook and apply generated functions. +................................................. + +According to tips from step1, we define ENABLE_VIR_NETWORK_DNSDEF_PARSE_HOOK to enable the hook and include the header-file. + +step3. Implement hook. +...................... + +Implement virNetworkDNSDefParseHook according to its delcaration and put the error-checking code into it. + +This implementation should be written into a C source file that includes "network_conf.h". + +In this example, "network_conf.c" is a good choice. + +Pre-hook for parse-function +--------------------------- + +This hook is used to change the common rule of passing some special arguments, including **parent** and **opaque**. +The common rule is: + +For each parse-function, the argument **parent** holds the pointer of its parent struct instance, and the argument **def** will be passed into child parse-function as their **parent**. + +The argument **opaque** holds an opaque data, and it will be passed from parent parse-function to child parse-function recursively. + +But sometimes, we should change this rule for some generated parse-functions. As in: + +:: + + int + virNetworkDHCPDefParseXML(xmlNodePtr node, + virNetworkDHCPDef *def, + const char *instname G_GNUC_UNUSED, + void *parent G_GNUC_UNUSED, + void *opaque G_GNUC_UNUSED) + { + ... ... + void *arg_parent G_GNUC_UNUSED = def; + void *arg_opaque G_GNUC_UNUSED = opaque; + + if (!def) + goto error; + + #ifdef ENABLE_VIR_NETWORK_DHCPDEF_PARSE_HOOK_SET_ARGS + virNetworkDHCPDefParseXMLSetArgs(node, parent, &arg_parent, &arg_opaque); + #endif + + if (nRangeNodes > 0) { + ... ... + if (virNetworkDHCPRangeDefParseXML(tnode, &def->ranges[i], instname, arg_parent, arg_opaque) < 0) + goto error; + } + ... ... + } + +The hook virNetworkDHCPDefParseXMLSetArgs has a chance to intercept **parent/opaque** in the chain. In this example, we need pass the pointer of virNetworkDef rather than that of virNetworkDHCPDef to virNetworkDHCPRangeDefParseXML as its **parent**, so we enable the hook virNetworkDHCPDefParseXMLSetArgs to implement it. + +The steps to enable the hook are just similar as mentioned in post-hook for parse-function. + +Pre-hook for format-function +---------------------------- + +The generated format-function has a pre-hook that can override the default xml output of any member and even the whole output of struct instance. + +E.g., for virNetworkForwardDefFormatBuf, we can implement the hook as below: + +:: + + int + virNetworkForwardDefFormatHook(const virNetworkForwardDef *def, + const void *parent, + const void *opaque G_GNUC_UNUSED, + virTristateBool *empty, + virTristateBool *shortcut, + virBuffer *devBuf, + virBuffer *typeBuf G_GNUC_UNUSED, + virBuffer *managedBuf, + virBuffer *driver_nameBuf G_GNUC_UNUSED, + virBuffer *natBuf G_GNUC_UNUSED, + virBuffer *pfsBuf G_GNUC_UNUSED, + virBuffer *ifsBuf G_GNUC_UNUSED) + { + if (def->type == VIR_NETWORK_FORWARD_NONE) { + *empty = VIR_TRISTATE_BOOL_YES; + return 0; + } + + if (!def->npfs) { + const char *dev = virNetworkDefForwardIf(parent, 0); + virBufferEscapeString(devBuf, " dev='%s'", dev); + } else { + virBufferIgnore(devBuf); + } + + if (def->type != VIR_NETWORK_FORWARD_HOSTDEV) + virBufferIgnore(managedBuf); + + if (!(def->nifs || def->npfs || def->nat.port.start || def->nat.port.end || + VIR_SOCKET_ADDR_VALID(&def->nat.addr.start) || + VIR_SOCKET_ADDR_VALID(&def->nat.addr.end) || + (def->driverName != VIR_NETWORK_FORWARD_DRIVER_NAME_DEFAULT) || + def->nat.natIPv6)) + *shortcut = VIR_TRISTATE_BOOL_YES; + + return 0; + } + +In the hook, we redefine four things: + +1) Redefine the xml output for the attribute **dev** by resetting *devBuf*. + +2) Indicate that the attribute **managed** should be ignored (by *virBufferIgnore*) under some conditions. + +3) Set **shortcut** to indicate that <forward>...</forward> should use the shortut form like <forward/> under some conditions. + +4) Set **empty** to indicate that <forward> shouldn't appear under some conditions. + +The arguments **shortcut** and **empty** are both the type of virTristateBool. + +By default, they are VIR_TRISTATE_BOOL_ABSENT which means they don't influence the default behavior of the generated format-function. + +Set them with VIR_TRISTATE_BOOL_YES or VIR_TRISTATE_BOOL_NO to turn on/off them respectively. + +Check-function +-------------- + +It is an auxiliary function for format-function. + +Before a format-function is called to output an xml node, its auxiliary check-function should be invoked to determine whether the xml node is empty. +For each struct with **genformat**, their check-functions are generated automatically. + +Sometimes the default implementation is not satisfied, we can redefine it. + +E.g., for virDomainGraphicsListenDef, we look through its check-function by calling command line as below: + + :: + + # ./scripts/xmlgen/go show virDomainGraphicsListenDef -kf + +We got the default implementation of virDomainGraphicsListenDefCheck and the tips. + +Since it doesn't satisfy our needs, we can define RESET_VIR_DOMAIN_GRAPHICS_LISTEN_DEF_CHECK and redefine it. -- 2.25.1