[libvirt] [PATCH 0/3] Libvirt RPC dispatching and unresponsive QEMU

If there is an unresponsive qemu process and libvirt access it's monitor, it will not get any response and this thread will block indefinitely, until the qemu process resumes or it's destroyed. If users continues executing APIs against that domain, libvirt will run out of worker threads and hangs (if those APIs will access monitor as well). Although, they will timeout in approx 30 seconds, which will free some workers, during that time is libvirt unable to process any request. Even worse - if the number of unresponsive qemu exceeds the size of worker thread pool, libvirt will hangs forever, and even restarting the daemon will not make it any better. This patch set heals the daemon on several levels, so nothing from written above will cause it to hangs: 1. RPC dispatching - all APIs are now annotated as 'high' or 'low' priority. Then a special thread pool is created. Low priority APIs will be still placed into usual pool, but high priority can be placed into this new pool if the former has no free worker. Which APIs should be marked high and which low? The splitting presented here is my guessing. It is not something written in stone, but from the logic of things it is not safe to annotate any API which is NOT guaranteed to end in reasonable small time as high priority call. 2. Job queue size limit - this sets bound on the number of threads blocked by a stuck qemu. Okay, there exists timeout on this, but if user application continue dispatching low priority calls it can still consume all (low priority) worker threads and therefore affect other users/VMs. Even if they timeout in approx 30 secs. 3. Run monitor re-connect in a separate thread per VM. If libvirtd is restarted, it tries to reconnect to all running qemu processes. This is potentially risky - one stuck qemu block daemon startup. However, putting the monitor startup code into one thread per VM allows libvirtd to startup, accept client connections and work with all VMs which monitor was successfully re-opened. Unresponsive qemu will hold job until we open the monitor. So clever user application can destroy such domain. All APIs requiring job will just fail in acquiring lock. Michal Privoznik (3): daemon: Create priority workers pool qemu: Introduce job queue size limit qemu: Deal with stucked qemu on daemon startup daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 10 +- daemon/libvirtd.conf | 6 + daemon/remote.c | 26 ++ daemon/remote.h | 2 + src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 7 + src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 17 ++ src/qemu/qemu_domain.h | 2 + src/qemu/qemu_driver.c | 23 +-- src/qemu/qemu_process.c | 89 ++++++- src/remote/qemu_protocol.x | 13 +- src/remote/remote_protocol.x | 544 +++++++++++++++++++++--------------------- src/rpc/gendispatch.pl | 48 ++++- src/rpc/virnetserver.c | 32 +++- src/rpc/virnetserver.h | 6 +- src/util/threadpool.c | 38 ++- src/util/threadpool.h | 1 + 20 files changed, 554 insertions(+), 318 deletions(-) -- 1.7.3.4

This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy). For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC. The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5. To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. --- daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 10 +- daemon/libvirtd.conf | 6 + daemon/remote.c | 26 ++ daemon/remote.h | 2 + src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 +- src/remote/remote_protocol.x | 544 +++++++++++++++++++++--------------------- src/rpc/gendispatch.pl | 48 ++++- src/rpc/virnetserver.c | 32 +++- src/rpc/virnetserver.h | 6 +- src/util/threadpool.c | 38 ++- src/util/threadpool.h | 1 + 13 files changed, 436 insertions(+), 293 deletions(-) diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers" let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 1a83326..293b8c4 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients; + int prio_workers; + int max_requests; int max_client_requests; @@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20; + data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5; @@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients); + GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests); @@ -1414,7 +1420,9 @@ int main(int argc, char **argv) { config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, - remoteClientInitHook))) { + remoteClientInitHook, + config->prio_workers, + remoteGetProcPriority))) { ret = VIR_DAEMON_ERR_INIT; goto cleanup; } diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20 + +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/daemon/remote.c b/daemon/remote.c index 939044c..f6c6f35 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -464,6 +464,32 @@ int remoteClientInitHook(virNetServerPtr srv ATTRIBUTE_UNUSED, return 0; } +int remoteGetProcPriority(virNetMessageHeaderPtr hdr) +{ + u_int prog = hdr->prog; + int proc = hdr->proc; + int *table = NULL; + size_t max = 0; + + switch (prog) { + case REMOTE_PROGRAM: + table = remoteProcsPriority; + max = remoteNProcs; + break; + case QEMU_PROGRAM: + table = qemuProcsPriority; + max = qemuNProcs; + break; + } + + if (!table || !max) + return 0; + if (proc >= max) + return 0; + + return table[proc]; +} + /*----- Functions. -----*/ static int diff --git a/daemon/remote.h b/daemon/remote.h index 5444e47..c624362 100644 --- a/daemon/remote.h +++ b/daemon/remote.h @@ -38,4 +38,6 @@ extern size_t qemuNProcs; int remoteClientInitHook(virNetServerPtr srv, virNetServerClientPtr client); +int remoteGetProcPriority(virNetMessageHeaderPtr hdr); + #endif /* __LIBVIRTD_REMOTE_H__ */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6f54b30..21e73a5 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, false, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1; enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..3dd23cc 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,290 +2201,300 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1; enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ - REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ - REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */ - - REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ - REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ - REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ - - REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */ - - REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ - REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ - REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen */ - REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 */ - REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */ - - REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ + REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:low */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen priority:high */ + + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen priority:low */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:low */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:low */ + REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:low */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen priority:low */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */ + + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen priority:high */ + + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:low */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:low */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen priority:low */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 priority:low */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:low */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen priority:low */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen priority:low */ + + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen priority:low */ /* * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..a8f9fc6 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -77,7 +77,8 @@ while (<PROTOCOL>) { UC_NAME => uc $name, args => "${structprefix}_${name}_args", args_members => [], - ret => "void" + ret => "void", + priority => 0 }; $collect_args_members = 1; @@ -97,7 +98,8 @@ while (<PROTOCOL>) { UC_NAME => uc $name, args => "void", ret => "${structprefix}_${name}_ret", - ret_members => [] + ret_members => [], + priority => 0 } } @@ -120,7 +122,8 @@ while (<PROTOCOL>) { name => $name, ProcName => $ProcName, UC_NAME => uc $name, - msg => "${structprefix}_${name}_msg" + msg => "${structprefix}_${name}_msg", + priority => 0 }; $collect_args_members = 0; @@ -141,17 +144,19 @@ while (<PROTOCOL>) { ProcName => $ProcName, UC_NAME => uc $name, args => "void", - ret => "void" + ret => "void", + priority => 0, } } if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\s+priority:(\S+)\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" } my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = $4; if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +176,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + # already initialized + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } } $calls[$id] = $calls{$name}; @@ -259,6 +274,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } } @@ -954,6 +970,28 @@ elsif ($opt_b) { } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; + + # Now we write priority table + + print "\nint ${structprefix}ProcsPriority[] = {\n"; + for ($id = 0 ; $id <= $#calls ; $id++) { + if (!defined $calls[$id]) { + print "0, /* Unkown proc $id */\n"; + next; + } + if ($calls[$id]->{priority}) { + print "1"; + } else { + print "0"; + } + if ($calls[$id]->{UC_NAME}) { + print ", /* ${procprefix}_PROC_$calls[$id]->{UC_NAME} */"; + } else { + print ","; + } + print "\n"; + } + print "};\n"; } # Bodies for client functions ("remote_client_bodies.h"). diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..b8150b7 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -72,9 +72,12 @@ struct _virNetServer { virMutex lock; virThreadPoolPtr workers; + virThreadPoolPtr prio_workers; bool privileged; + virNetServerPriorityProcFunc prio_func; + size_t nsignals; virNetServerSignalPtr *signals; int sigread; @@ -182,6 +185,7 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, { virNetServerPtr srv = opaque; virNetServerJobPtr job; + bool priority = false; int ret; VIR_DEBUG("server=%p client=%p message=%p", @@ -192,11 +196,25 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, return -1; } + if (srv->prio_func) + priority = srv->prio_func(&msg->header); + job->client = client; job->msg = msg; virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + ret = virThreadPoolSendJob(srv->workers, priority, job); + if (ret < -1) { + goto cleanup; + } else if (ret == -1) { + /* try placing job into priority pool */ + VIR_DEBUG("worker pool full, placing proc %d into priority pool", + msg->header.proc); + ret = virThreadPoolSendJob(srv->prio_workers, false, job); + } + +cleanup: + if (ret < 0) VIR_FREE(job); virNetServerUnlock(srv); @@ -277,7 +295,9 @@ virNetServerPtr virNetServerNew(size_t min_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, - virNetServerClientInitHook clientInitHook) + virNetServerClientInitHook clientInitHook, + size_t priority_workers, + virNetServerPriorityProcFunc func) { virNetServerPtr srv; struct sigaction sig_action; @@ -294,6 +314,13 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv))) goto error; + if (!(srv->prio_workers = virThreadPoolNew(priority_workers, + priority_workers, + virNetServerHandleJob, + srv))) + goto error; + + srv->prio_func = func; srv->nclients_max = max_clients; srv->sigwrite = srv->sigread = -1; srv->clientInitHook = clientInitHook; @@ -759,6 +786,7 @@ void virNetServerFree(virNetServerPtr srv) virNetServerServiceToggle(srv->services[i], false); virThreadPoolFree(srv->workers); + virThreadPoolFree(srv->prio_workers); for (i = 0 ; i < srv->nsignals ; i++) { sigaction(srv->signals[i]->signum, &srv->signals[i]->oldaction, NULL); diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..2b02737 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -37,12 +37,16 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv, virNetServerClientPtr client); +typedef int (*virNetServerPriorityProcFunc) (virNetMessageHeaderPtr hdr); + virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, - virNetServerClientInitHook clientInitHook); + virNetServerClientInitHook clientInitHook, + size_t priority_workers, + virNetServerPriorityProcFunc func); typedef int (*virNetServerAutoShutdownFunc)(virNetServerPtr srv, void *opaque); diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); } +/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2; virMutexLock(&pool->mutex); if (pool->quit) goto error; - if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + } - if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; } } @@ -227,5 +241,5 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, error: virMutexUnlock(&pool->mutex); - return -1; + return ret; } diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..d791b55 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -41,6 +41,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, void virThreadPoolFree(virThreadPoolPtr pool); int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; -- 1.7.3.4

On Tue, Aug 16, 2011 at 06:39:10PM +0200, Michal Privoznik wrote:
This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy).
For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC.
The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5.
To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen.
diff --git a/daemon/remote.c b/daemon/remote.c index 939044c..f6c6f35 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -464,6 +464,32 @@ int remoteClientInitHook(virNetServerPtr srv ATTRIBUTE_UNUSED, return 0; }
+int remoteGetProcPriority(virNetMessageHeaderPtr hdr) +{ + u_int prog = hdr->prog; + int proc = hdr->proc; + int *table = NULL; + size_t max = 0; + + switch (prog) { + case REMOTE_PROGRAM: + table = remoteProcsPriority; + max = remoteNProcs; + break; + case QEMU_PROGRAM: + table = qemuProcsPriority; + max = qemuNProcs; + break; + } + + if (!table || !max) + return 0; + if (proc >= max) + return 0; + + return table[proc]; +}
I don't think this is the right way provide the priority information. We already have a extensible table for information about procedures in virnetserverprogram.h struct _virNetServerProgramProc { virNetServerProgramDispatchFunc func; size_t arg_len; xdrproc_t arg_filter; size_t ret_len; xdrproc_t ret_filter; bool needAuth; }; We should be adding 'unsigned int priority' to the struct.
diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..a8f9fc6 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -954,6 +970,28 @@ elsif ($opt_b) { } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; + + # Now we write priority table + + print "\nint ${structprefix}ProcsPriority[] = {\n"; + for ($id = 0 ; $id <= $#calls ; $id++) { + if (!defined $calls[$id]) { + print "0, /* Unkown proc $id */\n"; + next; + } + if ($calls[$id]->{priority}) { + print "1"; + } else { + print "0"; + } + if ($calls[$id]->{UC_NAME}) { + print ", /* ${procprefix}_PROC_$calls[$id]->{UC_NAME} */"; + } else { + print ","; + } + print "\n"; + } + print "};\n"; }
And so instead of creating a new table here, we should just be writing the new priority field to the existing table.
# Bodies for client functions ("remote_client_bodies.h"). diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..b8150b7 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -72,9 +72,12 @@ struct _virNetServer { virMutex lock;
virThreadPoolPtr workers; + virThreadPoolPtr prio_workers;
bool privileged;
+ virNetServerPriorityProcFunc prio_func; + size_t nsignals; virNetServerSignalPtr *signals; int sigread; @@ -182,6 +185,7 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, { virNetServerPtr srv = opaque; virNetServerJobPtr job; + bool priority = false; int ret;
VIR_DEBUG("server=%p client=%p message=%p", @@ -192,11 +196,25 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, return -1; }
+ if (srv->prio_func) + priority = srv->prio_func(&msg->header); + job->client = client; job->msg = msg;
virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + ret = virThreadPoolSendJob(srv->workers, priority, job); + if (ret < -1) { + goto cleanup; + } else if (ret == -1) { + /* try placing job into priority pool */ + VIR_DEBUG("worker pool full, placing proc %d into priority pool", + msg->header.proc); + ret = virThreadPoolSendJob(srv->prio_workers, false, job); + } + +cleanup: + if (ret < 0) VIR_FREE(job); virNetServerUnlock(srv);
One thing that concerns me is that using 2 separate thread pools is inherantly racy. eg, between the time we check for free workers in the main pool, and place the job in the priority pool, workers in the main pool may have become free. This can lead to imbalances where the main pool is temporarily busy, and so a large number of jobs get queued up in the priority pool. I'm wondering if there is anyway we could have a single thread pool, but some of the workers in the pool are high priority workers. So the decision about who gets what jobs is taken when the job is served, rather than when the job is queued. Avoiding the double queues in this way would avoid getting into an unbalanced situation, where the priority queue is very full but the main queue is now empty.
diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); }
+/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2;
virMutexLock(&pool->mutex); if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + }
- if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; }
I don't think this check is correct, because it is only checking the free workers, against the current/max workers. It is not taking into account the possibility that there are queued jobs which have not yet been served. So it might look like there is a free worker, but there is already a pending job which could consume it. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On 23.08.2011 14:23, Daniel P. Berrange wrote:
On Tue, Aug 16, 2011 at 06:39:10PM +0200, Michal Privoznik wrote:
This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy).
For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC.
The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5.
To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen.
diff --git a/daemon/remote.c b/daemon/remote.c index 939044c..f6c6f35 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -464,6 +464,32 @@ int remoteClientInitHook(virNetServerPtr srv ATTRIBUTE_UNUSED, return 0; }
+int remoteGetProcPriority(virNetMessageHeaderPtr hdr) +{ + u_int prog = hdr->prog; + int proc = hdr->proc; + int *table = NULL; + size_t max = 0; + + switch (prog) { + case REMOTE_PROGRAM: + table = remoteProcsPriority; + max = remoteNProcs; + break; + case QEMU_PROGRAM: + table = qemuProcsPriority; + max = qemuNProcs; + break; + } + + if (!table || !max) + return 0; + if (proc >= max) + return 0; + + return table[proc]; +}
I don't think this is the right way provide the priority information. We already have a extensible table for information about procedures in virnetserverprogram.h
struct _virNetServerProgramProc { virNetServerProgramDispatchFunc func; size_t arg_len; xdrproc_t arg_filter; size_t ret_len; xdrproc_t ret_filter; bool needAuth; };
We should be adding 'unsigned int priority' to the struct.
diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..a8f9fc6 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -954,6 +970,28 @@ elsif ($opt_b) { } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; + + # Now we write priority table + + print "\nint ${structprefix}ProcsPriority[] = {\n"; + for ($id = 0 ; $id <= $#calls ; $id++) { + if (!defined $calls[$id]) { + print "0, /* Unkown proc $id */\n"; + next; + } + if ($calls[$id]->{priority}) { + print "1"; + } else { + print "0"; + } + if ($calls[$id]->{UC_NAME}) { + print ", /* ${procprefix}_PROC_$calls[$id]->{UC_NAME} */"; + } else { + print ","; + } + print "\n"; + } + print "};\n"; }
And so instead of creating a new table here, we should just be writing the new priority field to the existing table.
# Bodies for client functions ("remote_client_bodies.h"). diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..b8150b7 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -72,9 +72,12 @@ struct _virNetServer { virMutex lock;
virThreadPoolPtr workers; + virThreadPoolPtr prio_workers;
bool privileged;
+ virNetServerPriorityProcFunc prio_func; + size_t nsignals; virNetServerSignalPtr *signals; int sigread; @@ -182,6 +185,7 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, { virNetServerPtr srv = opaque; virNetServerJobPtr job; + bool priority = false; int ret;
VIR_DEBUG("server=%p client=%p message=%p", @@ -192,11 +196,25 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, return -1; }
+ if (srv->prio_func) + priority = srv->prio_func(&msg->header); + job->client = client; job->msg = msg;
virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + ret = virThreadPoolSendJob(srv->workers, priority, job); + if (ret < -1) { + goto cleanup; + } else if (ret == -1) { + /* try placing job into priority pool */ + VIR_DEBUG("worker pool full, placing proc %d into priority pool", + msg->header.proc); + ret = virThreadPoolSendJob(srv->prio_workers, false, job); + } + +cleanup: + if (ret < 0) VIR_FREE(job); virNetServerUnlock(srv);
One thing that concerns me is that using 2 separate thread pools is inherantly racy. eg, between the time we check for free workers in the main pool, and place the job in the priority pool, workers in the main pool may have become free. This can lead to imbalances where the main pool is temporarily busy, and so a large number of jobs get queued up in the priority pool.
I don't see why this is a problem. I mean - that's the purpose of high prio pool, isn't it? As soon as main pool will contain a free worker, HPC is sent to it instead of high prio pool. And because HPC are short-term calls, high prio queue will empty quickly.
I'm wondering if there is anyway we could have a single thread pool, but some of the workers in the pool are high priority workers. So the decision about who gets what jobs is taken when the job is served, rather than when the job is queued. Avoiding the double queues in this way would avoid getting into an unbalanced situation, where the priority queue is very full but the main queue is now empty.
diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); }
+/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2;
virMutexLock(&pool->mutex); if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + }
- if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; }
I don't think this check is correct, because it is only checking the free workers, against the current/max workers. It is not taking into account the possibility that there are queued jobs which have not yet been served. So it might look like there is a free worker, but there is already a pending job which could consume it.
I don't agree. Currently we allow expanding of pool only when placing a job into pool. During placing, pool is locked, so freeWorkers variable cannot change. Assume freeWorkers == 0; As soon as we can't expand the pool, job will wait on queue. So we may decide if we want to place job or not. If freeWorkers is not zero, there can't be any job on the queue, because it would be immediately taken by a free worker. What can be a problem though is when job finishes after we locked pool. In this case, freeWorkers will not get increased so we may do bad decision. But I don't see a simple way how this can be solved.
Regards, Daniel

On Tue, Aug 23, 2011 at 05:53:35PM +0200, Michal Privoznik wrote:
On 23.08.2011 14:23, Daniel P. Berrange wrote:
On Tue, Aug 16, 2011 at 06:39:10PM +0200, Michal Privoznik wrote:
diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); }
+/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2;
virMutexLock(&pool->mutex); if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + }
- if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; }
I don't think this check is correct, because it is only checking the free workers, against the current/max workers. It is not taking into account the possibility that there are queued jobs which have not yet been served. So it might look like there is a free worker, but there is already a pending job which could consume it.
I don't agree. Currently we allow expanding of pool only when placing a job into pool. During placing, pool is locked, so freeWorkers variable cannot change. Assume freeWorkers == 0; As soon as we can't expand the pool, job will wait on queue. So we may decide if we want to place job or not. If freeWorkers is not zero, there can't be any job on the queue, because it would be immediately taken by a free worker.
The problem I'm seeing involves a sequence of two calls to virThreadPoolSendJob(). Most of the time we will expect a liner sequence of virThreadPoolSendJob calls, which will result in something like this set of steps: * Initial condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==0 * Thread 1 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock * Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==1 * Thread 2 is a worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job * Current condition: freeWorkers==0, nWorkers==1, maxWorkers==5, queuedJobs==0 * Thread 3 calls virThreadPoolSendJob() 1. Acquires lock 2. Spawns new worker thread 3. Queues job 4. Notifies condition 5. Releases lock * Current condition: freeWorkers==0, nWorkers==2, maxWorkers==5, queuedJobs==1 * Thread 4 is a new worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job * Final condition: freeWorkers==0, nWorkers==2, maxWorkers==5, queuedJobs==0 But now consider what happens if two calls to virThreadPoolSendJob arrive concurrently. It is possible thread 3 will acquire the lock before thread 2 does. This results in the following set of steps: * Initial condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==0 * Thread 1 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock * Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==1 * Thread 3 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock * Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==2 * Thread 2 is a worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job * Final condition: freeWorkers==0, nWorkers==1, maxWorkers==5, queuedJobs==1 So we end up with 2 jobs and only one worker thread to process them. If that second job was a high priority job, it would be queued in the normal queue because 'freeWorkers' was still 1, even though there was a job in the queue ahead of it, so the effective 'freeWorkers' was 0. This is actually an existing bug in the virThreadPool code for queuing jobs, but it was harmless. When we start to differentiate between low and high priority jobs, then this bug becomes active and can result in a high priority job being blocked by low priority ones. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On Wed, Aug 24, 2011 at 10:12:36PM +0100, Daniel P. Berrange wrote:
On Tue, Aug 23, 2011 at 05:53:35PM +0200, Michal Privoznik wrote:
On 23.08.2011 14:23, Daniel P. Berrange wrote:
On Tue, Aug 16, 2011 at 06:39:10PM +0200, Michal Privoznik wrote:
diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); }
+/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2;
virMutexLock(&pool->mutex); if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) {
If we added another counter 'pool->jobQueueDepth', changed whenever a job is added or remove from the queue, then I think we could get the correct semantics by doing if ((pool->freeWorkers - pool->jobQueueDepth) <= 0) { ... }
+ if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + }
- if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; }
I don't think this check is correct, because it is only checking the free workers, against the current/max workers. It is not taking into account the possibility that there are queued jobs which have not yet been served. So it might look like there is a free worker, but there is already a pending job which could consume it.
I don't agree. Currently we allow expanding of pool only when placing a job into pool. During placing, pool is locked, so freeWorkers variable cannot change. Assume freeWorkers == 0; As soon as we can't expand the pool, job will wait on queue. So we may decide if we want to place job or not. If freeWorkers is not zero, there can't be any job on the queue, because it would be immediately taken by a free worker.
The problem I'm seeing involves a sequence of two calls to virThreadPoolSendJob(). Most of the time we will expect a liner sequence of virThreadPoolSendJob calls, which will result in something like this set of steps:
* Initial condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==0
* Thread 1 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock
* Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==1
* Thread 2 is a worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job
* Current condition: freeWorkers==0, nWorkers==1, maxWorkers==5, queuedJobs==0
* Thread 3 calls virThreadPoolSendJob() 1. Acquires lock 2. Spawns new worker thread 3. Queues job 4. Notifies condition 5. Releases lock
* Current condition: freeWorkers==0, nWorkers==2, maxWorkers==5, queuedJobs==1
* Thread 4 is a new worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job
* Final condition: freeWorkers==0, nWorkers==2, maxWorkers==5, queuedJobs==0
But now consider what happens if two calls to virThreadPoolSendJob arrive concurrently. It is possible thread 3 will acquire the lock before thread 2 does. This results in the following set of steps:
* Initial condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==0
* Thread 1 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock
* Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==1
* Thread 3 calls virThreadPoolSendJob() 1. Acquires lock 2. Queues job 3. Notifies condition 4. Releases lock
* Current condition: freeWorkers==1, nWorkers==1, maxWorkers==5, queuedJobs==2
* Thread 2 is a worker thread 1. Woken up from condition wait 2. Acquires lock 3. Decrements freeWorkers 5. Releases lock 6. Starts processing job
* Final condition: freeWorkers==0, nWorkers==1, maxWorkers==5, queuedJobs==1
So we end up with 2 jobs and only one worker thread to process them. If that second job was a high priority job, it would be queued in the normal queue because 'freeWorkers' was still 1, even though there was a job in the queue ahead of it, so the effective 'freeWorkers' was 0.
This is actually an existing bug in the virThreadPool code for queuing jobs, but it was harmless. When we start to differentiate between low and high priority jobs, then this bug becomes active and can result in a high priority job being blocked by low priority ones.
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

This patch creates an optional BeginJob queue size limit. When active, all other attempts above level will fail. To set this feature assign desired value to max_queued variable in qemu.conf. Setting it to 0 turns it off. --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_domain.c | 17 +++++++++++++++++ src/qemu/qemu_domain.h | 2 ++ 6 files changed, 33 insertions(+), 0 deletions(-) diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index d018ac2..6c145c7 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -51,6 +51,7 @@ module Libvirtd_qemu = | bool_entry "set_process_name" | int_entry "max_processes" | str_entry "lock_manager" + | int_entry "max_queued" (* Each enty in the config is one of the following three ... *) let entry = vnc_entry diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 79c6e85..4da5d5a 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -309,3 +309,10 @@ # disk), uncomment this # # lock_manager = "sanlock" + +# Set limit of maximum APIs queued on one domain. All other APIs +# over this threshold will fail on acquiring job lock. Specially, +# setting to zero turns this feature off. +# Note, that job lock is per domain. +# +# max_queued = 0 diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 443e08d..d1bf075 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -458,6 +458,10 @@ int qemudLoadDriverConfig(struct qemud_driver *driver, VIR_FREE(lockConf); } + p = virConfGetValue(conf, "max_queued"); + CHECK_TYPE("max_queued", VIR_CONF_LONG); + if (p) driver->max_queued = p->l; + virConfFree (conf); return 0; } diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 0a60d32..098e94e 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -108,6 +108,8 @@ struct qemud_driver { int maxProcesses; + int max_queued; + virCapsPtr caps; virDomainEventStatePtr domainEventState; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 675c6df..982bad6 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -713,6 +713,8 @@ qemuDomainObjBeginJobInternal(struct qemud_driver *driver, unsigned long long then; bool nested = job == QEMU_JOB_ASYNC_NESTED; + priv->jobs_queued++; + if (virTimeMs(&now) < 0) return -1; then = now + QEMU_JOB_WAIT_TIME; @@ -722,6 +724,11 @@ qemuDomainObjBeginJobInternal(struct qemud_driver *driver, qemuDriverUnlock(driver); retry: + if (driver->max_queued && + priv->jobs_queued > driver->max_queued) { + goto error; + } + while (!nested && !qemuDomainJobAllowed(priv, job)) { if (virCondWaitUntil(&priv->job.asyncCond, &obj->lock, then) < 0) goto error; @@ -761,9 +768,15 @@ error: if (errno == ETIMEDOUT) qemuReportError(VIR_ERR_OPERATION_TIMEOUT, "%s", _("cannot acquire state change lock")); + else if (driver->max_queued && + priv->jobs_queued > driver->max_queued) + qemuReportError(VIR_ERR_OPERATION_FAILED, + "%s", _("cannot acquire state change lock " + "due to max_queued limit")); else virReportSystemError(errno, "%s", _("cannot acquire job mutex")); + priv->jobs_queued--; if (driver_locked) { virDomainObjUnlock(obj); qemuDriverLock(driver); @@ -844,6 +857,8 @@ int qemuDomainObjEndJob(struct qemud_driver *driver, virDomainObjPtr obj) { qemuDomainObjPrivatePtr priv = obj->privateData; + priv->jobs_queued--; + qemuDomainObjResetJob(priv); qemuDomainObjSaveJob(driver, obj); virCondSignal(&priv->job.cond); @@ -856,6 +871,8 @@ qemuDomainObjEndAsyncJob(struct qemud_driver *driver, virDomainObjPtr obj) { qemuDomainObjPrivatePtr priv = obj->privateData; + priv->jobs_queued--; + qemuDomainObjResetAsyncJob(priv); qemuDomainObjSaveJob(driver, obj); virCondBroadcast(&priv->job.asyncCond); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index e12ca8e..55875fe 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -113,6 +113,8 @@ struct _qemuDomainObjPrivate { char *lockState; bool fakeReboot; + + int jobs_queued; }; struct qemuDomainWatchdogEvent -- 1.7.3.4

On Tue, Aug 16, 2011 at 06:39:11PM +0200, Michal Privoznik wrote:
This patch creates an optional BeginJob queue size limit. When active, all other attempts above level will fail. To set this feature assign desired value to max_queued variable in qemu.conf. Setting it to 0 turns it off. --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_domain.c | 17 +++++++++++++++++ src/qemu/qemu_domain.h | 2 ++ 6 files changed, 33 insertions(+), 0 deletions(-)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++--------- src/qemu/qemu_process.c | 87 ++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 421a98e..4574b6c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,26 +143,15 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { + if (vm->autostart && + !virDomainObjIsActive(vm) && + qemuDomainObjStart(data->conn, data->driver, vm, + false, false, + data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), + VIR_ERROR(_("Failed to autostart VM '%s': %s"), vm->def->name, err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, - false, false, - data->driver->autoStartBypassCache) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to autostart VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } - - if (qemuDomainObjEndJob(data->driver, vm) == 0) - vm = NULL; } if (vm) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 21e73a5..1daf6ae 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -820,6 +820,7 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; int ret = -1; + qemuMonitorPtr mon = NULL; if (virSecurityManagerSetSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to set security context for monitor for %s"), @@ -831,15 +832,29 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) * deleted while the monitor is active */ virDomainObjRef(vm); - priv->mon = qemuMonitorOpen(vm, - priv->monConfig, - priv->monJSON, - &monitorCallbacks); + ignore_value(virTimeMs(&priv->monStart)); + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + + mon = qemuMonitorOpen(vm, + priv->monConfig, + priv->monJSON, + &monitorCallbacks); + + qemuDriverLock(driver); + virDomainObjLock(vm); + priv->monStart = 0; /* Safe to ignore value since ref count was incremented above */ - if (priv->mon == NULL) + if (mon == NULL) ignore_value(virDomainObjUnref(vm)); + if (!virDomainObjIsActive(vm)) { + qemuMonitorClose(mon); + goto error; + } + priv->mon = mon; + if (virSecurityManagerClearSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to clear security context for monitor for %s"), vm->def->name); @@ -2484,21 +2499,25 @@ qemuProcessRecoverJob(struct qemud_driver *driver, struct qemuProcessReconnectData { virConnectPtr conn; struct qemud_driver *driver; + void *payload; }; /* * Open an existing VM's monitor, re-detect VCPU threads * and re-reserve the security labels in use */ static void -qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque) +qemuProcessReconnect(void *opaque) { - virDomainObjPtr obj = payload; struct qemuProcessReconnectData *data = opaque; struct qemud_driver *driver = data->driver; + virDomainObjPtr obj = data->payload; qemuDomainObjPrivatePtr priv; virConnectPtr conn = data->conn; struct qemuDomainJobObj oldjob; + VIR_FREE(data); + + qemuDriverLock(driver); virDomainObjLock(obj); qemuDomainObjRestoreJob(obj, &oldjob); @@ -2572,12 +2591,15 @@ qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opa if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); + return; error: if (!virDomainObjIsActive(obj)) { if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); return; } @@ -2591,6 +2613,55 @@ error: else virDomainObjUnlock(obj); } + qemuDriverUnlock(driver); +} + +static void +qemuProcessReconnectHelper(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virThread thread; + struct qemuProcessReconnectData *src = opaque; + struct qemuProcessReconnectData *data; + + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + return; + } + + memcpy(data, src, sizeof(*data)); + data->payload = payload; + + /* This iterator is called with driver being locked. + * However, we need to unlock it so qemuProcessReconnect, + * which will run in separate thread can lock it itself + * (if needed). The whole idea behind: qemuProcessReconnect: + * 1. lock driver + * 2. just before monitor reconnect, do lightweight MonitorEnter + * (unlock VM & driver) + * 3. Reconnect to monitor + * 4. do lightweight MonitorExit (lock driver & VM) + * 5. continue reconnect process + * 6. unlock driver + * + * It is necessary to NOT hold driver lock for the entire run + * of reconnect, otherwise we will get blocked if there is + * unresponsive qemu. + * However, iterating over hash table MUST be done on locked + * driver. + * + * NB, we can't do normal MonitorEnter & MonitorExit because + * these two lock the monitor lock, which does not exists in + * this early phase. + */ + qemuDriverUnlock(src->driver); + if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not create thread. QEMU initialization " + "might be incomplete")); + } + qemuDriverLock(src->driver); } /** @@ -2603,7 +2674,7 @@ void qemuProcessReconnectAll(virConnectPtr conn, struct qemud_driver *driver) { struct qemuProcessReconnectData data = {conn, driver}; - virHashForEach(driver->domains.objs, qemuProcessReconnect, &data); + virHashForEach(driver->domains.objs, qemuProcessReconnectHelper, &data); } int qemuProcessStart(virConnectPtr conn, -- 1.7.3.4

On Tue, Aug 16, 2011 at 18:39:12 +0200, Michal Privoznik wrote:
If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++--------- src/qemu/qemu_process.c | 87 ++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 25 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 421a98e..4574b6c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,26 +143,15 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq
virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { + if (vm->autostart && + !virDomainObjIsActive(vm) && + qemuDomainObjStart(data->conn, data->driver, vm, + false, false, + data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), + VIR_ERROR(_("Failed to autostart VM '%s': %s"), vm->def->name, err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, - false, false, - data->driver->autoStartBypassCache) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to autostart VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } - - if (qemuDomainObjEndJob(data->driver, vm) == 0) - vm = NULL; }
I think this is wrong. qemuDomainObjStart expect the job to be set so that it can enter qemu monitor. The patch should just move the if (vm->autostrt && !virDomainObjIsActive(vm)) check before BeginJob so that a job is started only if we're going to do something with the domain. Jirka

On Tue, Aug 16, 2011 at 06:39:12PM +0200, Michal Privoznik wrote:
If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++--------- src/qemu/qemu_process.c | 87 ++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 25 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 421a98e..4574b6c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,26 +143,15 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq
virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { + if (vm->autostart && + !virDomainObjIsActive(vm) && + qemuDomainObjStart(data->conn, data->driver, vm, + false, false, + data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), + VIR_ERROR(_("Failed to autostart VM '%s': %s"), vm->def->name, err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, - false, false, - data->driver->autoStartBypassCache) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to autostart VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } - - if (qemuDomainObjEndJob(data->driver, vm) == 0) - vm = NULL; }
I'm not really seeing why this change is safe. You're removing the job protection from the autostart code, but AFAICT, no where else in the caller of this method is changed to acquire the job instead.
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 21e73a5..1daf6ae 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -820,6 +820,7 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; int ret = -1; + qemuMonitorPtr mon = NULL;
if (virSecurityManagerSetSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to set security context for monitor for %s"), @@ -831,15 +832,29 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) * deleted while the monitor is active */ virDomainObjRef(vm);
- priv->mon = qemuMonitorOpen(vm, - priv->monConfig, - priv->monJSON, - &monitorCallbacks); + ignore_value(virTimeMs(&priv->monStart)); + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + + mon = qemuMonitorOpen(vm, + priv->monConfig, + priv->monJSON, + &monitorCallbacks); + + qemuDriverLock(driver); + virDomainObjLock(vm); + priv->monStart = 0;
I'm also not convinced it is safe to release the driver/domain locks in this way. At the very *least* you need to be acquiring an extra reference on the virDomainObjPtr, ideally using the EnterMonitor/ExitMonitor methods, otherwise some other thread may destroy the virDomainObjPtr while it is unlocked. Also what is the 'monStart' field used for ?
/* Safe to ignore value since ref count was incremented above */ - if (priv->mon == NULL) + if (mon == NULL) ignore_value(virDomainObjUnref(vm));
+ if (!virDomainObjIsActive(vm)) { + qemuMonitorClose(mon); + goto error; + } + priv->mon = mon; + if (virSecurityManagerClearSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to clear security context for monitor for %s"), vm->def->name); @@ -2484,21 +2499,25 @@ qemuProcessRecoverJob(struct qemud_driver *driver, struct qemuProcessReconnectData { virConnectPtr conn; struct qemud_driver *driver; + void *payload; }; /* * Open an existing VM's monitor, re-detect VCPU threads * and re-reserve the security labels in use */ static void -qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque) +qemuProcessReconnect(void *opaque) { - virDomainObjPtr obj = payload; struct qemuProcessReconnectData *data = opaque; struct qemud_driver *driver = data->driver; + virDomainObjPtr obj = data->payload; qemuDomainObjPrivatePtr priv; virConnectPtr conn = data->conn; struct qemuDomainJobObj oldjob;
+ VIR_FREE(data); + + qemuDriverLock(driver); virDomainObjLock(obj);
qemuDomainObjRestoreJob(obj, &oldjob); @@ -2572,12 +2591,15 @@ qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opa if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj);
+ qemuDriverUnlock(driver); + return;
error: if (!virDomainObjIsActive(obj)) { if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); return; }
@@ -2591,6 +2613,55 @@ error: else virDomainObjUnlock(obj); } + qemuDriverUnlock(driver); +} + +static void +qemuProcessReconnectHelper(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virThread thread; + struct qemuProcessReconnectData *src = opaque; + struct qemuProcessReconnectData *data; + + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + return; + } + + memcpy(data, src, sizeof(*data)); + data->payload = payload; + + /* This iterator is called with driver being locked. + * However, we need to unlock it so qemuProcessReconnect, + * which will run in separate thread can lock it itself + * (if needed). The whole idea behind: qemuProcessReconnect: + * 1. lock driver + * 2. just before monitor reconnect, do lightweight MonitorEnter + * (unlock VM & driver) + * 3. Reconnect to monitor + * 4. do lightweight MonitorExit (lock driver & VM) + * 5. continue reconnect process + * 6. unlock driver + * + * It is necessary to NOT hold driver lock for the entire run + * of reconnect, otherwise we will get blocked if there is + * unresponsive qemu. + * However, iterating over hash table MUST be done on locked + * driver. + * + * NB, we can't do normal MonitorEnter & MonitorExit because + * these two lock the monitor lock, which does not exists in + * this early phase. + */ + qemuDriverUnlock(src->driver); + if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not create thread. QEMU initialization " + "might be incomplete")); + } + qemuDriverLock(src->driver); }
This comment & code is not quite right. Since the 'qemuProcessReconnect' function runs in a separate thread, there is no need to unlock the src->driver here. Unlocking the driver is in fact dangerous, since this means something can (accidentally or not) change the driver->domains hash table, while we're in the middle of iterating over it. If you remove these qemuDriverUnlock/Lock calls here, it simply means that the thread you spawn will start, but immediately block waiting for the lock. When the thread that triggers reconnect finally unlocks the driver later, all the threads will be able to start executing. The other unsafe thing here though, is that the main thread owns the reference of virDomainObjPtr, and something could happily destroy this reference, before the thread you spawn gets a chance to run. So *before* spawning the thread, you need to start a job on the virDomainObjPtr. The thread will then actually do the job work and release the job when it is done. So this code should look something more like virDomainObjPtr obj = payload; ...begin job on obj... if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not create thread to reconnect to domain"); ...end job on obj... ...destroy the running guest PID if any.. } else { ...nothing. the thread ends the job... } Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On 23.08.2011 14:42, Daniel P. Berrange wrote:
On Tue, Aug 16, 2011 at 06:39:12PM +0200, Michal Privoznik wrote:
If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++--------- src/qemu/qemu_process.c | 87 ++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 25 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 421a98e..4574b6c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,26 +143,15 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq
virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { + if (vm->autostart && + !virDomainObjIsActive(vm) && + qemuDomainObjStart(data->conn, data->driver, vm, + false, false, + data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), + VIR_ERROR(_("Failed to autostart VM '%s': %s"), vm->def->name, err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, - false, false, - data->driver->autoStartBypassCache) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to autostart VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } - - if (qemuDomainObjEndJob(data->driver, vm) == 0) - vm = NULL; }
I'm not really seeing why this change is safe. You're removing the job protection from the autostart code, but AFAICT, no where else in the caller of this method is changed to acquire the job instead.
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 21e73a5..1daf6ae 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -820,6 +820,7 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; int ret = -1; + qemuMonitorPtr mon = NULL;
if (virSecurityManagerSetSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to set security context for monitor for %s"), @@ -831,15 +832,29 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) * deleted while the monitor is active */ virDomainObjRef(vm);
- priv->mon = qemuMonitorOpen(vm, - priv->monConfig, - priv->monJSON, - &monitorCallbacks); + ignore_value(virTimeMs(&priv->monStart)); + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + + mon = qemuMonitorOpen(vm, + priv->monConfig, + priv->monJSON, + &monitorCallbacks); + + qemuDriverLock(driver); + virDomainObjLock(vm); + priv->monStart = 0;
I'm also not convinced it is safe to release the driver/domain locks in this way. At the very *least* you need to be acquiring an extra reference on the virDomainObjPtr, ideally using the EnterMonitor/ExitMonitor methods, otherwise some other thread may destroy the virDomainObjPtr while it is unlocked. I cannot use EnterMonitor because it access priv->mon which does not exist at this time.
Also what is the 'monStart' field used for ?
For gathering statistics: virsh domcontrol <domain>
/* Safe to ignore value since ref count was incremented above */ - if (priv->mon == NULL) + if (mon == NULL) ignore_value(virDomainObjUnref(vm));
+ if (!virDomainObjIsActive(vm)) { + qemuMonitorClose(mon); + goto error; + } + priv->mon = mon; + if (virSecurityManagerClearSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to clear security context for monitor for %s"), vm->def->name); @@ -2484,21 +2499,25 @@ qemuProcessRecoverJob(struct qemud_driver *driver, struct qemuProcessReconnectData { virConnectPtr conn; struct qemud_driver *driver; + void *payload; }; /* * Open an existing VM's monitor, re-detect VCPU threads * and re-reserve the security labels in use */ static void -qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque) +qemuProcessReconnect(void *opaque) { - virDomainObjPtr obj = payload; struct qemuProcessReconnectData *data = opaque; struct qemud_driver *driver = data->driver; + virDomainObjPtr obj = data->payload; qemuDomainObjPrivatePtr priv; virConnectPtr conn = data->conn; struct qemuDomainJobObj oldjob;
+ VIR_FREE(data); + + qemuDriverLock(driver); virDomainObjLock(obj);
qemuDomainObjRestoreJob(obj, &oldjob); @@ -2572,12 +2591,15 @@ qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opa if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj);
+ qemuDriverUnlock(driver); + return;
error: if (!virDomainObjIsActive(obj)) { if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); return; }
@@ -2591,6 +2613,55 @@ error: else virDomainObjUnlock(obj); } + qemuDriverUnlock(driver); +} + +static void +qemuProcessReconnectHelper(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virThread thread; + struct qemuProcessReconnectData *src = opaque; + struct qemuProcessReconnectData *data; + + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + return; + } + + memcpy(data, src, sizeof(*data)); + data->payload = payload; + + /* This iterator is called with driver being locked. + * However, we need to unlock it so qemuProcessReconnect, + * which will run in separate thread can lock it itself + * (if needed). The whole idea behind: qemuProcessReconnect: + * 1. lock driver + * 2. just before monitor reconnect, do lightweight MonitorEnter + * (unlock VM & driver) + * 3. Reconnect to monitor + * 4. do lightweight MonitorExit (lock driver & VM) + * 5. continue reconnect process + * 6. unlock driver + * + * It is necessary to NOT hold driver lock for the entire run + * of reconnect, otherwise we will get blocked if there is + * unresponsive qemu. + * However, iterating over hash table MUST be done on locked + * driver. + * + * NB, we can't do normal MonitorEnter & MonitorExit because + * these two lock the monitor lock, which does not exists in + * this early phase. + */ + qemuDriverUnlock(src->driver); + if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not create thread. QEMU initialization " + "might be incomplete")); + } + qemuDriverLock(src->driver); }
This comment & code is not quite right. Since the 'qemuProcessReconnect' function runs in a separate thread, there is no need to unlock the src->driver here. Unlocking the driver is in fact dangerous, since this means something can (accidentally or not) change the driver->domains hash table, while we're in the middle of iterating over it.
If you remove these qemuDriverUnlock/Lock calls here, it simply means that the thread you spawn will start, but immediately block waiting for the lock. When the thread that triggers reconnect finally unlocks the driver later, all the threads will be able to start executing.
The other unsafe thing here though, is that the main thread owns the reference of virDomainObjPtr, and something could happily destroy this reference, before the thread you spawn gets a chance to run.
So *before* spawning the thread, you need to start a job on the virDomainObjPtr. The thread will then actually do the job work and release the job when it is done.
So this code should look something more like
virDomainObjPtr obj = payload;
...begin job on obj... if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not create thread to reconnect to domain"); ...end job on obj... ...destroy the running guest PID if any.. } else { ...nothing. the thread ends the job... }
Regards, Daniel

This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy). For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC. The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5. To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. --- daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 10 +- daemon/libvirtd.conf | 6 + daemon/remote.c | 26 ++ daemon/remote.h | 2 + src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 +- src/remote/remote_protocol.x | 544 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 ++- src/rpc/virnetserver.c | 32 +++- src/rpc/virnetserver.h | 6 +- src/rpc/virnetserverprogram.h | 1 + src/util/threadpool.c | 38 ++- src/util/threadpool.h | 1 + 14 files changed, 411 insertions(+), 291 deletions(-) diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers" let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 0530ba5..3a6233d 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients; + int prio_workers; + int max_requests; int max_client_requests; @@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20; + data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5; @@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients); + GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests); @@ -1433,7 +1439,9 @@ int main(int argc, char **argv) { config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, - remoteClientInitHook))) { + remoteClientInitHook, + config->prio_workers, + remoteGetProcPriority))) { ret = VIR_DAEMON_ERR_INIT; goto cleanup; } diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20 + +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/daemon/remote.c b/daemon/remote.c index 0f088c6..9b1433b 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -473,6 +473,32 @@ int remoteClientInitHook(virNetServerPtr srv ATTRIBUTE_UNUSED, return 0; } +unsigned int remoteGetProcPriority(virNetMessageHeaderPtr hdr) +{ + u_int prog = hdr->prog; + int proc = hdr->proc; + virNetServerProgramProc *table = NULL; + size_t max = 0; + + switch (prog) { + case REMOTE_PROGRAM: + table = remoteProcs; + max = remoteNProcs; + break; + case QEMU_PROGRAM: + table = qemuProcs; + max = qemuNProcs; + break; + } + + if (!table || !max) + return 0; + if (proc >= max) + return 0; + + return table[proc].priority; +} + /*----- Functions. -----*/ static int diff --git a/daemon/remote.h b/daemon/remote.h index 5444e47..1969f22 100644 --- a/daemon/remote.h +++ b/daemon/remote.h @@ -38,4 +38,6 @@ extern size_t qemuNProcs; int remoteClientInitHook(virNetServerPtr srv, virNetServerClientPtr client); +unsigned int remoteGetProcPriority(virNetMessageHeaderPtr hdr); + #endif /* __LIBVIRTD_REMOTE_H__ */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6f54b30..21e73a5 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, false, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1; enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..3dd23cc 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,290 +2201,300 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1; enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ - REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ - REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */ - - REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ - REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ - REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ - - REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */ - - REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ - REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ - REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen */ - REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 */ - REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */ - - REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ + REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:low */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen priority:high */ + + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen priority:low */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:low */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:low */ + REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:low */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen priority:low */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */ + + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen priority:high */ + + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:low */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:low */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen priority:low */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 priority:low */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:low */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen priority:low */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen priority:low */ + + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen priority:low */ /* * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..1ca3bc0 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { } if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\s+priority:(\S+)\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" } my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = $4; if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +172,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + $calls{$name}->{priority} = 0; + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } } $calls[$id] = $calls{$name}; @@ -259,6 +270,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } } @@ -927,7 +939,7 @@ elsif ($opt_b) { print "virNetServerProgramProc ${structprefix}Procs[] = {\n"; for ($id = 0 ; $id <= $#calls ; $id++) { - my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter); + my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter, $priority); if (defined $calls[$id] && !$calls[$id]->{msg}) { $comment = "/* Method $calls[$id]->{ProcName} => $id */"; @@ -950,7 +962,9 @@ elsif ($opt_b) { $retfilter = "xdr_void"; } - print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true \n},\n"; + $priority = defined $calls[$id]->{priority} ? $calls[$id]->{priority} : 0; + + print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true,\n $priority\n},\n"; } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..b8150b7 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -72,9 +72,12 @@ struct _virNetServer { virMutex lock; virThreadPoolPtr workers; + virThreadPoolPtr prio_workers; bool privileged; + virNetServerPriorityProcFunc prio_func; + size_t nsignals; virNetServerSignalPtr *signals; int sigread; @@ -182,6 +185,7 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, { virNetServerPtr srv = opaque; virNetServerJobPtr job; + bool priority = false; int ret; VIR_DEBUG("server=%p client=%p message=%p", @@ -192,11 +196,25 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, return -1; } + if (srv->prio_func) + priority = srv->prio_func(&msg->header); + job->client = client; job->msg = msg; virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + ret = virThreadPoolSendJob(srv->workers, priority, job); + if (ret < -1) { + goto cleanup; + } else if (ret == -1) { + /* try placing job into priority pool */ + VIR_DEBUG("worker pool full, placing proc %d into priority pool", + msg->header.proc); + ret = virThreadPoolSendJob(srv->prio_workers, false, job); + } + +cleanup: + if (ret < 0) VIR_FREE(job); virNetServerUnlock(srv); @@ -277,7 +295,9 @@ virNetServerPtr virNetServerNew(size_t min_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, - virNetServerClientInitHook clientInitHook) + virNetServerClientInitHook clientInitHook, + size_t priority_workers, + virNetServerPriorityProcFunc func) { virNetServerPtr srv; struct sigaction sig_action; @@ -294,6 +314,13 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv))) goto error; + if (!(srv->prio_workers = virThreadPoolNew(priority_workers, + priority_workers, + virNetServerHandleJob, + srv))) + goto error; + + srv->prio_func = func; srv->nclients_max = max_clients; srv->sigwrite = srv->sigread = -1; srv->clientInitHook = clientInitHook; @@ -759,6 +786,7 @@ void virNetServerFree(virNetServerPtr srv) virNetServerServiceToggle(srv->services[i], false); virThreadPoolFree(srv->workers); + virThreadPoolFree(srv->prio_workers); for (i = 0 ; i < srv->nsignals ; i++) { sigaction(srv->signals[i]->signum, &srv->signals[i]->oldaction, NULL); diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..1bdc105 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -37,12 +37,16 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv, virNetServerClientPtr client); +typedef unsigned int (*virNetServerPriorityProcFunc) (virNetMessageHeaderPtr hdr); + virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, - virNetServerClientInitHook clientInitHook); + virNetServerClientInitHook clientInitHook, + size_t priority_workers, + virNetServerPriorityProcFunc func); typedef int (*virNetServerAutoShutdownFunc)(virNetServerPtr srv, void *opaque); diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index ca31b7e..75d969c 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -53,6 +53,7 @@ struct _virNetServerProgramProc { size_t ret_len; xdrproc_t ret_filter; bool needAuth; + unsigned int priority; }; virNetServerProgramPtr virNetServerProgramNew(unsigned program, diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); } +/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2; virMutexLock(&pool->mutex); if (pool->quit) goto error; - if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + } - if (virThreadCreate(&pool->workers[pool->nWorkers - 1], - true, - virThreadPoolWorker, - pool) < 0) { - pool->nWorkers--; + if (virThreadCreate(&pool->workers[pool->nWorkers - 1], + true, + virThreadPoolWorker, + pool) < 0) { + pool->nWorkers--; + goto error; + } + } else if (only_if_free) { + ret = -1; goto error; } } @@ -227,5 +241,5 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, error: virMutexUnlock(&pool->mutex); - return -1; + return ret; } diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..d791b55 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -41,6 +41,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, void virThreadPoolFree(virThreadPoolPtr pool); int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; -- 1.7.3.4

On Tue, Aug 23, 2011 at 08:22:01PM +0200, Michal Privoznik wrote:
This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy).
For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC.
The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5.
To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. --- daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 10 +- daemon/libvirtd.conf | 6 + daemon/remote.c | 26 ++ daemon/remote.h | 2 + src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 +- src/remote/remote_protocol.x | 544 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 ++- src/rpc/virnetserver.c | 32 +++- src/rpc/virnetserver.h | 6 +- src/rpc/virnetserverprogram.h | 1 + src/util/threadpool.c | 38 ++- src/util/threadpool.h | 1 + 14 files changed, 411 insertions(+), 291 deletions(-)
diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 0530ba5..3a6233d 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -1433,7 +1439,9 @@ int main(int argc, char **argv) { config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, - remoteClientInitHook))) { + remoteClientInitHook, + config->prio_workers, + remoteGetProcPriority))) { ret = VIR_DAEMON_ERR_INIT; goto cleanup; }
diff --git a/daemon/remote.c b/daemon/remote.c index 0f088c6..9b1433b 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -473,6 +473,32 @@ int remoteClientInitHook(virNetServerPtr srv ATTRIBUTE_UNUSED, return 0; }
+unsigned int remoteGetProcPriority(virNetMessageHeaderPtr hdr) +{ + u_int prog = hdr->prog; + int proc = hdr->proc; + virNetServerProgramProc *table = NULL; + size_t max = 0; + + switch (prog) { + case REMOTE_PROGRAM: + table = remoteProcs; + max = remoteNProcs; + break; + case QEMU_PROGRAM: + table = qemuProcs; + max = qemuNProcs; + break; + } + + if (!table || !max) + return 0; + if (proc >= max) + return 0; + + return table[proc].priority; +}
This is still not quite the right design. Since the priority information is directly in the virNetServerProgramProc table, there should be no need to pass in a function callback to virNetServerNew(). It already has a list of programs, so it should be able to ask for the priority directly.
diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..b8150b7 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -182,6 +185,7 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, { virNetServerPtr srv = opaque; virNetServerJobPtr job; + bool priority = false; int ret;
VIR_DEBUG("server=%p client=%p message=%p", @@ -192,11 +196,25 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, return -1; }
+ if (srv->prio_func) + priority = srv->prio_func(&msg->header); +
IIUC, the problem you're facing is that the 'virNetServerDispatchNewMessage' method does not yet know what virNetServerProgramPtr is associated with this RPC call, because that lookup is not done until the virNetServerHandleJob method is run in the worker. To address this, we should put move that bit of code. So specifically - virNetServerJobPtr struct should gain a virNetServerProgramPtr - virNetServerDispatchNewMessage should take the following code from virNetServerHandleJob for (i = 0 ; i < srv->nprograms ; i++) { if (virNetServerProgramMatches(srv->programs[i], job->msg)) { prog = srv->programs[i]; break; } } if (!prog) { VIR_DEBUG("Cannot find program %d version %d", job->msg->header.prog, job->msg->header.vers); goto error; } virNetServerProgramRef(prog); and then store 'prog' in job->prog - virNetServerHandleJob should unref job->prog when it is done - virNetServerDispatchNewMessage can now simply call virNetServerProgramGetPriority(prog, xdr->header.proc); to obtain the priority of the procedure. We now do not need any special callback
diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..ad2d249 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -185,27 +185,41 @@ void virThreadPoolFree(virThreadPoolPtr pool) VIR_FREE(pool); }
+/* + * @only_if_free - place job in pool iff there is + * a free worker(s). + * + * Return: 0 on success, + * -1 if no free worker available but requested + * -2 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + bool only_if_free, void *jobData) { virThreadPoolJobPtr job; + int ret = -2;
virMutexLock(&pool->mutex); if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && - pool->nWorkers < pool->maxWorkers) { - if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { - virReportOOMError(); - goto error; - } + if (pool->freeWorkers == 0) { + if (pool->nWorkers < pool->maxWorkers) { + if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { + virReportOOMError(); + goto error; + }
From my previous mail, I think this needs to be
if ((pool->freeWorkers - pool->jobQueueDepth) <= 0) { .... } (where jobQueueDepth is a new counter we have to introduce) Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++++---- src/qemu/qemu_process.c | 123 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 127 insertions(+), 19 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index c8dda73..8c2e356 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,16 +143,18 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, + if (vm->autostart && + !virDomainObjIsActive(vm)) { + if (qemuDomainObjBeginJobWithDriver(data->driver, vm, + QEMU_JOB_MODIFY) < 0) { + err = virGetLastError(); + VIR_ERROR(_("Failed to start job on VM '%s': %s"), + vm->def->name, + err ? err->message : _("unknown error")); + goto cleanup; + } + + if (qemuDomainObjStart(data->conn, data->driver, vm, false, false, data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); @@ -165,6 +167,7 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq vm = NULL; } +cleanup: if (vm) virDomainObjUnlock(vm); } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 21e73a5..4cc8ae6 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -820,6 +820,7 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; int ret = -1; + qemuMonitorPtr mon = NULL; if (virSecurityManagerSetSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to set security context for monitor for %s"), @@ -831,15 +832,29 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) * deleted while the monitor is active */ virDomainObjRef(vm); - priv->mon = qemuMonitorOpen(vm, - priv->monConfig, - priv->monJSON, - &monitorCallbacks); + ignore_value(virTimeMs(&priv->monStart)); + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + + mon = qemuMonitorOpen(vm, + priv->monConfig, + priv->monJSON, + &monitorCallbacks); + + qemuDriverLock(driver); + virDomainObjLock(vm); + priv->monStart = 0; /* Safe to ignore value since ref count was incremented above */ - if (priv->mon == NULL) + if (mon == NULL) ignore_value(virDomainObjUnref(vm)); + if (!virDomainObjIsActive(vm)) { + qemuMonitorClose(mon); + goto error; + } + priv->mon = mon; + if (virSecurityManagerClearSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to clear security context for monitor for %s"), vm->def->name); @@ -2484,24 +2499,30 @@ qemuProcessRecoverJob(struct qemud_driver *driver, struct qemuProcessReconnectData { virConnectPtr conn; struct qemud_driver *driver; + void *payload; + struct qemuDomainJobObj oldjob; }; /* * Open an existing VM's monitor, re-detect VCPU threads * and re-reserve the security labels in use */ static void -qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque) +qemuProcessReconnect(void *opaque) { - virDomainObjPtr obj = payload; struct qemuProcessReconnectData *data = opaque; struct qemud_driver *driver = data->driver; + virDomainObjPtr obj = data->payload; qemuDomainObjPrivatePtr priv; virConnectPtr conn = data->conn; struct qemuDomainJobObj oldjob; + memcpy(&oldjob, &data->oldjob, sizeof(oldjob)); + + VIR_FREE(data); + + qemuDriverLock(driver); virDomainObjLock(obj); - qemuDomainObjRestoreJob(obj, &oldjob); VIR_DEBUG("Reconnect monitor to %p '%s'", obj, obj->def->name); @@ -2572,12 +2593,21 @@ qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opa if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + if (qemuDomainObjEndJob(driver, obj) == 0) + obj = NULL; + + qemuDriverUnlock(driver); + return; error: + if (qemuDomainObjEndJob(driver, obj) == 0) + obj = NULL; + if (!virDomainObjIsActive(obj)) { if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); return; } @@ -2591,6 +2621,81 @@ error: else virDomainObjUnlock(obj); } + qemuDriverUnlock(driver); +} + +static void +qemuProcessReconnectHelper(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virThread thread; + struct qemuProcessReconnectData *src = opaque; + struct qemuProcessReconnectData *data; + virDomainObjPtr obj = payload; + + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + return; + } + + memcpy(data, src, sizeof(*data)); + data->payload = payload; + + /* This iterator is called with driver being locked. + * We create a separate thread to run qemuProcessReconnect in it. + * However, qemuProcessReconnect needs to: + * 1. lock driver + * 2. just before monitor reconnect do lightweight MonitorEnter + * (increase VM refcount, unlock VM & driver) + * 3. reconnect to monitor + * 4. do lightweight MonitorExit (lock driver & VM) + * 5. continue reconnect process + * 6. EndJob + * 7. unlock driver + * + * It is necessary to NOT hold driver lock for the entire run + * of reconnect, otherwise we will get blocked if there is + * unresponsive qemu. + * However, iterating over hash table MUST be done on locked + * driver. + * + * NB, we can't do normal MonitorEnter & MonitorExit because + * these two lock the monitor lock, which does not exists in + * this early phase. + */ + + virDomainObjLock(obj); + + qemuDomainObjRestoreJob(obj, &data->oldjob); + + if (qemuDomainObjBeginJobWithDriver(src->driver, obj, QEMU_JOB_MODIFY) < 0) + goto error; + + if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not create thread. QEMU initialization " + "might be incomplete")); + if (qemuDomainObjEndJob(src->driver, obj) == 0) { + obj = NULL; + } else if (virDomainObjUnref(obj) > 0) { + /* We can't spawn a thread and thus connect to monitor. + * Kill qemu */ + qemuProcessStop(src->driver, obj, 0, VIR_DOMAIN_SHUTOFF_FAILED); + if (!obj->persistent) + virDomainRemoveInactive(&src->driver->domains, obj); + else + virDomainObjUnlock(obj); + } + goto error; + } + + virDomainObjUnlock(obj); + + return; + +error: + VIR_FREE(data); } /** @@ -2603,7 +2708,7 @@ void qemuProcessReconnectAll(virConnectPtr conn, struct qemud_driver *driver) { struct qemuProcessReconnectData data = {conn, driver}; - virHashForEach(driver->domains.objs, qemuProcessReconnect, &data); + virHashForEach(driver->domains.objs, qemuProcessReconnectHelper, &data); } int qemuProcessStart(virConnectPtr conn, -- 1.7.3.4

On Tue, Aug 23, 2011 at 08:22:18PM +0200, Michal Privoznik wrote:
If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. --- src/qemu/qemu_driver.c | 23 +++++---- src/qemu/qemu_process.c | 123 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 127 insertions(+), 19 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index c8dda73..8c2e356 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -143,16 +143,18 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq
virDomainObjLock(vm); virResetLastError(); - if (qemuDomainObjBeginJobWithDriver(data->driver, vm, - QEMU_JOB_MODIFY) < 0) { - err = virGetLastError(); - VIR_ERROR(_("Failed to start job on VM '%s': %s"), - vm->def->name, - err ? err->message : _("unknown error")); - } else { - if (vm->autostart && - !virDomainObjIsActive(vm) && - qemuDomainObjStart(data->conn, data->driver, vm, + if (vm->autostart && + !virDomainObjIsActive(vm)) { + if (qemuDomainObjBeginJobWithDriver(data->driver, vm, + QEMU_JOB_MODIFY) < 0) { + err = virGetLastError(); + VIR_ERROR(_("Failed to start job on VM '%s': %s"), + vm->def->name, + err ? err->message : _("unknown error")); + goto cleanup; + } + + if (qemuDomainObjStart(data->conn, data->driver, vm, false, false, data->driver->autoStartBypassCache) < 0) { err = virGetLastError(); @@ -165,6 +167,7 @@ qemuAutostartDomain(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaq vm = NULL; }
+cleanup: if (vm) virDomainObjUnlock(vm); } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 21e73a5..4cc8ae6 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -820,6 +820,7 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; int ret = -1; + qemuMonitorPtr mon = NULL;
if (virSecurityManagerSetSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to set security context for monitor for %s"), @@ -831,15 +832,29 @@ qemuConnectMonitor(struct qemud_driver *driver, virDomainObjPtr vm) * deleted while the monitor is active */ virDomainObjRef(vm);
- priv->mon = qemuMonitorOpen(vm, - priv->monConfig, - priv->monJSON, - &monitorCallbacks); + ignore_value(virTimeMs(&priv->monStart)); + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + + mon = qemuMonitorOpen(vm, + priv->monConfig, + priv->monJSON, + &monitorCallbacks); + + qemuDriverLock(driver); + virDomainObjLock(vm); + priv->monStart = 0;
/* Safe to ignore value since ref count was incremented above */ - if (priv->mon == NULL) + if (mon == NULL) ignore_value(virDomainObjUnref(vm));
+ if (!virDomainObjIsActive(vm)) { + qemuMonitorClose(mon); + goto error; + } + priv->mon = mon; + if (virSecurityManagerClearSocketLabel(driver->securityManager, vm) < 0) { VIR_ERROR(_("Failed to clear security context for monitor for %s"), vm->def->name); @@ -2484,24 +2499,30 @@ qemuProcessRecoverJob(struct qemud_driver *driver, struct qemuProcessReconnectData { virConnectPtr conn; struct qemud_driver *driver; + void *payload; + struct qemuDomainJobObj oldjob; }; /* * Open an existing VM's monitor, re-detect VCPU threads * and re-reserve the security labels in use */ static void -qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque) +qemuProcessReconnect(void *opaque) { - virDomainObjPtr obj = payload; struct qemuProcessReconnectData *data = opaque; struct qemud_driver *driver = data->driver; + virDomainObjPtr obj = data->payload; qemuDomainObjPrivatePtr priv; virConnectPtr conn = data->conn; struct qemuDomainJobObj oldjob;
+ memcpy(&oldjob, &data->oldjob, sizeof(oldjob)); + + VIR_FREE(data); + + qemuDriverLock(driver); virDomainObjLock(obj);
- qemuDomainObjRestoreJob(obj, &oldjob);
VIR_DEBUG("Reconnect monitor to %p '%s'", obj, obj->def->name);
@@ -2572,12 +2593,21 @@ qemuProcessReconnect(void *payload, const void *name ATTRIBUTE_UNUSED, void *opa if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj);
+ if (qemuDomainObjEndJob(driver, obj) == 0) + obj = NULL; + + qemuDriverUnlock(driver); + return;
error: + if (qemuDomainObjEndJob(driver, obj) == 0) + obj = NULL; + if (!virDomainObjIsActive(obj)) { if (virDomainObjUnref(obj) > 0) virDomainObjUnlock(obj); + qemuDriverUnlock(driver); return; }
@@ -2591,6 +2621,81 @@ error: else virDomainObjUnlock(obj); } + qemuDriverUnlock(driver); +} + +static void +qemuProcessReconnectHelper(void *payload, + const void *name ATTRIBUTE_UNUSED, + void *opaque) +{ + virThread thread; + struct qemuProcessReconnectData *src = opaque; + struct qemuProcessReconnectData *data; + virDomainObjPtr obj = payload; + + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + return; + } + + memcpy(data, src, sizeof(*data)); + data->payload = payload; + + /* This iterator is called with driver being locked. + * We create a separate thread to run qemuProcessReconnect in it. + * However, qemuProcessReconnect needs to: + * 1. lock driver + * 2. just before monitor reconnect do lightweight MonitorEnter + * (increase VM refcount, unlock VM & driver) + * 3. reconnect to monitor + * 4. do lightweight MonitorExit (lock driver & VM) + * 5. continue reconnect process + * 6. EndJob + * 7. unlock driver + * + * It is necessary to NOT hold driver lock for the entire run + * of reconnect, otherwise we will get blocked if there is + * unresponsive qemu. + * However, iterating over hash table MUST be done on locked + * driver. + * + * NB, we can't do normal MonitorEnter & MonitorExit because + * these two lock the monitor lock, which does not exists in + * this early phase. + */ + + virDomainObjLock(obj); + + qemuDomainObjRestoreJob(obj, &data->oldjob); + + if (qemuDomainObjBeginJobWithDriver(src->driver, obj, QEMU_JOB_MODIFY) < 0) + goto error; + + if (virThreadCreate(&thread, true, qemuProcessReconnect, data) < 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Could not create thread. QEMU initialization " + "might be incomplete")); + if (qemuDomainObjEndJob(src->driver, obj) == 0) { + obj = NULL; + } else if (virDomainObjUnref(obj) > 0) { + /* We can't spawn a thread and thus connect to monitor. + * Kill qemu */ + qemuProcessStop(src->driver, obj, 0, VIR_DOMAIN_SHUTOFF_FAILED); + if (!obj->persistent) + virDomainRemoveInactive(&src->driver->domains, obj); + else + virDomainObjUnlock(obj); + } + goto error; + } + + virDomainObjUnlock(obj); + + return; + +error: + VIR_FREE(data); }
/** @@ -2603,7 +2708,7 @@ void qemuProcessReconnectAll(virConnectPtr conn, struct qemud_driver *driver) { struct qemuProcessReconnectData data = {conn, driver}; - virHashForEach(driver->domains.objs, qemuProcessReconnect, &data); + virHashForEach(driver->domains.objs, qemuProcessReconnectHelper, &data); }
int qemuProcessStart(virConnectPtr conn,
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy). For high priority calls (HPC), there is new thread pool created. HPC tries to land in usual thread pool firstly. The condition here is it contains at least one free worker. As soon as it doesn't, HPCs are placed into the new pool. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC. The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5. To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. --- diff to v2: - leave the idea of 2 separate pools, and rewrite threadpool to allow priority workers. These can work only on priority jobs. Normal workers can work on both normal & priority job. Job is inherited from HPC. - Introduce jobQueueDepth to heal harmless bug with freeWorkers equal zero. - Leave out callback for proc priority. As this information is in program proc table. However, when queueing job, we didn't know to which program is job related. Program was looked up in job handling, so after queuing. Move this bit a step forward. NB: the current division of calls to HPC and normal ones I present here may be not quite precise enough. So any feedback is more than welcome. daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 7 + daemon/libvirtd.conf | 6 + src/qemu/qemu_driver.c | 2 +- src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 +- src/remote/remote_protocol.x | 544 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 ++- src/rpc/virnetserver.c | 61 +++-- src/rpc/virnetserver.h | 1 + src/rpc/virnetserverprogram.c | 11 + src/rpc/virnetserverprogram.h | 4 + src/util/threadpool.c | 158 ++++++++++-- src/util/threadpool.h | 4 +- 14 files changed, 507 insertions(+), 327 deletions(-) diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers" let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 423c3d7..7b445a2 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients; + int prio_workers; + int max_requests; int max_client_requests; @@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20; + data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5; @@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients); + GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests); @@ -1430,6 +1436,7 @@ int main(int argc, char **argv) { config->auth_unix_ro == REMOTE_AUTH_POLKIT; if (!(srv = virNetServerNew(config->min_workers, config->max_workers, + config->prio_workers, config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20 + +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b5268e4..c8bd4a8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -656,7 +656,7 @@ qemudStartup(int privileged) { virHashForEach(qemu_driver->domains.objs, qemuDomainSnapshotLoad, qemu_driver->snapshotDir); - qemu_driver->workerPool = virThreadPoolNew(0, 1, processWatchdogEvent, qemu_driver); + qemu_driver->workerPool = virThreadPoolNew(0, 1, 0, processWatchdogEvent, qemu_driver); if (!qemu_driver->workerPool) goto error; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f691bbb..146ba12 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, 0, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1; enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..787b409 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,290 +2201,300 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1; enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ - REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ - REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ - REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */ - - REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ - REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ - REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ - - REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ - REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */ - - REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ - REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ - REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ - REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen */ - REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen */ - REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ - REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ - REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ - REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen */ - REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 */ - REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ - REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen */ - - REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen */ - REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen */ - REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */ - - REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ - REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ + REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:low */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen priority:high */ + + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen priority:low */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:low */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:low */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:low */ + REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:low */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:low */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen priority:low */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */ + + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen priority:high */ + + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:low */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:low */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen priority:low */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 priority:low */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:low */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen priority:high */ + + REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen priority:low */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen priority:low */ + + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen priority:low */ /* * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..1ca3bc0 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { } if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\s+priority:(\S+)\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" } my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = $4; if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +172,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + $calls{$name}->{priority} = 0; + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } } $calls[$id] = $calls{$name}; @@ -259,6 +270,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } } @@ -927,7 +939,7 @@ elsif ($opt_b) { print "virNetServerProgramProc ${structprefix}Procs[] = {\n"; for ($id = 0 ; $id <= $#calls ; $id++) { - my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter); + my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter, $priority); if (defined $calls[$id] && !$calls[$id]->{msg}) { $comment = "/* Method $calls[$id]->{ProcName} => $id */"; @@ -950,7 +962,9 @@ elsif ($opt_b) { $retfilter = "xdr_void"; } - print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true \n},\n"; + $priority = defined $calls[$id]->{priority} ? $calls[$id]->{priority} : 0; + + print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true,\n $priority\n},\n"; } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..ab964e3 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -64,6 +64,7 @@ typedef virNetServerJob *virNetServerJobPtr; struct _virNetServerJob { virNetServerClientPtr client; virNetMessagePtr msg; + virNetServerProgramPtr prog; }; struct _virNetServer { @@ -128,38 +129,18 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job = jobOpaque; - virNetServerProgramPtr prog = NULL; - size_t i; - virNetServerLock(srv); VIR_DEBUG("server=%p client=%p message=%p", srv, job->client, job->msg); - for (i = 0 ; i < srv->nprograms ; i++) { - if (virNetServerProgramMatches(srv->programs[i], job->msg)) { - prog = srv->programs[i]; - break; - } - } - - if (!prog) { - VIR_DEBUG("Cannot find program %d version %d", - job->msg->header.prog, - job->msg->header.vers); - goto error; - } - - virNetServerProgramRef(prog); - virNetServerUnlock(srv); - - if (virNetServerProgramDispatch(prog, + if (virNetServerProgramDispatch(job->prog, srv, job->client, job->msg) < 0) goto error; virNetServerLock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetServerUnlock(srv); virNetServerClientFree(job->client); @@ -168,21 +149,23 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) error: virNetServerUnlock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetMessageFree(job->msg); virNetServerClientClose(job->client); virNetServerClientFree(job->client); VIR_FREE(job); } - static int virNetServerDispatchNewMessage(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job; - int ret; + virNetServerProgramPtr prog = NULL; + unsigned int priority = 0; + size_t i; + int ret = -1; VIR_DEBUG("server=%p client=%p message=%p", srv, client, msg); @@ -196,8 +179,32 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, job->msg = msg; virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + for (i = 0 ; i < srv->nprograms ; i++) { + if (virNetServerProgramMatches(srv->programs[i], job->msg)) { + prog = srv->programs[i]; + break; + } + } + + if (!prog) { + VIR_DEBUG("Cannot find program %d version %d", + job->msg->header.prog, + job->msg->header.vers); + goto cleanup; + } + + virNetServerProgramRef(prog); + job->prog = prog; + priority = virNetServerProgramGetPriority(prog, msg->header.proc); + + ret = virThreadPoolSendJob(srv->workers, priority, job); + +cleanup: + if (ret < 0) { VIR_FREE(job); + if (prog) + virNetServerProgramFree(prog); + } virNetServerUnlock(srv); return ret; @@ -274,6 +281,7 @@ static void virNetServerFatalSignal(int sig, siginfo_t *siginfo ATTRIBUTE_UNUSED virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, @@ -290,6 +298,7 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv->refs = 1; if (!(srv->workers = virThreadPoolNew(min_workers, max_workers, + priority_workers, virNetServerHandleJob, srv))) goto error; diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..cc9d039 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -39,6 +39,7 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv, virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index ca80ae0..33e036d 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -108,6 +108,17 @@ static virNetServerProgramProcPtr virNetServerProgramGetProc(virNetServerProgram return &prog->procs[procedure]; } +unsigned int +virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure) +{ + virNetServerProgramProcPtr proc = virNetServerProgramGetProc(prog, procedure); + + if (!proc) + return 0; + + return proc->priority; +} static int virNetServerProgramSendError(virNetServerProgramPtr prog, diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index ca31b7e..4f931ff 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -53,6 +53,7 @@ struct _virNetServerProgramProc { size_t ret_len; xdrproc_t ret_filter; bool needAuth; + unsigned int priority; }; virNetServerProgramPtr virNetServerProgramNew(unsigned program, @@ -63,6 +64,9 @@ virNetServerProgramPtr virNetServerProgramNew(unsigned program, int virNetServerProgramGetID(virNetServerProgramPtr prog); int virNetServerProgramGetVersion(virNetServerProgramPtr prog); +unsigned int virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure); + void virNetServerProgramRef(virNetServerProgramPtr prog); int virNetServerProgramMatches(virNetServerProgramPtr prog, diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..70a75c0 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -37,7 +37,9 @@ typedef struct _virThreadPoolJob virThreadPoolJob; typedef virThreadPoolJob *virThreadPoolJobPtr; struct _virThreadPoolJob { + virThreadPoolJobPtr prev; virThreadPoolJobPtr next; + unsigned int priority; void *data; }; @@ -47,7 +49,8 @@ typedef virThreadPoolJobList *virThreadPoolJobListPtr; struct _virThreadPoolJobList { virThreadPoolJobPtr head; - virThreadPoolJobPtr *tail; + virThreadPoolJobPtr tail; + virThreadPoolJobPtr firstPrio; }; @@ -57,6 +60,7 @@ struct _virThreadPool { virThreadPoolJobFunc jobFunc; void *jobOpaque; virThreadPoolJobList jobList; + size_t jobQueueDepth; virMutex mutex; virCond cond; @@ -66,33 +70,75 @@ struct _virThreadPool { size_t freeWorkers; size_t nWorkers; virThreadPtr workers; + + size_t nPrioWorkers; + virThreadPtr prioWorkers; + virCond prioCond; +}; + +struct virThreadPoolWorkerData { + virThreadPoolPtr pool; + virCondPtr cond; + bool priority; }; static void virThreadPoolWorker(void *opaque) { - virThreadPoolPtr pool = opaque; + struct virThreadPoolWorkerData *data = opaque; + virThreadPoolPtr pool = data->pool; + virCondPtr cond = data->cond; + bool priority = data->priority; + virThreadPoolJobPtr job = NULL; + + VIR_FREE(data); virMutexLock(&pool->mutex); while (1) { while (!pool->quit && - !pool->jobList.head) { - pool->freeWorkers++; - if (virCondWait(&pool->cond, &pool->mutex) < 0) { - pool->freeWorkers--; + ((!priority && !pool->jobList.head) || + (priority && !pool->jobList.firstPrio))) { + if (!priority) + pool->freeWorkers++; + if (virCondWait(cond, &pool->mutex) < 0) { + if (!priority) + pool->freeWorkers--; goto out; } - pool->freeWorkers--; + if (!priority) + pool->freeWorkers--; } if (pool->quit) break; - virThreadPoolJobPtr job = pool->jobList.head; - pool->jobList.head = pool->jobList.head->next; - job->next = NULL; - if (pool->jobList.tail == &job->next) - pool->jobList.tail = &pool->jobList.head; + if (priority) { + job = pool->jobList.firstPrio; + } else { + job = pool->jobList.head; + } + + if (job == pool->jobList.firstPrio) { + virThreadPoolJobPtr tmp = job->next; + while (tmp) { + if (tmp->priority) { + break; + } + tmp = tmp->next; + } + pool->jobList.firstPrio = tmp; + } + + if (job->prev) + job->prev->next = job->next; + else + pool->jobList.head = job->next; + if (job->next) + job->next->prev = job->prev; + else + pool->jobList.tail = job->prev; + + pool->jobQueueDepth--; virMutexUnlock(&pool->mutex); (pool->jobFunc)(job->data, pool->jobOpaque); @@ -101,19 +147,24 @@ static void virThreadPoolWorker(void *opaque) } out: - pool->nWorkers--; - if (pool->nWorkers == 0) + if (priority) + pool->nPrioWorkers--; + else + pool->nWorkers--; + if (pool->nWorkers == 0 && pool->nPrioWorkers==0) virCondSignal(&pool->quit_cond); virMutexUnlock(&pool->mutex); } virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, void *opaque) { virThreadPoolPtr pool; size_t i; + struct virThreadPoolWorkerData *data = NULL; if (minWorkers > maxWorkers) minWorkers = maxWorkers; @@ -123,8 +174,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, return NULL; } - pool->jobList.head = NULL; - pool->jobList.tail = &pool->jobList.head; + pool->jobList.tail = pool->jobList.head = NULL; pool->jobFunc = func; pool->jobOpaque = opaque; @@ -141,18 +191,51 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, pool->maxWorkers = maxWorkers; for (i = 0; i < minWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->cond; + if (virThreadCreate(&pool->workers[i], true, virThreadPoolWorker, - pool) < 0) { + data) < 0) { goto error; } pool->nWorkers++; } + if (prioWorkers) { + if (virCondInit(&pool->prioCond) < 0) + goto error; + if (VIR_ALLOC_N(pool->prioWorkers, prioWorkers) < 0) + goto error; + + for (i = 0; i < prioWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->prioCond; + data->priority = true; + + if (virThreadCreate(&pool->prioWorkers[i], + true, + virThreadPoolWorker, + data) < 0) { + goto error; + } + pool->nPrioWorkers++; + } + } + return pool; error: + VIR_FREE(data); virThreadPoolFree(pool); return NULL; @@ -161,17 +244,22 @@ error: void virThreadPoolFree(virThreadPoolPtr pool) { virThreadPoolJobPtr job; + bool priority = false; if (!pool) return; virMutexLock(&pool->mutex); pool->quit = true; - if (pool->nWorkers > 0) { + if (pool->nWorkers > 0) virCondBroadcast(&pool->cond); - ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + if (pool->nPrioWorkers > 0) { + priority = true; + virCondBroadcast(&pool->prioCond); } + ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + while ((job = pool->jobList.head)) { pool->jobList.head = pool->jobList.head->next; VIR_FREE(job); @@ -182,10 +270,19 @@ void virThreadPoolFree(virThreadPoolPtr pool) virMutexDestroy(&pool->mutex); ignore_value(virCondDestroy(&pool->quit_cond)); ignore_value(virCondDestroy(&pool->cond)); + if (priority) { + VIR_FREE(pool->prioWorkers); + ignore_value(virCondDestroy(&pool->prioCond)); + } VIR_FREE(pool); } +/* + * @priority - job priority + * Return: 0 on success, -1 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobData) { virThreadPoolJobPtr job; @@ -194,7 +291,7 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, if (pool->quit) goto error; - if (pool->freeWorkers == 0 && + if (pool->freeWorkers - pool->jobQueueDepth <= 0 && pool->nWorkers < pool->maxWorkers) { if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { virReportOOMError(); @@ -216,13 +313,26 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, } job->data = jobData; - job->next = NULL; - *pool->jobList.tail = job; - pool->jobList.tail = &(*pool->jobList.tail)->next; + job->priority = priority; + + job->prev = pool->jobList.tail; + if (pool->jobList.tail) + pool->jobList.tail->next = job; + pool->jobList.tail = job; + + if (!pool->jobList.head) + pool->jobList.head = job; + + if (priority && !pool->jobList.firstPrio) + pool->jobList.firstPrio = job; + + pool->jobQueueDepth++; virCondSignal(&pool->cond); - virMutexUnlock(&pool->mutex); + if (priority) + virCondSignal(&pool->prioCond); + virMutexUnlock(&pool->mutex); return 0; error: diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..c4612ad 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -35,12 +35,14 @@ typedef void (*virThreadPoolJobFunc)(void *jobdata, void *opaque); virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, - void *opaque) ATTRIBUTE_NONNULL(3); + void *opaque) ATTRIBUTE_NONNULL(4); void virThreadPoolFree(virThreadPoolPtr pool); int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; -- 1.7.3.4

On Fri, Aug 26, 2011 at 11:34:27AM +0200, Michal Privoznik wrote:
diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1;
enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */
How about we make the last argument optional, so we only have to include it if we want high priority. It is easy enough todo in the RPC parser.
+ REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ + REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen priority:low */ + REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:low */
high
+ REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen priority:high */
low
+ REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen priority:high */
low
+ REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen priority:low */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:low */
high
+ REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:low */
high
+ REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:low */
high
+ REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:low */
high
+ REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen priority:low */ + REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen priority:low */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */ + + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen priority:high */
low
+ REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen priority:high */
low
+ REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen priority:high */
low
+ REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen priority:high */
low
+ + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen priority:high */
low
+ REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen priority:high */
low
+ + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen priority:high */
low
+ REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen priority:low */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:low */
high
+ REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:low */
high
+ + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:low */
high
+ REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:low */
high
+ REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:low */
high
+ REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:low */
high
+ REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_DETACH_DEVICE_FLAGS = 161, /* autogen autogen priority:low */ + REMOTE_PROC_CPU_BASELINE = 162, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_JOB_INFO = 163, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:low */
high
+ REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen priority:low */
n/a
+ + REMOTE_PROC_DOMAIN_EVENT_WATCHDOG = 171, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen priority:low */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_REVERT_TO_SNAPSHOT = 192, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SNAPSHOT_DELETE = 193, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_INFO = 194, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /* autogen autogen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_MEMORY_PARAMETERS = 197, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_MEMORY_PARAMETERS = 198, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS = 199, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 priority:low */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:low */
high
+ REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207, /* autogen autogen priority:low */ + REMOTE_PROC_STORAGE_VOL_UPLOAD = 208, /* autogen autogen | writestream@1 priority:low */ + REMOTE_PROC_STORAGE_VOL_DOWNLOAD = 209, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 priority:low */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* autogen autogen priority:low */ + REMOTE_PROC_INTERFACE_CHANGE_BEGIN = 220, /* autogen autogen priority:high */
low
+ + REMOTE_PROC_INTERFACE_CHANGE_COMMIT = 221, /* autogen autogen priority:high */
low
+ REMOTE_PROC_INTERFACE_CHANGE_ROLLBACK = 222, /* autogen autogen priority:high */
low
+ REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS_FLAGS = 223, /* skipgen autogen priority:low */ + REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen priority:low */
n/a
+ REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen priority:low */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen priority:low */ + + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:low */
high
+ REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen priority:low */ + REMOTE_PROC_DOMAIN_BLOCK_PULL = 240, /* autogen autogen priority:low */ + + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB = 241 /* skipgen skipgen priority:low */
diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..1ca3bc0 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { }
if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\s+priority:(\S+)\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" }
To make priority optional use: if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)(\s+priority:(\S+)\s*\*\/\s*$/)?)) {
my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = $4;
And change to: my $priority = defined $4 ? $4 : "low"; thus we can leave out all the priority:low annotations by default The rest of the patch looks fine to me Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On 08/31/2011 04:45 AM, Daniel P. Berrange wrote:
+ REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */
high
Really, even though it might require a call to an external process like iscsiadm?
+ REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */
high
Same question?
+ REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */
high
Really, even though it might require a call to an external process like qemu-img? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org

On Wed, Aug 31, 2011 at 08:59:39AM -0600, Eric Blake wrote:
On 08/31/2011 04:45 AM, Daniel P. Berrange wrote:
+ REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:low */
high
Really, even though it might require a call to an external process like iscsiadm?
+ REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:low */
high
Same question?
+ REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:low */
high
Really, even though it might require a call to an external process like qemu-img?
None of those 3 APIs need to call out to any external process. They all just work off cached data. It is only refreshed when doing an explit virStoragePoolRefresh Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy). For high priority calls (HPC), there are some high priority workers (HPW) created in the pool. HPW can execute only HPC, although normal worker can process any call regardless priority. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC. The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5. To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. If not marked, the generator assumes low as default. --- diff to v3: -make 'priority:low' as default which can be left out -rearrange some APIs to be HPC or LPC daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 7 + daemon/libvirtd.conf | 6 + src/qemu/qemu_driver.c | 2 +- src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 ++- src/remote/remote_protocol.x | 294 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 +++- src/rpc/virnetserver.c | 60 +++++---- src/rpc/virnetserver.h | 1 + src/rpc/virnetserverprogram.c | 11 ++ src/rpc/virnetserverprogram.h | 4 + src/util/threadpool.c | 158 +++++++++++++++++++---- src/util/threadpool.h | 4 +- 14 files changed, 382 insertions(+), 201 deletions(-) diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers" let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 423c3d7..7b445a2 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients; + int prio_workers; + int max_requests; int max_client_requests; @@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20; + data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5; @@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients); + GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests); @@ -1430,6 +1436,7 @@ int main(int argc, char **argv) { config->auth_unix_ro == REMOTE_AUTH_POLKIT; if (!(srv = virNetServerNew(config->min_workers, config->max_workers, + config->prio_workers, config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20 + +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f21122d..b26a0cb 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -656,7 +656,7 @@ qemudStartup(int privileged) { virHashForEach(qemu_driver->domains.objs, qemuDomainSnapshotLoad, qemu_driver->snapshotDir); - qemu_driver->workerPool = virThreadPoolNew(0, 1, processWatchdogEvent, qemu_driver); + qemu_driver->workerPool = virThreadPoolNew(0, 1, 0, processWatchdogEvent, qemu_driver); if (!qemu_driver->workerPool) goto error; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 58b4d36..878805a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, 0, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1; enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..6ad83f3 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,189 +2201,195 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1; enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:high */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */ - REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:high */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:high */ REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:high */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:high */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */ - REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:high */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:high */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:high */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */ @@ -2393,8 +2399,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */ @@ -2402,22 +2408,22 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */ REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ @@ -2432,8 +2438,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */ REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:high */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ @@ -2443,7 +2449,7 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */ REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ @@ -2459,17 +2465,17 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ @@ -2481,10 +2487,16 @@ enum remote_procedure { * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..38cbb65 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { } if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(\|.*)?\s+(priority:(\S+))?\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" } my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = defined $5 ? $5 : "low"; if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +172,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + $calls{$name}->{priority} = 0; + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } } $calls[$id] = $calls{$name}; @@ -259,6 +270,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } } @@ -927,7 +939,7 @@ elsif ($opt_b) { print "virNetServerProgramProc ${structprefix}Procs[] = {\n"; for ($id = 0 ; $id <= $#calls ; $id++) { - my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter); + my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter, $priority); if (defined $calls[$id] && !$calls[$id]->{msg}) { $comment = "/* Method $calls[$id]->{ProcName} => $id */"; @@ -950,7 +962,9 @@ elsif ($opt_b) { $retfilter = "xdr_void"; } - print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true \n},\n"; + $priority = defined $calls[$id]->{priority} ? $calls[$id]->{priority} : 0; + + print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true,\n $priority\n},\n"; } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..bc9fa89 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -64,6 +64,7 @@ typedef virNetServerJob *virNetServerJobPtr; struct _virNetServerJob { virNetServerClientPtr client; virNetMessagePtr msg; + virNetServerProgramPtr prog; }; struct _virNetServer { @@ -128,38 +129,18 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job = jobOpaque; - virNetServerProgramPtr prog = NULL; - size_t i; - virNetServerLock(srv); VIR_DEBUG("server=%p client=%p message=%p", srv, job->client, job->msg); - for (i = 0 ; i < srv->nprograms ; i++) { - if (virNetServerProgramMatches(srv->programs[i], job->msg)) { - prog = srv->programs[i]; - break; - } - } - - if (!prog) { - VIR_DEBUG("Cannot find program %d version %d", - job->msg->header.prog, - job->msg->header.vers); - goto error; - } - - virNetServerProgramRef(prog); - virNetServerUnlock(srv); - - if (virNetServerProgramDispatch(prog, + if (virNetServerProgramDispatch(job->prog, srv, job->client, job->msg) < 0) goto error; virNetServerLock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetServerUnlock(srv); virNetServerClientFree(job->client); @@ -168,21 +149,23 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) error: virNetServerUnlock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetMessageFree(job->msg); virNetServerClientClose(job->client); virNetServerClientFree(job->client); VIR_FREE(job); } - static int virNetServerDispatchNewMessage(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job; - int ret; + virNetServerProgramPtr prog = NULL; + unsigned int priority = 0; + size_t i; + int ret = -1; VIR_DEBUG("server=%p client=%p message=%p", srv, client, msg); @@ -196,8 +179,31 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, job->msg = msg; virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + for (i = 0 ; i < srv->nprograms ; i++) { + if (virNetServerProgramMatches(srv->programs[i], job->msg)) { + prog = srv->programs[i]; + break; + } + } + + if (!prog) { + VIR_DEBUG("Cannot find program %d version %d", + job->msg->header.prog, + job->msg->header.vers); + goto cleanup; + } + + virNetServerProgramRef(prog); + job->prog = prog; + priority = virNetServerProgramGetPriority(prog, msg->header.proc); + + ret = virThreadPoolSendJob(srv->workers, priority, job); + +cleanup: + if (ret < 0) { VIR_FREE(job); + virNetServerProgramFree(prog); + } virNetServerUnlock(srv); return ret; @@ -274,6 +280,7 @@ static void virNetServerFatalSignal(int sig, siginfo_t *siginfo ATTRIBUTE_UNUSED virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, @@ -290,6 +297,7 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv->refs = 1; if (!(srv->workers = virThreadPoolNew(min_workers, max_workers, + priority_workers, virNetServerHandleJob, srv))) goto error; diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..cc9d039 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -39,6 +39,7 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv, virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index ca80ae0..33e036d 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -108,6 +108,17 @@ static virNetServerProgramProcPtr virNetServerProgramGetProc(virNetServerProgram return &prog->procs[procedure]; } +unsigned int +virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure) +{ + virNetServerProgramProcPtr proc = virNetServerProgramGetProc(prog, procedure); + + if (!proc) + return 0; + + return proc->priority; +} static int virNetServerProgramSendError(virNetServerProgramPtr prog, diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index ca31b7e..4f931ff 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -53,6 +53,7 @@ struct _virNetServerProgramProc { size_t ret_len; xdrproc_t ret_filter; bool needAuth; + unsigned int priority; }; virNetServerProgramPtr virNetServerProgramNew(unsigned program, @@ -63,6 +64,9 @@ virNetServerProgramPtr virNetServerProgramNew(unsigned program, int virNetServerProgramGetID(virNetServerProgramPtr prog); int virNetServerProgramGetVersion(virNetServerProgramPtr prog); +unsigned int virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure); + void virNetServerProgramRef(virNetServerProgramPtr prog); int virNetServerProgramMatches(virNetServerProgramPtr prog, diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..70a75c0 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -37,7 +37,9 @@ typedef struct _virThreadPoolJob virThreadPoolJob; typedef virThreadPoolJob *virThreadPoolJobPtr; struct _virThreadPoolJob { + virThreadPoolJobPtr prev; virThreadPoolJobPtr next; + unsigned int priority; void *data; }; @@ -47,7 +49,8 @@ typedef virThreadPoolJobList *virThreadPoolJobListPtr; struct _virThreadPoolJobList { virThreadPoolJobPtr head; - virThreadPoolJobPtr *tail; + virThreadPoolJobPtr tail; + virThreadPoolJobPtr firstPrio; }; @@ -57,6 +60,7 @@ struct _virThreadPool { virThreadPoolJobFunc jobFunc; void *jobOpaque; virThreadPoolJobList jobList; + size_t jobQueueDepth; virMutex mutex; virCond cond; @@ -66,33 +70,75 @@ struct _virThreadPool { size_t freeWorkers; size_t nWorkers; virThreadPtr workers; + + size_t nPrioWorkers; + virThreadPtr prioWorkers; + virCond prioCond; +}; + +struct virThreadPoolWorkerData { + virThreadPoolPtr pool; + virCondPtr cond; + bool priority; }; static void virThreadPoolWorker(void *opaque) { - virThreadPoolPtr pool = opaque; + struct virThreadPoolWorkerData *data = opaque; + virThreadPoolPtr pool = data->pool; + virCondPtr cond = data->cond; + bool priority = data->priority; + virThreadPoolJobPtr job = NULL; + + VIR_FREE(data); virMutexLock(&pool->mutex); while (1) { while (!pool->quit && - !pool->jobList.head) { - pool->freeWorkers++; - if (virCondWait(&pool->cond, &pool->mutex) < 0) { - pool->freeWorkers--; + ((!priority && !pool->jobList.head) || + (priority && !pool->jobList.firstPrio))) { + if (!priority) + pool->freeWorkers++; + if (virCondWait(cond, &pool->mutex) < 0) { + if (!priority) + pool->freeWorkers--; goto out; } - pool->freeWorkers--; + if (!priority) + pool->freeWorkers--; } if (pool->quit) break; - virThreadPoolJobPtr job = pool->jobList.head; - pool->jobList.head = pool->jobList.head->next; - job->next = NULL; - if (pool->jobList.tail == &job->next) - pool->jobList.tail = &pool->jobList.head; + if (priority) { + job = pool->jobList.firstPrio; + } else { + job = pool->jobList.head; + } + + if (job == pool->jobList.firstPrio) { + virThreadPoolJobPtr tmp = job->next; + while (tmp) { + if (tmp->priority) { + break; + } + tmp = tmp->next; + } + pool->jobList.firstPrio = tmp; + } + + if (job->prev) + job->prev->next = job->next; + else + pool->jobList.head = job->next; + if (job->next) + job->next->prev = job->prev; + else + pool->jobList.tail = job->prev; + + pool->jobQueueDepth--; virMutexUnlock(&pool->mutex); (pool->jobFunc)(job->data, pool->jobOpaque); @@ -101,19 +147,24 @@ static void virThreadPoolWorker(void *opaque) } out: - pool->nWorkers--; - if (pool->nWorkers == 0) + if (priority) + pool->nPrioWorkers--; + else + pool->nWorkers--; + if (pool->nWorkers == 0 && pool->nPrioWorkers==0) virCondSignal(&pool->quit_cond); virMutexUnlock(&pool->mutex); } virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, void *opaque) { virThreadPoolPtr pool; size_t i; + struct virThreadPoolWorkerData *data = NULL; if (minWorkers > maxWorkers) minWorkers = maxWorkers; @@ -123,8 +174,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, return NULL; } - pool->jobList.head = NULL; - pool->jobList.tail = &pool->jobList.head; + pool->jobList.tail = pool->jobList.head = NULL; pool->jobFunc = func; pool->jobOpaque = opaque; @@ -141,18 +191,51 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, pool->maxWorkers = maxWorkers; for (i = 0; i < minWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->cond; + if (virThreadCreate(&pool->workers[i], true, virThreadPoolWorker, - pool) < 0) { + data) < 0) { goto error; } pool->nWorkers++; } + if (prioWorkers) { + if (virCondInit(&pool->prioCond) < 0) + goto error; + if (VIR_ALLOC_N(pool->prioWorkers, prioWorkers) < 0) + goto error; + + for (i = 0; i < prioWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->prioCond; + data->priority = true; + + if (virThreadCreate(&pool->prioWorkers[i], + true, + virThreadPoolWorker, + data) < 0) { + goto error; + } + pool->nPrioWorkers++; + } + } + return pool; error: + VIR_FREE(data); virThreadPoolFree(pool); return NULL; @@ -161,17 +244,22 @@ error: void virThreadPoolFree(virThreadPoolPtr pool) { virThreadPoolJobPtr job; + bool priority = false; if (!pool) return; virMutexLock(&pool->mutex); pool->quit = true; - if (pool->nWorkers > 0) { + if (pool->nWorkers > 0) virCondBroadcast(&pool->cond); - ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + if (pool->nPrioWorkers > 0) { + priority = true; + virCondBroadcast(&pool->prioCond); } + ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + while ((job = pool->jobList.head)) { pool->jobList.head = pool->jobList.head->next; VIR_FREE(job); @@ -182,10 +270,19 @@ void virThreadPoolFree(virThreadPoolPtr pool) virMutexDestroy(&pool->mutex); ignore_value(virCondDestroy(&pool->quit_cond)); ignore_value(virCondDestroy(&pool->cond)); + if (priority) { + VIR_FREE(pool->prioWorkers); + ignore_value(virCondDestroy(&pool->prioCond)); + } VIR_FREE(pool); } +/* + * @priority - job priority + * Return: 0 on success, -1 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobData) { virThreadPoolJobPtr job; @@ -194,7 +291,7 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, if (pool->quit) goto error; - if (pool->freeWorkers == 0 && + if (pool->freeWorkers - pool->jobQueueDepth <= 0 && pool->nWorkers < pool->maxWorkers) { if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { virReportOOMError(); @@ -216,13 +313,26 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, } job->data = jobData; - job->next = NULL; - *pool->jobList.tail = job; - pool->jobList.tail = &(*pool->jobList.tail)->next; + job->priority = priority; + + job->prev = pool->jobList.tail; + if (pool->jobList.tail) + pool->jobList.tail->next = job; + pool->jobList.tail = job; + + if (!pool->jobList.head) + pool->jobList.head = job; + + if (priority && !pool->jobList.firstPrio) + pool->jobList.firstPrio = job; + + pool->jobQueueDepth++; virCondSignal(&pool->cond); - virMutexUnlock(&pool->mutex); + if (priority) + virCondSignal(&pool->prioCond); + virMutexUnlock(&pool->mutex); return 0; error: diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..c4612ad 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -35,12 +35,14 @@ typedef void (*virThreadPoolJobFunc)(void *jobdata, void *opaque); virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, - void *opaque) ATTRIBUTE_NONNULL(3); + void *opaque) ATTRIBUTE_NONNULL(4); void virThreadPoolFree(virThreadPoolPtr pool); int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; -- 1.7.3.4

On Mon, Sep 05, 2011 at 02:36:13PM +0200, Michal Privoznik wrote:
This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy).
For high priority calls (HPC), there are some high priority workers (HPW) created in the pool. HPW can execute only HPC, although normal worker can process any call regardless priority. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC.
The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5.
To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. If not marked, the generator assumes low as default. --- diff to v3: -make 'priority:low' as default which can be left out -rearrange some APIs to be HPC or LPC
daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 7 + daemon/libvirtd.conf | 6 + src/qemu/qemu_driver.c | 2 +- src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 ++- src/remote/remote_protocol.x | 294 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 +++- src/rpc/virnetserver.c | 60 +++++---- src/rpc/virnetserver.h | 1 + src/rpc/virnetserverprogram.c | 11 ++ src/rpc/virnetserverprogram.h | 4 + src/util/threadpool.c | 158 +++++++++++++++++++---- src/util/threadpool.h | 4 +- 14 files changed, 382 insertions(+), 201 deletions(-)
diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers"
let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 423c3d7..7b445a2 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients;
+ int prio_workers; + int max_requests; int max_client_requests;
@@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20;
+ data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5;
@@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients);
+ GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests);
@@ -1430,6 +1436,7 @@ int main(int argc, char **argv) { config->auth_unix_ro == REMOTE_AUTH_POLKIT; if (!(srv = virNetServerNew(config->min_workers, config->max_workers, + config->prio_workers, config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20
+ +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f21122d..b26a0cb 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -656,7 +656,7 @@ qemudStartup(int privileged) { virHashForEach(qemu_driver->domains.objs, qemuDomainSnapshotLoad, qemu_driver->snapshotDir);
- qemu_driver->workerPool = virThreadPoolNew(0, 1, processWatchdogEvent, qemu_driver); + qemu_driver->workerPool = virThreadPoolNew(0, 1, 0, processWatchdogEvent, qemu_driver); if (!qemu_driver->workerPool) goto error;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 58b4d36..878805a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, 0, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1;
enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..6ad83f3 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,189 +2201,195 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1;
enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */
- REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:high */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:high */
REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */
- REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:high */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:high */
REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:high */ REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:high */
REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:high */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:high */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */
- REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */
- REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */
- REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:high */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:high */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:high */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */
@@ -2393,8 +2399,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */
@@ -2402,22 +2408,22 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */
REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ @@ -2432,8 +2438,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */
REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:high */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ @@ -2443,7 +2449,7 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */
REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ @@ -2459,17 +2465,17 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */
- REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ @@ -2481,10 +2487,16 @@ enum remote_procedure { * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..38cbb65 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { }
if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(\|.*)?\s+(priority:(\S+))?\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" }
my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = defined $5 ? $5 : "low";
if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +172,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + $calls{$name}->{priority} = 0; + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } }
$calls[$id] = $calls{$name}; @@ -259,6 +270,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } }
@@ -927,7 +939,7 @@ elsif ($opt_b) {
print "virNetServerProgramProc ${structprefix}Procs[] = {\n"; for ($id = 0 ; $id <= $#calls ; $id++) { - my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter); + my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter, $priority);
if (defined $calls[$id] && !$calls[$id]->{msg}) { $comment = "/* Method $calls[$id]->{ProcName} => $id */"; @@ -950,7 +962,9 @@ elsif ($opt_b) { $retfilter = "xdr_void"; }
- print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true \n},\n"; + $priority = defined $calls[$id]->{priority} ? $calls[$id]->{priority} : 0; + + print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true,\n $priority\n},\n"; } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..bc9fa89 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -64,6 +64,7 @@ typedef virNetServerJob *virNetServerJobPtr; struct _virNetServerJob { virNetServerClientPtr client; virNetMessagePtr msg; + virNetServerProgramPtr prog; };
struct _virNetServer { @@ -128,38 +129,18 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job = jobOpaque; - virNetServerProgramPtr prog = NULL; - size_t i;
- virNetServerLock(srv); VIR_DEBUG("server=%p client=%p message=%p", srv, job->client, job->msg);
- for (i = 0 ; i < srv->nprograms ; i++) { - if (virNetServerProgramMatches(srv->programs[i], job->msg)) { - prog = srv->programs[i]; - break; - } - } - - if (!prog) { - VIR_DEBUG("Cannot find program %d version %d", - job->msg->header.prog, - job->msg->header.vers); - goto error; - } - - virNetServerProgramRef(prog); - virNetServerUnlock(srv); - - if (virNetServerProgramDispatch(prog, + if (virNetServerProgramDispatch(job->prog, srv, job->client, job->msg) < 0) goto error;
virNetServerLock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetServerUnlock(srv); virNetServerClientFree(job->client);
@@ -168,21 +149,23 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque)
error: virNetServerUnlock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetMessageFree(job->msg); virNetServerClientClose(job->client); virNetServerClientFree(job->client); VIR_FREE(job); }
- static int virNetServerDispatchNewMessage(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job; - int ret; + virNetServerProgramPtr prog = NULL; + unsigned int priority = 0; + size_t i; + int ret = -1;
VIR_DEBUG("server=%p client=%p message=%p", srv, client, msg); @@ -196,8 +179,31 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, job->msg = msg;
virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + for (i = 0 ; i < srv->nprograms ; i++) { + if (virNetServerProgramMatches(srv->programs[i], job->msg)) { + prog = srv->programs[i]; + break; + } + } + + if (!prog) { + VIR_DEBUG("Cannot find program %d version %d", + job->msg->header.prog, + job->msg->header.vers); + goto cleanup; + } + + virNetServerProgramRef(prog); + job->prog = prog; + priority = virNetServerProgramGetPriority(prog, msg->header.proc); + + ret = virThreadPoolSendJob(srv->workers, priority, job); + +cleanup: + if (ret < 0) { VIR_FREE(job); + virNetServerProgramFree(prog); + } virNetServerUnlock(srv);
return ret; @@ -274,6 +280,7 @@ static void virNetServerFatalSignal(int sig, siginfo_t *siginfo ATTRIBUTE_UNUSED
virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, @@ -290,6 +297,7 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv->refs = 1;
if (!(srv->workers = virThreadPoolNew(min_workers, max_workers, + priority_workers, virNetServerHandleJob, srv))) goto error; diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..cc9d039 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -39,6 +39,7 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv,
virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index ca80ae0..33e036d 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -108,6 +108,17 @@ static virNetServerProgramProcPtr virNetServerProgramGetProc(virNetServerProgram return &prog->procs[procedure]; }
+unsigned int +virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure) +{ + virNetServerProgramProcPtr proc = virNetServerProgramGetProc(prog, procedure); + + if (!proc) + return 0; + + return proc->priority; +}
static int virNetServerProgramSendError(virNetServerProgramPtr prog, diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index ca31b7e..4f931ff 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -53,6 +53,7 @@ struct _virNetServerProgramProc { size_t ret_len; xdrproc_t ret_filter; bool needAuth; + unsigned int priority; };
virNetServerProgramPtr virNetServerProgramNew(unsigned program, @@ -63,6 +64,9 @@ virNetServerProgramPtr virNetServerProgramNew(unsigned program, int virNetServerProgramGetID(virNetServerProgramPtr prog); int virNetServerProgramGetVersion(virNetServerProgramPtr prog);
+unsigned int virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure); + void virNetServerProgramRef(virNetServerProgramPtr prog);
int virNetServerProgramMatches(virNetServerProgramPtr prog, diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..70a75c0 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -37,7 +37,9 @@ typedef struct _virThreadPoolJob virThreadPoolJob; typedef virThreadPoolJob *virThreadPoolJobPtr;
struct _virThreadPoolJob { + virThreadPoolJobPtr prev; virThreadPoolJobPtr next; + unsigned int priority;
void *data; }; @@ -47,7 +49,8 @@ typedef virThreadPoolJobList *virThreadPoolJobListPtr;
struct _virThreadPoolJobList { virThreadPoolJobPtr head; - virThreadPoolJobPtr *tail; + virThreadPoolJobPtr tail; + virThreadPoolJobPtr firstPrio; };
@@ -57,6 +60,7 @@ struct _virThreadPool { virThreadPoolJobFunc jobFunc; void *jobOpaque; virThreadPoolJobList jobList; + size_t jobQueueDepth;
virMutex mutex; virCond cond; @@ -66,33 +70,75 @@ struct _virThreadPool { size_t freeWorkers; size_t nWorkers; virThreadPtr workers; + + size_t nPrioWorkers; + virThreadPtr prioWorkers; + virCond prioCond; +}; + +struct virThreadPoolWorkerData { + virThreadPoolPtr pool; + virCondPtr cond; + bool priority; };
static void virThreadPoolWorker(void *opaque) { - virThreadPoolPtr pool = opaque; + struct virThreadPoolWorkerData *data = opaque; + virThreadPoolPtr pool = data->pool; + virCondPtr cond = data->cond; + bool priority = data->priority; + virThreadPoolJobPtr job = NULL; + + VIR_FREE(data);
virMutexLock(&pool->mutex);
while (1) { while (!pool->quit && - !pool->jobList.head) { - pool->freeWorkers++; - if (virCondWait(&pool->cond, &pool->mutex) < 0) { - pool->freeWorkers--; + ((!priority && !pool->jobList.head) || + (priority && !pool->jobList.firstPrio))) { + if (!priority) + pool->freeWorkers++; + if (virCondWait(cond, &pool->mutex) < 0) { + if (!priority) + pool->freeWorkers--; goto out; } - pool->freeWorkers--; + if (!priority) + pool->freeWorkers--; }
if (pool->quit) break;
- virThreadPoolJobPtr job = pool->jobList.head; - pool->jobList.head = pool->jobList.head->next; - job->next = NULL; - if (pool->jobList.tail == &job->next) - pool->jobList.tail = &pool->jobList.head; + if (priority) { + job = pool->jobList.firstPrio; + } else { + job = pool->jobList.head; + } + + if (job == pool->jobList.firstPrio) { + virThreadPoolJobPtr tmp = job->next; + while (tmp) { + if (tmp->priority) { + break; + } + tmp = tmp->next; + } + pool->jobList.firstPrio = tmp; + } + + if (job->prev) + job->prev->next = job->next; + else + pool->jobList.head = job->next; + if (job->next) + job->next->prev = job->prev; + else + pool->jobList.tail = job->prev; + + pool->jobQueueDepth--;
virMutexUnlock(&pool->mutex); (pool->jobFunc)(job->data, pool->jobOpaque); @@ -101,19 +147,24 @@ static void virThreadPoolWorker(void *opaque) }
out: - pool->nWorkers--; - if (pool->nWorkers == 0) + if (priority) + pool->nPrioWorkers--; + else + pool->nWorkers--; + if (pool->nWorkers == 0 && pool->nPrioWorkers==0) virCondSignal(&pool->quit_cond); virMutexUnlock(&pool->mutex); }
virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, void *opaque) { virThreadPoolPtr pool; size_t i; + struct virThreadPoolWorkerData *data = NULL;
if (minWorkers > maxWorkers) minWorkers = maxWorkers; @@ -123,8 +174,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, return NULL; }
- pool->jobList.head = NULL; - pool->jobList.tail = &pool->jobList.head; + pool->jobList.tail = pool->jobList.head = NULL;
pool->jobFunc = func; pool->jobOpaque = opaque; @@ -141,18 +191,51 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers,
pool->maxWorkers = maxWorkers; for (i = 0; i < minWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->cond; + if (virThreadCreate(&pool->workers[i], true, virThreadPoolWorker, - pool) < 0) { + data) < 0) { goto error; } pool->nWorkers++; }
+ if (prioWorkers) { + if (virCondInit(&pool->prioCond) < 0) + goto error; + if (VIR_ALLOC_N(pool->prioWorkers, prioWorkers) < 0) + goto error; + + for (i = 0; i < prioWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->prioCond; + data->priority = true; + + if (virThreadCreate(&pool->prioWorkers[i], + true, + virThreadPoolWorker, + data) < 0) { + goto error; + } + pool->nPrioWorkers++; + } + } + return pool;
error: + VIR_FREE(data); virThreadPoolFree(pool); return NULL;
@@ -161,17 +244,22 @@ error: void virThreadPoolFree(virThreadPoolPtr pool) { virThreadPoolJobPtr job; + bool priority = false;
if (!pool) return;
virMutexLock(&pool->mutex); pool->quit = true; - if (pool->nWorkers > 0) { + if (pool->nWorkers > 0) virCondBroadcast(&pool->cond); - ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + if (pool->nPrioWorkers > 0) { + priority = true; + virCondBroadcast(&pool->prioCond); }
+ ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + while ((job = pool->jobList.head)) { pool->jobList.head = pool->jobList.head->next; VIR_FREE(job); @@ -182,10 +270,19 @@ void virThreadPoolFree(virThreadPoolPtr pool) virMutexDestroy(&pool->mutex); ignore_value(virCondDestroy(&pool->quit_cond)); ignore_value(virCondDestroy(&pool->cond)); + if (priority) { + VIR_FREE(pool->prioWorkers); + ignore_value(virCondDestroy(&pool->prioCond)); + } VIR_FREE(pool); }
+/* + * @priority - job priority + * Return: 0 on success, -1 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobData) { virThreadPoolJobPtr job; @@ -194,7 +291,7 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && + if (pool->freeWorkers - pool->jobQueueDepth <= 0 && pool->nWorkers < pool->maxWorkers) { if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { virReportOOMError(); @@ -216,13 +313,26 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, }
job->data = jobData; - job->next = NULL; - *pool->jobList.tail = job; - pool->jobList.tail = &(*pool->jobList.tail)->next; + job->priority = priority; + + job->prev = pool->jobList.tail; + if (pool->jobList.tail) + pool->jobList.tail->next = job; + pool->jobList.tail = job; + + if (!pool->jobList.head) + pool->jobList.head = job; + + if (priority && !pool->jobList.firstPrio) + pool->jobList.firstPrio = job; + + pool->jobQueueDepth++;
virCondSignal(&pool->cond); - virMutexUnlock(&pool->mutex); + if (priority) + virCondSignal(&pool->prioCond);
+ virMutexUnlock(&pool->mutex); return 0;
error: diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..c4612ad 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -35,12 +35,14 @@ typedef void (*virThreadPoolJobFunc)(void *jobdata, void *opaque);
virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, - void *opaque) ATTRIBUTE_NONNULL(3); + void *opaque) ATTRIBUTE_NONNULL(4);
void virThreadPoolFree(virThreadPoolPtr pool);
int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK;
Okay, the suggesting from Dan are in, ACK, please push :) Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

On 05.09.2011 17:00, Daniel Veillard wrote:
On Mon, Sep 05, 2011 at 02:36:13PM +0200, Michal Privoznik wrote:
This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy).
For high priority calls (HPC), there are some high priority workers (HPW) created in the pool. HPW can execute only HPC, although normal worker can process any call regardless priority. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC.
The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5.
To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. If not marked, the generator assumes low as default. --- diff to v3: -make 'priority:low' as default which can be left out -rearrange some APIs to be HPC or LPC
daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 7 + daemon/libvirtd.conf | 6 + src/qemu/qemu_driver.c | 2 +- src/qemu/qemu_process.c | 2 +- src/remote/qemu_protocol.x | 13 ++- src/remote/remote_protocol.x | 294 +++++++++++++++++++++-------------------- src/rpc/gendispatch.pl | 20 +++- src/rpc/virnetserver.c | 60 +++++---- src/rpc/virnetserver.h | 1 + src/rpc/virnetserverprogram.c | 11 ++ src/rpc/virnetserverprogram.h | 4 + src/util/threadpool.c | 158 +++++++++++++++++++---- src/util/threadpool.h | 4 +- 14 files changed, 382 insertions(+), 201 deletions(-)
diff --git a/daemon/libvirtd.aug b/daemon/libvirtd.aug index 3f47ebb..ce00db5 100644 --- a/daemon/libvirtd.aug +++ b/daemon/libvirtd.aug @@ -57,6 +57,7 @@ module Libvirtd = | int_entry "max_clients" | int_entry "max_requests" | int_entry "max_client_requests" + | int_entry "prio_workers"
let logging_entry = int_entry "log_level" | str_entry "log_filters" diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 423c3d7..7b445a2 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -134,6 +134,8 @@ struct daemonConfig { int max_workers; int max_clients;
+ int prio_workers; + int max_requests; int max_client_requests;
@@ -886,6 +888,8 @@ daemonConfigNew(bool privileged ATTRIBUTE_UNUSED) data->max_workers = 20; data->max_clients = 20;
+ data->prio_workers = 5; + data->max_requests = 20; data->max_client_requests = 5;
@@ -1042,6 +1046,8 @@ daemonConfigLoad(struct daemonConfig *data, GET_CONF_INT (conf, filename, max_workers); GET_CONF_INT (conf, filename, max_clients);
+ GET_CONF_INT (conf, filename, prio_workers); + GET_CONF_INT (conf, filename, max_requests); GET_CONF_INT (conf, filename, max_client_requests);
@@ -1430,6 +1436,7 @@ int main(int argc, char **argv) { config->auth_unix_ro == REMOTE_AUTH_POLKIT; if (!(srv = virNetServerNew(config->min_workers, config->max_workers, + config->prio_workers, config->max_clients, config->mdns_adv ? config->mdns_name : NULL, use_polkit_dbus, diff --git a/daemon/libvirtd.conf b/daemon/libvirtd.conf index 95e43dd..da3983e 100644 --- a/daemon/libvirtd.conf +++ b/daemon/libvirtd.conf @@ -257,6 +257,12 @@ #min_workers = 5 #max_workers = 20
+ +# The number of priority workers. If all workers from above +# pool will stuck, some calls marked as high priority +# (notably domainDestroy) can be executed in this pool. +#prio_workers = 5 + # Total global limit on concurrent RPC calls. Should be # at least as large as max_workers. Beyond this, RPC requests # will be read into memory and queued. This directly impact diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f21122d..b26a0cb 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -656,7 +656,7 @@ qemudStartup(int privileged) { virHashForEach(qemu_driver->domains.objs, qemuDomainSnapshotLoad, qemu_driver->snapshotDir);
- qemu_driver->workerPool = virThreadPoolNew(0, 1, processWatchdogEvent, qemu_driver); + qemu_driver->workerPool = virThreadPoolNew(0, 1, 0, processWatchdogEvent, qemu_driver); if (!qemu_driver->workerPool) goto error;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 58b4d36..878805a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -576,7 +576,7 @@ qemuProcessHandleWatchdog(qemuMonitorPtr mon ATTRIBUTE_UNUSED, * deleted before handling watchdog event is finished. */ virDomainObjRef(vm); - if (virThreadPoolSendJob(driver->workerPool, wdEvent) < 0) { + if (virThreadPoolSendJob(driver->workerPool, 0, wdEvent) < 0) { if (virDomainObjUnref(vm) == 0) vm = NULL; VIR_FREE(wdEvent); diff --git a/src/remote/qemu_protocol.x b/src/remote/qemu_protocol.x index 3279405..39f9adf 100644 --- a/src/remote/qemu_protocol.x +++ b/src/remote/qemu_protocol.x @@ -52,9 +52,14 @@ const QEMU_PROGRAM = 0x20008087; const QEMU_PROTOCOL_VERSION = 1;
enum qemu_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether - * it handles src/remote. */ - QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen */ - QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen */ + * it handles src/remote. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. */ + QEMU_PROC_MONITOR_COMMAND = 1, /* skipgen skipgen priority:low */ + QEMU_PROC_DOMAIN_ATTACH = 2 /* autogen autogen priority:low */ }; diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 8f68808..6ad83f3 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2201,189 +2201,195 @@ const REMOTE_PROGRAM = 0x20008086; const REMOTE_PROTOCOL_VERSION = 1;
enum remote_procedure { - /* Each function must have a two-word comment. The first word is + /* Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point * of view. A readstream transfers data from daemon to src/remote. The * <offset> specifies at which offset the stream parameter is inserted * in the function parameter list. */ - REMOTE_PROC_OPEN = 1, /* skipgen skipgen */ - REMOTE_PROC_CLOSE = 2, /* skipgen skipgen */ - REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen */ - REMOTE_PROC_GET_VERSION = 4, /* autogen autogen */ - REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen */ - REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen */ + REMOTE_PROC_OPEN = 1, /* skipgen skipgen priority:high */ + REMOTE_PROC_CLOSE = 2, /* skipgen skipgen priority:high */ + REMOTE_PROC_GET_TYPE = 3, /* autogen skipgen priority:high */ + REMOTE_PROC_GET_VERSION = 4, /* autogen autogen priority:high */ + REMOTE_PROC_GET_MAX_VCPUS = 5, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_GET_INFO = 6, /* autogen autogen priority:high */ REMOTE_PROC_GET_CAPABILITIES = 7, /* autogen autogen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE = 8, /* autogen autogen */ REMOTE_PROC_DOMAIN_CREATE = 9, /* autogen skipgen */ REMOTE_PROC_DOMAIN_CREATE_XML = 10, /* autogen autogen */
- REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DEFINE_XML = 11, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_DESTROY = 12, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_DETACH_DEVICE = 13, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_XML_DESC = 14, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_AUTOSTART = 15, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_INFO = 16, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen */ - - REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen */ - REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen */ + REMOTE_PROC_DOMAIN_GET_MAX_MEMORY = 17, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_MAX_VCPUS = 18, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_OS_TYPE = 19, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_GET_VCPUS = 20, /* skipgen skipgen priority:high */ + + REMOTE_PROC_LIST_DEFINED_DOMAINS = 21, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_ID = 22, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_NAME = 23, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID = 24, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_DOMAINS = 25, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_PIN_VCPU = 26, /* autogen autogen */ REMOTE_PROC_DOMAIN_REBOOT = 27, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESUME = 28, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SET_AUTOSTART = 29, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SET_MAX_MEMORY = 30, /* autogen autogen priority:high */
REMOTE_PROC_DOMAIN_SET_MEMORY = 31, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_VCPUS = 32, /* autogen autogen */ REMOTE_PROC_DOMAIN_SHUTDOWN = 33, /* autogen autogen */ REMOTE_PROC_DOMAIN_SUSPEND = 34, /* autogen autogen */ - REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen */ - REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen */ - REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE = 35, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_NETWORKS = 36, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DOMAINS = 37, /* autogen skipgen priority:high */ + REMOTE_PROC_LIST_NETWORKS = 38, /* autogen autogen priority:high */ REMOTE_PROC_NETWORK_CREATE = 39, /* autogen autogen */ REMOTE_PROC_NETWORK_CREATE_XML = 40, /* autogen autogen */
- REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen */ - REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen */ - REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen */ - REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen */ - REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen */ - REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen */ - - REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen */ + REMOTE_PROC_NETWORK_DEFINE_XML = 41, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_DESTROY = 42, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_XML_DESC = 43, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_AUTOSTART = 44, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_GET_BRIDGE_NAME = 45, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_NAME = 46, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_LOOKUP_BY_UUID = 47, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_SET_AUTOSTART = 48, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_UNDEFINE = 49, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_NETWORKS = 50, /* autogen autogen priority:high */ + + REMOTE_PROC_NUM_OF_DOMAINS = 51, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NETWORKS = 52, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_CORE_DUMP = 53, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE = 54, /* autogen autogen */ REMOTE_PROC_DOMAIN_SAVE = 55, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_TYPE = 56, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_GET_SCHEDULER_PARAMETERS = 57, /* skipgen autogen */ REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS = 58, /* autogen autogen */ - REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen */ - REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen */ + REMOTE_PROC_GET_HOSTNAME = 59, /* autogen autogen priority:high */ + REMOTE_PROC_SUPPORTS_FEATURE = 60, /* autogen autogen priority:high */
REMOTE_PROC_DOMAIN_MIGRATE_PREPARE = 61, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PERFORM = 62, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH = 63, /* autogen autogen */ REMOTE_PROC_DOMAIN_BLOCK_STATS = 64, /* autogen autogen */ - REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen */ - REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen */ - REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen */ - REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen */ - - REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen */ - REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen */ + REMOTE_PROC_DOMAIN_INTERFACE_STATS = 65, /* autogen autogen priority:high */ + REMOTE_PROC_AUTH_LIST = 66, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_INIT = 67, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_START = 68, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_SASL_STEP = 69, /* skipgen skipgen priority:high */ + REMOTE_PROC_AUTH_POLKIT = 70, /* skipgen skipgen priority:high */ + + REMOTE_PROC_NUM_OF_STORAGE_POOLS = 71, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_STORAGE_POOLS = 72, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_DEFINED_STORAGE_POOLS = 73, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_STORAGE_POOLS = 74, /* autogen autogen priority:high */ REMOTE_PROC_FIND_STORAGE_POOL_SOURCES = 75, /* autogen skipgen */ REMOTE_PROC_STORAGE_POOL_CREATE_XML = 76, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DEFINE_XML = 77, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_CREATE = 78, /* autogen autogen */ REMOTE_PROC_STORAGE_POOL_BUILD = 79, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_DESTROY = 80, /* autogen autogen priority:high */
REMOTE_PROC_STORAGE_POOL_DELETE = 81, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_UNDEFINE = 82, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_POOL_REFRESH = 83, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen */ - - REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_NAME = 84, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_UUID = 85, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LOOKUP_BY_VOLUME = 86, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_INFO = 87, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_XML_DESC = 88, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_GET_AUTOSTART = 89, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_SET_AUTOSTART = 90, /* autogen autogen priority:high */ + + REMOTE_PROC_STORAGE_POOL_NUM_OF_VOLUMES = 91, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_LIST_VOLUMES = 92, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML = 93, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_DELETE = 94, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen */ - REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen */ - - REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen */ - REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_NAME = 95, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_KEY = 96, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_LOOKUP_BY_PATH = 97, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_INFO = 98, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_XML_DESC = 99, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_VOL_GET_PATH = 100, /* autogen autogen priority:high */ + + REMOTE_PROC_NODE_GET_CELLS_FREE_MEMORY = 101, /* autogen skipgen priority:high */ + REMOTE_PROC_NODE_GET_FREE_MEMORY = 102, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_PEEK = 103, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MEMORY_PEEK = 104, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER = 105, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER = 106, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_LIFECYCLE = 107, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE2 = 108, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH2 = 109, /* autogen autogen */ - REMOTE_PROC_GET_URI = 110, /* autogen skipgen */ + REMOTE_PROC_GET_URI = 110, /* autogen skipgen priority:high */
- REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen */ - REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen */ + REMOTE_PROC_NODE_NUM_OF_DEVICES = 111, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_LIST_DEVICES = 112, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LOOKUP_BY_NAME = 113, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_GET_XML_DESC = 114, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen */ - REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_GET_PARENT = 115, /* skipgen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_NUM_OF_CAPS = 116, /* autogen autogen priority:high */ + REMOTE_PROC_NODE_DEVICE_LIST_CAPS = 117, /* autogen autogen priority:high */ REMOTE_PROC_NODE_DEVICE_DETTACH = 118, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RE_ATTACH = 119, /* autogen skipgen */ REMOTE_PROC_NODE_DEVICE_RESET = 120, /* autogen skipgen */
- REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_SECURITY_LABEL = 121, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_SECURITY_MODEL = 122, /* skipgen skipgen priority:high */ REMOTE_PROC_NODE_DEVICE_CREATE_XML = 123, /* autogen autogen */ - REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen */ + REMOTE_PROC_NODE_DEVICE_DESTROY = 124, /* autogen autogen priority:high */ REMOTE_PROC_STORAGE_VOL_CREATE_XML_FROM = 125, /* autogen autogen */ - REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen */ - REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen */ - REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen */ + REMOTE_PROC_NUM_OF_INTERFACES = 126, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_INTERFACES = 127, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_NAME = 128, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_LOOKUP_BY_MAC_STRING = 129, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_GET_XML_DESC = 130, /* autogen autogen */
- REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen */ - REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DEFINE_XML = 131, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_UNDEFINE = 132, /* autogen autogen priority:high */ REMOTE_PROC_INTERFACE_CREATE = 133, /* autogen autogen */ - REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen */ + REMOTE_PROC_INTERFACE_DESTROY = 134, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_XML_FROM_NATIVE = 135, /* autogen autogen */ REMOTE_PROC_DOMAIN_XML_TO_NATIVE = 136, /* autogen autogen */ - REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen */ - REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen */ - REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen */ - REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen */ - - REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen */ - REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen */ - REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen */ - REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen */ - REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen */ - REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen */ + REMOTE_PROC_NUM_OF_DEFINED_INTERFACES = 137, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_DEFINED_INTERFACES = 138, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_SECRETS = 139, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_SECRETS = 140, /* autogen autogen priority:high */ + + REMOTE_PROC_SECRET_LOOKUP_BY_UUID = 141, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_DEFINE_XML = 142, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_XML_DESC = 143, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_SET_VALUE = 144, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_GET_VALUE = 145, /* skipgen skipgen priority:high */ + REMOTE_PROC_SECRET_UNDEFINE = 146, /* autogen autogen priority:high */ + REMOTE_PROC_SECRET_LOOKUP_BY_USAGE = 147, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL = 148, /* autogen autogen | writestream@1 */ - REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen */ - REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen */ - - REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen */ - REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen */ - REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen */ - REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen */ - REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen */ - REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen */ + REMOTE_PROC_IS_SECURE = 149, /* autogen skipgen priority:high */ + REMOTE_PROC_DOMAIN_IS_ACTIVE = 150, /* autogen autogen priority:high */ + + REMOTE_PROC_DOMAIN_IS_PERSISTENT = 151, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_ACTIVE = 152, /* autogen autogen priority:high */ + REMOTE_PROC_NETWORK_IS_PERSISTENT = 153, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_ACTIVE = 154, /* autogen autogen priority:high */ + REMOTE_PROC_STORAGE_POOL_IS_PERSISTENT = 155, /* autogen autogen priority:high */ + REMOTE_PROC_INTERFACE_IS_ACTIVE = 156, /* autogen autogen priority:high */ + REMOTE_PROC_GET_LIB_VERSION = 157, /* autogen autogen priority:high */ + REMOTE_PROC_CPU_COMPARE = 158, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MEMORY_STATS = 159, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_ATTACH_DEVICE_FLAGS = 160, /* autogen autogen */
@@ -2393,8 +2399,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_ABORT_JOB = 164, /* autogen autogen */ REMOTE_PROC_STORAGE_VOL_WIPE = 165, /* autogen autogen */ REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_DOWNTIME = 166, /* autogen autogen */ - REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_EVENTS_REGISTER_ANY = 167, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_EVENTS_DEREGISTER_ANY = 168, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_EVENT_REBOOT = 169, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_RTC_CHANGE = 170, /* autogen autogen */
@@ -2402,22 +2408,22 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_IO_ERROR = 172, /* autogen autogen */ REMOTE_PROC_DOMAIN_EVENT_GRAPHICS = 173, /* autogen autogen */ REMOTE_PROC_DOMAIN_UPDATE_DEVICE_FLAGS = 174, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen */ - REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen */ - REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen */ - REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen */ - REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen */ - REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen */ - - REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_NAME = 175, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_LOOKUP_BY_UUID = 176, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_GET_XML_DESC = 177, /* autogen autogen priority:high */ + REMOTE_PROC_NUM_OF_NWFILTERS = 178, /* autogen autogen priority:high */ + REMOTE_PROC_LIST_NWFILTERS = 179, /* autogen autogen priority:high */ + REMOTE_PROC_NWFILTER_DEFINE_XML = 180, /* autogen autogen priority:high */ + + REMOTE_PROC_NWFILTER_UNDEFINE = 181, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_MANAGED_SAVE = 182, /* autogen autogen */ REMOTE_PROC_DOMAIN_HAS_MANAGED_SAVE_IMAGE = 183, /* autogen autogen */ REMOTE_PROC_DOMAIN_MANAGED_SAVE_REMOVE = 184, /* autogen autogen */ REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML = 185, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen */ + REMOTE_PROC_DOMAIN_SNAPSHOT_GET_XML_DESC = 186, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_NUM = 187, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_NAMES = 188, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SNAPSHOT_LOOKUP_BY_NAME = 189, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_HAS_CURRENT_SNAPSHOT = 190, /* autogen autogen */
REMOTE_PROC_DOMAIN_SNAPSHOT_CURRENT = 191, /* autogen autogen */ @@ -2432,8 +2438,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS = 200, /* autogen autogen */
REMOTE_PROC_DOMAIN_OPEN_CONSOLE = 201, /* autogen autogen | readstream@2 */ - REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen */ - REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen */ + REMOTE_PROC_DOMAIN_IS_UPDATED = 202, /* autogen autogen priority:high */ + REMOTE_PROC_GET_SYSINFO = 203, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204, /* autogen autogen */ REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206, /* skipgen skipgen */ @@ -2443,7 +2449,7 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_INJECT_NMI = 210, /* autogen autogen */
REMOTE_PROC_DOMAIN_SCREENSHOT = 211, /* autogen autogen | readstream@1 */ - REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_STATE = 212, /* skipgen skipgen priority:high */ REMOTE_PROC_DOMAIN_MIGRATE_BEGIN3 = 213, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE3 = 214, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_PREPARE_TUNNEL3 = 215, /* autogen skipgen | writestream@1 */ @@ -2459,17 +2465,17 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_CONTROL_ERROR = 224, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_PIN_VCPU_FLAGS = 225, /* autogen autogen */ REMOTE_PROC_DOMAIN_SEND_KEY = 226, /* autogen autogen */ - REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen */ - REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen */ + REMOTE_PROC_NODE_GET_CPU_STATS = 227, /* skipgen skipgen priority:high */ + REMOTE_PROC_NODE_GET_MEMORY_STATS = 228, /* skipgen skipgen priority:high */ + REMOTE_PROC_DOMAIN_GET_CONTROL_INFO = 229, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_GET_VCPU_PIN_INFO = 230, /* skipgen skipgen */
- REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ + REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ - REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen */ - REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen */ + REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_GET_XML_DESC = 235, /* autogen autogen priority:high */ + REMOTE_PROC_DOMAIN_SAVE_IMAGE_DEFINE_XML = 236, /* autogen autogen priority:high */ REMOTE_PROC_DOMAIN_BLOCK_JOB_ABORT = 237, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_BLOCK_JOB_INFO = 238, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_BLOCK_JOB_SET_SPEED = 239, /* autogen autogen */ @@ -2481,10 +2487,16 @@ enum remote_procedure { * Notice how the entries are grouped in sets of 10 ? * Nice isn't it. Please keep it this way when adding more. * - * Each function must have a two-word comment. The first word is + * Each function must have a three-word comment. The first word is * whether gendispatch.pl handles daemon, the second whether * it handles src/remote. Additional flags can be specified after a * pipe. + * The last argument describes priority of API. There are two accepted + * values: low, high; Each API that might eventually access hypervisor's + * monitor (and thus block) MUST fall into low priority. However, there + * are some exceptions to this rule, e.g. domainDestroy. Other APIs MAY + * be marked as high priority. If in doubt, it's safe to choose low. + * Low is taken as default, and thus can be left out. * * The (readstream|writestream)@<offset> flag lets daemon and src/remote * create a stream. The direction is defined from the src/remote point diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 0d344e8..38cbb65 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -146,12 +146,13 @@ while (<PROTOCOL>) { }
if ($opt_b or $opt_k) { - if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(.*)\*\/\s*$/)) { + if (!($flags =~ m/^\s*\/\*\s*(\S+)\s+(\S+)\s*(\|.*)?\s+(priority:(\S+))?\s*\*\/\s*$/)) { die "invalid generator flags for ${procprefix}_PROC_${name}" }
my $genmode = $opt_b ? $1 : $2; my $genflags = $3; + my $priority = defined $5 ? $5 : "low";
if ($genmode eq "autogen") { push(@autogen, $ProcName); @@ -171,6 +172,16 @@ while (<PROTOCOL>) { } else { $calls{$name}->{streamflag} = "none"; } + + # for now, we distinguish only two levels of prioroty: + # low (0) and high (1) + if ($priority eq "high") { + $calls{$name}->{priority} = 1; + } elsif ($priority eq "low") { + $calls{$name}->{priority} = 0; + } else { + die "invalid priority ${priority} for ${procprefix}_PROC_${name}" + } }
$calls[$id] = $calls{$name}; @@ -259,6 +270,7 @@ if ($opt_d) { print "$_:\n"; print " name $calls{$_}->{name} ($calls{$_}->{ProcName})\n"; print " $calls{$_}->{args} -> $calls{$_}->{ret}\n"; + print " priority -> $calls{$_}->{priority}\n"; } }
@@ -927,7 +939,7 @@ elsif ($opt_b) {
print "virNetServerProgramProc ${structprefix}Procs[] = {\n"; for ($id = 0 ; $id <= $#calls ; $id++) { - my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter); + my ($comment, $name, $argtype, $arglen, $argfilter, $retlen, $retfilter, $priority);
if (defined $calls[$id] && !$calls[$id]->{msg}) { $comment = "/* Method $calls[$id]->{ProcName} => $id */"; @@ -950,7 +962,9 @@ elsif ($opt_b) { $retfilter = "xdr_void"; }
- print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true \n},\n"; + $priority = defined $calls[$id]->{priority} ? $calls[$id]->{priority} : 0; + + print "{ $comment\n ${name},\n $arglen,\n (xdrproc_t)$argfilter,\n $retlen,\n (xdrproc_t)$retfilter,\n true,\n $priority\n},\n"; } print "};\n"; print "size_t ${structprefix}NProcs = ARRAY_CARDINALITY(${structprefix}Procs);\n"; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 1a49dbb..bc9fa89 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -64,6 +64,7 @@ typedef virNetServerJob *virNetServerJobPtr; struct _virNetServerJob { virNetServerClientPtr client; virNetMessagePtr msg; + virNetServerProgramPtr prog; };
struct _virNetServer { @@ -128,38 +129,18 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job = jobOpaque; - virNetServerProgramPtr prog = NULL; - size_t i;
- virNetServerLock(srv); VIR_DEBUG("server=%p client=%p message=%p", srv, job->client, job->msg);
- for (i = 0 ; i < srv->nprograms ; i++) { - if (virNetServerProgramMatches(srv->programs[i], job->msg)) { - prog = srv->programs[i]; - break; - } - } - - if (!prog) { - VIR_DEBUG("Cannot find program %d version %d", - job->msg->header.prog, - job->msg->header.vers); - goto error; - } - - virNetServerProgramRef(prog); - virNetServerUnlock(srv); - - if (virNetServerProgramDispatch(prog, + if (virNetServerProgramDispatch(job->prog, srv, job->client, job->msg) < 0) goto error;
virNetServerLock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetServerUnlock(srv); virNetServerClientFree(job->client);
@@ -168,21 +149,23 @@ static void virNetServerHandleJob(void *jobOpaque, void *opaque)
error: virNetServerUnlock(srv); - virNetServerProgramFree(prog); + virNetServerProgramFree(job->prog); virNetMessageFree(job->msg); virNetServerClientClose(job->client); virNetServerClientFree(job->client); VIR_FREE(job); }
- static int virNetServerDispatchNewMessage(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { virNetServerPtr srv = opaque; virNetServerJobPtr job; - int ret; + virNetServerProgramPtr prog = NULL; + unsigned int priority = 0; + size_t i; + int ret = -1;
VIR_DEBUG("server=%p client=%p message=%p", srv, client, msg); @@ -196,8 +179,31 @@ static int virNetServerDispatchNewMessage(virNetServerClientPtr client, job->msg = msg;
virNetServerLock(srv); - if ((ret = virThreadPoolSendJob(srv->workers, job)) < 0) + for (i = 0 ; i < srv->nprograms ; i++) { + if (virNetServerProgramMatches(srv->programs[i], job->msg)) { + prog = srv->programs[i]; + break; + } + } + + if (!prog) { + VIR_DEBUG("Cannot find program %d version %d", + job->msg->header.prog, + job->msg->header.vers); + goto cleanup; + } + + virNetServerProgramRef(prog); + job->prog = prog; + priority = virNetServerProgramGetPriority(prog, msg->header.proc); + + ret = virThreadPoolSendJob(srv->workers, priority, job); + +cleanup: + if (ret < 0) { VIR_FREE(job); + virNetServerProgramFree(prog); + } virNetServerUnlock(srv);
return ret; @@ -274,6 +280,7 @@ static void virNetServerFatalSignal(int sig, siginfo_t *siginfo ATTRIBUTE_UNUSED
virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus ATTRIBUTE_UNUSED, @@ -290,6 +297,7 @@ virNetServerPtr virNetServerNew(size_t min_workers, srv->refs = 1;
if (!(srv->workers = virThreadPoolNew(min_workers, max_workers, + priority_workers, virNetServerHandleJob, srv))) goto error; diff --git a/src/rpc/virnetserver.h b/src/rpc/virnetserver.h index 324cfb7..cc9d039 100644 --- a/src/rpc/virnetserver.h +++ b/src/rpc/virnetserver.h @@ -39,6 +39,7 @@ typedef int (*virNetServerClientInitHook)(virNetServerPtr srv,
virNetServerPtr virNetServerNew(size_t min_workers, size_t max_workers, + size_t priority_workers, size_t max_clients, const char *mdnsGroupName, bool connectDBus, diff --git a/src/rpc/virnetserverprogram.c b/src/rpc/virnetserverprogram.c index ca80ae0..33e036d 100644 --- a/src/rpc/virnetserverprogram.c +++ b/src/rpc/virnetserverprogram.c @@ -108,6 +108,17 @@ static virNetServerProgramProcPtr virNetServerProgramGetProc(virNetServerProgram return &prog->procs[procedure]; }
+unsigned int +virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure) +{ + virNetServerProgramProcPtr proc = virNetServerProgramGetProc(prog, procedure); + + if (!proc) + return 0; + + return proc->priority; +}
static int virNetServerProgramSendError(virNetServerProgramPtr prog, diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index ca31b7e..4f931ff 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -53,6 +53,7 @@ struct _virNetServerProgramProc { size_t ret_len; xdrproc_t ret_filter; bool needAuth; + unsigned int priority; };
virNetServerProgramPtr virNetServerProgramNew(unsigned program, @@ -63,6 +64,9 @@ virNetServerProgramPtr virNetServerProgramNew(unsigned program, int virNetServerProgramGetID(virNetServerProgramPtr prog); int virNetServerProgramGetVersion(virNetServerProgramPtr prog);
+unsigned int virNetServerProgramGetPriority(virNetServerProgramPtr prog, + int procedure); + void virNetServerProgramRef(virNetServerProgramPtr prog);
int virNetServerProgramMatches(virNetServerProgramPtr prog, diff --git a/src/util/threadpool.c b/src/util/threadpool.c index 8217591..70a75c0 100644 --- a/src/util/threadpool.c +++ b/src/util/threadpool.c @@ -37,7 +37,9 @@ typedef struct _virThreadPoolJob virThreadPoolJob; typedef virThreadPoolJob *virThreadPoolJobPtr;
struct _virThreadPoolJob { + virThreadPoolJobPtr prev; virThreadPoolJobPtr next; + unsigned int priority;
void *data; }; @@ -47,7 +49,8 @@ typedef virThreadPoolJobList *virThreadPoolJobListPtr;
struct _virThreadPoolJobList { virThreadPoolJobPtr head; - virThreadPoolJobPtr *tail; + virThreadPoolJobPtr tail; + virThreadPoolJobPtr firstPrio; };
@@ -57,6 +60,7 @@ struct _virThreadPool { virThreadPoolJobFunc jobFunc; void *jobOpaque; virThreadPoolJobList jobList; + size_t jobQueueDepth;
virMutex mutex; virCond cond; @@ -66,33 +70,75 @@ struct _virThreadPool { size_t freeWorkers; size_t nWorkers; virThreadPtr workers; + + size_t nPrioWorkers; + virThreadPtr prioWorkers; + virCond prioCond; +}; + +struct virThreadPoolWorkerData { + virThreadPoolPtr pool; + virCondPtr cond; + bool priority; };
static void virThreadPoolWorker(void *opaque) { - virThreadPoolPtr pool = opaque; + struct virThreadPoolWorkerData *data = opaque; + virThreadPoolPtr pool = data->pool; + virCondPtr cond = data->cond; + bool priority = data->priority; + virThreadPoolJobPtr job = NULL; + + VIR_FREE(data);
virMutexLock(&pool->mutex);
while (1) { while (!pool->quit && - !pool->jobList.head) { - pool->freeWorkers++; - if (virCondWait(&pool->cond, &pool->mutex) < 0) { - pool->freeWorkers--; + ((!priority && !pool->jobList.head) || + (priority && !pool->jobList.firstPrio))) { + if (!priority) + pool->freeWorkers++; + if (virCondWait(cond, &pool->mutex) < 0) { + if (!priority) + pool->freeWorkers--; goto out; } - pool->freeWorkers--; + if (!priority) + pool->freeWorkers--; }
if (pool->quit) break;
- virThreadPoolJobPtr job = pool->jobList.head; - pool->jobList.head = pool->jobList.head->next; - job->next = NULL; - if (pool->jobList.tail == &job->next) - pool->jobList.tail = &pool->jobList.head; + if (priority) { + job = pool->jobList.firstPrio; + } else { + job = pool->jobList.head; + } + + if (job == pool->jobList.firstPrio) { + virThreadPoolJobPtr tmp = job->next; + while (tmp) { + if (tmp->priority) { + break; + } + tmp = tmp->next; + } + pool->jobList.firstPrio = tmp; + } + + if (job->prev) + job->prev->next = job->next; + else + pool->jobList.head = job->next; + if (job->next) + job->next->prev = job->prev; + else + pool->jobList.tail = job->prev; + + pool->jobQueueDepth--;
virMutexUnlock(&pool->mutex); (pool->jobFunc)(job->data, pool->jobOpaque); @@ -101,19 +147,24 @@ static void virThreadPoolWorker(void *opaque) }
out: - pool->nWorkers--; - if (pool->nWorkers == 0) + if (priority) + pool->nPrioWorkers--; + else + pool->nWorkers--; + if (pool->nWorkers == 0 && pool->nPrioWorkers==0) virCondSignal(&pool->quit_cond); virMutexUnlock(&pool->mutex); }
virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, void *opaque) { virThreadPoolPtr pool; size_t i; + struct virThreadPoolWorkerData *data = NULL;
if (minWorkers > maxWorkers) minWorkers = maxWorkers; @@ -123,8 +174,7 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers, return NULL; }
- pool->jobList.head = NULL; - pool->jobList.tail = &pool->jobList.head; + pool->jobList.tail = pool->jobList.head = NULL;
pool->jobFunc = func; pool->jobOpaque = opaque; @@ -141,18 +191,51 @@ virThreadPoolPtr virThreadPoolNew(size_t minWorkers,
pool->maxWorkers = maxWorkers; for (i = 0; i < minWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->cond; + if (virThreadCreate(&pool->workers[i], true, virThreadPoolWorker, - pool) < 0) { + data) < 0) { goto error; } pool->nWorkers++; }
+ if (prioWorkers) { + if (virCondInit(&pool->prioCond) < 0) + goto error; + if (VIR_ALLOC_N(pool->prioWorkers, prioWorkers) < 0) + goto error; + + for (i = 0; i < prioWorkers; i++) { + if (VIR_ALLOC(data) < 0) { + virReportOOMError(); + goto error; + } + data->pool = pool; + data->cond = &pool->prioCond; + data->priority = true; + + if (virThreadCreate(&pool->prioWorkers[i], + true, + virThreadPoolWorker, + data) < 0) { + goto error; + } + pool->nPrioWorkers++; + } + } + return pool;
error: + VIR_FREE(data); virThreadPoolFree(pool); return NULL;
@@ -161,17 +244,22 @@ error: void virThreadPoolFree(virThreadPoolPtr pool) { virThreadPoolJobPtr job; + bool priority = false;
if (!pool) return;
virMutexLock(&pool->mutex); pool->quit = true; - if (pool->nWorkers > 0) { + if (pool->nWorkers > 0) virCondBroadcast(&pool->cond); - ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + if (pool->nPrioWorkers > 0) { + priority = true; + virCondBroadcast(&pool->prioCond); }
+ ignore_value(virCondWait(&pool->quit_cond, &pool->mutex)); + while ((job = pool->jobList.head)) { pool->jobList.head = pool->jobList.head->next; VIR_FREE(job); @@ -182,10 +270,19 @@ void virThreadPoolFree(virThreadPoolPtr pool) virMutexDestroy(&pool->mutex); ignore_value(virCondDestroy(&pool->quit_cond)); ignore_value(virCondDestroy(&pool->cond)); + if (priority) { + VIR_FREE(pool->prioWorkers); + ignore_value(virCondDestroy(&pool->prioCond)); + } VIR_FREE(pool); }
+/* + * @priority - job priority + * Return: 0 on success, -1 otherwise + */ int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobData) { virThreadPoolJobPtr job; @@ -194,7 +291,7 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, if (pool->quit) goto error;
- if (pool->freeWorkers == 0 && + if (pool->freeWorkers - pool->jobQueueDepth <= 0 && pool->nWorkers < pool->maxWorkers) { if (VIR_EXPAND_N(pool->workers, pool->nWorkers, 1) < 0) { virReportOOMError(); @@ -216,13 +313,26 @@ int virThreadPoolSendJob(virThreadPoolPtr pool, }
job->data = jobData; - job->next = NULL; - *pool->jobList.tail = job; - pool->jobList.tail = &(*pool->jobList.tail)->next; + job->priority = priority; + + job->prev = pool->jobList.tail; + if (pool->jobList.tail) + pool->jobList.tail->next = job; + pool->jobList.tail = job; + + if (!pool->jobList.head) + pool->jobList.head = job; + + if (priority && !pool->jobList.firstPrio) + pool->jobList.firstPrio = job; + + pool->jobQueueDepth++;
virCondSignal(&pool->cond); - virMutexUnlock(&pool->mutex); + if (priority) + virCondSignal(&pool->prioCond);
+ virMutexUnlock(&pool->mutex); return 0;
error: diff --git a/src/util/threadpool.h b/src/util/threadpool.h index 8b8c676..c4612ad 100644 --- a/src/util/threadpool.h +++ b/src/util/threadpool.h @@ -35,12 +35,14 @@ typedef void (*virThreadPoolJobFunc)(void *jobdata, void *opaque);
virThreadPoolPtr virThreadPoolNew(size_t minWorkers, size_t maxWorkers, + size_t prioWorkers, virThreadPoolJobFunc func, - void *opaque) ATTRIBUTE_NONNULL(3); + void *opaque) ATTRIBUTE_NONNULL(4);
void virThreadPoolFree(virThreadPoolPtr pool);
int virThreadPoolSendJob(virThreadPoolPtr pool, + unsigned int priority, void *jobdata) ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK;
Okay, the suggesting from Dan are in, ACK,
please push :)
Daniel
Thank you all for comments & review. Pushed.
participants (5)
-
Daniel P. Berrange
-
Daniel Veillard
-
Eric Blake
-
Jiri Denemark
-
Michal Privoznik