March 2014 - CIM - libvirt List Archives

[PATCH v3 0/8] Add Controller Device Support
by John Ferlan 06 May '14

06 May '14

I know this is a long cover letter, but it covers a lot of ground... I was just about done with all the merges this morning - I ran the cimtest one more time and found that there's a couple of tests which associate RASDs and Pools. So I jumped into the rabbit hole and wound my way through the maze of twisty little passages. I even heard a babbling brook (it's an old computer game reference). So here's where I'm at - I have cimtest completely passing again which keeps me happy. Although I have a pile of cimtest related changes that will also need to be made, but I have to wait to do them until I get a libvirt-cim revision number to compare against so as to be able to add conditional code to support controllers devices, rasds, and pools. For now support is only for KVM. We've already determined LXC doesn't have the concept of a controller and I have no way to test Xen. I also don't have the cycles to dig through the Xen source code and determine what that driver supports. If someone wants to add that - great - have at it! I want to put it out there for review - knowing that I probably still have a bit of cleanup left. I think I'm going to need to add KVM specific code to a number of places, but I figured I needed to get everything merged first then I could determine that. I'm also hoping it'll be pointed out to me :-). I kept the changes split even though I know only changes 1 & 2 can be compiled alone and run without any new cimtest failures. Each step of changes can compile successfully/cleanly - although once you hit change 3, you'll need to regenerate your Makefile Changes 1 & 2 are more or less Xu's previous patch #1. I split it for ease of review - there's no real reason, although you will note I have left out the addition of CONTROLLER to the cim_res_types until patch 4... With that patch 2 will have cimtest failures. Change 3 adds the MOF's and adjusts the install resulting in cimtest failures because of "missing" links between MOF and code. Change 4 adds the bulk of the code required to fill in all the fields for the mofs from the xml/data structures. Change 6 adds the various associations between the new elements Change 7 adds the MOF's and modifies install/build to include ControllerPool's Change 8 adds the code and associations for Controller Pools. Without it there are a few cimtest failures. My plan is to allow review of Changes 3->8 separately, but when it comes time to push - 3-8 could be squashed together so that the ability to bisect history and run cimtest without too many failures isn't lost. Although I'm also considering combining 3-6 and 7-8 for "functionality" likeness. I'll take suggestions from the community if it's felt strongly to keep the split as is, then so bit it. The following is a list of the differences between this set of changes and what Xu posted as his v2. All I did was pull down his changes into a clean branch, then 'git am' those changes. Then after making all my changes a simple 'git diff' of the branches let me know what was different. Makefile.am * Copyright date * Add controller pool libvirt-cim.spec.in * Order - kept closer to DisplayController/PointingDevice * Add controller pool libxkutil/device_parsing.c * Copyright date * Additional CU_DEBUG messages - useful in debugging free() issues * Merged sgio/rawio changes * cleanup_controller_device(): * Free new controller fields for queues, ports, vectors, address * Changed controller 'type' from "char *" to "uint16_t", no need to free * parse_controller_device(): * Use controller_protocol_type_StrToID() to store type integer * Fail in "index" is not present * Convert/store "index" as uint64_t and free "index" * Added controller fields for queues, ports, vectors, address * Format id as "controller:<type_str>:<index#>" libxkutil/device_parsing.h * Copyright date * Add CONTROLLER_INDEX_NOT_SET as -1 * Change type to uint16_t, add uint64_t index, and char */device_address fields libxkutil/xmlgen.c * Copyright date * Change how we store fields * Be sure to convert the "type" to it's string representation * Handle missing/default Index value (eg from define system from vdev) * Print out model, ports, vectors, queues, and address if present schema/Controller.mof * Remove {Xen|LXC}_Controller schema/Controller.registration * Remove {Xen|LXC}_Controller schema/ControllerPool.mof * Add ControllerPool mof schema/ControllerPool.registration * Add ControllerPool registration schema/ResourceAllocationSettingData.mof * Copyright date * Merged rawio/sgio changes * Remove {Xen|LXC}_ControllerResourceAllocationSettingData * Add comments and properties we care about schema/ResourceAllocationSettingData.registration * Copyright date * Remove {Xen|LXC}_ControllerResourceAllocationSettingData src/Virt_Device.c * Copyright date * Set the Virtual Controller Logical Device properties: * ProtocolSupported <== String from controller_device->type (pci, usb, etc.) * ProtocolDescription <== If available, string from model * Adjusted some debug messages * Removed incorrect setting of "Controller" property (twice) src/Virt_DevicePool.c * Add ControllerPool support src/Virt_ElementAllocatedFromPool.c * Copyright date * Remove Xen/LXC associations * Add ControllerPool src/Virt_ElementCapabilities.c * Copyright date * Add ControllerPool src/Virt_ElementSettingData.c * Copyright date * Remove Xen/LXC associations src/Virt_HostedResourcePool.c * Copyright date * Add ControllerPool src/Virt_RASD.c * Copyright date * Merged sgio/rawio changes * set_controller_rasd_params(): * Set the fields properly based on defintion of structure/mof * ResourceSubType <== String-ified type (pci, usb, etc) * Other fields set if available in data structure * Changed vdev_device -> type to use CIM_RES_TYPE_OTHER - this is the parent structure to the controller_device structure src/Virt_ResourceAllocationFromPool.c * Copyright date * Controller RASD Association * ControllerPool Association src/Virt_ResourcePoolConfigurationService.c * Copyright date * Add ControllerPool src/Virt_ServiceAffectsElement.c * Copyright date * Add "Controller" to list that use get_device_by_ref() call * Remove Xen/LXC associations src/Virt_SettingsDefineState.c * Copyright date * Remove Xen/LXC associations src/Virt_SystemDevice.c * Copyright date * Remove Xen/LXC associations src/Virt_VSSDComponent.c * Copyright date * Remove Xen/LXC associations src/Virt_VirtualSystemManagementService.c * Copyright date * Merged sgio/rawio changes * Fix controller_rasd_to_vdev(): * Handle the controller_device->type properly to get 'type_str' * Build dev->id based on Index: NOTE: I found through debugging cimtest code that on input the dev->id is NULL. It cannot be left that way since a subsequent call to add_device_nodup() would core on the STREQC(ptr->id, dev->id). Thus I formulated an InstanceID with the -1. If there's another similarly "undefined" element, it'll go through the overriding code and be removed. * Set other fields if present in data * Following Boris' earlier advice, mimic the method that disks, networks, and graphics devices use to set things up. * Similarly for resource_del, resource_add, and resource_mod - allow the code to do it's magic for controller devices. * HMM: Given what I discovered about dev->id in controller_rasd_to_vdev() described above, I wonder if the check in resource_add could ever happen. I guess it's "safe" to keep... src/svpc_types.h * Copyright date * Add some pointers to where to find the details we're describing * Add CIM_controller_protocol_type enum to describe the various values * Add controller_protocol_type_StrToID() and controller_protocol_type_IDToStr() to handle the "supported" conversions John Ferlan (3): Associations Add MOFS and change install for ControllerPools Add code and associations for ControllerPool Xu Wang (5): Add virtual controller device types Parse/Store controller XML tags Add virtual controller object definitions to mofs Set fields in mofs for Controller Device/RASD VSMS: Support for domains with controller devices Makefile.am | 6 +- libvirt-cim.spec.in | 4 + libxkutil/device_parsing.c | 119 ++++++++++++++++++- libxkutil/device_parsing.h | 17 ++- libxkutil/xmlgen.c | 74 +++++++++++- schema/Controller.mof | 7 ++ schema/Controller.registration | 4 + schema/ControllerPool.mof | 6 + schema/ControllerPool.registration | 3 + schema/ResourceAllocationSettingData.mof | 37 +++++- schema/ResourceAllocationSettingData.registration | 3 +- src/Virt_Device.c | 68 ++++++++++- src/Virt_DevicePool.c | 55 ++++++++- src/Virt_ElementAllocatedFromPool.c | 6 +- src/Virt_ElementCapabilities.c | 4 +- src/Virt_ElementSettingData.c | 3 +- src/Virt_HostedResourcePool.c | 3 +- src/Virt_RASD.c | 57 +++++++++- src/Virt_ResourceAllocationFromPool.c | 2 + src/Virt_ResourcePoolConfigurationCapabilities.c | 2 +- src/Virt_ResourcePoolConfigurationService.c | 2 + src/Virt_ServiceAffectsElement.c | 8 +- src/Virt_SettingsDefineState.c | 4 +- src/Virt_SystemDevice.c | 3 +- src/Virt_VSSDComponent.c | 3 +- src/Virt_VirtualSystemManagementService.c | 87 +++++++++++++- src/svpc_types.h | 132 +++++++++++++++++++++- 27 files changed, 695 insertions(+), 24 deletions(-) create mode 100644 schema/Controller.mof create mode 100644 schema/Controller.registration create mode 100644 schema/ControllerPool.mof create mode 100644 schema/ControllerPool.registration -- 1.8.5.3

2 16

[PATCH V2 0/5] Add Controller Device Support
by Xu Wang 27 Mar '14

27 Mar '14

These patches are based on V1. An unexpectable accident makes me only one hand could work now so sorry everything from me takes so long time. Updates from V1: 1. Added 2 break logic in switch. 2. Changed CIM_RES_TYPE_CONTROLLER into 32771. 3. Added XEN/KVM/LXC_Controller classes for cimtest. Most of cimtest testcases have passed except the following: -------------------------------------------------------------------- HostSystem - 02_hostsystem_to_rasd.py: FAIL ERROR - Failed to get associators information for KVM_SettingsDefineState ERROR - Exception: u'KVM_Controller' -------------------------------------------------------------------- -------------------------------------------------------------------- RASD - 03_rasd_errs.py: FAIL ERROR - Expected 6 RASDs, got 7 -------------------------------------------------------------------- -------------------------------------------------------------------- ResourceAllocationFromPool - 01_forward.py: FAIL ERROR - 7 RASD insts != 6 pool insts -------------------------------------------------------------------- ResourceAllocationFromPool - 02_reverse.py: FAIL ERROR - 7 RASD insts != 6 pool insts -------------------------------------------------------------------- -------------------------------------------------------------------- SettingsDefine - 01_forward.py: FAIL ERROR - 6 device insts != 7 RASD insts -------------------------------------------------------------------- SettingsDefine - 02_reverse.py: FAIL ERROR - u'KVM_Controller' -------------------------------------------------------------------- -------------------------------------------------------------------- SystemDevice - 01_forward.py: FAIL 01_forward.py:29: DeprecationWarning: the sets module is deprecated from sets import Set ERROR - Device Class mismatch ERROR - Exception Expected Device class list: ['KVM_DisplayController', 'KVM_LogicalDisk', 'KVM_Memory', 'KVM_NetworkPort', 'KVM_PointingDevice', 'KVM_Processor'] Got: [u'KVM_Controller', u'KVM_DisplayController', u'KVM_LogicalDisk', u'KVM_Memory', u'KVM_NetworkPort', u'KVM_PointingDevice', u'KVM_Processor'] -------------------------------------------------------------------- -------------------------------------------------------------------- VirtualSystemSettingDataComponent - 02_reverse.py: FAIL ERROR - Unexpected RASD instance type ERROR - Mistmatching association value -------------------------------------------------------------------- I think they are caused by the parameter set of cimtest (such as KVM_Controller or ControllerResourceAllocationSettingData should be added into some arrays. But if you find they are caused by my coding errors, please let me know, thanks). I'll fix these issues of cimtest later. Dear John and Boris, could you help me testing it on RHEL6.5? I just have an upgraded version and introduced some packages from CentOS source (I am worried about they may influence the testing result). I'll rebuild my developing environment later using pure RHEL 6.5 version. Xu Wang (5): libxutil: Controller Support RASD: Schema and Provider Support for Controller RASDs VSMS: Support for domains with controller devices Device: CIM_LogicalDevice for controllers Virt_Device: Add a device class for controllers Makefile.am | 2 + libvirt-cim.spec.in | 2 + libxkutil/device_parsing.c | 70 ++++++++++++++++++++- libxkutil/device_parsing.h | 9 +++ libxkutil/xmlgen.c | 30 +++++++++ schema/Controller.mof | 16 +++++ schema/Controller.registration | 5 ++ schema/ResourceAllocationSettingData.mof | 27 ++++++++ schema/ResourceAllocationSettingData.registration | 3 + src/Virt_Device.c | 36 ++++++++++- src/Virt_ElementAllocatedFromPool.c | 6 ++ src/Virt_ElementSettingData.c | 3 + src/Virt_RASD.c | 24 +++++++ src/Virt_ServiceAffectsElement.c | 6 ++- src/Virt_SettingsDefineState.c | 6 ++ src/Virt_SystemDevice.c | 3 + src/Virt_VSSDComponent.c | 3 + src/Virt_VirtualSystemManagementService.c | 44 +++++++++++++ src/svpc_types.h | 4 +- 19 files changed, 295 insertions(+), 4 deletions(-) create mode 100644 schema/Controller.mof create mode 100644 schema/Controller.registration

2 11

[PATCH 0/2] Add rawio/sgio tag property support
by Xu Wang 26 Mar '14

26 Mar '14

They are needed by virtio-scsi so add them into libvirt-cim support Xu Wang (2): Add rawio property support Add sgio property support libxkutil/device_parsing.c | 8 ++++++++ libxkutil/device_parsing.h | 2 ++ libxkutil/xmlgen.c | 4 ++++ schema/ResourceAllocationSettingData.mof | 6 ++++++ src/Virt_RASD.c | 12 ++++++++++++ src/Virt_VirtualSystemManagementService.c | 10 ++++++++++ 6 files changed, 42 insertions(+), 0 deletions(-)

2 4

[PATCH 0/3] Resolved recent/new Coverity warnings
by John Ferlan 26 Mar '14

26 Mar '14

For virlog.c and commandtest.c - it seems recent changes within the modules have allowed Coverity to dig deeper and find different issues. Not sure why Coverity has that "ability"... Since I was making changes, figured I'd clean up the lxc_controller fd leak as well - these should clean the Jenkins build environment. John Ferlan (3): Coverity: Resolve a CHECKED_RETURN message Coverity: Resolve a FORWARD_NULL Coverity: Resolve a RESOURCE_LEAK src/lxc/lxc_controller.c | 14 ++++++++++---- src/util/virlog.c | 2 +- tests/commandtest.c | 1 + 3 files changed, 12 insertions(+), 5 deletions(-) -- 1.8.5.3

1 4

[PATCH 0/6] Add Virtual Controller Device
by John Ferlan 22 Mar '14

22 Mar '14

This is rework of the Controller patch series submitted by Xu Wang. I have built upon the constructs put in place by Xu with respect to reading and writing the XML, but for the remainder of the code I changed where the virtual controller lives. Xu's patches made a Controller RASD which had some issues regarding associations with other devices and pools. Rather than go with a RASD model, I have chosen to make a KVM_Controller virtual device to mimic a CIM_Controller device. This solution keeps things at a lower level and has been able to pass the cimtest as well as the update a defined domain with a changed value test. This second test is what fails for RHEL7. As it turns out, it seems as long as a 'pci' device with model 'pci-root' is created, then things seem to work. The code relies on the previous patch code Xu created to read/write the XML file with some adjustments. Perhaps the only controversial patch (for me at least) is 6/6. I figured that after we've read everything and just before we go to create or update the guest that we need to make sure that at least the 'pci' device with model 'pci-root' exists. This is similar to the add_default_devs() code, except that it's run after all that code prior to any add or update of the guest just before the "system_to_xml()" call is made. I figure this is the last gasp to ensure that at least the 'pci' device is there which seems to be used by a number of "silently added" libvirt devices. John Ferlan (4): Add virtual controller object defs Change static API to global API Controller Device Details VSMS: Determine if default controller exists for KVM Xu Wang (2): Add virtual controller device types Parse/Store controller XML tags Makefile.am | 2 + libvirt-cim.spec.in | 2 + libxkutil/device_parsing.c | 119 +++++++++++++++++++++++++++- libxkutil/device_parsing.h | 15 ++++ libxkutil/xmlgen.c | 72 +++++++++++++++++ schema/Controller.mof | 47 +++++++++++ schema/Controller.registration | 19 +++++ src/Virt_Device.c | 84 ++++++++++++++++++++ src/Virt_RASD.c | 21 +++-- src/Virt_RASD.h | 4 + src/Virt_VirtualSystemManagementService.c | 106 ++++++++++++++++++++++++- src/svpc_types.h | 127 ++++++++++++++++++++++++++++++ 12 files changed, 602 insertions(+), 16 deletions(-) create mode 100644 schema/Controller.mof create mode 100644 schema/Controller.registration -- 1.8.5.3

2 9

[REPOST MERGED PATCH 0/3] Repost controller series merged
by John Ferlan 18 Mar '14

18 Mar '14

Changes since last time - 1. Address review comments 2. I did not add CONTROLLER_INDEX_NOT_SET. Since that's a -1 value and the index is an unsigned value - it just didn't seem right. Furthermore, if libvirt doesn't find the 'index' value, it defaults to using 0 when parsing the XML - so I think following that model is better. I have yet to chase down all the libvirt paths to see what happens, but I think if someone adds a controller without defining an index and that index conflicts with something already there for that named/type of controller, then libvirt will reject the xml for the guest to start requiring the "user" to fix it. 3. I didn't yet do it, but I think the 'master' may need to be removed. The libvirt documented example is: <devices> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0' bus='0' slot='4' function='7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0' bus='0' slot='4' function='0' multifunction='on'/> </controller> ... </devices> This examples shows that the type is 'usb' and the 'index' is 0 for both which would violate our namespace rule. The 'model' is the same too. So unless we incorporate the address into the name, then there's a conflict. I'll have to think about this one some more and of course take advice! 4. Just realized I forgot to switch the mof values for queues, ports, and vectors back to string types... I'll do that, but didn't want to lose my current cover letter. Xu Wang (3): libxutil, xmlgen: Add Controller Support RASD: Schema and Provider Support for Controller RASDs VSMS: Support for domains with controller devices libxkutil/device_parsing.c | 105 +++++++++++++++++++++- libxkutil/device_parsing.h | 15 ++++ libxkutil/xmlgen.c | 52 +++++++++++ schema/ResourceAllocationSettingData.mof | 41 +++++++++ schema/ResourceAllocationSettingData.registration | 1 + src/Virt_ElementSettingData.c | 1 + src/Virt_RASD.c | 81 +++++++++++++++-- src/Virt_SettingsDefineState.c | 1 + src/Virt_VSSDComponent.c | 1 + src/Virt_VirtualSystemManagementService.c | 76 ++++++++++++++++ src/svpc_types.h | 4 +- 11 files changed, 368 insertions(+), 10 deletions(-) -- 1.8.5.3

2 11

[REWORK 0/7] Rework the controller patch series
by John Ferlan 14 Mar '14

14 Mar '14

I took the "liberty" of making some adjustments based on the base series. I have left the base code "as is" in patches 1, 3, and 6. If I simply apply 1 & 2 and more specifically - if I modify patch 1 to just add the "break;"'s in 'do_parse()' and 'device_to_xml()', then cimtest has "new" failures, such as: -------------------------------------------------------------------- ComputerSystem - 41_cs_to_settingdefinestate.py: FAIL ERROR - Exception details: KVM_SystemDevice returned 0 Logical Devices objects ERROR - Exception details is Failed to get SystemDevice information CIM_ERR_FAILED: Couldn't get device instances -------------------------------------------------------------------- Memory - 02_defgetmem.py: FAIL ERROR - No devices returned ERROR - Exception: 'int' object is not iterable CIM_ERR_FAILED: Couldn't get device instances -------------------------------------------------------------------- Processor - 02_definesys_get_procs.py: FAIL ERROR - No devices returned ERROR - Exception: 'int' object is not iterable CIM_ERR_FAILED: Couldn't get device instances -------------------------------------------------------------------- RASD - 03_rasd_errs.py: FAIL ERROR - Expected 6 RASDs, got 7 -------------------------------------------------------------------- ResourceAllocationFromPool - 01_forward.py: FAIL ERROR - 7 RASD insts != 6 pool insts -------------------------------------------------------------------- ResourceAllocationFromPool - 02_reverse.py: FAIL ERROR - 7 RASD insts != 6 pool insts -------------------------------------------------------------------- SettingsDefine - 01_forward.py: FAIL ERROR - 6 device insts != 7 RASD insts -------------------------------------------------------------------- SettingsDefine - 02_reverse.py: FAIL ERROR - Got 9 RASDs, expected 7 ERROR - Failed to verify RASDs -------------------------------------------------------------------- VirtualSystemManagementService - 16_removeresource.py: FAIL ERROR - No devices returned ERROR - UnboundLocalError : local variable 'ccn' referenced before assignment Traceback (most recent call last): File "/home/jferlan/git/cimtest.work/suites/libvirt-cim/lib/XenKvmLib/const.py", line 141, in do_try rc = f() File "16_removeresource.py", line 95, in main if ccn == input: UnboundLocalError: local variable 'ccn' referenced before assignment ERROR - None CIM_ERR_FAILED: Couldn't get device instances -------------------------------------------------------------------- VirtualSystemManagementService - 31_unset_netrasd.py: FAIL ERROR - CIMError : (1, u"CIM_ERR_FAILED: Couldn't get device instances") Traceback (most recent call last): ... assoc_names = cim.AssociatorNames(inst_name, AssocClass=a_class, ResultClass=r_class) ... CIMError: (1, u"CIM_ERR_FAILED: Couldn't get device instances") ERROR - None -------------------------------------------------------------------- VirtualSystemManagementService - 32_modify_cdrom_media.py: FAIL ERROR - CIMError : (1, u"CIM_ERR_FAILED: Couldn't get device instances") Traceback (most recent call last): ... assoc_names = cim.AssociatorNames(inst_name, AssocClass=a_class, ResultClass=r_class) ... CIMError: (1, u"CIM_ERR_FAILED: Couldn't get device instances") ERROR - None -------------------------------------------------------------------- I haven't dug in too deeply but there's some disconnect between the new 'controller' device and other parts. It took a bit of time to just get things where there are now - I can continue with this, but I wanted to "share" in hopes that someone else can also look and give me some hints. I'm hoping it's something simple and not that I have to have something else - although I'm beginning to think I need a logical device... I have not completed the 3 of 3 changes to manage all the new fields. I figured I could add those eventually, but I did want to make sure I was setting up the instance id correctly as described in the code reviews thus far. John Ferlan (4): Adjustments to patch 1/3 Rename set_rasd_device_address Changes to 2/3 to support more fields Changes to 3 of 3 from code review Xu Wang (3): libxutil: Controller Support RASD: Schema and Provider Support for Controller RASDs VSMS: Support for domains with controller devices libxkutil/device_parsing.c | 99 ++++++++++++++++++++++- libxkutil/device_parsing.h | 15 ++++ libxkutil/xmlgen.c | 47 +++++++++++ schema/ResourceAllocationSettingData.mof | 45 +++++++++++ schema/ResourceAllocationSettingData.registration | 1 + src/Virt_ElementSettingData.c | 1 + src/Virt_RASD.c | 81 +++++++++++++++++-- src/Virt_SettingsDefineState.c | 1 + src/Virt_VSSDComponent.c | 1 + src/Virt_VirtualSystemManagementService.c | 70 ++++++++++++++++ src/svpc_types.h | 4 +- 11 files changed, 355 insertions(+), 10 deletions(-) -- 1.8.5.3

3 13

[PATCH 0/3] Add Controller Device Support
by Xu Wang 14 Mar '14

14 Mar '14

Add Xen/KVM/LXC_ControllerResourceAllocationSettingData classes for controller device operation. So far there are two variables (type and model) could be operated by calling API. Xu Wang (3): libxutil: Controller Support RASD: Schema and Provider Support for Controller RASDs VSMS: Support for domains with controller devices libxkutil/device_parsing.c | 62 ++++++++++++++++++++- libxkutil/device_parsing.h | 9 +++ libxkutil/xmlgen.c | 28 +++++++++ schema/ResourceAllocationSettingData.mof | 27 +++++++++ schema/ResourceAllocationSettingData.registration | 3 + src/Virt_RASD.c | 24 ++++++++ src/Virt_VirtualSystemManagementService.c | 44 +++++++++++++++ src/svpc_types.h | 4 +- 8 files changed, 199 insertions(+), 2 deletions(-)

3 21