- CIM - libvirt List Archives

[PATCH 00/47] Solution to solve unsupported tag missing issue
by Xu Wang 19 Oct '13

19 Oct '13

These patches are based on commit 04bfeb825e5e155ffa72119253de608ccf3bd72b. So I may need more work on rebasing. Most of bugs reported recently are about some tags libvirt-cim doesn't support will be dropped after resource updated. So a new member was added into every virt_device and domain structure. And a new structure named unknown device was added to save those tags new added. The original implementaion is reading every member of virt_device structure from xml and save it. But the defect of this is some tags were not a member of virt_device were dropped. After resource updated, every data in the virt_device will be used to regenerate new xml. Hence the tags unsupported above, disappeared. So I added a member into every virt_device and domain structure, 'others'. It's a link list and used to save all data read from xml. Another new structure 'unknown_device' was added to save data except libvirt-cim could recognize. The new implementation is, firstly parse_*_device() could read all nodes and properties from xml and save them into 'others' link list. Then every member of virt_device will fetch data from others link list and save it. So that nodes in the 'others' link list could be saved until they are used to re- generate xml. After resource updating finished, libvirt-cim will call *_xml() to generate xml. The new process of generating xml like this, firstly all data in the members of virt_device will be restored into 'others' link list, then a function (others_to_xml) will use this link list to generate xml. Some points I have updated, 1. 'others' link list has to be processed in _get_proc_device() and _get_mem_device(). They should be copied into new data structure. 2. If resource updating happened, others field should be cleared because this device has been changed and they are useless. Besides above some logic may be a little strange or boring. Implemention like that is just to be compatible with upper layer functions (to make changes as less as possible). After keep a balance I decided to devide the whole xml into several parts: <domain>------------------------others in domain to save unsupported sub-nodes of domain -some fields (devices, mem, vcpu...) will be skiped because -they have their own parsing functions. <name>xxx</name> <uuid>xxx</uuid> <vcpu>xxx</vcpu> <mem>xxx</mem> <devices> <disk>xxx</disk>------- others in virt_device to save unsupported tags of this device <emulator>xxx</emulator> ...----------- unknown_device to save unsupported device except like <disk>,<emu>,etc. </devices> </domain> Hence, all nodes read from xml will be restored after xml generation. Xu Wang (47): Add others member for saving unsupported tag and unknown device Add others and unknown_device clean up Add basic operations for reading data from xml node Fix xml parsing algorithm for parse_fs_device() Fix xml parsing algorithm for parse_block_device() Fix xml parsing algorithm for parse_vsi_device() Fix xml parsing algorithm for parse_net_device() Fix xml parsing algorithm for parse_vcpu_device() Fix xml parsing algorithm for parse_emu_device() Fix xml parsing algorithm for parse_mem_device() Fix xml parsing algorithm for parse_graphics_device() Fix xml parsing algorithm for parse_input_device() Add parse_unknown_device() Add parse_devices() for unknown type in get_dominfo_from_xml() Fix xml parsing algorithm in parse_domain() Fix xml parsing algorithm in parse_os() Fix xml parsing algorithm in parse_feature() Add dup function for device copy Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del Add basic functions about converting others link list to xml Fix xml generation algorithm in disk_block_xml() Fix xml generation algorithm in disk_file_xml() Fix xml generation algorithm in disk_fs_xml() Fix xml generation algorithm in set_net_vsi() Fix xml generation algorithm in set_net_source() Fix xml generation algorithm in bridge_net_to_xml() Fix xml generation algorithm in net_xml() Fix xml generation algorithm in vcpu_xml() Fix xml generation algorithm in cputune_xml() Fix xml generation algorithm in mem_xml() Fix xml generation algorithm in emu_xml() Fix xml generation algorithm in graphics_vnc_xml() Fix xml generation algorithm in graphics_pty_xml() Fix xml generation algorithm in input_xml() Fix xml generation algorithm in system_xml() Fix xml generation algorithm in _xenpv_os_xml() Fix xml generation algorithm in _fv_bootlist_xml() Fix xml generation algorithm in _xenfv_os_xml() Fix xml generation algorithm in _kvm_os_xml() Fix xml generation algorithm in _lxc_os_xml() Fix xml generation algorithm in os_xml() Fix xml generation algorithm in features_xml() Add functions for xml generation of unknown devices Fix xml generation algorithm in system_to_xml() Add cleanup_others() calling during rasd_to_vdev() Add others handling in _get_mem_device() Add dup_others() into _get_proc_device() libxkutil/device_parsing.c | 1351 +++++++++++++++++++++----- libxkutil/device_parsing.h | 56 ++ libxkutil/xmlgen.c | 1470 +++++++++++++++++++++++------ src/Virt_VirtualSystemManagementService.c | 20 +- src/svpc_types.h | 1 + 5 files changed, 2346 insertions(+), 552 deletions(-)

4 52

[PATCHv2 0/3] Console Fixes and Enhancements
by Viktor Mihajlovski 11 Oct '13

11 Oct '13

Enabling cimtest for the Console RASDs required a few improvements. The XML generation code was simplified, a logical device class for consoles had to be added and a bug in the KVM redirection service was fixed. V2 Changes: - Smaller change in Virt_Device.c (2/3) - Simpler console check in Virt_KVMRedirectionSAP.c (3/3) Viktor Mihajlovski (3): libxkutil: Simplify XML handling of consoles Virt_Device: Add a device class for consoles KVMRedirectionSAP: Only return redirection SAPs for VNC graphics libxkutil/device_parsing.c | 12 +----- libxkutil/xmlgen.c | 76 --------------------------------- schema/DisplayController.mof | 15 +++++++ schema/DisplayController.registration | 3 ++ src/Virt_Device.c | 6 ++- src/Virt_ElementSettingData.c | 3 ++ src/Virt_KVMRedirectionSAP.c | 26 ++++++----- src/Virt_SettingsDefineState.c | 6 +++ src/Virt_SystemDevice.c | 3 ++ src/Virt_VSSDComponent.c | 3 ++ 10 files changed, 53 insertions(+), 100 deletions(-) -- 1.7.9.5

2 5

Re: [Libvirt-cim] [PATCH] cimtest: Add type == 'bridge' handling branch in vsms_util.py
by Xu Wang 10 Oct '13

10 Oct '13

Yeah, I found this issue has been solved by the early patch from John after git pull. Sorry working on other things so long that I am out of date :-) Please drop this patch and I think my works on libvirt-cim could work well based on upstream of cimtest. Thanks, Xu Wang 于 2013/10/2 0:34, Viktor Mihajlovski 写道: > On 09/26/2013 09:28 AM, Xu Wang wrote: >> The name of interface should be read based on the type of it. There >> are two basic types for interface device (bridge and network). The >> name should be fetched in different way. I don't clear if Xen could >> support network type so I didn't make any change for it and just >> enhanced the logic in "virt == 'KVM'". I'll submit a set of patches >> and they solved XFAIL of VirtualSystemManagementService/ >> 22_addmulti_brg_interface.py. So it's necessary to fix it. >> >> Signed-off-by: Xu Wang <gesaint(a)linux.vnet.ibm.com> >> --- >> suites/libvirt-cim/lib/XenKvmLib/vsms_util.py | 13 +++++++++++-- >> 1 files changed, 11 insertions(+), 2 deletions(-) >> >> diff --git a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> index 075c09f..7839a38 100644 >> --- a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> +++ b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> @@ -189,10 +189,19 @@ def add_net_res(server, service, virt, cxml, >> vssd_ref, nasd, attr): >> % attr['nmac']) >> >> if virt == "KVM": >> - name = cxml.get_value_xpath( >> + if attr['ntype'] == "network": >> + name = cxml.get_value_xpath( >> '/domain/devices/interface/source/@network[. = "%s"]' >> % attr['net_name']) >> - >> + elif attr['ntype'] == "bridge": >> + br = get_bridge_from_network_xml(attr['net_name'], server, virt) >> + name = cxml.get_value_xpath( >> + '/domain/devices/interface/source/@bridge[. = "%s"]' >> + % br) >> + if name != None: >> + name = attr['net_name'] >> + else: >> + logger.error("Unrecognized interface type: %s" % type) >> else: >> # For Xen, network interfaces are converted to bridge interfaces. >> br = get_bridge_from_network_xml(attr['net_name'], server, virt) >> > Hi Xu Wang, > > I was trying to apply the patch on master but failed doing so. Maybe you > will need to rebase it onto current upstream? I have seen that John > Ferlan has committed something in May which looks similar to your > changes, naturally leading to a conflict. >

1 0

[PATCH 0/3] Console Fixes and Enhancements
by Viktor Mihajlovski 03 Oct '13

03 Oct '13

Enabling cimtest for the Console RASDs required a few improvements. The XML generation code was simplified, a logical device class for consoles had to be added and a bug in the KVM redirection service was fixed. Viktor Mihajlovski (3): libxkutil: Simplify XML handling of consoles Virt_Device: Add a device class for consoles KVMRedirectionSAP: Only return redirection SAPs for VNC graphics libxkutil/device_parsing.c | 12 +----- libxkutil/xmlgen.c | 76 --------------------------------- schema/DisplayController.mof | 15 +++++++ schema/DisplayController.registration | 3 ++ src/Virt_Device.c | 18 ++++---- src/Virt_ElementSettingData.c | 3 ++ src/Virt_KVMRedirectionSAP.c | 23 ++++++---- src/Virt_SettingsDefineState.c | 6 +++ src/Virt_SystemDevice.c | 3 ++ src/Virt_VSSDComponent.c | 3 ++ 10 files changed, 59 insertions(+), 103 deletions(-) -- 1.7.9.5

2 7

[PATCH] Make sure provider revision is a number
by Viktor Mihajlovski 03 Oct '13

03 Oct '13

Commit 4a7fae9 changed the revision returned by get_provider_version to a string. However, the revision has to be a number because (since git is used) it's basically the number of commits in HEAD. Comparing a string against an int in Python 2.x will always result in the string value being greater than the int value. Since there are a lot of compares against revision numbers all over the test scripts, this change is probably the most economic. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- Not being a python guy I got wacked myself when I was testing my (soon to come) cimtest changes against a pre-console libvirt-cim. '1272' > 1276 silently returned True leading to epic failures :-( suites/libvirt-cim/lib/XenKvmLib/const.py | 4 +++- suites/libvirt-cim/lib/XenKvmLib/reporting.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/suites/libvirt-cim/lib/XenKvmLib/const.py b/suites/libvirt-cim/lib/XenKvmLib/const.py index a8d0254..a454b2f 100755 --- a/suites/libvirt-cim/lib/XenKvmLib/const.py +++ b/suites/libvirt-cim/lib/XenKvmLib/const.py @@ -179,7 +179,9 @@ def get_provider_version(virt, ip): revision = revision.strip("+") if revision.isdigit(): revision = int(revision) + else: + raise Exception("revision %s is not a digit", revision) - return str(revision), str(changeset) + return revision, changeset diff --git a/suites/libvirt-cim/lib/XenKvmLib/reporting.py b/suites/libvirt-cim/lib/XenKvmLib/reporting.py index 67ec974..88375b0 100644 --- a/suites/libvirt-cim/lib/XenKvmLib/reporting.py +++ b/suites/libvirt-cim/lib/XenKvmLib/reporting.py @@ -101,7 +101,7 @@ def get_env_data(ip, virt): rev, changeset = get_provider_version(virt, ip) cimtest_revision, cimtest_changeset = get_cimtest_version() - lc_ver = "Libvirt-cim revision: %s\nLibvirt-cim changeset: %s\n" % \ + lc_ver = "Libvirt-cim revision: %d\nLibvirt-cim changeset: %s\n" % \ (rev, changeset) cimtest_ver = "Cimtest revision: %s\nCimtest changeset: %s\n" % \ (cimtest_revision, cimtest_changeset) -- 1.7.9.5

2 2

[PATCH] cimtest: Add type == 'bridge' handling branch in vsms_util.py
by Xu Wang 02 Oct '13

02 Oct '13

The name of interface should be read based on the type of it. There are two basic types for interface device (bridge and network). The name should be fetched in different way. I don't clear if Xen could support network type so I didn't make any change for it and just enhanced the logic in "virt == 'KVM'". I'll submit a set of patches and they solved XFAIL of VirtualSystemManagementService/ 22_addmulti_brg_interface.py. So it's necessary to fix it. Signed-off-by: Xu Wang <gesaint(a)linux.vnet.ibm.com> --- suites/libvirt-cim/lib/XenKvmLib/vsms_util.py | 13 +++++++++++-- 1 files changed, 11 insertions(+), 2 deletions(-) diff --git a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py index 075c09f..7839a38 100644 --- a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py +++ b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py @@ -189,10 +189,19 @@ def add_net_res(server, service, virt, cxml, vssd_ref, nasd, attr): % attr['nmac']) if virt == "KVM": - name = cxml.get_value_xpath( + if attr['ntype'] == "network": + name = cxml.get_value_xpath( '/domain/devices/interface/source/@network[. = "%s"]' % attr['net_name']) - + elif attr['ntype'] == "bridge": + br = get_bridge_from_network_xml(attr['net_name'], server, virt) + name = cxml.get_value_xpath( + '/domain/devices/interface/source/@bridge[. = "%s"]' + % br) + if name != None: + name = attr['net_name'] + else: + logger.error("Unrecognized interface type: %s" % type) else: # For Xen, network interfaces are converted to bridge interfaces. br = get_bridge_from_network_xml(attr['net_name'], server, virt) -- 1.7.1

2 1

[PATCHv2 0/7] Support for full function consoles
by Viktor Mihajlovski 13 Sep '13

13 Sep '13

Motivation: the current libvirt-cim support for consoles is very limited in that it doesn't allow to specify the target and source types supported by libvirt. Having full support of the libvirt console feature is however mandatory for architectures like s390. The current implementation using a Graphics_RASD to represent consoles cannot easily be extended without breaking the existing implementation (or without being entirely unusable). Therefore a new RASD class is added to represent consoles in the model. Since there's no SVPC resource type for a console we use the combination of ResourceType = 1 (Other) and OtherResourceType = "console" for the new RASD instances. In order to be backward compatible, the only console type supported by older libvirt-cim versions (target type 'pty') will still be returned as Graphics_RASD. In the long run, this should be deprecated. V2 Changes: - Squashed old patches 2 and 5 and moved that one to position 4 - Fix double frees - Fix NULL pointer accesses - Improve error handling - Improve error messages - Some cosmetic changes Thilo Boehm (5): schema: New SVPC types for chardev/consoles libxkutil: Console Support RASD: Schema and Provider Support for Console RASDs VSMS: Support for domains with console devices VSMS: add default console Viktor Mihajlovski (2): VSMS: Set resource types for default devices Device: CIM_LogicalDevice for consoles libxkutil/device_parsing.c | 314 ++++++++++++++++++- libxkutil/device_parsing.h | 43 ++- libxkutil/xmlgen.c | 191 ++++++++++- schema/ResourceAllocationSettingData.mof | 246 ++++++++++++++- schema/ResourceAllocationSettingData.registration | 5 +- src/Virt_Device.c | 33 ++ src/Virt_RASD.c | 149 ++++++++- src/Virt_VirtualSystemManagementService.c | 348 +++++++++++++++++++-- src/svpc_types.h | 102 +++++- 9 files changed, 1392 insertions(+), 39 deletions(-) -- 1.7.9.5

2 10

[PATCH 0/2] Remove memory leaks in XML device parsing
by Viktor Mihajlovski 12 Sep '13

12 Sep '13

Discovered a few leaks in device_parsing.c running xml_parse_test under valgrind. The series applies on master as well as on top of the last series I sent out. Viktor Mihajlovski (2): libxkutil: Plug memory leaks in device parsing xml_parse_test: Call cleanup_dominfo before exiting libxkutil/device_parsing.c | 30 ++++++++++++++++++++---------- libxkutil/xml_parse_test.c | 2 ++ 2 files changed, 22 insertions(+), 10 deletions(-) -- 1.7.9.5

2 6

[PATCH 0/8] Support for full function consoles
by Viktor Mihajlovski 11 Sep '13

11 Sep '13

Motivation: the current libvirt-cim support for consoles is very limited in that it doesn't allow to specify the target and source types supported by libvirt. Having full support of the libvirt console feature is however mandatory for architectures like s390. The current implementation using a Graphics_RASD to represent consoles cannot easily be extended without breaking the existing implementation (or without being entirely unusable). Therefore a new RASD class is added to represent consoles in the model. Since there's no SVPC resource type for a console we use the combination of ResourceType = 1 (Other) and OtherResourceType = "console" for the new RASD instances. In order to be backward compatible, the only console type supported by older libvirt-cim versions (target type 'pty') will still be returned as Graphics_RASD. In the long run, this should be deprecated. Thilo Boehm (6): schema: Add CIM_ResourceAllocationSettingData for console resources schema: New SVPC types for chardev/consoles libxkutil: Console Support RASD: Provider Support for Console RASDs VSMS: Support for domains with console devices VSMS: add default console Viktor Mihajlovski (2): VSMS: Set resource types for default devices Device: CIM_LogicalDevice for consoles libxkutil/device_parsing.c | 316 ++++++++++++++++++- libxkutil/device_parsing.h | 43 ++- libxkutil/xmlgen.c | 191 +++++++++++- schema/ResourceAllocationSettingData.mof | 247 ++++++++++++++- schema/ResourceAllocationSettingData.registration | 5 +- src/Virt_Device.c | 33 ++ src/Virt_RASD.c | 148 ++++++++- src/Virt_VirtualSystemManagementService.c | 336 +++++++++++++++++++-- src/svpc_types.h | 102 ++++++- 9 files changed, 1384 insertions(+), 37 deletions(-) -- 1.7.9.5

2 26

[PATCH 0/2] Make cimtest more s390-friendly
by Viktor Mihajlovski 11 Sep '13

11 Sep '13

cimtest currently doesn't work too well on s390 systems, the two main areas of failure are the attempt to define systems with IDE hard drivers and VGA graphics, both not supported on this host type. Changing from IDE to virtio disks fixed a lot of the test FAILs with no adverse impacts to the x86 runs on our systems at least. It would be nice if somebody could verify this. Further, we do not add the graphics RASD to the default guest definition if the hypervisor host is an s390 system. In principle, we could have removed the graphics RASD altogether, because libvirt-cim will define a default graphics device for x86 but since it is only a small change, it seems safer to do it this way. Thilo Boehm (1): cimtest: Use vda instead of hda for disk RASDs Viktor Mihajlovski (1): cimtest: Don't generate graphics for s390 .../cimtest/ElementAllocatedFromPool/01_forward.py | 2 +- .../cimtest/ElementAllocatedFromPool/02_reverse.py | 2 +- .../ElementAllocatedFromPool/03_reverse_errs.py | 2 +- .../ElementAllocatedFromPool/04_forward_errs.py | 2 +- .../cimtest/HostedAccessPoint/01_forward.py | 2 +- .../cimtest/HostedAccessPoint/02_reverse.py | 2 +- .../02_guest_add_mod_rem_rasd_ind.py | 2 +- .../cimtest/ServiceAccessBySAP/01_forward.py | 2 +- .../cimtest/ServiceAccessBySAP/02_reverse.py | 2 +- .../cimtest/SettingsDefine/01_forward.py | 2 +- .../cimtest/SettingsDefine/02_reverse.py | 2 +- .../cimtest/SettingsDefine/03_sds_fwd_errs.py | 2 +- .../cimtest/SettingsDefine/04_sds_rev_errs.py | 2 +- .../libvirt-cim/cimtest/SystemDevice/01_forward.py | 2 +- .../libvirt-cim/cimtest/SystemDevice/03_fwderrs.py | 2 +- suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py | 2 +- .../13_refconfig_additional_devs.py | 2 +- .../14_define_sys_disk.py | 2 +- .../30_dynamic_disk_mod.py | 2 +- .../02_reverse.py | 2 +- .../03_vssdc_fwd_errs.py | 2 +- suites/libvirt-cim/lib/XenKvmLib/const.py | 2 +- suites/libvirt-cim/lib/XenKvmLib/vsms.py | 2 +- suites/libvirt-cim/lib/XenKvmLib/vxml.py | 5 +++-- suites/libvirt-cim/lib/XenKvmLib/xm_virt_util.py | 7 +++++++ 25 files changed, 33 insertions(+), 25 deletions(-) -- 1.7.9.5

2 4