October 2013 - CIM - libvirt List Archives

[PATCH 0/5] Persistent Device Address Support
by Viktor Mihajlovski 21 Nov '13

21 Nov '13

This patch set is adding persistent device address support to libvirt-cim. Now it is possible to specify device bus addresses as supported by libvirt when defining or modifying domains. This applies on top of current master but I would recommend to first apply the little/big endian series posted last week. Reviewed-by: Daniel Hansel <daniel.hansel(a)linux.vnet.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy(a)linux.vnet.ibm.com> Viktor Mihajlovski (5): RASD/schema: Add properties for device address representation libxkutil: Support for device addresses RASD: Support for device address properties VSMS: Support for device addresses VSMS: Improve device cleanup libxkutil/device_parsing.c | 113 +++++++++++++++++++++++++++++ libxkutil/device_parsing.h | 11 +++ libxkutil/xmlgen.c | 28 +++++++ schema/ResourceAllocationSettingData.mof | 12 +++ src/Virt_RASD.c | 73 +++++++++++++++++++ src/Virt_VirtualSystemManagementService.c | 86 +++++++++++++++------- 6 files changed, 295 insertions(+), 28 deletions(-) -- 1.7.9.5

2 14

[PATCH 0/2] cimtest support for consoles
by Viktor Mihajlovski 16 Nov '13

16 Nov '13

This series extends the KVM cimtests with the verification of the console instrumentation. If the tested libvirt-cim provider has the necessary release level, the test domains are instantiated with a console. The other patch deals with the fact that certain device types (like graphics) are supported on s390. I ran the tests on an x86 system as well seeing no regressions, but an independent test run would be welcome... Viktor Mihajlovski (2): cimtest: Adding support for ConsoleRASD cimtest: Fix s390 failures .../cimtest/ElementSettingData/01_forward.py | 8 +++-- .../cimtest/HostSystem/02_hostsystem_to_rasd.py | 2 ++ .../cimtest/HostSystem/04_hs_to_EAPF.py | 24 +++++++++---- .../cimtest/HostedAccessPoint/01_forward.py | 5 +++ .../cimtest/HostedAccessPoint/02_reverse.py | 5 +++ .../cimtest/KVMRedirectionSAP/01_enum_KVMredSAP.py | 5 +++ .../cimtest/KVMRedirectionSAP/02_ipv6_support.py | 5 +++ .../02_guest_add_mod_rem_rasd_ind.py | 9 +++-- .../RedirectionService/03_RedirectionSAP_errs.py | 5 +++ .../ResourceAllocationFromPool/01_forward.py | 3 ++ .../ResourceAllocationFromPool/02_reverse.py | 3 ++ .../cimtest/ServiceAccessBySAP/01_forward.py | 5 +++ .../cimtest/ServiceAccessBySAP/02_reverse.py | 6 ++++ .../cimtest/ServiceAffectsElement/01_forward.py | 6 ++-- .../cimtest/ServiceAffectsElement/02_reverse.py | 12 ++++--- .../libvirt-cim/cimtest/SystemDevice/01_forward.py | 16 +++++++-- suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py | 11 ++++-- suites/libvirt-cim/cimtest/VSSD/07_autostart.py | 5 +++ .../01_forward.py | 4 ++- .../02_reverse.py | 9 +++-- suites/libvirt-cim/lib/XenKvmLib/devices.py | 6 ++++ suites/libvirt-cim/lib/XenKvmLib/pool.py | 4 ++- suites/libvirt-cim/lib/XenKvmLib/rasd.py | 21 ++++++++++- suites/libvirt-cim/lib/XenKvmLib/vsms.py | 36 +++++++++++++++++++ suites/libvirt-cim/lib/XenKvmLib/vxml.py | 38 ++++++++++++++------ 25 files changed, 215 insertions(+), 38 deletions(-) -- 1.7.9.5

2 7

[PATCH V2 00/48] Solution to solve unsupported tag missing issue
by Xu Wang 14 Nov '13

14 Nov '13

What's new in V2: 1. Rebase commit to the current upstream. 2. Update console device support. 3. Add id and parent_id to handle there are more than two sub-node but with the same issue. 4. Add status (ACTIVE/INACTIVE) to identify if a node was fetched. 5. Adjust the original designing plan. Now after fetched a node from link list just change its status from ACTIVE to INACTIVE instead of delete it. Purpose of this update is to make link list keep the structure of xml and regenerate xml easily. 6. Necessary comments added. 7. Add function (compare_param_int/compare_param_str) to adjust the result of comparation (if any parameter is NULL/-1 the comparation result is TRUE instead of FALSE. 8. Bugs fixing. Such as attribute 'arch' under <type> of <os>, <dev> of <target> parsing, etc. -------------------------------------------------------------------------- These patches are based on commit 04bfeb825e5e155ffa72119253de608ccf3bd72b. So I may need more work on rebasing. Most of bugs reported recently are about some tags libvirt-cim doesn't support will be dropped after resource updated. So a new member was added into every virt_device and domain structure. And a new structure named unknown device was added to save those tags new added. The original implementaion is reading every member of virt_device structure from xml and save it. But the defect of this is some tags were not a member of virt_device were dropped. After resource updated, every data in the virt_device will be used to regenerate new xml. Hence the tags unsupported above, disappeared. So I added a member into every virt_device and domain structure, 'others'. It's a link list and used to save all data read from xml. Another new structure 'unknown_device' was added to save data except libvirt-cim could recognize. The new implementation is, firstly parse_*_device() could read all nodes and properties from xml and save them into 'others' link list. Then every member of virt_device will fetch data from others link list and save it. So that nodes in the 'others' link list could be saved until they are used to re- generate xml. After resource updating finished, libvirt-cim will call *_xml() to generate xml. The new process of generating xml like this, firstly all data in the members of virt_device will be restored into 'others' link list, then a function (others_to_xml) will use this link list to generate xml. Some points I have updated, 1. 'others' link list has to be processed in _get_proc_device() and _get_mem_device(). They should be copied into new data structure. 2. If resource updating happened, others field should be cleared because this device has been changed and they are useless. Besides above some logic may be a little strange or boring. Implemention like that is just to be compatible with upper layer functions (to make changes as less as possible). After keep a balance I decided to devide the whole xml into several parts: <domain>------------------------others in domain to save unsupported sub-nodes of domain -some fields (devices, mem, vcpu...) will be skiped because -they have their own parsing functions. <name>xxx</name> <uuid>xxx</uuid> <vcpu>xxx</vcpu> <mem>xxx</mem> <devices> <disk>xxx</disk>------- others in virt_device to save unsupported tags of this device <emulator>xxx</emulator> ...----------- unknown_device to save unsupported device except like <disk>,<emu>,etc. </devices> </domain> Hence, all nodes read from xml will be restored after xml generation. Xu Wang (48): Add others member for saving unsupported tag and unknown device Add others and unknown_device clean up Add basic operations for reading data from xml node Fix xml parsing algorithm for parse_fs_device() Fix xml parsing algorithm for parse_block_device() Fix xml parsing algorithm for parse_vsi_device() Fix xml parsing algorithm for parse_net_device() Fix xml parsing algorithm for parse_vcpu_device() Fix xml parsing algorithm for parse_emu_device() Fix xml parsing algorithm for parse_mem_device() Fix xml parsing algorithm for parse_console_device() Fix xml parsing algorithm for parse_graphics_device() Fix xml parsing algorithm for parse_input_device() Add parse_unknown_device() Add parse_devices() for unknown type in get_dominfo_from_xml() Fix xml parsing algorithm in parse_domain() Fix xml parsing algorithm in parse_os() Fix xml parsing algorithm in parse_feature() Add dup function for device copy Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del Add basic functions about converting others link list to xml Fix xml generation algorithm in console_xml() Fix xml generation algorithm in disk_block_xml() Fix xml generation algorithm in disk_file_xml() Fix xml generation algorithm in disk_fs_xml() Fix xml generation algorithm in set_net_vsi() Fix xml generation algorithm in set_net_source() Fix xml generation algorithm in bridge_net_to_xml() Fix xml generation algorithm in net_xml() Fix xml generation algorithm in vcpu_xml() Fix xml generation algorithm in cputune_xml() Fix xml generation algorithm in mem_xml() Fix xml generation algorithm in emu_xml() Fix xml generation algorithm in graphics_vnc_xml() Fix xml generation algorithm in input_xml() Fix xml generation algorithm in system_xml() Fix xml generation algorithm in _xenpv_os_xml() Fix xml generation algorithm in _fv_bootlist_xml() Fix xml generation algorithm in _xenfv_os_xml() Fix xml generation algorithm in _kvm_os_xml() Fix xml generation algorithm in _lxc_os_xml() Fix xml generation algorithm in os_xml() Fix xml generation algorithm in features_xml() Add functions for xml generation of unknown devices Fix xml generation algorithm in system_to_xml() Add cleanup_others() calling during rasd_to_vdev() Add others handling in _get_mem_device() Add dup_others() into _get_proc_device() libxkutil/device_parsing.c | 1959 ++++++++++++++++++++++----- libxkutil/device_parsing.h | 64 + libxkutil/xmlgen.c | 2103 +++++++++++++++++++++++------ src/Virt_VirtualSystemManagementService.c | 20 +- src/svpc_types.h | 1 + 5 files changed, 3445 insertions(+), 702 deletions(-)

3 55

[PATCH 0/4] Little/big endian and memory fixes.
by Viktor Mihajlovski 14 Nov '13

14 Nov '13

This series addresses a number of problems encountered testing libvirt-cim on s390. The first three patches deal with endian-unsafe memory access, which of course only surfaces on big endian systems. The last patch addresses a potential heap corruption/double free issue also hit during testing. Thilo Boehm (2): FilterEntry: Fix endianness issues VSSM: Fix endianness issue in domain properties Viktor Mihajlovski (2): VSDC: Fix endianess issues libxkutil: clean entire device structure to avoid memory corruption libxkutil/device_parsing.c | 4 +-- libxkutil/device_parsing.h | 8 ++--- src/Virt_FilterEntry.c | 46 +++++++++++++++-------------- src/Virt_SettingsDefineCapabilities.c | 16 +++++----- src/Virt_VirtualSystemManagementService.c | 4 +-- 5 files changed, 40 insertions(+), 38 deletions(-) -- 1.7.9.5

2 12

[PATCH 00/47] Solution to solve unsupported tag missing issue
by Xu Wang 19 Oct '13

19 Oct '13

These patches are based on commit 04bfeb825e5e155ffa72119253de608ccf3bd72b. So I may need more work on rebasing. Most of bugs reported recently are about some tags libvirt-cim doesn't support will be dropped after resource updated. So a new member was added into every virt_device and domain structure. And a new structure named unknown device was added to save those tags new added. The original implementaion is reading every member of virt_device structure from xml and save it. But the defect of this is some tags were not a member of virt_device were dropped. After resource updated, every data in the virt_device will be used to regenerate new xml. Hence the tags unsupported above, disappeared. So I added a member into every virt_device and domain structure, 'others'. It's a link list and used to save all data read from xml. Another new structure 'unknown_device' was added to save data except libvirt-cim could recognize. The new implementation is, firstly parse_*_device() could read all nodes and properties from xml and save them into 'others' link list. Then every member of virt_device will fetch data from others link list and save it. So that nodes in the 'others' link list could be saved until they are used to re- generate xml. After resource updating finished, libvirt-cim will call *_xml() to generate xml. The new process of generating xml like this, firstly all data in the members of virt_device will be restored into 'others' link list, then a function (others_to_xml) will use this link list to generate xml. Some points I have updated, 1. 'others' link list has to be processed in _get_proc_device() and _get_mem_device(). They should be copied into new data structure. 2. If resource updating happened, others field should be cleared because this device has been changed and they are useless. Besides above some logic may be a little strange or boring. Implemention like that is just to be compatible with upper layer functions (to make changes as less as possible). After keep a balance I decided to devide the whole xml into several parts: <domain>------------------------others in domain to save unsupported sub-nodes of domain -some fields (devices, mem, vcpu...) will be skiped because -they have their own parsing functions. <name>xxx</name> <uuid>xxx</uuid> <vcpu>xxx</vcpu> <mem>xxx</mem> <devices> <disk>xxx</disk>------- others in virt_device to save unsupported tags of this device <emulator>xxx</emulator> ...----------- unknown_device to save unsupported device except like <disk>,<emu>,etc. </devices> </domain> Hence, all nodes read from xml will be restored after xml generation. Xu Wang (47): Add others member for saving unsupported tag and unknown device Add others and unknown_device clean up Add basic operations for reading data from xml node Fix xml parsing algorithm for parse_fs_device() Fix xml parsing algorithm for parse_block_device() Fix xml parsing algorithm for parse_vsi_device() Fix xml parsing algorithm for parse_net_device() Fix xml parsing algorithm for parse_vcpu_device() Fix xml parsing algorithm for parse_emu_device() Fix xml parsing algorithm for parse_mem_device() Fix xml parsing algorithm for parse_graphics_device() Fix xml parsing algorithm for parse_input_device() Add parse_unknown_device() Add parse_devices() for unknown type in get_dominfo_from_xml() Fix xml parsing algorithm in parse_domain() Fix xml parsing algorithm in parse_os() Fix xml parsing algorithm in parse_feature() Add dup function for device copy Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del Add basic functions about converting others link list to xml Fix xml generation algorithm in disk_block_xml() Fix xml generation algorithm in disk_file_xml() Fix xml generation algorithm in disk_fs_xml() Fix xml generation algorithm in set_net_vsi() Fix xml generation algorithm in set_net_source() Fix xml generation algorithm in bridge_net_to_xml() Fix xml generation algorithm in net_xml() Fix xml generation algorithm in vcpu_xml() Fix xml generation algorithm in cputune_xml() Fix xml generation algorithm in mem_xml() Fix xml generation algorithm in emu_xml() Fix xml generation algorithm in graphics_vnc_xml() Fix xml generation algorithm in graphics_pty_xml() Fix xml generation algorithm in input_xml() Fix xml generation algorithm in system_xml() Fix xml generation algorithm in _xenpv_os_xml() Fix xml generation algorithm in _fv_bootlist_xml() Fix xml generation algorithm in _xenfv_os_xml() Fix xml generation algorithm in _kvm_os_xml() Fix xml generation algorithm in _lxc_os_xml() Fix xml generation algorithm in os_xml() Fix xml generation algorithm in features_xml() Add functions for xml generation of unknown devices Fix xml generation algorithm in system_to_xml() Add cleanup_others() calling during rasd_to_vdev() Add others handling in _get_mem_device() Add dup_others() into _get_proc_device() libxkutil/device_parsing.c | 1351 +++++++++++++++++++++----- libxkutil/device_parsing.h | 56 ++ libxkutil/xmlgen.c | 1470 +++++++++++++++++++++++------ src/Virt_VirtualSystemManagementService.c | 20 +- src/svpc_types.h | 1 + 5 files changed, 2346 insertions(+), 552 deletions(-)

4 52

[PATCHv2 0/3] Console Fixes and Enhancements
by Viktor Mihajlovski 11 Oct '13

11 Oct '13

Enabling cimtest for the Console RASDs required a few improvements. The XML generation code was simplified, a logical device class for consoles had to be added and a bug in the KVM redirection service was fixed. V2 Changes: - Smaller change in Virt_Device.c (2/3) - Simpler console check in Virt_KVMRedirectionSAP.c (3/3) Viktor Mihajlovski (3): libxkutil: Simplify XML handling of consoles Virt_Device: Add a device class for consoles KVMRedirectionSAP: Only return redirection SAPs for VNC graphics libxkutil/device_parsing.c | 12 +----- libxkutil/xmlgen.c | 76 --------------------------------- schema/DisplayController.mof | 15 +++++++ schema/DisplayController.registration | 3 ++ src/Virt_Device.c | 6 ++- src/Virt_ElementSettingData.c | 3 ++ src/Virt_KVMRedirectionSAP.c | 26 ++++++----- src/Virt_SettingsDefineState.c | 6 +++ src/Virt_SystemDevice.c | 3 ++ src/Virt_VSSDComponent.c | 3 ++ 10 files changed, 53 insertions(+), 100 deletions(-) -- 1.7.9.5

2 5

Re: [Libvirt-cim] [PATCH] cimtest: Add type == 'bridge' handling branch in vsms_util.py
by Xu Wang 10 Oct '13

10 Oct '13

Yeah, I found this issue has been solved by the early patch from John after git pull. Sorry working on other things so long that I am out of date :-) Please drop this patch and I think my works on libvirt-cim could work well based on upstream of cimtest. Thanks, Xu Wang 于 2013/10/2 0:34, Viktor Mihajlovski 写道: > On 09/26/2013 09:28 AM, Xu Wang wrote: >> The name of interface should be read based on the type of it. There >> are two basic types for interface device (bridge and network). The >> name should be fetched in different way. I don't clear if Xen could >> support network type so I didn't make any change for it and just >> enhanced the logic in "virt == 'KVM'". I'll submit a set of patches >> and they solved XFAIL of VirtualSystemManagementService/ >> 22_addmulti_brg_interface.py. So it's necessary to fix it. >> >> Signed-off-by: Xu Wang <gesaint(a)linux.vnet.ibm.com> >> --- >> suites/libvirt-cim/lib/XenKvmLib/vsms_util.py | 13 +++++++++++-- >> 1 files changed, 11 insertions(+), 2 deletions(-) >> >> diff --git a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> index 075c09f..7839a38 100644 >> --- a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> +++ b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py >> @@ -189,10 +189,19 @@ def add_net_res(server, service, virt, cxml, >> vssd_ref, nasd, attr): >> % attr['nmac']) >> >> if virt == "KVM": >> - name = cxml.get_value_xpath( >> + if attr['ntype'] == "network": >> + name = cxml.get_value_xpath( >> '/domain/devices/interface/source/@network[. = "%s"]' >> % attr['net_name']) >> - >> + elif attr['ntype'] == "bridge": >> + br = get_bridge_from_network_xml(attr['net_name'], server, virt) >> + name = cxml.get_value_xpath( >> + '/domain/devices/interface/source/@bridge[. = "%s"]' >> + % br) >> + if name != None: >> + name = attr['net_name'] >> + else: >> + logger.error("Unrecognized interface type: %s" % type) >> else: >> # For Xen, network interfaces are converted to bridge interfaces. >> br = get_bridge_from_network_xml(attr['net_name'], server, virt) >> > Hi Xu Wang, > > I was trying to apply the patch on master but failed doing so. Maybe you > will need to rebase it onto current upstream? I have seen that John > Ferlan has committed something in May which looks similar to your > changes, naturally leading to a conflict. >

1 0

[PATCH 0/3] Console Fixes and Enhancements
by Viktor Mihajlovski 03 Oct '13

03 Oct '13

Enabling cimtest for the Console RASDs required a few improvements. The XML generation code was simplified, a logical device class for consoles had to be added and a bug in the KVM redirection service was fixed. Viktor Mihajlovski (3): libxkutil: Simplify XML handling of consoles Virt_Device: Add a device class for consoles KVMRedirectionSAP: Only return redirection SAPs for VNC graphics libxkutil/device_parsing.c | 12 +----- libxkutil/xmlgen.c | 76 --------------------------------- schema/DisplayController.mof | 15 +++++++ schema/DisplayController.registration | 3 ++ src/Virt_Device.c | 18 ++++---- src/Virt_ElementSettingData.c | 3 ++ src/Virt_KVMRedirectionSAP.c | 23 ++++++---- src/Virt_SettingsDefineState.c | 6 +++ src/Virt_SystemDevice.c | 3 ++ src/Virt_VSSDComponent.c | 3 ++ 10 files changed, 59 insertions(+), 103 deletions(-) -- 1.7.9.5

2 7

[PATCH] Make sure provider revision is a number
by Viktor Mihajlovski 03 Oct '13

03 Oct '13

Commit 4a7fae9 changed the revision returned by get_provider_version to a string. However, the revision has to be a number because (since git is used) it's basically the number of commits in HEAD. Comparing a string against an int in Python 2.x will always result in the string value being greater than the int value. Since there are a lot of compares against revision numbers all over the test scripts, this change is probably the most economic. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- Not being a python guy I got wacked myself when I was testing my (soon to come) cimtest changes against a pre-console libvirt-cim. '1272' > 1276 silently returned True leading to epic failures :-( suites/libvirt-cim/lib/XenKvmLib/const.py | 4 +++- suites/libvirt-cim/lib/XenKvmLib/reporting.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/suites/libvirt-cim/lib/XenKvmLib/const.py b/suites/libvirt-cim/lib/XenKvmLib/const.py index a8d0254..a454b2f 100755 --- a/suites/libvirt-cim/lib/XenKvmLib/const.py +++ b/suites/libvirt-cim/lib/XenKvmLib/const.py @@ -179,7 +179,9 @@ def get_provider_version(virt, ip): revision = revision.strip("+") if revision.isdigit(): revision = int(revision) + else: + raise Exception("revision %s is not a digit", revision) - return str(revision), str(changeset) + return revision, changeset diff --git a/suites/libvirt-cim/lib/XenKvmLib/reporting.py b/suites/libvirt-cim/lib/XenKvmLib/reporting.py index 67ec974..88375b0 100644 --- a/suites/libvirt-cim/lib/XenKvmLib/reporting.py +++ b/suites/libvirt-cim/lib/XenKvmLib/reporting.py @@ -101,7 +101,7 @@ def get_env_data(ip, virt): rev, changeset = get_provider_version(virt, ip) cimtest_revision, cimtest_changeset = get_cimtest_version() - lc_ver = "Libvirt-cim revision: %s\nLibvirt-cim changeset: %s\n" % \ + lc_ver = "Libvirt-cim revision: %d\nLibvirt-cim changeset: %s\n" % \ (rev, changeset) cimtest_ver = "Cimtest revision: %s\nCimtest changeset: %s\n" % \ (cimtest_revision, cimtest_changeset) -- 1.7.9.5

2 2

[PATCH] cimtest: Add type == 'bridge' handling branch in vsms_util.py
by Xu Wang 02 Oct '13

02 Oct '13

The name of interface should be read based on the type of it. There are two basic types for interface device (bridge and network). The name should be fetched in different way. I don't clear if Xen could support network type so I didn't make any change for it and just enhanced the logic in "virt == 'KVM'". I'll submit a set of patches and they solved XFAIL of VirtualSystemManagementService/ 22_addmulti_brg_interface.py. So it's necessary to fix it. Signed-off-by: Xu Wang <gesaint(a)linux.vnet.ibm.com> --- suites/libvirt-cim/lib/XenKvmLib/vsms_util.py | 13 +++++++++++-- 1 files changed, 11 insertions(+), 2 deletions(-) diff --git a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py index 075c09f..7839a38 100644 --- a/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py +++ b/suites/libvirt-cim/lib/XenKvmLib/vsms_util.py @@ -189,10 +189,19 @@ def add_net_res(server, service, virt, cxml, vssd_ref, nasd, attr): % attr['nmac']) if virt == "KVM": - name = cxml.get_value_xpath( + if attr['ntype'] == "network": + name = cxml.get_value_xpath( '/domain/devices/interface/source/@network[. = "%s"]' % attr['net_name']) - + elif attr['ntype'] == "bridge": + br = get_bridge_from_network_xml(attr['net_name'], server, virt) + name = cxml.get_value_xpath( + '/domain/devices/interface/source/@bridge[. = "%s"]' + % br) + if name != None: + name = attr['net_name'] + else: + logger.error("Unrecognized interface type: %s" % type) else: # For Xen, network interfaces are converted to bridge interfaces. br = get_bridge_from_network_xml(attr['net_name'], server, virt) -- 1.7.1

2 1