[libvirt] [RFC PATCHv2] network: try to eliminate default network conflict during package install
by Laine Stump
Sometimes libvirt is installed on a host that is already using the
network 192.168.122.0/24. If the libvirt-daemon-config-network package
is installed, this creates a conflict, since that package has been
hard-coded to create a virtual network that also uses
192.168.122.0/24. In the past libvirt has attempted to warn of /
remediate this situation by checking for conflicting routes when the
network is started, but it turns out that isn't always useful (for
example in the case that the *other* interface/network creating the
conflict hasn't yet been started at the time libvirtd start its own
networks).
This patch attempts to catch the problem earlier - at install
time. During the %post install script for
libvirt-daemon-config-network, we use a case statement to look through
the output of "ip route show" for a route that exactly matches
192.168.122.0/24, and if found we search for a similar route that
*doesn't* match (e.g. 192.168.123.0/24). When we find an available
route, we just replace all occurrences of "122" in the default.xml
that is being created with the newly found 192.168 subnet. This could
obviously be made more complicated - examine the template defaul.xml
to automatically determine the existing network address and mask
rather than hard coding it in the specfile, etc, but this scripting is
simpler and gets the job done as long as we continue to use
192.168.122.0/24 in the template. (If anyone with mad bash skillz
wants to suggest something to do that, by all means please do).
This is intended to at least "further reduce" occurrence of the
problems detailed in:
https://bugzilla.redhat.com/show_bug.cgi?id=811967
---
Difference from V1: fixed some typos in the commit message as pointed
out by Eric. Also switched from using comparison loops for route
matching, to using a case statement as a low cost replacement for grep
(again, as suggested by Eric).
Unless there is an objection, I will push this patch later this
afternoon so that we can try it out. In the off chance that it causes
any disasters, we have plenty of time to revert before the next
release :-)
libvirt.spec.in | 31 ++++++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/libvirt.spec.in b/libvirt.spec.in
index a6a58cf..90da0c2 100644
--- a/libvirt.spec.in
+++ b/libvirt.spec.in
@@ -1728,8 +1728,37 @@ fi
%if %{with_network}
%post daemon-config-network
if test $1 -eq 1 && test ! -f %{_sysconfdir}/libvirt/qemu/networks/default.xml ; then
+ # see if the network used by default network creates a conflict,
+ # and try to resolve it
+ # NB: 192.168.122.0/24 is used in the default.xml template file;
+ # do not modify any of those values here without also modifying
+ # them in the template.
+ orig_sub=122
+ sub=${orig_sub}
+ nl='
+'
+ routes="${nl}$(ip route show | cut -d' ' -f1)"
+ case ${routes} in
+ *"${nl}192.168.${orig_sub}.0/24${nl}"*)
+ # there was a match, so we need to look for an unused subnet
+ for new_sub in $(seq 123 254); do
+ case ${routes} in
+ *"${nl}192.168.${new_sub}.0/24${nl}"*)
+ ;;
+ *)
+ sub=$new_sub
+ break;
+ ;;
+ esac
+ done
+ ;;
+ *)
+ ;;
+ esac
+
UUID=`/usr/bin/uuidgen`
- sed -e "s,</name>,</name>\n <uuid>$UUID</uuid>," \
+ sed -e "s/${orig_sub}/${sub}/g" \
+ -e "s,</name>,</name>\n <uuid>$UUID</uuid>," \
< %{_datadir}/libvirt/networks/default.xml \
> %{_sysconfdir}/libvirt/qemu/networks/default.xml
ln -s ../default.xml %{_sysconfdir}/libvirt/qemu/networks/autostart/default.xml
--
1.9.3
10 years, 2 months
[libvirt] Inconsistent behavior between x86_64 and ppc64 when creating guests with NUMA node placement
by Michael Turek
Hello all,
I was recently trying out NUMA placement for my guests on both x86_64
and ppc64 machines. When booting a guest on the x86_64 machine, the
following specs were valid (obviously, just notable excepts from the xml):
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>4</vcpu>
...
<cpu>
<topology sockets='4' cores='1' threads='1'/>
<numa>
<cell cpus='0-2' memory='6144'/>
<cell cpus='3' memory='2048'/>
</numa>
</cpu>
However, on ppc64 this causes the following error:
error: Failed to create domain from sample_guest.xml
error: internal error: early end of file from monitor: possible problem:
2014-09-11T18:44:25.502140Z qemu-system-ppc64: total memory for NUMA
nodes (8388608) should equal RAM size (200000000)
The 200000000 is actually 8192 MB in bytes and hexidecimal. This is
apparently just an issue with the error message.
The following specs work on ppc64:
<cpu>
<topology sockets='4' cores='1' threads='1'/>
<numa>
<cell cpus='0-2' memory='6291456'/>
<cell cpus='3' memory='2097152'/>
</numa>
</cpu>
Note that the memory for each cell is 6144*1024 and 2048*1024
respectively. The issue is that the memory size for each NUMA cell
should be specified in KiB, not MB
(http://libvirt.org/formatdomain.html#resPartition "|memory| specifies
the node memory in kibibyte").
In short, it seems that specifying NUMA cell memory in MB works on
x86_64 but not on ppc64. Does anyone have any insight to what's causing
this, or if I'm misunderstanding something? Any help is appreciated,
thank you!
Regards,
Mike Turek
10 years, 2 months
[libvirt] [PATCH RFC] network: try to eliminate default network conflict during package install
by Laine Stump
Sometimes libvirt is installed on a host that is already using the
network 192.168.122.0/24. If the libvirt-daemon-config-network package
is installed, this creates a conflict, since that package has been
hard-coded to create a virtual network that also uses
192.168.122.0/24. In the past libvirt has attempted to warn of /
remediate this situation by checking for conflicting routes when the
network is started, but it turns out that isn't always useful (for
example in the case that the *other* interface/network creating the
conflict hasn't yet been started at the time libvirtd start its owm
networks).
This patch attempts to catch the problem earlier - at install
time. During the %post install for libvirt-daemon-config-network, we
look through the output of "ip route show" for a route that exactly
matches 192.1 68.122.0/24, and if found we search for a similar route
that *doesn't* match (e.g. 192.168.123.0/24). When we find an
available route, we just replace all occurences of "122" in the
default.xml that is being created with ${new_sub}. This could
obviously be made more complicated - automatically determine the
existing network address and mask from examining the template
default.xml, etc, but this scripting is simpler and gets the job done
as long as we continue to use 192.168.122.0/24 in the template. (If
anyone with mad bash skillz wants to suggest something to do that, by
all means please do).
This is intended to at least "further reduce" the problems detailed in:
https://bugzilla.redhat.com/show_bug.cgi?id=811967
I acknowledge that it doesn't help for cases of pre-built cloud images
(or live images that are created on real hardware and then run in a
virtual machine), but it should at least eliminate the troubles
encountered by individuals doing one-off installs (and could be used
to stifle complaints for live images, as long as libvirtd was running
on the system where the live image compose took place (or the compose
was itself done in a virtual machine that had a 192.168.122.0/24
interface address).
---
The question here is: "Will this help some people's situation without
causing new problems for anyone else?" I wouldn't mind pushing this
patch, but also wouldn't mind if it was just the catalyst for
discussion that leads to a better solution. We do need *some kind* of
solution though, as more and more people are installing OSes that
include the libvirt package in virtual machines, and are running into
this problem with increasing frequency.
libvirt.spec.in | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/libvirt.spec.in b/libvirt.spec.in
index a6a58cf..539d9ef 100644
--- a/libvirt.spec.in
+++ b/libvirt.spec.in
@@ -1728,8 +1728,32 @@ fi
%if %{with_network}
%post daemon-config-network
if test $1 -eq 1 && test ! -f %{_sysconfdir}/libvirt/qemu/networks/default.xml ; then
+ # see if the network used by default network creates a conflict,
+ # and try to resolve it
+ orig_sub=122
+ sub=${orig_sub}
+ net=192.168.${sub}.0/24
+ routes=$(ip route show | cut -d' ' -f1)
+ for route in $routes; do
+ if [ "${net}" = "${route}" ]; then
+ # there was a match, so we need to look for an unused subnet
+ for new_sub in $(seq 123 254); do
+ new_net="192.168.${new_sub}.0/24"
+ usable=yes
+ for route in ${routes}; do
+ [ "${new_net}" = "${route}" ] && usable=no
+ done
+ if [ "${usable}" = "yes" ]; then
+ sub=${new_sub}
+ break;
+ fi
+ done
+ fi
+ done
+
UUID=`/usr/bin/uuidgen`
- sed -e "s,</name>,</name>\n <uuid>$UUID</uuid>," \
+ sed -e "s/${orig_sub}/${sub}/g" \
+ -e "s,</name>,</name>\n <uuid>$UUID</uuid>," \
< %{_datadir}/libvirt/networks/default.xml \
> %{_sysconfdir}/libvirt/qemu/networks/default.xml
ln -s ../default.xml %{_sysconfdir}/libvirt/qemu/networks/autostart/default.xml
--
1.9.3
10 years, 2 months
[libvirt] [PATCH v4 0/7] wrap up virDomainBlockCopy
by Eric Blake
diff to v3:
patch 1 now consumes the destination in the common helper instead
of copying it
patch 2 rebased accordingly
patch 3 didn't need much work
patch 4 finally tested
patches 5-7 are new
Also available at git://repo.or.cz/libvirt/ericb.git branch blockcopy,
http://repo.or.cz/w/libvirt/ericb.git/shortlog/refs/heads/blockcopy
Eric Blake (7):
blockcopy: tweak how rebase calls into copy
blockcopy: add qemu implementation of new API
blockcopy: add qemu implementation of new tunables
blockjob: allow finer bandwidth tuning for set speed
blockjob: improve handling of bandwidth in virsh
blockjob: automatically use bytes for bandwidth in virsh
blockjob: allow forcing of bytes scale in virsh
include/libvirt/libvirt.h.in | 31 ++++--
src/libvirt.c | 83 +++++++++-----
src/qemu/qemu_driver.c | 255 +++++++++++++++++++++++++++++--------------
src/qemu/qemu_migration.c | 2 +-
src/qemu/qemu_monitor.c | 8 +-
src/qemu/qemu_monitor.h | 2 +
src/qemu/qemu_monitor_json.c | 4 +
src/qemu/qemu_monitor_json.h | 2 +
tests/qemumonitorjsontest.c | 2 +-
tools/virsh-domain.c | 226 ++++++++++++++++++++++++++++++--------
tools/virsh.pod | 50 +++++----
11 files changed, 481 insertions(+), 184 deletions(-)
--
1.9.3
10 years, 2 months
[libvirt] [PATCHv4 0/8] bulk stats: QEMU implementation
by Francesco Romani
This patchset enhances the QEMU support
for the new bulk stats API to include
equivalents of these APIs:
virDomainBlockInfo
virDomainGetInfo - for balloon stats
virDomainGetCPUStats
virDomainBlockStatsFlags
virDomainInterfaceStats
virDomainGetVcpusFlags
virDomainGetVcpus
This subset of API is the one oVirt relies on.
Scale/stress test on an oVirt test environment is in progress.
The patchset is organized as follows:
- the first patch enhances the internal stats gathering API
to accomodate the needs of the groups which extract information
using QEMU monitor jobs.
- the next five patches implement the bulk stats groups, extracting
helpers where do refactoring to extract internal helpers every time
it is feasible and convenient.
- the seventh patch enhances the virsh domstats command with options
to use the new bulk stats.
- the last patch enhances the block stats group adding the wr_highest_offset
information, needed by oVirt for thin provisioned disks.
ChangeLog
v4: fixes and cleanups
- addressed reviewers comments (Peter, Wang Rui).
- pushed domain check into group stats functions. This follows
the strategy to gather and report as much data as possible,
silently skipping errors along the way.
- moved the block allocation patch to the end of the series.
v3: more polishing and fixes after first review
- addressed Eric's comments.
- squashed patches which extracts helpers with patches which
use them.
- changed gathering strategy: now code tries to reap as much
information as possible instead to give up and bail out with
error. Only critical errors cause the bulk stats to fail.
- moved away from the transfer semantics. I find it error-prone
and not flexible enough, I'd like to avoid as much as possible.
- rearranged helpers to have one single QEMU query job with
many monitor jobs nested inside.
- fixed docs.
- implemented missing virsh domstats bits.
in v2: polishing and optimizations.
- incorporated feedback from Li Wei (thanks).
- added documentation.
- optimized block group to gather all the information with just
one call to QEMU monitor.
- stripped to bare bones merged the 'block info' group into the
'block' group - oVirt actually needs just one stat from there.
- reorganized the keys to be more consistent and shorter.
Francesco Romani (8):
qemu: bulk stats: extend internal collection API
qemu: bulk stats: implement CPU stats group
qemu: bulk stats: implement balloon group
qemu: bulk stats: implement VCPU group
qemu: bulk stats: implement interface group
qemu: bulk stats: implement block group
virsh: add options to query bulk stats group
qemu: bulk stats: add block allocation information
include/libvirt/libvirt.h.in | 5 +
src/libvirt.c | 61 +++++
src/qemu/qemu_driver.c | 575 +++++++++++++++++++++++++++++++++++++------
src/qemu/qemu_monitor.c | 26 ++
src/qemu/qemu_monitor.h | 21 ++
src/qemu/qemu_monitor_json.c | 227 ++++++++++++-----
src/qemu/qemu_monitor_json.h | 4 +
tools/virsh-domain-monitor.c | 35 +++
tools/virsh.pod | 4 +-
9 files changed, 822 insertions(+), 136 deletions(-)
--
1.9.3
10 years, 2 months
[libvirt] [PATCHv2 0/2] conf: snapshot: Don't default-snapshot empty drives
by Peter Krempa
Series contains a duplicate submission of patch 1/2 with my ohter series.
Peter Krempa (2):
util: Add function to check if a virStorageSource is "empty"
conf: snapshot: Don't default-snapshot empty drives
src/conf/snapshot_conf.c | 8 +++++++-
src/libvirt_private.syms | 1 +
src/util/virstoragefile.c | 20 ++++++++++++++++++++
src/util/virstoragefile.h | 1 +
4 files changed, 29 insertions(+), 1 deletion(-)
--
2.1.0
10 years, 2 months
[libvirt] [PATCHv2 0/2] Couple of NVRAM fixes
by Michal Privoznik
diff to v1:
-patch 1/2 (2/2 in previous version) updated according to Laszlo's review
Michal Privoznik (2):
virDomainUndefineFlags: Allow NVRAM unlinking
formatdomain: Update <loader/> example to match the rest
docs/formatdomain.html.in | 2 +-
include/libvirt/libvirt.h.in | 2 ++
src/qemu/qemu_driver.c | 20 +++++++++++++++++++-
tools/virsh-domain.c | 20 ++++++++++++++++----
tools/virsh.pod | 6 +++++-
5 files changed, 43 insertions(+), 7 deletions(-)
--
1.8.5.5
10 years, 2 months
[libvirt] [PATCH 0/9] More Coverity fixes
by John Ferlan
There are two repeats from the last series (1 & 2).
For patch 1, I went with my suggestion - I'm open to others
For patch 2, Coverity was complaining more about the way nparams
would be overwritten - fix that by adding a new variable
New patches
3 & 4 -> eblake helped out with these - especially the mgetgroups oddity
5 -> Fallout from fixing 4
6 -> virTimeFieldsThen() and the "offset = 0". I'd be OK with deleting the
code, but it just feels like someone had it on a todo list to come
back to some day
7 & 8 -> Fairly straightforward
9 -> This was an interesting case - it seems from what was being done
that I have the right "answer". I did go all the way back to the
initial submission of the code and it did the same thing, except it
was using an unsigned long instead of int and well thus wouldn't
ever hit the condition since we're grabbing the big endian int value
This gets me down to 5 issues
John Ferlan (9):
remote_driver: Resolve Coverity RESOURCE_LEAK
virsh: Resolve Coverity NEGATIVE_RETURNS
daemon: Resolve Coverity RESOURCE_LEAK
virutil: Resolve Coverity RESOURCE_LEAK
virfile: Resolve Coverity RESOURCE_LEAK
virtime: Resolve Coverity DEADCODE
qemu: Resolve Coverity FORWARD_NULL
libxl: Resolve Coverity CHECKED_RETURN
virstoragefile: Resolve Coverity DEADCODE
daemon/libvirtd.c | 8 ++++----
src/libxl/libxl_driver.c | 3 ++-
src/qemu/qemu_capabilities.c | 2 +-
src/remote/remote_driver.c | 20 +++++++++++++++++++-
src/util/virfile.c | 7 +++++--
src/util/virstoragefile.c | 2 +-
src/util/virtime.c | 2 ++
src/util/virutil.c | 1 +
tools/virsh-domain.c | 7 ++++---
9 files changed, 39 insertions(+), 13 deletions(-)
--
1.9.3
10 years, 2 months
[libvirt] [PATCH v2 0/2] parallels: use parallels SDK instead of prlctl tool
by Dmitry Guryanov
This patchset begins reworking of parallels driver. We have
published Opensource version of parallels SDK (under LGPL license),
so libvirt can link with it.
Dmitry Guryanov (2):
parallels: build with parallels SDK
parallels: login to parallels SDK
configure.ac | 24 ++--
po/POTFILES.in | 1 +
src/Makefile.am | 8 +-
src/parallels/parallels_driver.c | 40 ++++++-
src/parallels/parallels_sdk.c | 241 +++++++++++++++++++++++++++++++++++++++
src/parallels/parallels_sdk.h | 30 +++++
src/parallels/parallels_utils.h | 4 +
7 files changed, 334 insertions(+), 14 deletions(-)
create mode 100644 src/parallels/parallels_sdk.c
create mode 100644 src/parallels/parallels_sdk.h
--
1.9.3
10 years, 2 months
[libvirt] [libvirt-python][PATCH] generator: Free strings after libvirt_charPtrWrap
by Michal Privoznik
Up till bb3301ba the wrapper was freeing the passed strings for us.
However that changed after the commit. So now we don't free any
strings which results in memory leaks as reported upstream [1]:
==14265== 2,407 bytes in 1 blocks are definitely lost in loss record 1,457 of 1,550
==14265== at 0x4C2845D: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==14265== by 0x5C46624: xdr_string (in /usr/lib64/libc-2.17.so)
==14265== by 0xCFD9FCD: xdr_remote_nonnull_string (remote_protocol.c:31)
==14265== by 0xCFDC2C8: xdr_remote_domain_get_xml_desc_ret (remote_protocol.c:1617)
==14265== by 0xCFF0811: virNetMessageDecodePayload (virnetmessage.c:407)
==14265== by 0xCFE68FB: virNetClientProgramCall (virnetclientprogram.c:379)
==14265== by 0xCFBE8B1: callFull.isra.2 (remote_driver.c:6578)
==14265== by 0xCFC7F04: remoteDomainGetXMLDesc (remote_driver.c:6600)
==14265== by 0xCF8167C: virDomainGetXMLDesc (libvirt.c:4380)
==14265== by 0xCC2C4DF: libvirt_virDomainGetXMLDesc (libvirt.c:1141)
==14265== by 0x4F12B93: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==14265== by 0x4F141AC: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
The python documentation clearly advise us to call free() [2]. From
an example in their docs:
PyObject *res;
char *buf = (char *) malloc(BUFSIZ); /* for I/O */
if (buf == NULL)
return PyErr_NoMemory();
...Do some I/O operation involving buf...
res = PyString_FromString(buf);
free(buf); /* malloc'ed */
return res;
Moreover, instead of using VIR_FREE() (which we are not exporting),
I'll just go with bare free().
1: https://www.redhat.com/archives/libvir-list/2014-September/msg00736.html
2: https://docs.python.org/2/c-api/memory.html
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
While the fix seems okay, for some reason I'm seeing another valgrind warning
after the fix is applied:
==21247== Invalid read of size 4
==21247== at 0x4EBFB93: PyObject_Free (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4EB613E: insertdict_by_entry (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4EB7BAF: dict_set_item_by_hash_or_entry (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4EBC4CB: _PyModule_Clear (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F2916A: PyImport_Cleanup (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F34B0D: Py_Finalize (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F460A5: Py_Main (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x54462BF: (below main) (in /lib64/libc-2.19.so)
==21247== Address 0x69be020 is 2,016 bytes inside a block of size 4,390 free'd
==21247== at 0x4C2B5AC: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==21247== by 0x7F1EC2D: libvirt_virDomainGetXMLDesc (in /home/zippy/work/libvirt/libvirt-python.git/build/lib.linux-x86_64-2.7/libvirtmod.so)
==21247== by 0x4F180BB: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F199FF: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F18AB4: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F199FF: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F19AF8: PyEval_EvalCode (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F32EBE: run_mod (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F340E1: PyRun_FileExFlags (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F352E6: PyRun_SimpleFileExFlags (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x4F4670E: Py_Main (in /usr/lib64/libpython2.7.so.1.0)
==21247== by 0x54462BF: (below main) (in /lib64/libc-2.19.so)
This happens when using our examples/dominfo.py.
generator.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/generator.py b/generator.py
index a798274..8ee046a 100755
--- a/generator.py
+++ b/generator.py
@@ -708,12 +708,16 @@ def print_function_wrapper(module, name, output, export, include):
else:
c_call = "\n c_retval = %s(%s);\n" % (name, c_call)
ret_convert = " py_retval = libvirt_%sWrap((%s) c_retval);\n" % (n,c)
+ if n == "charPtr":
+ ret_convert = ret_convert + " free(c_retval);\n"
ret_convert = ret_convert + " return py_retval;\n"
elif ret[0] in py_return_types:
(f, t, n, c) = py_return_types[ret[0]]
c_return = " %s c_retval;\n" % (ret[0])
c_call = "\n c_retval = %s(%s);\n" % (name, c_call)
ret_convert = " py_retval = libvirt_%sWrap((%s) c_retval);\n" % (n,c)
+ if n == "charPtr":
+ ret_convert = ret_convert + " free(c_retval);\n"
ret_convert = ret_convert + " return py_retval;\n"
else:
if ret[0] in skipped_types:
--
1.8.5.5
10 years, 2 months
