11:29 a.m.
This is an update of Eren's v4 patch series to provide
Docker image support. In this v5 I have incorporated
fixes for all of the feedback I gave Eren against the
v4, so I think this series is ready for merging now.
One thing I still want to look at separately is how
applicable this design is to other container formats,
in particular the 'appc' specification
https://github.com/appc/spec/blob/master/spec/discovery.md
Before we make a release containing the docker impl, I
want to be confident we've not done anything silly
which will cause us compat problems if we want to
extend to cover appc later.
Daniel P Berrange (1):
Add virt-sandbox-image
Daniel P. Berrange (1):
Rename 'name' to 'template' to disambiguate
Eren Yagdiran (18):
Fix docker authentication handling
Image: Add Hooking Mechanism
Image: virt-sandbox-image default dir constants
Image: Add download function
Image: Refactor create function
Image: Add delete function
Image: Add get_command function to Source
Image: Add run args
Image: Add check_connect function
Image: Add get_disk function to Source
Image: Add run function
Image: Add network support
Image: man file for virt-sandbox-image
Add config for environment variables
Add environment parameter to virt-sandbox
init-common: Exporting custom environment variables
Image: Add custom environment support
Image: Add Volume Support
.gitignore | 1 +
bin/Makefile.am | 8 +-
bin/virt-sandbox-image | 8 +
bin/virt-sandbox-image.pod | 169 +++++++++++
bin/virt-sandbox.c | 14 +
configure.ac | 3 +
libvirt-sandbox.spec.in | 3 +
libvirt-sandbox/Makefile.am | 2 +-
libvirt-sandbox/image/Makefile.am | 10 +
libvirt-sandbox/image/__init__.py | 0
libvirt-sandbox/image/cli.py | 293 ++++++++++++++++++
libvirt-sandbox/image/sources/DockerSource.py | 416 ++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Makefile.am | 9 +
libvirt-sandbox/image/sources/Source.py | 117 ++++++++
libvirt-sandbox/image/sources/__init__.py | 0
libvirt-sandbox/libvirt-sandbox-config.c | 171 ++++++++++-
libvirt-sandbox/libvirt-sandbox-config.h | 13 +
libvirt-sandbox/libvirt-sandbox-init-common.c | 18 ++
libvirt-sandbox/libvirt-sandbox.sym | 9 +
libvirt-sandbox/tests/test-config.c | 10 +
po/POTFILES.in | 1 +
21 files changed, 1272 insertions(+), 3 deletions(-)
create mode 100644 bin/virt-sandbox-image
create mode 100644 bin/virt-sandbox-image.pod
create mode 100644 libvirt-sandbox/image/Makefile.am
create mode 100644 libvirt-sandbox/image/__init__.py
create mode 100755 libvirt-sandbox/image/cli.py
create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
create mode 100644 libvirt-sandbox/image/sources/Makefile.am
create mode 100644 libvirt-sandbox/image/sources/Source.py
create mode 100644 libvirt-sandbox/image/sources/__init__.py
--
2.4.3
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 01/20] Add virt-sandbox-image
From: Daniel P Berrange <berrange(a)redhat.com>
virt-sandbox-image.py is a python script that lets you download Docker
images easily. It is a proof of concept code and consumes Docker Rest API.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
bin/Makefile.am | 3 +-
bin/virt-sandbox-image | 8 +
configure.ac | 2 +
libvirt-sandbox.spec.in | 2 +
libvirt-sandbox/Makefile.am | 2 +-
libvirt-sandbox/image/Makefile.am | 8 +
libvirt-sandbox/image/__init__.py | 0
libvirt-sandbox/image/cli.py | 394 ++++++++++++++++++++++++++++++++++++++
po/POTFILES.in | 1 +
9 files changed, 418 insertions(+), 2 deletions(-)
create mode 100644 bin/virt-sandbox-image
create mode 100644 libvirt-sandbox/image/Makefile.am
create mode 100644 libvirt-sandbox/image/__init__.py
create mode 100644 libvirt-sandbox/image/cli.py
diff --git a/bin/Makefile.am b/bin/Makefile.am
index 416f86f..deedcf6 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -3,7 +3,8 @@ bin_PROGRAMS = virt-sandbox
libexec_PROGRAMS = virt-sandbox-service-util
-bin_SCRIPTS = virt-sandbox-service
+bin_SCRIPTS = virt-sandbox-service \
+ virt-sandbox-image
virtsandboxcompdir = $(datarootdir)/bash-completion/completions/
diff --git a/bin/virt-sandbox-image b/bin/virt-sandbox-image
new file mode 100644
index 0000000..7e0d76b
--- /dev/null
+++ b/bin/virt-sandbox-image
@@ -0,0 +1,8 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+
+from libvirt_sandbox.image import cli
+import sys
+
+if __name__ == '__main__':
+ sys.exit(cli.main())
diff --git a/configure.ac b/configure.ac
index 71c4392..a8376ca 100644
--- a/configure.ac
+++ b/configure.ac
@@ -124,10 +124,12 @@ dnl Should be in m4/virt-gettext.m4 but intltoolize is too
dnl dumb to find it there
IT_PROG_INTLTOOL([0.35.0])
+AM_PATH_PYTHON
AC_OUTPUT(Makefile
libvirt-sandbox/Makefile
libvirt-sandbox/tests/Makefile
+ libvirt-sandbox/image/Makefile
bin/Makefile
examples/Makefile
docs/Makefile
diff --git a/libvirt-sandbox.spec.in b/libvirt-sandbox.spec.in
index 4978242..54fde55 100644
--- a/libvirt-sandbox.spec.in
+++ b/libvirt-sandbox.spec.in
@@ -98,7 +98,9 @@ rm -rf $RPM_BUILD_ROOT
%dir %{_sysconfdir}/libvirt-sandbox/services
%{_bindir}/virt-sandbox
%{_bindir}/virt-sandbox-service
+%{_bindir}/virt-sandbox-image
%{_libexecdir}/virt-sandbox-service-util
+%{python_sitelib}/libvirt_sandbox
%{_mandir}/man1/virt-sandbox.1*
%{_mandir}/man1/virt-sandbox-service.1*
%{_mandir}/man1/virt-sandbox-service-*.1*
diff --git a/libvirt-sandbox/Makefile.am b/libvirt-sandbox/Makefile.am
index 597803e..b303078 100644
--- a/libvirt-sandbox/Makefile.am
+++ b/libvirt-sandbox/Makefile.am
@@ -2,7 +2,7 @@
EXTRA_DIST = libvirt-sandbox.sym
CLEANFILES =
-SUBDIRS = tests
+SUBDIRS = tests image
rundir = $(localstatedir)/run
diff --git a/libvirt-sandbox/image/Makefile.am b/libvirt-sandbox/image/Makefile.am
new file mode 100644
index 0000000..7c8da51
--- /dev/null
+++ b/libvirt-sandbox/image/Makefile.am
@@ -0,0 +1,8 @@
+
+pythonsandboxdir = $(pythondir)/libvirt_sandbox
+pythonsandbox_DATA = __init__.py
+
+pythonimagedir = $(pythondir)/libvirt_sandbox/image
+pythonimage_DATA = __init__.py cli.py
+
+EXTRA_DIST = $(pythonimage_DATA)
diff --git a/libvirt-sandbox/image/__init__.py b/libvirt-sandbox/image/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
new file mode 100644
index 0000000..ec96c7e
--- /dev/null
+++ b/libvirt-sandbox/image/cli.py
@@ -0,0 +1,394 @@
+#!/usr/bin/python -Es
+#
+# Authors: Daniel P. Berrange <berrange(a)redhat.com>
+#
+# Copyright (C) 2013 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+#
+
+import argparse
+import gettext
+import hashlib
+import json
+import os
+import os.path
+import shutil
+import sys
+import urllib2
+import subprocess
+
+default_index_server = "index.docker.io"
+default_template_dir = "/var/lib/libvirt/templates"
+
+debug = True
+verbose = True
+
+gettext.bindtextdomain("libvirt-sandbox", "/usr/share/locale")
+gettext.textdomain("libvirt-sandbox")
+try:
+ gettext.install("libvirt-sandbox",
+ localedir="/usr/share/locale",
+ unicode=False,
+ codeset = 'utf-8')
+except IOError:
+ import __builtin__
+ __builtin__.__dict__['_'] = unicode
+
+
+def debug(msg):
+ sys.stderr.write(msg)
+
+def info(msg):
+ sys.stdout.write(msg)
+
+def get_url(server, path, headers):
+ url = "https://" + server + path
+ debug(" Fetching %s..." % url)
+ req = urllib2.Request(url=url)
+
+ if json:
+ req.add_header("Accept", "application/json")
+
+ for h in headers.keys():
+ req.add_header(h, headers[h])
+
+ return urllib2.urlopen(req)
+
+def get_json(server, path, headers):
+ try:
+ res = get_url(server, path, headers)
+ data = json.loads(res.read())
+ debug("OK\n")
+ return (data, res)
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+def save_data(server, path, headers, dest, checksum=None, datalen=None):
+ try:
+ res = get_url(server, path, headers)
+
+ csum = None
+ if checksum is not None:
+ csum = hashlib.sha256()
+
+ pattern = [".", "o", "O", "o"]
+ patternIndex = 0
+ donelen = 0
+
+ with open(dest, "w") as f:
+ while 1:
+ buf = res.read(1024*64)
+ if not buf:
+ break
+ if csum is not None:
+ csum.update(buf)
+ f.write(buf)
+
+ if datalen is not None:
+ donelen = donelen + len(buf)
+ debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
+ pattern[patternIndex], (donelen/1024), (datalen/1024)
+ ))
+ patternIndex = (patternIndex + 1) % 4
+
+ debug("\x1b[K")
+ if csum is not None:
+ csumstr = "sha256:" + csum.hexdigest()
+ if csumstr != checksum:
+ debug("FAIL checksum '%s' does not match '%s'"
% (csumstr, checksum))
+ os.remove(dest)
+ raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
+ debug("OK\n")
+ return res
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+
+def download_template(name, server, destdir):
+ tag = "latest"
+
+ offset = name.find(':')
+ if offset != -1:
+ tag = name[offset + 1:]
+ name = name[0:offset]
+
+ # First we must ask the index server about the image name. THe
+ # index server will return an auth token we can use when talking
+ # to the registry server. We need this token even when anonymous
+ try:
+ (data, res) = get_json(server, "/v1/repositories/" + name +
"/images",
+ {"X-Docker-Token": "true"})
+ except urllib2.HTTPError, e:
+ raise ValueError(["Image '%s' does not exist" % name])
+
+ registryserver = res.info().getheader('X-Docker-Endpoints')
+ token = res.info().getheader('X-Docker-Token')
+ checksums = {}
+ for layer in data:
+ pass
+ # XXX Checksums here don't appear to match the data in
+ # image download later. Find out what these are sums of
+ #checksums[layer["id"]] = layer["checksum"]
+
+ # Now we ask the registry server for the list of tags associated
+ # with the image. Tags usually reflect some kind of version of
+ # the image, but they aren't officially "versions". There is
+ # always a "latest" tag which is the most recent upload
+ #
+ # We must pass in the auth token from the index server. This
+ # token can only be used once, and we're given a cookie back
+ # in return to use for later RPC calls.
+ (data, res) = get_json(registryserver, "/v1/repositories/" + name +
"/tags",
+ { "Authorization": "Token " + token })
+
+ cookie = res.info().getheader('Set-Cookie')
+
+ if not tag in data:
+ raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, name)])
+ imagetagid = data[tag]
+
+ # Only base images are self-contained, most images reference one
+ # or more parents, in a linear stack. Here we are getting the list
+ # of layers for the image with the tag we used.
+ (data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
+ { "Cookie": cookie })
+
+ if data[0] != imagetagid:
+ raise ValueError(["Expected first layer id '%s' to match image id
'%s'",
+ data[0], imagetagid])
+
+ try:
+ createdFiles = []
+ createdDirs = []
+
+ for layerid in data:
+ templatedir = destdir + "/" + layerid
+ if not os.path.exists(templatedir):
+ os.mkdir(templatedir)
+ createdDirs.append(templatedir)
+
+ jsonfile = templatedir + "/template.json"
+ datafile = templatedir + "/template.tar.gz"
+
+ if not os.path.exists(jsonfile) or not os.path.exists(datafile):
+ # The '/json' URL gives us some metadata about the layer
+ res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
+ { "Cookie": cookie }, jsonfile)
+ createdFiles.append(jsonfile)
+ layersize = int(res.info().getheader("x-docker-size"))
+
+ datacsum = None
+ if layerid in checksums:
+ datacsum = checksums[layerid]
+
+ # and the '/layer' URL is the actual payload, provided
+ # as a tar.gz archive
+ save_data(registryserver, "/v1/images/" + layerid +
"/layer",
+ { "Cookie": cookie }, datafile, datacsum, layersize)
+ createdFiles.append(datafile)
+
+ # Strangely the 'json' data for a layer doesn't include
+ # its actual name, so we save that in a json file of our own
+ index = {
+ "name": name,
+ }
+
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ with open(indexfile, "w") as f:
+ f.write(json.dumps(index))
+ except Exception, e:
+ for f in createdFiles:
+ try:
+ os.remove(f)
+ except:
+ pass
+ for d in createdDirs:
+ try:
+ os.rmdir(d)
+ except:
+ pass
+
+
+def delete_template(name, destdir):
+ imageusage = {}
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(destdir)
+ for imagetagid in imagedirs:
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile, "r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = destdir + "/" + imagetagid + "/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile, "r") as f:
+ template = json.load(f)
+
+ parent = template.get("parent", None)
+ if parent:
+ if parent not in imageusage:
+ imageusage[parent] = []
+ imageusage[parent].append(imagetagid)
+ imageparent[imagetagid] = parent
+
+ if not name in imagenames:
+ raise ValueError(["Image %s does not exist locally" % name])
+
+ imagetagid = imagenames[name]
+ while imagetagid != None:
+ debug("Remove %s\n" % imagetagid)
+ parent = imageparent.get(imagetagid, None)
+
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ if os.path.exists(indexfile):
+ os.remove(indexfile)
+ jsonfile = destdir + "/" + imagetagid + "/template.json"
+ if os.path.exists(jsonfile):
+ os.remove(jsonfile)
+ datafile = destdir + "/" + imagetagid + "/template.tar.gz"
+ if os.path.exists(datafile):
+ os.remove(datafile)
+ imagedir = destdir + "/" + imagetagid
+ os.rmdir(imagedir)
+
+ if parent:
+ if len(imageusage[parent]) != 1:
+ debug("Parent %s is shared\n" % parent)
+ parent = None
+ imagetagid = parent
+
+
+def get_image_list(name, destdir):
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(destdir)
+ for imagetagid in imagedirs:
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile, "r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = destdir + "/" + imagetagid + "/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile, "r") as f:
+ template = json.load(f)
+
+ parent = template.get("parent", None)
+ if parent:
+ imageparent[imagetagid] = parent
+
+ if not name in imagenames:
+ raise ValueError(["Image %s does not exist locally" % name])
+
+ imagetagid = imagenames[name]
+ imagelist = []
+ while imagetagid != None:
+ imagelist.append(imagetagid)
+ parent = imageparent.get(imagetagid, None)
+ imagetagid = parent
+
+ return imagelist
+
+def create_template(name, imagepath, format, destdir):
+ if not format in ["qcow2"]:
+ raise ValueError(["Unsupported image format %s" % format])
+
+ imagelist = get_image_list(name, destdir)
+ imagelist.reverse()
+
+ parentImage = None
+ for imagetagid in imagelist:
+ templateImage = destdir + "/" + imagetagid + "/template." +
format
+ cmd = ["qemu-img", "create", "-f",
"qcow2"]
+ if parentImage is not None:
+ cmd.append("-o")
+ cmd.append("backing_fmt=qcow2,backing_file=%s" % parentImage)
+ cmd.append(templateImage)
+ if parentImage is None:
+ cmd.append("10G")
+ debug("Run %s\n" % " ".join(cmd))
+ subprocess.call(cmd)
+ parentImage = templateImage
+
+def download(args):
+ info("Downloading %s from %s to %s\n" % (args.name, default_index_server,
default_template_dir))
+ download_template(args.name, default_index_server, default_template_dir)
+
+def delete(args):
+ info("Deleting %s from %s\n" % (args.name, default_template_dir))
+ delete_template(args.name, default_template_dir)
+
+def create(args):
+ info("Creating %s from %s in format %s\n" % (args.imagepath, args.name,
args.format))
+ create_template(args.name, args.imagepath, args.format, default_template_dir)
+
+def requires_name(parser):
+ parser.add_argument("name",
+ help=_("name of the template"))
+
+def gen_download_args(subparser):
+ parser = subparser.add_parser("download",
+ help=_("Download template data"))
+ requires_name(parser)
+ parser.set_defaults(func=download)
+
+def gen_delete_args(subparser):
+ parser = subparser.add_parser("delete",
+ help=_("Delete template data"))
+ requires_name(parser)
+ parser.set_defaults(func=delete)
+
+def gen_create_args(subparser):
+ parser = subparser.add_parser("create",
+ help=_("Create image from template data"))
+ requires_name(parser)
+ parser.add_argument("imagepath",
+ help=_("path for image"))
+ parser.add_argument("format",
+ help=_("format"))
+ parser.set_defaults(func=create)
+
+def main():
+ parser = argparse.ArgumentParser(description='Sandbox Container Image Tool')
+
+ subparser = parser.add_subparsers(help=_("commands"))
+ gen_download_args(subparser)
+ gen_delete_args(subparser)
+ gen_create_args(subparser)
+
+ try:
+ args = parser.parse_args()
+ args.func(args)
+ sys.exit(0)
+ except KeyboardInterrupt, e:
+ sys.exit(0)
+ except ValueError, e:
+ for line in e:
+ for l in line:
+ sys.stderr.write("%s: %s\n" % (sys.argv[0], l))
+ sys.stderr.flush()
+ sys.exit(1)
+ except IOError, e:
+ sys.stderr.write("%s: %s: %s\n" % (sys.argv[0], e.filename, e.reason))
+ sys.stderr.flush()
+ sys.exit(1)
+ except OSError, e:
+ sys.stderr.write("%s: %s\n" % (sys.argv[0], e))
+ sys.stderr.flush()
+ sys.exit(1)
diff --git a/po/POTFILES.in b/po/POTFILES.in
index afcb050..724c49c 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -11,3 +11,4 @@ libvirt-sandbox/libvirt-sandbox-context-interactive.c
libvirt-sandbox/libvirt-sandbox-init-common.c
libvirt-sandbox/libvirt-sandbox-rpcpacket.c
libvirt-sandbox/libvirt-sandbox-util.c
+libvirt-sandbox/image/cli.py
--
2.4.3
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 02/20] Rename 'name' to 'template' to disambiguate
Multiple objects have names, and it is desirable to reserve
the 'name' arg to refer to the name of the sandbox instance,
so rename 'name' to 'template'.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index ec96c7e..89eef1a 100644
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -327,37 +327,37 @@ def create_template(name, imagepath, format, destdir):
parentImage = templateImage
def download(args):
- info("Downloading %s from %s to %s\n" % (args.name, default_index_server,
default_template_dir))
- download_template(args.name, default_index_server, default_template_dir)
+ info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
+ download_template(args.template, default_index_server, default_template_dir)
def delete(args):
- info("Deleting %s from %s\n" % (args.name, default_template_dir))
- delete_template(args.name, default_template_dir)
+ info("Deleting %s from %s\n" % (args.template, default_template_dir))
+ delete_template(args.template, default_template_dir)
def create(args):
- info("Creating %s from %s in format %s\n" % (args.imagepath, args.name,
args.format))
- create_template(args.name, args.imagepath, args.format, default_template_dir)
+ info("Creating %s from %s in format %s\n" % (args.imagepath, args.template,
args.format))
+ create_template(args.template, args.imagepath, args.format, default_template_dir)
-def requires_name(parser):
- parser.add_argument("name",
+def requires_template(parser):
+ parser.add_argument("template",
help=_("name of the template"))
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
- requires_name(parser)
+ requires_template(parser)
parser.set_defaults(func=download)
def gen_delete_args(subparser):
parser = subparser.add_parser("delete",
help=_("Delete template data"))
- requires_name(parser)
+ requires_template(parser)
parser.set_defaults(func=delete)
def gen_create_args(subparser):
parser = subparser.add_parser("create",
help=_("Create image from template data"))
- requires_name(parser)
+ requires_template(parser)
parser.add_argument("imagepath",
help=_("path for image"))
parser.add_argument("format",
--
2.4.3
4:02 a.m.
New subject: [libvirt] [PATCH sandbox v5 02/20] Rename 'name' to 'template' to disambiguate
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
Multiple objects have names, and it is desirable to reserve
the 'name' arg to refer to the name of the sandbox instance,
so rename 'name' to 'template'.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index ec96c7e..89eef1a 100644
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -327,37 +327,37 @@ def create_template(name, imagepath, format, destdir):
parentImage = templateImage
def download(args):
- info("Downloading %s from %s to %s\n" % (args.name, default_index_server,
default_template_dir))
- download_template(args.name, default_index_server, default_template_dir)
+ info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
+ download_template(args.template, default_index_server, default_template_dir)
def delete(args):
- info("Deleting %s from %s\n" % (args.name, default_template_dir))
- delete_template(args.name, default_template_dir)
+ info("Deleting %s from %s\n" % (args.template, default_template_dir))
+ delete_template(args.template, default_template_dir)
def create(args):
- info("Creating %s from %s in format %s\n" % (args.imagepath, args.name,
args.format))
- create_template(args.name, args.imagepath, args.format, default_template_dir)
+ info("Creating %s from %s in format %s\n" % (args.imagepath,
args.template, args.format))
+ create_template(args.template, args.imagepath, args.format, default_template_dir)
-def requires_name(parser):
- parser.add_argument("name",
+def requires_template(parser):
+ parser.add_argument("template",
help=_("name of the template"))
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
- requires_name(parser)
+ requires_template(parser)
parser.set_defaults(func=download)
def gen_delete_args(subparser):
parser = subparser.add_parser("delete",
help=_("Delete template data"))
- requires_name(parser)
+ requires_template(parser)
parser.set_defaults(func=delete)
def gen_create_args(subparser):
parser = subparser.add_parser("create",
help=_("Create image from template data"))
- requires_name(parser)
+ requires_template(parser)
parser.add_argument("imagepath",
help=_("path for image"))
parser.add_argument("format",
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 03/20] Fix docker authentication handling
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Authentication fix for Docker REST API.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 89eef1a..ea04820 100644
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -1,8 +1,10 @@
#!/usr/bin/python -Es
#
# Authors: Daniel P. Berrange <berrange(a)redhat.com>
+# Eren Yagdiran <erenyagdiran(a)gmail.com>
#
# Copyright (C) 2013 Red Hat, Inc.
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -166,7 +168,7 @@ def download_template(name, server, destdir):
# or more parents, in a linear stack. Here we are getting the list
# of layers for the image with the tag we used.
(data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
- { "Cookie": cookie })
+ { "Authorization": "Token " + token })
if data[0] != imagetagid:
raise ValueError(["Expected first layer id '%s' to match image id
'%s'",
@@ -188,9 +190,9 @@ def download_template(name, server, destdir):
if not os.path.exists(jsonfile) or not os.path.exists(datafile):
# The '/json' URL gives us some metadata about the layer
res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
- { "Cookie": cookie }, jsonfile)
+ { "Authorization": "Token " + token
}, jsonfile)
createdFiles.append(jsonfile)
- layersize = int(res.info().getheader("x-docker-size"))
+ layersize = int(res.info().getheader("Content-Length"))
datacsum = None
if layerid in checksums:
@@ -199,7 +201,7 @@ def download_template(name, server, destdir):
# and the '/layer' URL is the actual payload, provided
# as a tar.gz archive
save_data(registryserver, "/v1/images/" + layerid +
"/layer",
- { "Cookie": cookie }, datafile, datacsum, layersize)
+ { "Authorization": "Token " + token },
datafile, datacsum, layersize)
createdFiles.append(datafile)
# Strangely the 'json' data for a layer doesn't include
--
2.4.3
4:03 a.m.
New subject: [libvirt] [PATCH sandbox v5 03/20] Fix docker authentication handling
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Authentication fix for Docker REST API.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 89eef1a..ea04820 100644
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -1,8 +1,10 @@
#!/usr/bin/python -Es
#
# Authors: Daniel P. Berrange <berrange(a)redhat.com>
+# Eren Yagdiran <erenyagdiran(a)gmail.com>
#
# Copyright (C) 2013 Red Hat, Inc.
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -166,7 +168,7 @@ def download_template(name, server, destdir):
# or more parents, in a linear stack. Here we are getting the list
# of layers for the image with the tag we used.
(data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
- { "Cookie": cookie })
+ { "Authorization": "Token " + token })
if data[0] != imagetagid:
raise ValueError(["Expected first layer id '%s' to match image id
'%s'",
@@ -188,9 +190,9 @@ def download_template(name, server, destdir):
if not os.path.exists(jsonfile) or not os.path.exists(datafile):
# The '/json' URL gives us some metadata about the layer
res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
- { "Cookie": cookie }, jsonfile)
+ { "Authorization": "Token " + token
}, jsonfile)
createdFiles.append(jsonfile)
- layersize = int(res.info().getheader("x-docker-size"))
+ layersize = int(res.info().getheader("Content-Length"))
datacsum = None
if layerid in checksums:
@@ -199,7 +201,7 @@ def download_template(name, server, destdir):
# and the '/layer' URL is the actual payload, provided
# as a tar.gz archive
save_data(registryserver, "/v1/images/" + layerid +
"/layer",
- { "Cookie": cookie }, datafile, datacsum,
layersize)
+ { "Authorization": "Token " + token },
datafile, datacsum, layersize)
createdFiles.append(datafile)
# Strangely the 'json' data for a layer doesn't include
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 04/20] Image: Add Hooking Mechanism
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Define a 'Source' class which is an abstract base to use for
different template repository sources. Initially there will be
a docker source which can talk to the Docker repository REST
API, but others may follow.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
.gitignore | 1 +
configure.ac | 1 +
libvirt-sandbox/image/Makefile.am | 2 ++
libvirt-sandbox/image/cli.py | 15 ++++++++------
libvirt-sandbox/image/sources/Makefile.am | 8 ++++++++
libvirt-sandbox/image/sources/Source.py | 33 +++++++++++++++++++++++++++++++
libvirt-sandbox/image/sources/__init__.py | 0
7 files changed, 54 insertions(+), 6 deletions(-)
mode change 100644 => 100755 libvirt-sandbox/image/cli.py
create mode 100644 libvirt-sandbox/image/sources/Makefile.am
create mode 100644 libvirt-sandbox/image/sources/Source.py
create mode 100644 libvirt-sandbox/image/sources/__init__.py
diff --git a/.gitignore b/.gitignore
index f77ea12..390d65c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@ libvirt-sandbox/LibvirtSandbox-1.0.typelib
*.lo
*.la
*.o
+*.pyc
*~
libvirt-sandbox/libvirt-sandbox-init-lxc
libvirt-sandbox/libvirt-sandbox-init-common
diff --git a/configure.ac b/configure.ac
index a8376ca..92d65f4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -130,6 +130,7 @@ AC_OUTPUT(Makefile
libvirt-sandbox/Makefile
libvirt-sandbox/tests/Makefile
libvirt-sandbox/image/Makefile
+ libvirt-sandbox/image/sources/Makefile
bin/Makefile
examples/Makefile
docs/Makefile
diff --git a/libvirt-sandbox/image/Makefile.am b/libvirt-sandbox/image/Makefile.am
index 7c8da51..344a881 100644
--- a/libvirt-sandbox/image/Makefile.am
+++ b/libvirt-sandbox/image/Makefile.am
@@ -1,4 +1,6 @@
+SUBDIRS = sources
+
pythonsandboxdir = $(pythondir)/libvirt_sandbox
pythonsandbox_DATA = __init__.py
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
old mode 100644
new mode 100755
index ea04820..3bf7d58
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -1,5 +1,5 @@
#!/usr/bin/python -Es
-#
+# -*- coding: utf-8 -*-
# Authors: Daniel P. Berrange <berrange(a)redhat.com>
# Eren Yagdiran <erenyagdiran(a)gmail.com>
#
@@ -32,11 +32,14 @@ import sys
import urllib2
import subprocess
-default_index_server = "index.docker.io"
-default_template_dir = "/var/lib/libvirt/templates"
-
-debug = True
-verbose = True
+import importlib
+def dynamic_source_loader(name):
+ name = name[0].upper() + name[1:]
+ modname = "libvirt_sandbox.image.sources." + name + "Source"
+ mod = importlib.import_module(modname)
+ classname = name + "Source"
+ classimpl = getattr(mod, classname)
+ return classimpl()
gettext.bindtextdomain("libvirt-sandbox", "/usr/share/locale")
gettext.textdomain("libvirt-sandbox")
diff --git a/libvirt-sandbox/image/sources/Makefile.am
b/libvirt-sandbox/image/sources/Makefile.am
new file mode 100644
index 0000000..48d0f33
--- /dev/null
+++ b/libvirt-sandbox/image/sources/Makefile.am
@@ -0,0 +1,8 @@
+
+pythonimagedir = $(pythondir)/libvirt_sandbox/image/sources
+pythonimage_DATA = \
+ __init__.py \
+ Source.py \
+ $(NULL)
+
+EXTRA_DIST = $(pythonimage_DATA)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
new file mode 100644
index 0000000..f12b0eb
--- /dev/null
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -0,0 +1,33 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Author: Eren Yagdiran <erenyagdiran(a)gmail.com>
+
+from abc import ABCMeta, abstractmethod
+
+class Source():
+ '''The Source class defines the base interface for
+ all image provider source implementations. An image
+ provide source is able to download templates from
+ a repository, convert them to a host specific image
+ format and report commands used to invoke them.'''
+
+ __metaclass__ = ABCMeta
+ def __init__(self):
+ pass
diff --git a/libvirt-sandbox/image/sources/__init__.py
b/libvirt-sandbox/image/sources/__init__.py
new file mode 100644
index 0000000..e69de29
--
2.4.3
6:42 a.m.
New subject: [libvirt] [PATCH sandbox v5 04/20] Image: Add Hooking Mechanism
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Define a 'Source' class which is an abstract base to use for
different template repository sources. Initially there will be
a docker source which can talk to the Docker repository REST
API, but others may follow.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
.gitignore | 1 +
configure.ac | 1 +
libvirt-sandbox/image/Makefile.am | 2 ++
libvirt-sandbox/image/cli.py | 15 ++++++++------
libvirt-sandbox/image/sources/Makefile.am | 8 ++++++++
libvirt-sandbox/image/sources/Source.py | 33 +++++++++++++++++++++++++++++++
libvirt-sandbox/image/sources/__init__.py | 0
7 files changed, 54 insertions(+), 6 deletions(-)
mode change 100644 => 100755 libvirt-sandbox/image/cli.py
create mode 100644 libvirt-sandbox/image/sources/Makefile.am
create mode 100644 libvirt-sandbox/image/sources/Source.py
create mode 100644 libvirt-sandbox/image/sources/__init__.py
diff --git a/.gitignore b/.gitignore
index f77ea12..390d65c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@ libvirt-sandbox/LibvirtSandbox-1.0.typelib
*.lo
*.la
*.o
+*.pyc
*~
libvirt-sandbox/libvirt-sandbox-init-lxc
libvirt-sandbox/libvirt-sandbox-init-common
diff --git a/configure.ac b/configure.ac
index a8376ca..92d65f4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -130,6 +130,7 @@ AC_OUTPUT(Makefile
libvirt-sandbox/Makefile
libvirt-sandbox/tests/Makefile
libvirt-sandbox/image/Makefile
+ libvirt-sandbox/image/sources/Makefile
bin/Makefile
examples/Makefile
docs/Makefile
diff --git a/libvirt-sandbox/image/Makefile.am b/libvirt-sandbox/image/Makefile.am
index 7c8da51..344a881 100644
--- a/libvirt-sandbox/image/Makefile.am
+++ b/libvirt-sandbox/image/Makefile.am
@@ -1,4 +1,6 @@
+SUBDIRS = sources
+
pythonsandboxdir = $(pythondir)/libvirt_sandbox
pythonsandbox_DATA = __init__.py
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
old mode 100644
new mode 100755
index ea04820..3bf7d58
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -1,5 +1,5 @@
#!/usr/bin/python -Es
-#
+# -*- coding: utf-8 -*-
# Authors: Daniel P. Berrange <berrange(a)redhat.com>
# Eren Yagdiran <erenyagdiran(a)gmail.com>
#
@@ -32,11 +32,14 @@ import sys
import urllib2
import subprocess
-default_index_server = "index.docker.io"
-default_template_dir = "/var/lib/libvirt/templates"
-
-debug = True
-verbose = True
+import importlib
+def dynamic_source_loader(name):
+ name = name[0].upper() + name[1:]
+ modname = "libvirt_sandbox.image.sources." + name + "Source"
+ mod = importlib.import_module(modname)
+ classname = name + "Source"
+ classimpl = getattr(mod, classname)
+ return classimpl()
gettext.bindtextdomain("libvirt-sandbox", "/usr/share/locale")
gettext.textdomain("libvirt-sandbox")
diff --git a/libvirt-sandbox/image/sources/Makefile.am
b/libvirt-sandbox/image/sources/Makefile.am
new file mode 100644
index 0000000..48d0f33
--- /dev/null
+++ b/libvirt-sandbox/image/sources/Makefile.am
@@ -0,0 +1,8 @@
+
+pythonimagedir = $(pythondir)/libvirt_sandbox/image/sources
+pythonimage_DATA = \
+ __init__.py \
+ Source.py \
+ $(NULL)
+
+EXTRA_DIST = $(pythonimage_DATA)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
new file mode 100644
index 0000000..f12b0eb
--- /dev/null
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -0,0 +1,33 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Author: Eren Yagdiran <erenyagdiran(a)gmail.com>
+
+from abc import ABCMeta, abstractmethod
+
+class Source():
+ '''The Source class defines the base interface for
+ all image provider source implementations. An image
+ provide source is able to download templates from
+ a repository, convert them to a host specific image
+ format and report commands used to invoke them.'''
+
+ __metaclass__ = ABCMeta
+ def __init__(self):
+ pass
diff --git a/libvirt-sandbox/image/sources/__init__.py
b/libvirt-sandbox/image/sources/__init__.py
new file mode 100644
index 0000000..e69de29
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 05/20] Image: virt-sandbox-image default dir constants
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Define some constants to refer to the default image and
template storage directories
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 3bf7d58..de34321 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -32,6 +32,16 @@ import sys
import urllib2
import subprocess
+if os.geteuid() == 0:
+ default_template_dir = "/var/lib/libvirt/templates"
+ default_image_dir = "/var/lib/libvirt/images"
+else:
+ default_template_dir = os.environ['HOME'] +
"/.local/share/libvirt/templates"
+ default_image_dir = os.environ['HOME'] +
"/.local/share/libvirt/images"
+
+debug = False
+verbose = False
+
import importlib
def dynamic_source_loader(name):
name = name[0].upper() + name[1:]
--
2.4.3
6:43 a.m.
New subject: [libvirt] [PATCH sandbox v5 05/20] Image: virt-sandbox-image default dir constants
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Define some constants to refer to the default image and
template storage directories
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 3bf7d58..de34321 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -32,6 +32,16 @@ import sys
import urllib2
import subprocess
+if os.geteuid() == 0:
+ default_template_dir = "/var/lib/libvirt/templates"
+ default_image_dir = "/var/lib/libvirt/images"
+else:
+ default_template_dir = os.environ['HOME'] +
"/.local/share/libvirt/templates"
+ default_image_dir = os.environ['HOME'] +
"/.local/share/libvirt/images"
+
+debug = False
+verbose = False
+
import importlib
def dynamic_source_loader(name):
name = name[0].upper() + name[1:]
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 06/20] Image: Add download function
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Refactor download function from virt-sandbox-image to use
the newly introduced Source abstract class. The docker-specific
download code is moved to a new DockerSource class.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 204 ++++---------------------
libvirt-sandbox/image/sources/DockerSource.py | 209 ++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Makefile.am | 1 +
libvirt-sandbox/image/sources/Source.py | 15 ++
4 files changed, 257 insertions(+), 172 deletions(-)
create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index de34321..7af617e 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -69,176 +69,6 @@ def debug(msg):
def info(msg):
sys.stdout.write(msg)
-def get_url(server, path, headers):
- url = "https://" + server + path
- debug(" Fetching %s..." % url)
- req = urllib2.Request(url=url)
-
- if json:
- req.add_header("Accept", "application/json")
-
- for h in headers.keys():
- req.add_header(h, headers[h])
-
- return urllib2.urlopen(req)
-
-def get_json(server, path, headers):
- try:
- res = get_url(server, path, headers)
- data = json.loads(res.read())
- debug("OK\n")
- return (data, res)
- except Exception, e:
- debug("FAIL %s\n" % str(e))
- raise
-
-def save_data(server, path, headers, dest, checksum=None, datalen=None):
- try:
- res = get_url(server, path, headers)
-
- csum = None
- if checksum is not None:
- csum = hashlib.sha256()
-
- pattern = [".", "o", "O", "o"]
- patternIndex = 0
- donelen = 0
-
- with open(dest, "w") as f:
- while 1:
- buf = res.read(1024*64)
- if not buf:
- break
- if csum is not None:
- csum.update(buf)
- f.write(buf)
-
- if datalen is not None:
- donelen = donelen + len(buf)
- debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
- pattern[patternIndex], (donelen/1024), (datalen/1024)
- ))
- patternIndex = (patternIndex + 1) % 4
-
- debug("\x1b[K")
- if csum is not None:
- csumstr = "sha256:" + csum.hexdigest()
- if csumstr != checksum:
- debug("FAIL checksum '%s' does not match '%s'"
% (csumstr, checksum))
- os.remove(dest)
- raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
- debug("OK\n")
- return res
- except Exception, e:
- debug("FAIL %s\n" % str(e))
- raise
-
-
-def download_template(name, server, destdir):
- tag = "latest"
-
- offset = name.find(':')
- if offset != -1:
- tag = name[offset + 1:]
- name = name[0:offset]
-
- # First we must ask the index server about the image name. THe
- # index server will return an auth token we can use when talking
- # to the registry server. We need this token even when anonymous
- try:
- (data, res) = get_json(server, "/v1/repositories/" + name +
"/images",
- {"X-Docker-Token": "true"})
- except urllib2.HTTPError, e:
- raise ValueError(["Image '%s' does not exist" % name])
-
- registryserver = res.info().getheader('X-Docker-Endpoints')
- token = res.info().getheader('X-Docker-Token')
- checksums = {}
- for layer in data:
- pass
- # XXX Checksums here don't appear to match the data in
- # image download later. Find out what these are sums of
- #checksums[layer["id"]] = layer["checksum"]
-
- # Now we ask the registry server for the list of tags associated
- # with the image. Tags usually reflect some kind of version of
- # the image, but they aren't officially "versions". There is
- # always a "latest" tag which is the most recent upload
- #
- # We must pass in the auth token from the index server. This
- # token can only be used once, and we're given a cookie back
- # in return to use for later RPC calls.
- (data, res) = get_json(registryserver, "/v1/repositories/" + name +
"/tags",
- { "Authorization": "Token " + token })
-
- cookie = res.info().getheader('Set-Cookie')
-
- if not tag in data:
- raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, name)])
- imagetagid = data[tag]
-
- # Only base images are self-contained, most images reference one
- # or more parents, in a linear stack. Here we are getting the list
- # of layers for the image with the tag we used.
- (data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
- { "Authorization": "Token " + token })
-
- if data[0] != imagetagid:
- raise ValueError(["Expected first layer id '%s' to match image id
'%s'",
- data[0], imagetagid])
-
- try:
- createdFiles = []
- createdDirs = []
-
- for layerid in data:
- templatedir = destdir + "/" + layerid
- if not os.path.exists(templatedir):
- os.mkdir(templatedir)
- createdDirs.append(templatedir)
-
- jsonfile = templatedir + "/template.json"
- datafile = templatedir + "/template.tar.gz"
-
- if not os.path.exists(jsonfile) or not os.path.exists(datafile):
- # The '/json' URL gives us some metadata about the layer
- res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
- { "Authorization": "Token " + token
}, jsonfile)
- createdFiles.append(jsonfile)
- layersize = int(res.info().getheader("Content-Length"))
-
- datacsum = None
- if layerid in checksums:
- datacsum = checksums[layerid]
-
- # and the '/layer' URL is the actual payload, provided
- # as a tar.gz archive
- save_data(registryserver, "/v1/images/" + layerid +
"/layer",
- { "Authorization": "Token " + token },
datafile, datacsum, layersize)
- createdFiles.append(datafile)
-
- # Strangely the 'json' data for a layer doesn't include
- # its actual name, so we save that in a json file of our own
- index = {
- "name": name,
- }
-
- indexfile = destdir + "/" + imagetagid + "/index.json"
- with open(indexfile, "w") as f:
- f.write(json.dumps(index))
- except Exception, e:
- for f in createdFiles:
- try:
- os.remove(f)
- except:
- pass
- for d in createdDirs:
- try:
- os.rmdir(d)
- except:
- pass
-
-
def delete_template(name, destdir):
imageusage = {}
imageparent = {}
@@ -342,8 +172,16 @@ def create_template(name, imagepath, format, destdir):
parentImage = templateImage
def download(args):
- info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
- download_template(args.template, default_index_server, default_template_dir)
+ try:
+ dynamic_source_loader(args.source).download_template(templatename=args.template,
+
templatedir=args.template_dir,
+ registry=args.registry,
+ username=args.username,
+ password=args.password)
+ except IOError,e:
+ print "Source %s cannot be found in given path" %args.source
+ except Exception,e:
+ print "Download Error %s" % str(e)
def delete(args):
info("Deleting %s from %s\n" % (args.template, default_template_dir))
@@ -357,10 +195,32 @@ def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
+def requires_source(parser):
+ parser.add_argument("-s","--source",
+ default="docker",
+ help=_("name of the template"))
+
+def requires_auth_conn(parser):
+ parser.add_argument("-r","--registry",
+ help=_("Url of the custom registry"))
+ parser.add_argument("-u","--username",
+ help=_("Username for the custom registry"))
+ parser.add_argument("-p","--password",
+ help=_("Password for the custom registry"))
+
+def requires_template_dir(parser):
+ global default_template_dir
+ parser.add_argument("-t","--template-dir",
+ default=default_template_dir,
+ help=_("Template directory for saving templates"))
+
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
requires_template(parser)
+ requires_source(parser)
+ requires_auth_conn(parser)
+ requires_template_dir(parser)
parser.set_defaults(func=download)
def gen_delete_args(subparser):
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
new file mode 100644
index 0000000..37b40dc
--- /dev/null
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -0,0 +1,209 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Author: Eren Yagdiran <erenyagdiran(a)gmail.com>
+#
+
+from Source import Source
+import urllib2
+import sys
+import json
+import traceback
+import os
+import subprocess
+import shutil
+
+class DockerSource(Source):
+
+ www_auth_username = None
+ www_auth_password = None
+
+ def __init__(self):
+ self.default_index_server = "index.docker.io"
+
+ def _check_cert_validate(self):
+ major = sys.version_info.major
+ SSL_WARNING = "SSL certificates couldn't be validated by default. You
need to have 2.7.9/3.4.3 or higher"
+ SSL_WARNING +="\nSee https://bugs.python.org/issue22417\n"
+ py2_7_9_hexversion = 34015728
+ py3_4_3_hexversion = 50594800
+ if (major == 2 and sys.hexversion < py2_7_9_hexversion) or (major == 3 and
sys.hexversion < py3_4_3_hexversion):
+ sys.stderr.write(SSL_WARNING)
+
+ def download_template(self, templatename, templatedir,
+ registry=None, username=None, password=None):
+ if registry is None:
+ registry = self.default_index_server
+
+ if username is not None:
+ self.www_auth_username = username
+ self.www_auth_password = password
+
+ self._check_cert_validate()
+ tag = "latest"
+ offset = templatename.find(':')
+ if offset != -1:
+ tag = templatename[offset + 1:]
+ templatename = templatename[0:offset]
+ try:
+ (data, res) = self._get_json(registry, "/v1/repositories/" +
templatename + "/images",
+ {"X-Docker-Token": "true"})
+ except urllib2.HTTPError, e:
+ raise ValueError(["Image '%s' does not exist" %
templatename])
+
+ registryendpoint = res.info().getheader('X-Docker-Endpoints')
+ token = res.info().getheader('X-Docker-Token')
+ checksums = {}
+ for layer in data:
+ pass
+ (data, res) = self._get_json(registryendpoint, "/v1/repositories/" +
templatename + "/tags",
+ { "Authorization": "Token " + token })
+
+ cookie = res.info().getheader('Set-Cookie')
+
+ if not tag in data:
+ raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, templatename)])
+ imagetagid = data[tag]
+
+ (data, res) = self._get_json(registryendpoint, "/v1/images/" +
imagetagid + "/ancestry",
+ { "Authorization": "Token "+token })
+
+ if data[0] != imagetagid:
+ raise ValueError(["Expected first layer id '%s' to match image
id '%s'",
+ data[0], imagetagid])
+
+ try:
+ createdFiles = []
+ createdDirs = []
+
+ for layerid in data:
+ layerdir = templatedir + "/" + layerid
+ if not os.path.exists(layerdir):
+ os.makedirs(layerdir)
+ createdDirs.append(layerdir)
+
+ jsonfile = layerdir + "/template.json"
+ datafile = layerdir + "/template.tar.gz"
+
+ if not os.path.exists(jsonfile) or not os.path.exists(datafile):
+ res = self._save_data(registryendpoint, "/v1/images/" +
layerid + "/json",
+ { "Authorization": "Token " + token
}, jsonfile)
+ createdFiles.append(jsonfile)
+
+ layersize = int(res.info().getheader("Content-Length"))
+
+ datacsum = None
+ if layerid in checksums:
+ datacsum = checksums[layerid]
+
+ self._save_data(registryendpoint, "/v1/images/" + layerid +
"/layer",
+ { "Authorization": "Token "+token },
datafile, datacsum, layersize)
+ createdFiles.append(datafile)
+
+ index = {
+ "name": templatename,
+ }
+
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ print("Index file " + indexfile)
+ with open(indexfile, "w") as f:
+ f.write(json.dumps(index))
+ except Exception as e:
+ traceback.print_exc()
+ for f in createdFiles:
+ try:
+ os.remove(f)
+ except:
+ pass
+ for d in createdDirs:
+ try:
+ shutil.rmtree(d)
+ except:
+ pass
+ def _save_data(self,server, path, headers, dest, checksum=None, datalen=None):
+ try:
+ res = self._get_url(server, path, headers)
+
+ csum = None
+ if checksum is not None:
+ csum = hashlib.sha256()
+
+ pattern = [".", "o", "O", "o"]
+ patternIndex = 0
+ donelen = 0
+
+ with open(dest, "w") as f:
+ while 1:
+ buf = res.read(1024*64)
+ if not buf:
+ break
+ if csum is not None:
+ csum.update(buf)
+ f.write(buf)
+
+ if datalen is not None:
+ donelen = donelen + len(buf)
+ debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
+ pattern[patternIndex], (donelen/1024), (datalen/1024)
+ ))
+ patternIndex = (patternIndex + 1) % 4
+
+ debug("\x1b[K")
+ if csum is not None:
+ csumstr = "sha256:" + csum.hexdigest()
+ if csumstr != checksum:
+ debug("FAIL checksum '%s' does not match
'%s'" % (csumstr, checksum))
+ os.remove(dest)
+ raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
+ debug("OK\n")
+ return res
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+ def _get_url(self,server, path, headers):
+ url = "https://" + server + path
+ debug("Fetching %s..." % url)
+
+ req = urllib2.Request(url=url)
+ if json:
+ req.add_header("Accept", "application/json")
+ for h in headers.keys():
+ req.add_header(h, headers[h])
+
+ #www Auth header starts
+ if self.www_auth_username is not None:
+ base64string = base64.encodestring('%s:%s' % (self.www_auth_username,
self.www_auth_password)).replace('\n', '')
+ req.add_header("Authorization", "Basic %s" %
base64string)
+ #www Auth header finish
+
+ return urllib2.urlopen(req)
+
+ def _get_json(self,server, path, headers):
+ try:
+ res = self._get_url(server, path, headers)
+ data = json.loads(res.read())
+ debug("OK\n")
+ return (data, res)
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+def debug(msg):
+ sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Makefile.am
b/libvirt-sandbox/image/sources/Makefile.am
index 48d0f33..069557d 100644
--- a/libvirt-sandbox/image/sources/Makefile.am
+++ b/libvirt-sandbox/image/sources/Makefile.am
@@ -3,6 +3,7 @@ pythonimagedir = $(pythondir)/libvirt_sandbox/image/sources
pythonimage_DATA = \
__init__.py \
Source.py \
+ DockerSource.py \
$(NULL)
EXTRA_DIST = $(pythonimage_DATA)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index f12b0eb..81f5176 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -31,3 +31,18 @@ class Source():
__metaclass__ = ABCMeta
def __init__(self):
pass
+
+ @abstractmethod
+ def download_template(self, templatename, templatedir,
+ registry=None, username=None, password=None):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to store the template
+ :param registry: optional hostname of image registry server
+ :param username: optional username to authenticate against registry server
+ :param password: optional password to authenticate against registry server
+
+ Download a template from the registry, storing it in the local
+ filesystem
+ """
+ pass
--
2.4.3
6:50 a.m.
New subject: [libvirt] [PATCH sandbox v5 06/20] Image: Add download function
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Refactor download function from virt-sandbox-image to use
the newly introduced Source abstract class. The docker-specific
download code is moved to a new DockerSource class.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 204 ++++---------------------
libvirt-sandbox/image/sources/DockerSource.py | 209 ++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Makefile.am | 1 +
libvirt-sandbox/image/sources/Source.py | 15 ++
4 files changed, 257 insertions(+), 172 deletions(-)
create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index de34321..7af617e 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -69,176 +69,6 @@ def debug(msg):
def info(msg):
sys.stdout.write(msg)
-def get_url(server, path, headers):
- url = "https://" + server + path
- debug(" Fetching %s..." % url)
- req = urllib2.Request(url=url)
-
- if json:
- req.add_header("Accept", "application/json")
-
- for h in headers.keys():
- req.add_header(h, headers[h])
-
- return urllib2.urlopen(req)
-
-def get_json(server, path, headers):
- try:
- res = get_url(server, path, headers)
- data = json.loads(res.read())
- debug("OK\n")
- return (data, res)
- except Exception, e:
- debug("FAIL %s\n" % str(e))
- raise
-
-def save_data(server, path, headers, dest, checksum=None, datalen=None):
- try:
- res = get_url(server, path, headers)
-
- csum = None
- if checksum is not None:
- csum = hashlib.sha256()
-
- pattern = [".", "o", "O", "o"]
- patternIndex = 0
- donelen = 0
-
- with open(dest, "w") as f:
- while 1:
- buf = res.read(1024*64)
- if not buf:
- break
- if csum is not None:
- csum.update(buf)
- f.write(buf)
-
- if datalen is not None:
- donelen = donelen + len(buf)
- debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
- pattern[patternIndex], (donelen/1024), (datalen/1024)
- ))
- patternIndex = (patternIndex + 1) % 4
-
- debug("\x1b[K")
- if csum is not None:
- csumstr = "sha256:" + csum.hexdigest()
- if csumstr != checksum:
- debug("FAIL checksum '%s' does not match '%s'"
% (csumstr, checksum))
- os.remove(dest)
- raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
- debug("OK\n")
- return res
- except Exception, e:
- debug("FAIL %s\n" % str(e))
- raise
-
-
-def download_template(name, server, destdir):
- tag = "latest"
-
- offset = name.find(':')
- if offset != -1:
- tag = name[offset + 1:]
- name = name[0:offset]
-
- # First we must ask the index server about the image name. THe
- # index server will return an auth token we can use when talking
- # to the registry server. We need this token even when anonymous
- try:
- (data, res) = get_json(server, "/v1/repositories/" + name +
"/images",
- {"X-Docker-Token": "true"})
- except urllib2.HTTPError, e:
- raise ValueError(["Image '%s' does not exist" % name])
-
- registryserver = res.info().getheader('X-Docker-Endpoints')
- token = res.info().getheader('X-Docker-Token')
- checksums = {}
- for layer in data:
- pass
- # XXX Checksums here don't appear to match the data in
- # image download later. Find out what these are sums of
- #checksums[layer["id"]] = layer["checksum"]
-
- # Now we ask the registry server for the list of tags associated
- # with the image. Tags usually reflect some kind of version of
- # the image, but they aren't officially "versions". There is
- # always a "latest" tag which is the most recent upload
- #
- # We must pass in the auth token from the index server. This
- # token can only be used once, and we're given a cookie back
- # in return to use for later RPC calls.
- (data, res) = get_json(registryserver, "/v1/repositories/" + name +
"/tags",
- { "Authorization": "Token " + token })
-
- cookie = res.info().getheader('Set-Cookie')
-
- if not tag in data:
- raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, name)])
- imagetagid = data[tag]
-
- # Only base images are self-contained, most images reference one
- # or more parents, in a linear stack. Here we are getting the list
- # of layers for the image with the tag we used.
- (data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
- { "Authorization": "Token " + token })
-
- if data[0] != imagetagid:
- raise ValueError(["Expected first layer id '%s' to match image id
'%s'",
- data[0], imagetagid])
-
- try:
- createdFiles = []
- createdDirs = []
-
- for layerid in data:
- templatedir = destdir + "/" + layerid
- if not os.path.exists(templatedir):
- os.mkdir(templatedir)
- createdDirs.append(templatedir)
-
- jsonfile = templatedir + "/template.json"
- datafile = templatedir + "/template.tar.gz"
-
- if not os.path.exists(jsonfile) or not os.path.exists(datafile):
- # The '/json' URL gives us some metadata about the layer
- res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
- { "Authorization": "Token " + token
}, jsonfile)
- createdFiles.append(jsonfile)
- layersize = int(res.info().getheader("Content-Length"))
-
- datacsum = None
- if layerid in checksums:
- datacsum = checksums[layerid]
-
- # and the '/layer' URL is the actual payload, provided
- # as a tar.gz archive
- save_data(registryserver, "/v1/images/" + layerid +
"/layer",
- { "Authorization": "Token " + token },
datafile, datacsum, layersize)
- createdFiles.append(datafile)
-
- # Strangely the 'json' data for a layer doesn't include
- # its actual name, so we save that in a json file of our own
- index = {
- "name": name,
- }
-
- indexfile = destdir + "/" + imagetagid + "/index.json"
- with open(indexfile, "w") as f:
- f.write(json.dumps(index))
- except Exception, e:
- for f in createdFiles:
- try:
- os.remove(f)
- except:
- pass
- for d in createdDirs:
- try:
- os.rmdir(d)
- except:
- pass
-
-
def delete_template(name, destdir):
imageusage = {}
imageparent = {}
@@ -342,8 +172,16 @@ def create_template(name, imagepath, format, destdir):
parentImage = templateImage
def download(args):
- info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
- download_template(args.template, default_index_server, default_template_dir)
+ try:
+
dynamic_source_loader(args.source).download_template(templatename=args.template,
+
templatedir=args.template_dir,
+ registry=args.registry,
+ username=args.username,
+ password=args.password)
+ except IOError,e:
+ print "Source %s cannot be found in given path" %args.source
+ except Exception,e:
+ print "Download Error %s" % str(e)
def delete(args):
info("Deleting %s from %s\n" % (args.template, default_template_dir))
@@ -357,10 +195,32 @@ def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
+def requires_source(parser):
+ parser.add_argument("-s","--source",
+ default="docker",
+ help=_("name of the template"))
+
+def requires_auth_conn(parser):
+ parser.add_argument("-r","--registry",
+ help=_("Url of the custom registry"))
This wording really sounds docker-specific. The registry word only fits
docker terminology, would surely not apply to virt-builder or appc case.
Something like "images storage" would may be more generic. The problem
is that "repository" has a special meaning in the docker terminology.
+ parser.add_argument("-u","--username",
+ help=_("Username for the custom registry"))
+ parser.add_argument("-p","--password",
+ help=_("Password for the custom registry"))
+
+def requires_template_dir(parser):
+ global default_template_dir
+ parser.add_argument("-t","--template-dir",
+ default=default_template_dir,
+ help=_("Template directory for saving templates"))
+
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
requires_template(parser)
+ requires_source(parser)
+ requires_auth_conn(parser)
+ requires_template_dir(parser)
parser.set_defaults(func=download)
def gen_delete_args(subparser):
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
new file mode 100644
index 0000000..37b40dc
--- /dev/null
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -0,0 +1,209 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 Universitat Politècnica de Catalunya.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Author: Eren Yagdiran <erenyagdiran(a)gmail.com>
+#
+
+from Source import Source
+import urllib2
+import sys
+import json
+import traceback
+import os
+import subprocess
+import shutil
+
+class DockerSource(Source):
+
+ www_auth_username = None
+ www_auth_password = None
+
+ def __init__(self):
+ self.default_index_server = "index.docker.io"
+
+ def _check_cert_validate(self):
+ major = sys.version_info.major
+ SSL_WARNING = "SSL certificates couldn't be validated by default. You
need to have 2.7.9/3.4.3 or higher"
+ SSL_WARNING +="\nSee https://bugs.python.org/issue22417\n"
+ py2_7_9_hexversion = 34015728
+ py3_4_3_hexversion = 50594800
+ if (major == 2 and sys.hexversion < py2_7_9_hexversion) or (major == 3 and
sys.hexversion < py3_4_3_hexversion):
+ sys.stderr.write(SSL_WARNING)
+
+ def download_template(self, templatename, templatedir,
+ registry=None, username=None, password=None):
+ if registry is None:
+ registry = self.default_index_server
+
+ if username is not None:
+ self.www_auth_username = username
+ self.www_auth_password = password
+
+ self._check_cert_validate()
+ tag = "latest"
+ offset = templatename.find(':')
+ if offset != -1:
+ tag = templatename[offset + 1:]
+ templatename = templatename[0:offset]
+ try:
+ (data, res) = self._get_json(registry, "/v1/repositories/" +
templatename + "/images",
+ {"X-Docker-Token": "true"})
+ except urllib2.HTTPError, e:
+ raise ValueError(["Image '%s' does not exist" %
templatename])
+
+ registryendpoint = res.info().getheader('X-Docker-Endpoints')
+ token = res.info().getheader('X-Docker-Token')
+ checksums = {}
+ for layer in data:
+ pass
+ (data, res) = self._get_json(registryendpoint, "/v1/repositories/" +
templatename + "/tags",
+ { "Authorization": "Token " + token })
+
+ cookie = res.info().getheader('Set-Cookie')
+
+ if not tag in data:
+ raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, templatename)])
+ imagetagid = data[tag]
+
+ (data, res) = self._get_json(registryendpoint, "/v1/images/" +
imagetagid + "/ancestry",
+ { "Authorization": "Token "+token })
+
+ if data[0] != imagetagid:
+ raise ValueError(["Expected first layer id '%s' to match image
id '%s'",
+ data[0], imagetagid])
+
+ try:
+ createdFiles = []
+ createdDirs = []
+
+ for layerid in data:
+ layerdir = templatedir + "/" + layerid
+ if not os.path.exists(layerdir):
+ os.makedirs(layerdir)
+ createdDirs.append(layerdir)
+
+ jsonfile = layerdir + "/template.json"
+ datafile = layerdir + "/template.tar.gz"
+
+ if not os.path.exists(jsonfile) or not os.path.exists(datafile):
+ res = self._save_data(registryendpoint, "/v1/images/" +
layerid + "/json",
+ { "Authorization": "Token " + token
}, jsonfile)
+ createdFiles.append(jsonfile)
+
+ layersize = int(res.info().getheader("Content-Length"))
+
+ datacsum = None
+ if layerid in checksums:
+ datacsum = checksums[layerid]
+
+ self._save_data(registryendpoint, "/v1/images/" + layerid
+ "/layer",
+ { "Authorization": "Token "+token },
datafile, datacsum, layersize)
+ createdFiles.append(datafile)
+
+ index = {
+ "name": templatename,
+ }
+
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ print("Index file " + indexfile)
+ with open(indexfile, "w") as f:
+ f.write(json.dumps(index))
+ except Exception as e:
+ traceback.print_exc()
+ for f in createdFiles:
+ try:
+ os.remove(f)
+ except:
+ pass
+ for d in createdDirs:
+ try:
+ shutil.rmtree(d)
+ except:
+ pass
+ def _save_data(self,server, path, headers, dest, checksum=None, datalen=None):
+ try:
+ res = self._get_url(server, path, headers)
+
+ csum = None
+ if checksum is not None:
+ csum = hashlib.sha256()
+
+ pattern = [".", "o", "O", "o"]
+ patternIndex = 0
+ donelen = 0
+
+ with open(dest, "w") as f:
+ while 1:
+ buf = res.read(1024*64)
+ if not buf:
+ break
+ if csum is not None:
+ csum.update(buf)
+ f.write(buf)
+
+ if datalen is not None:
+ donelen = donelen + len(buf)
+ debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
+ pattern[patternIndex], (donelen/1024), (datalen/1024)
+ ))
+ patternIndex = (patternIndex + 1) % 4
+
+ debug("\x1b[K")
+ if csum is not None:
+ csumstr = "sha256:" + csum.hexdigest()
+ if csumstr != checksum:
+ debug("FAIL checksum '%s' does not match
'%s'" % (csumstr, checksum))
+ os.remove(dest)
+ raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
+ debug("OK\n")
+ return res
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+ def _get_url(self,server, path, headers):
+ url = "https://" + server + path
+ debug("Fetching %s..." % url)
+
+ req = urllib2.Request(url=url)
+ if json:
+ req.add_header("Accept", "application/json")
+ for h in headers.keys():
+ req.add_header(h, headers[h])
+
+ #www Auth header starts
+ if self.www_auth_username is not None:
+ base64string = base64.encodestring('%s:%s' %
(self.www_auth_username, self.www_auth_password)).replace('\n', '')
+ req.add_header("Authorization", "Basic %s" %
base64string)
+ #www Auth header finish
+
+ return urllib2.urlopen(req)
+
+ def _get_json(self,server, path, headers):
+ try:
+ res = self._get_url(server, path, headers)
+ data = json.loads(res.read())
+ debug("OK\n")
+ return (data, res)
+ except Exception, e:
+ debug("FAIL %s\n" % str(e))
+ raise
+
+def debug(msg):
+ sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Makefile.am
b/libvirt-sandbox/image/sources/Makefile.am
index 48d0f33..069557d 100644
--- a/libvirt-sandbox/image/sources/Makefile.am
+++ b/libvirt-sandbox/image/sources/Makefile.am
@@ -3,6 +3,7 @@ pythonimagedir = $(pythondir)/libvirt_sandbox/image/sources
pythonimage_DATA = \
__init__.py \
Source.py \
+ DockerSource.py \
$(NULL)
EXTRA_DIST = $(pythonimage_DATA)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index f12b0eb..81f5176 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -31,3 +31,18 @@ class Source():
__metaclass__ = ABCMeta
def __init__(self):
pass
+
+ @abstractmethod
+ def download_template(self, templatename, templatedir,
+ registry=None, username=None, password=None):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to store the template
+ :param registry: optional hostname of image registry server
+ :param username: optional username to authenticate against registry server
+ :param password: optional password to authenticate against registry server
+
+ Download a template from the registry, storing it in the local
+ filesystem
+ """
+ pass
ACK, but may need some thinking on the "Registry" word.
--
Cedric
6:55 a.m.
New subject: [libvirt] [PATCH sandbox v5 06/20] Image: Add download function
On Wed, Sep 09, 2015 at 01:50:11PM +0200, Cedric Bosdonnat wrote:
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
> From: Eren Yagdiran <erenyagdiran(a)gmail.com>
>
> Refactor download function from virt-sandbox-image to use
> the newly introduced Source abstract class. The docker-specific
> download code is moved to a new DockerSource class.
>
> Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
> ---
> libvirt-sandbox/image/cli.py | 204 ++++---------------------
> libvirt-sandbox/image/sources/DockerSource.py | 209 ++++++++++++++++++++++++++
> libvirt-sandbox/image/sources/Makefile.am | 1 +
> libvirt-sandbox/image/sources/Source.py | 15 ++
> 4 files changed, 257 insertions(+), 172 deletions(-)
> create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
>
> diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
> index de34321..7af617e 100755
> --- a/libvirt-sandbox/image/cli.py
> +++ b/libvirt-sandbox/image/cli.py
> @@ -69,176 +69,6 @@ def debug(msg):
> def info(msg):
> sys.stdout.write(msg)
>
> -def get_url(server, path, headers):
> - url = "https://" + server + path
> - debug(" Fetching %s..." % url)
> - req = urllib2.Request(url=url)
> -
> - if json:
> - req.add_header("Accept", "application/json")
> -
> - for h in headers.keys():
> - req.add_header(h, headers[h])
> -
> - return urllib2.urlopen(req)
> -
> -def get_json(server, path, headers):
> - try:
> - res = get_url(server, path, headers)
> - data = json.loads(res.read())
> - debug("OK\n")
> - return (data, res)
> - except Exception, e:
> - debug("FAIL %s\n" % str(e))
> - raise
> -
> -def save_data(server, path, headers, dest, checksum=None, datalen=None):
> - try:
> - res = get_url(server, path, headers)
> -
> - csum = None
> - if checksum is not None:
> - csum = hashlib.sha256()
> -
> - pattern = [".", "o", "O", "o"]
> - patternIndex = 0
> - donelen = 0
> -
> - with open(dest, "w") as f:
> - while 1:
> - buf = res.read(1024*64)
> - if not buf:
> - break
> - if csum is not None:
> - csum.update(buf)
> - f.write(buf)
> -
> - if datalen is not None:
> - donelen = donelen + len(buf)
> - debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
> - pattern[patternIndex], (donelen/1024), (datalen/1024)
> - ))
> - patternIndex = (patternIndex + 1) % 4
> -
> - debug("\x1b[K")
> - if csum is not None:
> - csumstr = "sha256:" + csum.hexdigest()
> - if csumstr != checksum:
> - debug("FAIL checksum '%s' does not match
'%s'" % (csumstr, checksum))
> - os.remove(dest)
> - raise IOError("Checksum '%s' for data does not match
'%s'" % (csumstr, checksum))
> - debug("OK\n")
> - return res
> - except Exception, e:
> - debug("FAIL %s\n" % str(e))
> - raise
> -
> -
> -def download_template(name, server, destdir):
> - tag = "latest"
> -
> - offset = name.find(':')
> - if offset != -1:
> - tag = name[offset + 1:]
> - name = name[0:offset]
> -
> - # First we must ask the index server about the image name. THe
> - # index server will return an auth token we can use when talking
> - # to the registry server. We need this token even when anonymous
> - try:
> - (data, res) = get_json(server, "/v1/repositories/" + name +
"/images",
> - {"X-Docker-Token": "true"})
> - except urllib2.HTTPError, e:
> - raise ValueError(["Image '%s' does not exist" % name])
> -
> - registryserver = res.info().getheader('X-Docker-Endpoints')
> - token = res.info().getheader('X-Docker-Token')
> - checksums = {}
> - for layer in data:
> - pass
> - # XXX Checksums here don't appear to match the data in
> - # image download later. Find out what these are sums of
> - #checksums[layer["id"]] = layer["checksum"]
> -
> - # Now we ask the registry server for the list of tags associated
> - # with the image. Tags usually reflect some kind of version of
> - # the image, but they aren't officially "versions". There is
> - # always a "latest" tag which is the most recent upload
> - #
> - # We must pass in the auth token from the index server. This
> - # token can only be used once, and we're given a cookie back
> - # in return to use for later RPC calls.
> - (data, res) = get_json(registryserver, "/v1/repositories/" + name +
"/tags",
> - { "Authorization": "Token " + token
})
> -
> - cookie = res.info().getheader('Set-Cookie')
> -
> - if not tag in data:
> - raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, name)])
> - imagetagid = data[tag]
> -
> - # Only base images are self-contained, most images reference one
> - # or more parents, in a linear stack. Here we are getting the list
> - # of layers for the image with the tag we used.
> - (data, res) = get_json(registryserver, "/v1/images/" + imagetagid +
"/ancestry",
> - { "Authorization": "Token " + token
})
> -
> - if data[0] != imagetagid:
> - raise ValueError(["Expected first layer id '%s' to match image
id '%s'",
> - data[0], imagetagid])
> -
> - try:
> - createdFiles = []
> - createdDirs = []
> -
> - for layerid in data:
> - templatedir = destdir + "/" + layerid
> - if not os.path.exists(templatedir):
> - os.mkdir(templatedir)
> - createdDirs.append(templatedir)
> -
> - jsonfile = templatedir + "/template.json"
> - datafile = templatedir + "/template.tar.gz"
> -
> - if not os.path.exists(jsonfile) or not os.path.exists(datafile):
> - # The '/json' URL gives us some metadata about the layer
> - res = save_data(registryserver, "/v1/images/" + layerid +
"/json",
> - { "Authorization": "Token " +
token }, jsonfile)
> - createdFiles.append(jsonfile)
> - layersize = int(res.info().getheader("Content-Length"))
> -
> - datacsum = None
> - if layerid in checksums:
> - datacsum = checksums[layerid]
> -
> - # and the '/layer' URL is the actual payload, provided
> - # as a tar.gz archive
> - save_data(registryserver, "/v1/images/" + layerid +
"/layer",
> - { "Authorization": "Token " + token
}, datafile, datacsum, layersize)
> - createdFiles.append(datafile)
> -
> - # Strangely the 'json' data for a layer doesn't include
> - # its actual name, so we save that in a json file of our own
> - index = {
> - "name": name,
> - }
> -
> - indexfile = destdir + "/" + imagetagid + "/index.json"
> - with open(indexfile, "w") as f:
> - f.write(json.dumps(index))
> - except Exception, e:
> - for f in createdFiles:
> - try:
> - os.remove(f)
> - except:
> - pass
> - for d in createdDirs:
> - try:
> - os.rmdir(d)
> - except:
> - pass
> -
> -
> def delete_template(name, destdir):
> imageusage = {}
> imageparent = {}
> @@ -342,8 +172,16 @@ def create_template(name, imagepath, format, destdir):
> parentImage = templateImage
>
> def download(args):
> - info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
> - download_template(args.template, default_index_server, default_template_dir)
> + try:
> +
dynamic_source_loader(args.source).download_template(templatename=args.template,
> +
templatedir=args.template_dir,
> +
registry=args.registry,
> +
username=args.username,
> +
password=args.password)
> + except IOError,e:
> + print "Source %s cannot be found in given path" %args.source
> + except Exception,e:
> + print "Download Error %s" % str(e)
>
> def delete(args):
> info("Deleting %s from %s\n" % (args.template,
default_template_dir))
> @@ -357,10 +195,32 @@ def requires_template(parser):
> parser.add_argument("template",
> help=_("name of the template"))
>
> +def requires_source(parser):
> + parser.add_argument("-s","--source",
> + default="docker",
> + help=_("name of the template"))
> +
> +def requires_auth_conn(parser):
> + parser.add_argument("-r","--registry",
> + help=_("Url of the custom registry"))
This wording really sounds docker-specific. The registry word only fits
docker terminology, would surely not apply to virt-builder or appc case.
Something like "images storage" would may be more generic. The problem
is that "repository" has a special meaning in the docker terminology.
Yeah, I was thinking we might need some way to let source subclasses
add custom arguments. eg so we'd have --docker-registry.
The appc spec doesn't have a concept of a central registry in the
same way as docker. Instead the image name encodes the domain
name, eg example.com/someapp, and there's a protocol to use
this to identify the server, and then resolve the download URL
ACK, but may need some thinking on the "Registry" word.
I'm inclined to merge this, and then fix it afterwards to be more
generic
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
7:19 a.m.
New subject: [libvirt] [PATCH sandbox v5 06/20] Image: Add download function
On Wed, 2015-09-09 at 12:55 +0100, Daniel P. Berrange wrote:
On Wed, Sep 09, 2015 at 01:50:11PM +0200, Cedric Bosdonnat wrote:
> On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
> > From: Eren Yagdiran <erenyagdiran(a)gmail.com>
> >
> > Refactor download function from virt-sandbox-image to use
> > the newly introduced Source abstract class. The docker-specific
> > download code is moved to a new DockerSource class.
> >
> > Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
> > ---
> > libvirt-sandbox/image/cli.py | 204 ++++---------------------
> > libvirt-sandbox/image/sources/DockerSource.py | 209
++++++++++++++++++++++++++
> > libvirt-sandbox/image/sources/Makefile.am | 1 +
> > libvirt-sandbox/image/sources/Source.py | 15 ++
> > 4 files changed, 257 insertions(+), 172 deletions(-)
> > create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
> >
> > diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
> > index de34321..7af617e 100755
> > --- a/libvirt-sandbox/image/cli.py
> > +++ b/libvirt-sandbox/image/cli.py
> > @@ -69,176 +69,6 @@ def debug(msg):
> > def info(msg):
> > sys.stdout.write(msg)
> >
> > -def get_url(server, path, headers):
> > - url = "https://" + server + path
> > - debug(" Fetching %s..." % url)
> > - req = urllib2.Request(url=url)
> > -
> > - if json:
> > - req.add_header("Accept", "application/json")
> > -
> > - for h in headers.keys():
> > - req.add_header(h, headers[h])
> > -
> > - return urllib2.urlopen(req)
> > -
> > -def get_json(server, path, headers):
> > - try:
> > - res = get_url(server, path, headers)
> > - data = json.loads(res.read())
> > - debug("OK\n")
> > - return (data, res)
> > - except Exception, e:
> > - debug("FAIL %s\n" % str(e))
> > - raise
> > -
> > -def save_data(server, path, headers, dest, checksum=None, datalen=None):
> > - try:
> > - res = get_url(server, path, headers)
> > -
> > - csum = None
> > - if checksum is not None:
> > - csum = hashlib.sha256()
> > -
> > - pattern = [".", "o", "O",
"o"]
> > - patternIndex = 0
> > - donelen = 0
> > -
> > - with open(dest, "w") as f:
> > - while 1:
> > - buf = res.read(1024*64)
> > - if not buf:
> > - break
> > - if csum is not None:
> > - csum.update(buf)
> > - f.write(buf)
> > -
> > - if datalen is not None:
> > - donelen = donelen + len(buf)
> > - debug("\x1b[s%s (%5d Kb of %5d Kb)\x1b8" % (
> > - pattern[patternIndex], (donelen/1024), (datalen/1024)
> > - ))
> > - patternIndex = (patternIndex + 1) % 4
> > -
> > - debug("\x1b[K")
> > - if csum is not None:
> > - csumstr = "sha256:" + csum.hexdigest()
> > - if csumstr != checksum:
> > - debug("FAIL checksum '%s' does not match
'%s'" % (csumstr, checksum))
> > - os.remove(dest)
> > - raise IOError("Checksum '%s' for data does not
match '%s'" % (csumstr, checksum))
> > - debug("OK\n")
> > - return res
> > - except Exception, e:
> > - debug("FAIL %s\n" % str(e))
> > - raise
> > -
> > -
> > -def download_template(name, server, destdir):
> > - tag = "latest"
> > -
> > - offset = name.find(':')
> > - if offset != -1:
> > - tag = name[offset + 1:]
> > - name = name[0:offset]
> > -
> > - # First we must ask the index server about the image name. THe
> > - # index server will return an auth token we can use when talking
> > - # to the registry server. We need this token even when anonymous
> > - try:
> > - (data, res) = get_json(server, "/v1/repositories/" + name +
"/images",
> > - {"X-Docker-Token":
"true"})
> > - except urllib2.HTTPError, e:
> > - raise ValueError(["Image '%s' does not exist" %
name])
> > -
> > - registryserver = res.info().getheader('X-Docker-Endpoints')
> > - token = res.info().getheader('X-Docker-Token')
> > - checksums = {}
> > - for layer in data:
> > - pass
> > - # XXX Checksums here don't appear to match the data in
> > - # image download later. Find out what these are sums of
> > - #checksums[layer["id"]] = layer["checksum"]
> > -
> > - # Now we ask the registry server for the list of tags associated
> > - # with the image. Tags usually reflect some kind of version of
> > - # the image, but they aren't officially "versions". There
is
> > - # always a "latest" tag which is the most recent upload
> > - #
> > - # We must pass in the auth token from the index server. This
> > - # token can only be used once, and we're given a cookie back
> > - # in return to use for later RPC calls.
> > - (data, res) = get_json(registryserver, "/v1/repositories/" +
name + "/tags",
> > - { "Authorization": "Token " +
token })
> > -
> > - cookie = res.info().getheader('Set-Cookie')
> > -
> > - if not tag in data:
> > - raise ValueError(["Tag '%s' does not exist for image
'%s'" % (tag, name)])
> > - imagetagid = data[tag]
> > -
> > - # Only base images are self-contained, most images reference one
> > - # or more parents, in a linear stack. Here we are getting the list
> > - # of layers for the image with the tag we used.
> > - (data, res) = get_json(registryserver, "/v1/images/" +
imagetagid + "/ancestry",
> > - { "Authorization": "Token " +
token })
> > -
> > - if data[0] != imagetagid:
> > - raise ValueError(["Expected first layer id '%s' to match
image id '%s'",
> > - data[0], imagetagid])
> > -
> > - try:
> > - createdFiles = []
> > - createdDirs = []
> > -
> > - for layerid in data:
> > - templatedir = destdir + "/" + layerid
> > - if not os.path.exists(templatedir):
> > - os.mkdir(templatedir)
> > - createdDirs.append(templatedir)
> > -
> > - jsonfile = templatedir + "/template.json"
> > - datafile = templatedir + "/template.tar.gz"
> > -
> > - if not os.path.exists(jsonfile) or not os.path.exists(datafile):
> > - # The '/json' URL gives us some metadata about the
layer
> > - res = save_data(registryserver, "/v1/images/" +
layerid + "/json",
> > - { "Authorization": "Token
" + token }, jsonfile)
> > - createdFiles.append(jsonfile)
> > - layersize =
int(res.info().getheader("Content-Length"))
> > -
> > - datacsum = None
> > - if layerid in checksums:
> > - datacsum = checksums[layerid]
> > -
> > - # and the '/layer' URL is the actual payload,
provided
> > - # as a tar.gz archive
> > - save_data(registryserver, "/v1/images/" + layerid +
"/layer",
> > - { "Authorization": "Token " +
token }, datafile, datacsum, layersize)
> > - createdFiles.append(datafile)
> > -
> > - # Strangely the 'json' data for a layer doesn't include
> > - # its actual name, so we save that in a json file of our own
> > - index = {
> > - "name": name,
> > - }
> > -
> > - indexfile = destdir + "/" + imagetagid +
"/index.json"
> > - with open(indexfile, "w") as f:
> > - f.write(json.dumps(index))
> > - except Exception, e:
> > - for f in createdFiles:
> > - try:
> > - os.remove(f)
> > - except:
> > - pass
> > - for d in createdDirs:
> > - try:
> > - os.rmdir(d)
> > - except:
> > - pass
> > -
> > -
> > def delete_template(name, destdir):
> > imageusage = {}
> > imageparent = {}
> > @@ -342,8 +172,16 @@ def create_template(name, imagepath, format, destdir):
> > parentImage = templateImage
> >
> > def download(args):
> > - info("Downloading %s from %s to %s\n" % (args.template,
default_index_server, default_template_dir))
> > - download_template(args.template, default_index_server,
default_template_dir)
> > + try:
> > +
dynamic_source_loader(args.source).download_template(templatename=args.template,
> > +
templatedir=args.template_dir,
> > +
registry=args.registry,
> > +
username=args.username,
> > +
password=args.password)
> > + except IOError,e:
> > + print "Source %s cannot be found in given path"
%args.source
> > + except Exception,e:
> > + print "Download Error %s" % str(e)
> >
> > def delete(args):
> > info("Deleting %s from %s\n" % (args.template,
default_template_dir))
> > @@ -357,10 +195,32 @@ def requires_template(parser):
> > parser.add_argument("template",
> > help=_("name of the template"))
> >
> > +def requires_source(parser):
> > + parser.add_argument("-s","--source",
> > + default="docker",
> > + help=_("name of the template"))
> > +
> > +def requires_auth_conn(parser):
> > + parser.add_argument("-r","--registry",
> > + help=_("Url of the custom registry"))
>
> This wording really sounds docker-specific. The registry word only fits
> docker terminology, would surely not apply to virt-builder or appc case.
> Something like "images storage" would may be more generic. The problem
> is that "repository" has a special meaning in the docker terminology.
Yeah, I was thinking we might need some way to let source subclasses
add custom arguments. eg so we'd have --docker-registry.
The appc spec doesn't have a concept of a central registry in the
same way as docker. Instead the image name encodes the domain
name, eg example.com/someapp, and there's a protocol to use
this to identify the server, and then resolve the download URL
> ACK, but may need some thinking on the "Registry" word.
I'm inclined to merge this, and then fix it afterwards to be more
generic
Sure... but then we surely don't want to get a release out with the
temporary parameter.
--
Cedric
Regards,
Daniel
7:28 a.m.
New subject: [libvirt] [PATCH sandbox v5 06/20] Image: Add download function
On Wed, Sep 09, 2015 at 02:19:20PM +0200, Cedric Bosdonnat wrote:
On Wed, 2015-09-09 at 12:55 +0100, Daniel P. Berrange wrote:
> On Wed, Sep 09, 2015 at 01:50:11PM +0200, Cedric Bosdonnat wrote:
> > This wording really sounds docker-specific. The registry word only fits
> > docker terminology, would surely not apply to virt-builder or appc case.
> > Something like "images storage" would may be more generic. The
problem
> > is that "repository" has a special meaning in the docker
terminology.
>
> Yeah, I was thinking we might need some way to let source subclasses
> add custom arguments. eg so we'd have --docker-registry.
>
> The appc spec doesn't have a concept of a central registry in the
> same way as docker. Instead the image name encodes the domain
> name, eg example.com/someapp, and there's a protocol to use
> this to identify the server, and then resolve the download URL
>
> > ACK, but may need some thinking on the "Registry" word.
>
> I'm inclined to merge this, and then fix it afterwards to be more
> generic
Sure... but then we surely don't want to get a release out with the
temporary parameter.
Oh definitely not. I'm not intending todo a release immediately. I want
some time to play around with this code and clean up more stuff before
we release it and get stuck with the syntax.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 07/20] Image: Refactor create function
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Move the docker-related code to the DockerSource and use
the Source mechanism
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 76 +++++-----------------
libvirt-sandbox/image/sources/DockerSource.py | 90 +++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 15 +++++
3 files changed, 122 insertions(+), 59 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 7af617e..f3c0ab7 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -118,59 +118,6 @@ def delete_template(name, destdir):
parent = None
imagetagid = parent
-
-def get_image_list(name, destdir):
- imageparent = {}
- imagenames = {}
- imagedirs = os.listdir(destdir)
- for imagetagid in imagedirs:
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- with open(indexfile, "r") as f:
- index = json.load(f)
- imagenames[index["name"]] = imagetagid
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- with open(jsonfile, "r") as f:
- template = json.load(f)
-
- parent = template.get("parent", None)
- if parent:
- imageparent[imagetagid] = parent
-
- if not name in imagenames:
- raise ValueError(["Image %s does not exist locally" % name])
-
- imagetagid = imagenames[name]
- imagelist = []
- while imagetagid != None:
- imagelist.append(imagetagid)
- parent = imageparent.get(imagetagid, None)
- imagetagid = parent
-
- return imagelist
-
-def create_template(name, imagepath, format, destdir):
- if not format in ["qcow2"]:
- raise ValueError(["Unsupported image format %s" % format])
-
- imagelist = get_image_list(name, destdir)
- imagelist.reverse()
-
- parentImage = None
- for imagetagid in imagelist:
- templateImage = destdir + "/" + imagetagid + "/template." +
format
- cmd = ["qemu-img", "create", "-f",
"qcow2"]
- if parentImage is not None:
- cmd.append("-o")
- cmd.append("backing_fmt=qcow2,backing_file=%s" % parentImage)
- cmd.append(templateImage)
- if parentImage is None:
- cmd.append("10G")
- debug("Run %s\n" % " ".join(cmd))
- subprocess.call(cmd)
- parentImage = templateImage
-
def download(args):
try:
dynamic_source_loader(args.source).download_template(templatename=args.template,
@@ -188,8 +135,13 @@ def delete(args):
delete_template(args.template, default_template_dir)
def create(args):
- info("Creating %s from %s in format %s\n" % (args.imagepath, args.template,
args.format))
- create_template(args.template, args.imagepath, args.format, default_template_dir)
+ try:
+ dynamic_source_loader(args.source).create_template(templatename=args.template,
+
templatedir=args.template_dir,
+ connect=args.connect,
+ format=args.format)
+ except Exception,e:
+ print "Create Error %s" % str(e)
def requires_template(parser):
parser.add_argument("template",
@@ -200,6 +152,10 @@ def requires_source(parser):
default="docker",
help=_("name of the template"))
+def requires_connect(parser):
+ parser.add_argument("-c","--connect",
+ help=_("Connect string for libvirt"))
+
def requires_auth_conn(parser):
parser.add_argument("-r","--registry",
help=_("Url of the custom registry"))
@@ -233,10 +189,12 @@ def gen_create_args(subparser):
parser = subparser.add_parser("create",
help=_("Create image from template data"))
requires_template(parser)
- parser.add_argument("imagepath",
- help=_("path for image"))
- parser.add_argument("format",
- help=_("format"))
+ requires_source(parser)
+ requires_connect(parser)
+ requires_template_dir(parser)
+ parser.add_argument("-f","--format",
+ default="qcow2",
+ help=_("format format for image"))
parser.set_defaults(func=create)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 37b40dc..c1c8a7d 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -205,5 +205,95 @@ class DockerSource(Source):
debug("FAIL %s\n" % str(e))
raise
+ def create_template(self, templatename, templatedir, connect=None, format=None):
+ if format is None:
+ format = self.default_disk_format
+ self._check_disk_format(format)
+ imagelist = self._get_image_list(templatename,templatedir)
+ imagelist.reverse()
+
+ parentImage = None
+ for imagetagid in imagelist:
+ templateImage = templatedir + "/" + imagetagid +
"/template." + format
+ cmd =
["qemu-img","create","-f","qcow2"]
+ if parentImage is not None:
+ cmd.append("-o")
+ cmd.append("backing_fmt=qcow2,backing_file=%s" % parentImage)
+ cmd.append(templateImage)
+ if parentImage is None:
+ cmd.append("10G")
+ subprocess.call(cmd)
+
+ if parentImage is None:
+ self._format_disk(templateImage,format,connect)
+
+ self._extract_tarballs(templatedir + "/" + imagetagid +
"/template.",format,connect)
+ parentImage = templateImage
+
+
+ def _check_disk_format(self,format):
+ supportedFormats = ['qcow2']
+ if not format in supportedFormats:
+ raise ValueError(["Unsupported image format %s" % format])
+
+ def _get_image_list(self,templatename,destdir):
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(destdir)
+ for imagetagid in imagedirs:
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile,"r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = destdir + "/" + imagetagid + "/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile,"r") as f:
+ template = json.load(f)
+ parent = template.get("parent",None)
+ if parent:
+ imageparent[imagetagid] = parent
+ if not templatename in imagenames:
+ raise ValueError(["Image %s does not exist locally"
%templatename])
+ imagetagid = imagenames[templatename]
+ imagelist = []
+ while imagetagid != None:
+ imagelist.append(imagetagid)
+ parent = imageparent.get(imagetagid,None)
+ imagetagid = parent
+ return imagelist
+
+ def _format_disk(self,disk,format,connect):
+ cmd = ['virt-sandbox']
+ if connect is not None:
+ cmd.append("-c")
+ cmd.append(connect)
+ cmd.append("-p")
+ params = ['--disk=file:disk_image=%s,format=%s' %(disk,format),
+ '/sbin/mkfs.ext3',
+ '/dev/disk/by-tag/disk_image']
+ cmd = cmd + params
+ subprocess.call(cmd)
+
+ def _extract_tarballs(self,directory,format,connect):
+ tempdir = "/mnt"
+ tarfile = directory + "tar.gz"
+ diskfile = directory + "qcow2"
+ cmd = ['virt-sandbox']
+ if connect is not None:
+ cmd.append("-c")
+ cmd.append(connect)
+ cmd.append("-p")
+ params = ['-m',
+ 'host-image:/mnt=%s,format=%s' %(diskfile,format),
+ '--',
+ '/bin/tar',
+ 'zxf',
+ '%s' %tarfile,
+ '-C',
+ '/mnt']
+ cmd = cmd + params
+ subprocess.call(cmd)
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 81f5176..436eef6 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -46,3 +46,18 @@ class Source():
filesystem
"""
pass
+
+ @abstractmethod
+ def create_template(self, templatename, templatedir,
+ connect=None, format=None):
+ """
+ :param templatename: name of the template image to create
+ :param templatedir: local directory path in which to store the template
+ :param connect: libvirt connection URI
+ :param format: disk image format
+
+ Create a set of local disk images populated with the content
+ of a template. The images creation process will be isolated
+ inside a sandbox using the requested libvirt connection URI.
+ """
+ pass
--
2.4.3
6:53 a.m.
New subject: [libvirt] [PATCH sandbox v5 07/20] Image: Refactor create function
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Move the docker-related code to the DockerSource and use
the Source mechanism
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 76 +++++-----------------
libvirt-sandbox/image/sources/DockerSource.py | 90 +++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 15 +++++
3 files changed, 122 insertions(+), 59 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 7af617e..f3c0ab7 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -118,59 +118,6 @@ def delete_template(name, destdir):
parent = None
imagetagid = parent
-
-def get_image_list(name, destdir):
- imageparent = {}
- imagenames = {}
- imagedirs = os.listdir(destdir)
- for imagetagid in imagedirs:
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- with open(indexfile, "r") as f:
- index = json.load(f)
- imagenames[index["name"]] = imagetagid
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- with open(jsonfile, "r") as f:
- template = json.load(f)
-
- parent = template.get("parent", None)
- if parent:
- imageparent[imagetagid] = parent
-
- if not name in imagenames:
- raise ValueError(["Image %s does not exist locally" % name])
-
- imagetagid = imagenames[name]
- imagelist = []
- while imagetagid != None:
- imagelist.append(imagetagid)
- parent = imageparent.get(imagetagid, None)
- imagetagid = parent
-
- return imagelist
-
-def create_template(name, imagepath, format, destdir):
- if not format in ["qcow2"]:
- raise ValueError(["Unsupported image format %s" % format])
-
- imagelist = get_image_list(name, destdir)
- imagelist.reverse()
-
- parentImage = None
- for imagetagid in imagelist:
- templateImage = destdir + "/" + imagetagid + "/template." +
format
- cmd = ["qemu-img", "create", "-f",
"qcow2"]
- if parentImage is not None:
- cmd.append("-o")
- cmd.append("backing_fmt=qcow2,backing_file=%s" % parentImage)
- cmd.append(templateImage)
- if parentImage is None:
- cmd.append("10G")
- debug("Run %s\n" % " ".join(cmd))
- subprocess.call(cmd)
- parentImage = templateImage
-
def download(args):
try:
dynamic_source_loader(args.source).download_template(templatename=args.template,
@@ -188,8 +135,13 @@ def delete(args):
delete_template(args.template, default_template_dir)
def create(args):
- info("Creating %s from %s in format %s\n" % (args.imagepath,
args.template, args.format))
- create_template(args.template, args.imagepath, args.format, default_template_dir)
+ try:
+ dynamic_source_loader(args.source).create_template(templatename=args.template,
+
templatedir=args.template_dir,
+ connect=args.connect,
+ format=args.format)
+ except Exception,e:
+ print "Create Error %s" % str(e)
def requires_template(parser):
parser.add_argument("template",
@@ -200,6 +152,10 @@ def requires_source(parser):
default="docker",
help=_("name of the template"))
+def requires_connect(parser):
+ parser.add_argument("-c","--connect",
+ help=_("Connect string for libvirt"))
+
def requires_auth_conn(parser):
parser.add_argument("-r","--registry",
help=_("Url of the custom registry"))
@@ -233,10 +189,12 @@ def gen_create_args(subparser):
parser = subparser.add_parser("create",
help=_("Create image from template data"))
requires_template(parser)
- parser.add_argument("imagepath",
- help=_("path for image"))
- parser.add_argument("format",
- help=_("format"))
+ requires_source(parser)
+ requires_connect(parser)
+ requires_template_dir(parser)
+ parser.add_argument("-f","--format",
+ default="qcow2",
+ help=_("format format for image"))
parser.set_defaults(func=create)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 37b40dc..c1c8a7d 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -205,5 +205,95 @@ class DockerSource(Source):
debug("FAIL %s\n" % str(e))
raise
+ def create_template(self, templatename, templatedir, connect=None, format=None):
+ if format is None:
+ format = self.default_disk_format
+ self._check_disk_format(format)
+ imagelist = self._get_image_list(templatename,templatedir)
+ imagelist.reverse()
+
+ parentImage = None
+ for imagetagid in imagelist:
+ templateImage = templatedir + "/" + imagetagid +
"/template." + format
+ cmd =
["qemu-img","create","-f","qcow2"]
+ if parentImage is not None:
+ cmd.append("-o")
+ cmd.append("backing_fmt=qcow2,backing_file=%s" % parentImage)
+ cmd.append(templateImage)
+ if parentImage is None:
+ cmd.append("10G")
+ subprocess.call(cmd)
+
+ if parentImage is None:
+ self._format_disk(templateImage,format,connect)
+
+ self._extract_tarballs(templatedir + "/" + imagetagid +
"/template.",format,connect)
+ parentImage = templateImage
+
+
+ def _check_disk_format(self,format):
+ supportedFormats = ['qcow2']
+ if not format in supportedFormats:
+ raise ValueError(["Unsupported image format %s" % format])
+
+ def _get_image_list(self,templatename,destdir):
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(destdir)
+ for imagetagid in imagedirs:
+ indexfile = destdir + "/" + imagetagid + "/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile,"r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = destdir + "/" + imagetagid +
"/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile,"r") as f:
+ template = json.load(f)
+ parent = template.get("parent",None)
+ if parent:
+ imageparent[imagetagid] = parent
+ if not templatename in imagenames:
+ raise ValueError(["Image %s does not exist locally"
%templatename])
+ imagetagid = imagenames[templatename]
+ imagelist = []
+ while imagetagid != None:
+ imagelist.append(imagetagid)
+ parent = imageparent.get(imagetagid,None)
+ imagetagid = parent
+ return imagelist
+
+ def _format_disk(self,disk,format,connect):
+ cmd = ['virt-sandbox']
+ if connect is not None:
+ cmd.append("-c")
+ cmd.append(connect)
+ cmd.append("-p")
+ params = ['--disk=file:disk_image=%s,format=%s' %(disk,format),
+ '/sbin/mkfs.ext3',
+ '/dev/disk/by-tag/disk_image']
+ cmd = cmd + params
+ subprocess.call(cmd)
+
+ def _extract_tarballs(self,directory,format,connect):
+ tempdir = "/mnt"
Is it safe to assume we'll be able to use /mnt as a temporary mount
folder? I'ld be more inclined in using a really temporary folder.
--
Cedric
+ tarfile = directory + "tar.gz"
+ diskfile = directory + "qcow2"
+ cmd = ['virt-sandbox']
+ if connect is not None:
+ cmd.append("-c")
+ cmd.append(connect)
+ cmd.append("-p")
+ params = ['-m',
+ 'host-image:/mnt=%s,format=%s' %(diskfile,format),
+ '--',
+ '/bin/tar',
+ 'zxf',
+ '%s' %tarfile,
+ '-C',
+ '/mnt']
+ cmd = cmd + params
+ subprocess.call(cmd)
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 81f5176..436eef6 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -46,3 +46,18 @@ class Source():
filesystem
"""
pass
+
+ @abstractmethod
+ def create_template(self, templatename, templatedir,
+ connect=None, format=None):
+ """
+ :param templatename: name of the template image to create
+ :param templatedir: local directory path in which to store the template
+ :param connect: libvirt connection URI
+ :param format: disk image format
+
+ Create a set of local disk images populated with the content
+ of a template. The images creation process will be isolated
+ inside a sandbox using the requested libvirt connection URI.
+ """
+ pass
6:58 a.m.
New subject: [libvirt] [PATCH sandbox v5 07/20] Image: Refactor create function
On Wed, Sep 09, 2015 at 01:53:06PM +0200, Cedric Bosdonnat wrote:
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
> From: Eren Yagdiran <erenyagdiran(a)gmail.com>
>
> Move the docker-related code to the DockerSource and use
> the Source mechanism
>
> Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
> ---
> libvirt-sandbox/image/cli.py | 76 +++++-----------------
> libvirt-sandbox/image/sources/DockerSource.py | 90 +++++++++++++++++++++++++++
> libvirt-sandbox/image/sources/Source.py | 15 +++++
> 3 files changed, 122 insertions(+), 59 deletions(-)
>
> + def _extract_tarballs(self,directory,format,connect):
> + tempdir = "/mnt"
Is it safe to assume we'll be able to use /mnt as a temporary mount
folder? I'ld be more inclined in using a really temporary folder.
Amuzingly that variable is set and then never used again, so
can just be deleted
--
Cedric
> + tarfile = directory + "tar.gz"
> + diskfile = directory + "qcow2"
> + cmd = ['virt-sandbox']
> + if connect is not None:
> + cmd.append("-c")
> + cmd.append(connect)
> + cmd.append("-p")
> + params = ['-m',
> + 'host-image:/mnt=%s,format=%s' %(diskfile,format),
> + '--',
> + '/bin/tar',
> + 'zxf',
> + '%s' %tarfile,
> + '-C',
> + '/mnt']
> + cmd = cmd + params
> + subprocess.call(cmd)
> +
> def debug(msg):
> sys.stderr.write(msg)
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 08/20] Image: Add delete function
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Refactoring delete function from virt-sandbox-image to DockerSource. Delete function
can delete templates by name.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 58 ++++-----------------------
libvirt-sandbox/image/sources/DockerSource.py | 50 +++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 10 +++++
3 files changed, 67 insertions(+), 51 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index f3c0ab7..5490c4b 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -69,55 +69,6 @@ def debug(msg):
def info(msg):
sys.stdout.write(msg)
-def delete_template(name, destdir):
- imageusage = {}
- imageparent = {}
- imagenames = {}
- imagedirs = os.listdir(destdir)
- for imagetagid in imagedirs:
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- with open(indexfile, "r") as f:
- index = json.load(f)
- imagenames[index["name"]] = imagetagid
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- with open(jsonfile, "r") as f:
- template = json.load(f)
-
- parent = template.get("parent", None)
- if parent:
- if parent not in imageusage:
- imageusage[parent] = []
- imageusage[parent].append(imagetagid)
- imageparent[imagetagid] = parent
-
- if not name in imagenames:
- raise ValueError(["Image %s does not exist locally" % name])
-
- imagetagid = imagenames[name]
- while imagetagid != None:
- debug("Remove %s\n" % imagetagid)
- parent = imageparent.get(imagetagid, None)
-
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- os.remove(indexfile)
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- os.remove(jsonfile)
- datafile = destdir + "/" + imagetagid + "/template.tar.gz"
- if os.path.exists(datafile):
- os.remove(datafile)
- imagedir = destdir + "/" + imagetagid
- os.rmdir(imagedir)
-
- if parent:
- if len(imageusage[parent]) != 1:
- debug("Parent %s is shared\n" % parent)
- parent = None
- imagetagid = parent
-
def download(args):
try:
dynamic_source_loader(args.source).download_template(templatename=args.template,
@@ -131,8 +82,11 @@ def download(args):
print "Download Error %s" % str(e)
def delete(args):
- info("Deleting %s from %s\n" % (args.template, default_template_dir))
- delete_template(args.template, default_template_dir)
+ try:
+ dynamic_source_loader(args.source).delete_template(templatename=args.template,
+
templatedir=args.template_dir)
+ except Exception,e:
+ print "Delete Error %s", str(e)
def create(args):
try:
@@ -183,6 +137,8 @@ def gen_delete_args(subparser):
parser = subparser.add_parser("delete",
help=_("Delete template data"))
requires_template(parser)
+ requires_source(parser)
+ requires_template_dir(parser)
parser.set_defaults(func=delete)
def gen_create_args(subparser):
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index c1c8a7d..ab18b52 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -295,5 +295,55 @@ class DockerSource(Source):
cmd = cmd + params
subprocess.call(cmd)
+ def delete_template(self, templatename, templatedir):
+ imageusage = {}
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(templatedir)
+ for imagetagid in imagedirs:
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile,"r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = templatedir + "/" + imagetagid +
"/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile,"r") as f:
+ template = json.load(f)
+
+ parent = template.get("parent",None)
+ if parent:
+ if parent not in imageusage:
+ imageusage[parent] = []
+ imageusage[parent].append(imagetagid)
+ imageparent[imagetagid] = parent
+
+
+ if not templatename in imagenames:
+ raise ValueError(["Image %s does not exist locally"
%templatename])
+
+ imagetagid = imagenames[templatename]
+ while imagetagid != None:
+ debug("Remove %s\n" % imagetagid)
+ parent = imageparent.get(imagetagid,None)
+
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ if os.path.exists(indexfile):
+ os.remove(indexfile)
+ jsonfile = templatedir + "/" + imagetagid +
"/template.json"
+ if os.path.exists(jsonfile):
+ os.remove(jsonfile)
+ datafile = templatedir + "/" + imagetagid +
"/template.tar.gz"
+ if os.path.exists(datafile):
+ os.remove(datafile)
+ imagedir = templatedir + "/" + imagetagid
+ shutil.rmtree(imagedir)
+
+ if parent:
+ if len(imageusage[parent]) != 1:
+ debug("Parent %s is shared\n" % parent)
+ parent = None
+ imagetagid = parent
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 436eef6..4aea5c9 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -61,3 +61,13 @@ class Source():
inside a sandbox using the requested libvirt connection URI.
"""
pass
+
+ @abstractmethod
+ def delete_template(self, templatename, templatedir):
+ """
+ :param templatename: name of the template image to delete
+ :param templatedir: local directory path from which to delete template
+
+ Delete all local files associated with the template
+ """
+ pass
--
2.4.3
6:57 a.m.
New subject: [libvirt] [PATCH sandbox v5 08/20] Image: Add delete function
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Refactoring delete function from virt-sandbox-image to DockerSource. Delete function
can delete templates by name.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 58 ++++-----------------------
libvirt-sandbox/image/sources/DockerSource.py | 50 +++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 10 +++++
3 files changed, 67 insertions(+), 51 deletions(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index f3c0ab7..5490c4b 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -69,55 +69,6 @@ def debug(msg):
def info(msg):
sys.stdout.write(msg)
-def delete_template(name, destdir):
- imageusage = {}
- imageparent = {}
- imagenames = {}
- imagedirs = os.listdir(destdir)
- for imagetagid in imagedirs:
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- with open(indexfile, "r") as f:
- index = json.load(f)
- imagenames[index["name"]] = imagetagid
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- with open(jsonfile, "r") as f:
- template = json.load(f)
-
- parent = template.get("parent", None)
- if parent:
- if parent not in imageusage:
- imageusage[parent] = []
- imageusage[parent].append(imagetagid)
- imageparent[imagetagid] = parent
-
- if not name in imagenames:
- raise ValueError(["Image %s does not exist locally" % name])
-
- imagetagid = imagenames[name]
- while imagetagid != None:
- debug("Remove %s\n" % imagetagid)
- parent = imageparent.get(imagetagid, None)
-
- indexfile = destdir + "/" + imagetagid + "/index.json"
- if os.path.exists(indexfile):
- os.remove(indexfile)
- jsonfile = destdir + "/" + imagetagid + "/template.json"
- if os.path.exists(jsonfile):
- os.remove(jsonfile)
- datafile = destdir + "/" + imagetagid + "/template.tar.gz"
- if os.path.exists(datafile):
- os.remove(datafile)
- imagedir = destdir + "/" + imagetagid
- os.rmdir(imagedir)
-
- if parent:
- if len(imageusage[parent]) != 1:
- debug("Parent %s is shared\n" % parent)
- parent = None
- imagetagid = parent
-
def download(args):
try:
dynamic_source_loader(args.source).download_template(templatename=args.template,
@@ -131,8 +82,11 @@ def download(args):
print "Download Error %s" % str(e)
def delete(args):
- info("Deleting %s from %s\n" % (args.template, default_template_dir))
- delete_template(args.template, default_template_dir)
+ try:
+ dynamic_source_loader(args.source).delete_template(templatename=args.template,
+
templatedir=args.template_dir)
+ except Exception,e:
+ print "Delete Error %s", str(e)
def create(args):
try:
@@ -183,6 +137,8 @@ def gen_delete_args(subparser):
parser = subparser.add_parser("delete",
help=_("Delete template data"))
requires_template(parser)
+ requires_source(parser)
+ requires_template_dir(parser)
parser.set_defaults(func=delete)
def gen_create_args(subparser):
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index c1c8a7d..ab18b52 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -295,5 +295,55 @@ class DockerSource(Source):
cmd = cmd + params
subprocess.call(cmd)
+ def delete_template(self, templatename, templatedir):
+ imageusage = {}
+ imageparent = {}
+ imagenames = {}
+ imagedirs = os.listdir(templatedir)
+ for imagetagid in imagedirs:
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ if os.path.exists(indexfile):
+ with open(indexfile,"r") as f:
+ index = json.load(f)
+ imagenames[index["name"]] = imagetagid
+ jsonfile = templatedir + "/" + imagetagid +
"/template.json"
+ if os.path.exists(jsonfile):
+ with open(jsonfile,"r") as f:
+ template = json.load(f)
+
+ parent = template.get("parent",None)
+ if parent:
+ if parent not in imageusage:
+ imageusage[parent] = []
+ imageusage[parent].append(imagetagid)
+ imageparent[imagetagid] = parent
+
+
+ if not templatename in imagenames:
+ raise ValueError(["Image %s does not exist locally"
%templatename])
+
+ imagetagid = imagenames[templatename]
+ while imagetagid != None:
+ debug("Remove %s\n" % imagetagid)
+ parent = imageparent.get(imagetagid,None)
+
+ indexfile = templatedir + "/" + imagetagid +
"/index.json"
+ if os.path.exists(indexfile):
+ os.remove(indexfile)
+ jsonfile = templatedir + "/" + imagetagid +
"/template.json"
+ if os.path.exists(jsonfile):
+ os.remove(jsonfile)
+ datafile = templatedir + "/" + imagetagid +
"/template.tar.gz"
+ if os.path.exists(datafile):
+ os.remove(datafile)
+ imagedir = templatedir + "/" + imagetagid
+ shutil.rmtree(imagedir)
+
+ if parent:
+ if len(imageusage[parent]) != 1:
+ debug("Parent %s is shared\n" % parent)
+ parent = None
+ imagetagid = parent
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 436eef6..4aea5c9 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -61,3 +61,13 @@ class Source():
inside a sandbox using the requested libvirt connection URI.
"""
pass
+
+ @abstractmethod
+ def delete_template(self, templatename, templatedir):
+ """
+ :param templatename: name of the template image to delete
+ :param templatedir: local directory path from which to delete template
+
+ Delete all local files associated with the template
+ """
+ pass
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 09/20] Image: Add get_command function to Source
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Provide a way to know how a template can be started depending on the used source
DockerSource will need to parse the topmost config file in order to find the igniter
command
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/sources/DockerSource.py | 31 +++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 12 +++++++++++
2 files changed, 43 insertions(+)
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index ab18b52..2607011 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -29,6 +29,16 @@ import os
import subprocess
import shutil
+class DockerConfParser():
+
+ def __init__(self,jsonfile):
+ with open(jsonfile) as json_file:
+ self.json_data = json.load(json_file)
+ def getCommand(self):
+ return self.json_data['config']['Cmd']
+ def getEntrypoint(self):
+ return self.json_data['config']['Entrypoint']
+
class DockerSource(Source):
www_auth_username = None
@@ -345,5 +355,26 @@ class DockerSource(Source):
parent = None
imagetagid = parent
+ def _get_template_data(self, templatename, templatedir):
+ imageList = self._get_image_list(templatename, templatedir)
+ toplayer = imageList[0]
+ diskfile = templatedir + "/" + toplayer + "/template.qcow2"
+ configfile = templatedir + "/" + toplayer + "/template.json"
+ return configfile, diskfile
+
+ def get_command(self, templatename, templatedir, userargs):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ cmd = configParser.getCommand()
+ entrypoint = configParser.getEntrypoint()
+ if entrypoint is None:
+ entrypoint = []
+ if cmd is None:
+ cmd = []
+ if userargs is not None and len(userargs) > 0:
+ return entrypoint + userargs
+ else:
+ return entrypoint + cmd
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 4aea5c9..4305d0b 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -71,3 +71,15 @@ class Source():
Delete all local files associated with the template
"""
pass
+
+ @abstractmethod
+ def get_command(self, templatename, templatedir, userargs):
+ """
+ :param templatename: name of the template image to query
+ :param templatedir: local directory path in which templates are stored
+ :param userargs: user specified arguments to run
+
+ Get the command line to invoke in the container. If userargs
+ is specified, then this should override the default args in
+ the image"""
+ pass
--
2.4.3
6:59 a.m.
New subject: [libvirt] [PATCH sandbox v5 09/20] Image: Add get_command function to Source
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Provide a way to know how a template can be started depending on the used source
DockerSource will need to parse the topmost config file in order to find the igniter
command
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/sources/DockerSource.py | 31 +++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Source.py | 12 +++++++++++
2 files changed, 43 insertions(+)
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index ab18b52..2607011 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -29,6 +29,16 @@ import os
import subprocess
import shutil
+class DockerConfParser():
+
+ def __init__(self,jsonfile):
+ with open(jsonfile) as json_file:
+ self.json_data = json.load(json_file)
+ def getCommand(self):
+ return self.json_data['config']['Cmd']
+ def getEntrypoint(self):
+ return self.json_data['config']['Entrypoint']
+
class DockerSource(Source):
www_auth_username = None
@@ -345,5 +355,26 @@ class DockerSource(Source):
parent = None
imagetagid = parent
+ def _get_template_data(self, templatename, templatedir):
+ imageList = self._get_image_list(templatename, templatedir)
+ toplayer = imageList[0]
+ diskfile = templatedir + "/" + toplayer + "/template.qcow2"
+ configfile = templatedir + "/" + toplayer +
"/template.json"
+ return configfile, diskfile
+
+ def get_command(self, templatename, templatedir, userargs):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ cmd = configParser.getCommand()
+ entrypoint = configParser.getEntrypoint()
+ if entrypoint is None:
+ entrypoint = []
+ if cmd is None:
+ cmd = []
+ if userargs is not None and len(userargs) > 0:
+ return entrypoint + userargs
+ else:
+ return entrypoint + cmd
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 4aea5c9..4305d0b 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -71,3 +71,15 @@ class Source():
Delete all local files associated with the template
"""
pass
+
+ @abstractmethod
+ def get_command(self, templatename, templatedir, userargs):
+ """
+ :param templatename: name of the template image to query
+ :param templatedir: local directory path in which templates are stored
+ :param userargs: user specified arguments to run
+
+ Get the command line to invoke in the container. If userargs
+ is specified, then this should override the default args in
+ the image"""
+ pass
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 10/20] Image: Add run args
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Commandline parameters for running a template
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 5490c4b..6f9a5e7 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -101,6 +101,10 @@ def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
+def requires_name(parser):
+ parser.add_argument("-n","--name",
+ help=_("Name of the running sandbox"))
+
def requires_source(parser):
parser.add_argument("-s","--source",
default="docker",
@@ -124,6 +128,12 @@ def requires_template_dir(parser):
default=default_template_dir,
help=_("Template directory for saving templates"))
+def requires_image_dir(parser):
+ global default_image_dir
+ parser.add_argument("-I","--image-dir",
+ default=default_image_dir,
+ help=_("Image directory for saving images"))
+
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
@@ -153,6 +163,20 @@ def gen_create_args(subparser):
help=_("format format for image"))
parser.set_defaults(func=create)
+def gen_run_args(subparser):
+ parser = subparser.add_parser("run",
+ help=_("Run a already built image"))
+ requires_name(parser)
+ requires_template(parser)
+ requires_source(parser)
+ requires_connect(parser)
+ requires_template_dir(parser)
+ requires_image_dir(parser)
+ parser.add_argument("args",
+ nargs=argparse.REMAINDER,
+ help=_("command arguments to run"))
+ parser.set_defaults(func=run)
+
def main():
parser = argparse.ArgumentParser(description='Sandbox Container Image Tool')
@@ -160,6 +184,7 @@ def main():
gen_download_args(subparser)
gen_delete_args(subparser)
gen_create_args(subparser)
+ gen_run_args(subparser)
try:
args = parser.parse_args()
--
2.4.3
7:01 a.m.
New subject: [libvirt] [PATCH sandbox v5 10/20] Image: Add run args
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Commandline parameters for running a template
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 5490c4b..6f9a5e7 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -101,6 +101,10 @@ def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
+def requires_name(parser):
+ parser.add_argument("-n","--name",
+ help=_("Name of the running sandbox"))
+
def requires_source(parser):
parser.add_argument("-s","--source",
default="docker",
@@ -124,6 +128,12 @@ def requires_template_dir(parser):
default=default_template_dir,
help=_("Template directory for saving templates"))
+def requires_image_dir(parser):
+ global default_image_dir
+ parser.add_argument("-I","--image-dir",
+ default=default_image_dir,
+ help=_("Image directory for saving images"))
+
def gen_download_args(subparser):
parser = subparser.add_parser("download",
help=_("Download template data"))
@@ -153,6 +163,20 @@ def gen_create_args(subparser):
help=_("format format for image"))
parser.set_defaults(func=create)
+def gen_run_args(subparser):
+ parser = subparser.add_parser("run",
+ help=_("Run a already built image"))
I'm not a native english speaker, but it sounds like it should be:
"Run an already built image"
+ requires_name(parser)
+ requires_template(parser)
+ requires_source(parser)
+ requires_connect(parser)
+ requires_template_dir(parser)
+ requires_image_dir(parser)
+ parser.add_argument("args",
+ nargs=argparse.REMAINDER,
+ help=_("command arguments to run"))
+ parser.set_defaults(func=run)
+
def main():
parser = argparse.ArgumentParser(description='Sandbox Container Image
Tool')
@@ -160,6 +184,7 @@ def main():
gen_download_args(subparser)
gen_delete_args(subparser)
gen_create_args(subparser)
+ gen_run_args(subparser)
try:
args = parser.parse_args()
ACK with the typo fixed.
--
Cedric
7:16 a.m.
New subject: [libvirt] [PATCH sandbox v5 10/20] Image: Add run args
On Wed, Sep 09, 2015 at 02:01:41PM +0200, Cedric Bosdonnat wrote:
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
> From: Eren Yagdiran <erenyagdiran(a)gmail.com>
>
> Commandline parameters for running a template
>
> Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
> ---
> libvirt-sandbox/image/cli.py | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
> index 5490c4b..6f9a5e7 100755
> --- a/libvirt-sandbox/image/cli.py
> +++ b/libvirt-sandbox/image/cli.py
> @@ -101,6 +101,10 @@ def requires_template(parser):
> parser.add_argument("template",
> help=_("name of the template"))
>
> +def requires_name(parser):
> + parser.add_argument("-n","--name",
> + help=_("Name of the running sandbox"))
> +
> def requires_source(parser):
> parser.add_argument("-s","--source",
> default="docker",
> @@ -124,6 +128,12 @@ def requires_template_dir(parser):
> default=default_template_dir,
> help=_("Template directory for saving
templates"))
>
> +def requires_image_dir(parser):
> + global default_image_dir
> + parser.add_argument("-I","--image-dir",
> + default=default_image_dir,
> + help=_("Image directory for saving images"))
> +
> def gen_download_args(subparser):
> parser = subparser.add_parser("download",
> help=_("Download template data"))
> @@ -153,6 +163,20 @@ def gen_create_args(subparser):
> help=_("format format for image"))
> parser.set_defaults(func=create)
>
> +def gen_run_args(subparser):
> + parser = subparser.add_parser("run",
> + help=_("Run a already built image"))
I'm not a native english speaker, but it sounds like it should be:
"Run an already built image"
Yes, you're right.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 11/20] Image: Add check_connect function
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Check if user-specified connect argument is valid
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 6f9a5e7..2672a20 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -105,6 +105,12 @@ def requires_name(parser):
parser.add_argument("-n","--name",
help=_("Name of the running sandbox"))
+def check_connect(connectstr):
+ supportedDrivers =
['lxc:///','qemu:///session','qemu:///system']
+ if not connectstr in supportedDrivers:
+ raise ValueError("URI '%s' is not supported by
virt-sandbox-image" % connectstr)
+ return True
+
def requires_source(parser):
parser.add_argument("-s","--source",
default="docker",
--
2.4.3
7:02 a.m.
New subject: [libvirt] [PATCH sandbox v5 11/20] Image: Add check_connect function
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Check if user-specified connect argument is valid
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 6f9a5e7..2672a20 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -105,6 +105,12 @@ def requires_name(parser):
parser.add_argument("-n","--name",
help=_("Name of the running sandbox"))
+def check_connect(connectstr):
+ supportedDrivers =
['lxc:///','qemu:///session','qemu:///system']
+ if not connectstr in supportedDrivers:
+ raise ValueError("URI '%s' is not supported by
virt-sandbox-image" % connectstr)
+ return True
+
def requires_source(parser):
parser.add_argument("-s","--source",
default="docker",
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 12/20] Image: Add get_disk function to Source
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Provide a way to know which disk image to use for the sandbox depending on the used
source
DockerSource will need to locate the topmost disk image among all the layers images
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/sources/DockerSource.py | 12 ++++++++++++
libvirt-sandbox/image/sources/Source.py | 12 ++++++++++++
2 files changed, 24 insertions(+)
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 2607011..31c1d80 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -362,6 +362,18 @@ class DockerSource(Source):
configfile = templatedir + "/" + toplayer + "/template.json"
return configfile, diskfile
+ def get_disk(self,templatename, templatedir, imagedir, sandboxname):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ tempfile = imagedir + "/" + sandboxname + ".qcow2"
+ if not os.path.exists(imagedir):
+ os.makedirs(imagedir)
+ cmd =
["qemu-img","create","-q","-f","qcow2"]
+ cmd.append("-o")
+ cmd.append("backing_fmt=qcow2,backing_file=%s" % diskfile)
+ cmd.append(tempfile)
+ subprocess.call(cmd)
+ return tempfile
+
def get_command(self, templatename, templatedir, userargs):
configfile, diskfile = self._get_template_data(templatename, templatedir)
configParser = DockerConfParser(configfile)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 4305d0b..a5d3844 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -83,3 +83,15 @@ class Source():
is specified, then this should override the default args in
the image"""
pass
+
+ @abstractmethod
+ def get_disk(self,templatename, templatedir, imagedir, sandboxname):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+ :param imagedir: local directory in which to storage disk image
+
+ Creates an instance private disk image, backed by the content
+ of a template.
+ """
+ pass
--
2.4.3
7:03 a.m.
New subject: [libvirt] [PATCH sandbox v5 12/20] Image: Add get_disk function to Source
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Provide a way to know which disk image to use for the sandbox depending on the used
source
DockerSource will need to locate the topmost disk image among all the layers images
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/sources/DockerSource.py | 12 ++++++++++++
libvirt-sandbox/image/sources/Source.py | 12 ++++++++++++
2 files changed, 24 insertions(+)
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 2607011..31c1d80 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -362,6 +362,18 @@ class DockerSource(Source):
configfile = templatedir + "/" + toplayer +
"/template.json"
return configfile, diskfile
+ def get_disk(self,templatename, templatedir, imagedir, sandboxname):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ tempfile = imagedir + "/" + sandboxname + ".qcow2"
+ if not os.path.exists(imagedir):
+ os.makedirs(imagedir)
+ cmd =
["qemu-img","create","-q","-f","qcow2"]
+ cmd.append("-o")
+ cmd.append("backing_fmt=qcow2,backing_file=%s" % diskfile)
+ cmd.append(tempfile)
+ subprocess.call(cmd)
+ return tempfile
+
def get_command(self, templatename, templatedir, userargs):
configfile, diskfile = self._get_template_data(templatename, templatedir)
configParser = DockerConfParser(configfile)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 4305d0b..a5d3844 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -83,3 +83,15 @@ class Source():
is specified, then this should override the default args in
the image"""
pass
+
+ @abstractmethod
+ def get_disk(self,templatename, templatedir, imagedir, sandboxname):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+ :param imagedir: local directory in which to storage disk image
+
+ Creates an instance private disk image, backed by the content
+ of a template.
+ """
+ pass
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 13/20] Image: Add run function
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Run an already-built template
If there is no execution command specified by user, source.get_command will
find the command to invoke
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 2672a20..a06eb9c 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -31,6 +31,8 @@ import shutil
import sys
import urllib2
import subprocess
+import random
+import string
if os.geteuid() == 0:
default_template_dir = "/var/lib/libvirt/templates"
@@ -97,6 +99,37 @@ def create(args):
except Exception,e:
print "Create Error %s" % str(e)
+def run(args):
+ try:
+ if args.connect is not None:
+ check_connect(args.connect)
+ source = dynamic_source_loader(args.source)
+ name = args.name
+ if name is None:
+ randomid = ''.join(random.choice(string.lowercase) for i in
range(10))
+ name = args.template + ":" + randomid
+
+ diskfile = source.get_disk(templatename=args.template,
+ templatedir=args.template_dir,
+ imagedir=args.image_dir,
+ sandboxname=name)
+
+ format = "qcow2"
+ commandToRun = source.get_command(args.template, args.template_dir, args.args)
+ if len(commandToRun) == 0:
+ commandToRun = ["/bin/sh"]
+ cmd = ['virt-sandbox', '--name', name]
+ if args.connect is not None:
+ cmd.append("-c")
+ cmd.append(args.connect)
+ params = ['-m','host-image:/=%s,format=%s' %(diskfile,format)]
+ cmd = cmd + params + ['--'] + commandToRun
+ subprocess.call(cmd)
+ os.unlink(diskfile)
+
+ except Exception,e:
+ print "Run Error %s" % str(e)
+
def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
--
2.4.3
7:06 a.m.
New subject: [libvirt] [PATCH sandbox v5 13/20] Image: Add run function
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Run an already-built template
If there is no execution command specified by user, source.get_command will
find the command to invoke
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 2672a20..a06eb9c 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -31,6 +31,8 @@ import shutil
import sys
import urllib2
import subprocess
+import random
+import string
if os.geteuid() == 0:
default_template_dir = "/var/lib/libvirt/templates"
@@ -97,6 +99,37 @@ def create(args):
except Exception,e:
print "Create Error %s" % str(e)
+def run(args):
+ try:
+ if args.connect is not None:
+ check_connect(args.connect)
+ source = dynamic_source_loader(args.source)
+ name = args.name
+ if name is None:
+ randomid = ''.join(random.choice(string.lowercase) for i in
range(10))
+ name = args.template + ":" + randomid
+
+ diskfile = source.get_disk(templatename=args.template,
+ templatedir=args.template_dir,
+ imagedir=args.image_dir,
+ sandboxname=name)
+
+ format = "qcow2"
+ commandToRun = source.get_command(args.template, args.template_dir, args.args)
+ if len(commandToRun) == 0:
+ commandToRun = ["/bin/sh"]
+ cmd = ['virt-sandbox', '--name', name]
+ if args.connect is not None:
+ cmd.append("-c")
+ cmd.append(args.connect)
+ params = ['-m','host-image:/=%s,format=%s' %(diskfile,format)]
+ cmd = cmd + params + ['--'] + commandToRun
+ subprocess.call(cmd)
+ os.unlink(diskfile)
+
+ except Exception,e:
+ print "Run Error %s" % str(e)
+
def requires_template(parser):
parser.add_argument("template",
help=_("name of the template"))
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 14/20] Image: Add network support
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Virt-sandbox-image will pass exact network arguments to virt-sandbox
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index a06eb9c..30e2558 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -123,6 +123,12 @@ def run(args):
cmd.append("-c")
cmd.append(args.connect)
params = ['-m','host-image:/=%s,format=%s' %(diskfile,format)]
+
+ networkArgs = args.network
+ if networkArgs is not None:
+ params.append('-N')
+ params.append(networkArgs)
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
@@ -214,6 +220,8 @@ def gen_run_args(subparser):
parser.add_argument("args",
nargs=argparse.REMAINDER,
help=_("command arguments to run"))
+ parser.add_argument("-N","--network",
+ help=_("Network params for running template"))
parser.set_defaults(func=run)
def main():
--
2.4.3
7:08 a.m.
New subject: [libvirt] [PATCH sandbox v5 14/20] Image: Add network support
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Virt-sandbox-image will pass exact network arguments to virt-sandbox
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index a06eb9c..30e2558 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -123,6 +123,12 @@ def run(args):
cmd.append("-c")
cmd.append(args.connect)
params = ['-m','host-image:/=%s,format=%s' %(diskfile,format)]
+
+ networkArgs = args.network
+ if networkArgs is not None:
+ params.append('-N')
+ params.append(networkArgs)
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
@@ -214,6 +220,8 @@ def gen_run_args(subparser):
parser.add_argument("args",
nargs=argparse.REMAINDER,
help=_("command arguments to run"))
+ parser.add_argument("-N","--network",
+ help=_("Network params for running template"))
parser.set_defaults(func=run)
def main():
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 15/20] Image: man file for virt-sandbox-image
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
bin/Makefile.am | 5 ++
bin/virt-sandbox-image.pod | 169 +++++++++++++++++++++++++++++++++++++++++++++
libvirt-sandbox.spec.in | 1 +
3 files changed, 175 insertions(+)
create mode 100644 bin/virt-sandbox-image.pod
diff --git a/bin/Makefile.am b/bin/Makefile.am
index deedcf6..398e90c 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -20,6 +20,7 @@ POD_FILES = \
virt-sandbox-service-delete.pod \
virt-sandbox-service-reload.pod \
virt-sandbox-service-upgrade.pod \
+ virt-sandbox-image.pod \
$(NULL)
EXTRA_DIST = $(bin_SCRIPTS) $(POD_FILES) virt-sandbox-service-bash-completion.sh
virt-sandbox-service.logrotate
EXTRA_DIST += virt-sandbox-service-bash-completion.sh
@@ -34,6 +35,7 @@ man1_MANS = \
virt-sandbox-service-delete.1 \
virt-sandbox-service-reload.1 \
virt-sandbox-service-upgrade.1 \
+ virt-sandbox-image.1 \
$(NULL)
POD2MAN = pod2man -c "Virtualization Support" -r
"$(PACKAGE)-$(VERSION)"
@@ -65,6 +67,9 @@ virt-sandbox-service-reload.1: virt-sandbox-service-reload.pod Makefile
virt-sandbox-service-upgrade.1: virt-sandbox-service-upgrade.pod Makefile
$(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
+virt-sandbox-image.1: virt-sandbox-image.pod Makefile
+ $(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
+
CLEANFILES = $(man1_MANS)
virt_sandbox_SOURCES = virt-sandbox.c
diff --git a/bin/virt-sandbox-image.pod b/bin/virt-sandbox-image.pod
new file mode 100644
index 0000000..85954b0
--- /dev/null
+++ b/bin/virt-sandbox-image.pod
@@ -0,0 +1,169 @@
+=head1 NAME
+
+virt-sandbox-image - Sandbox Container Image Tool
+
+=head1 SYNOPSIS
+
+ {download,create,run,delete}
+
+ commands:
+
+ download Download template data
+
+ create Create image from template data
+
+ run Run an already built image
+
+ delete Delete template data
+
+=head1 DESCRIPTION
+
+virt-sandbox-image.py is a sandbox container image tool developed in python.
+This tool can download,create,run and delete templates which are provided by
+different sources. This tool comes with Docker source by default. Other sources
+can be implemented by extending source class
+
+=head1 OPTIONS
+
+=over 4
+
+=item B<download -s source -r registry -u username -p password -t template_directory
template>
+
+Download a template by given name with a specified source.
+
+=over 6
+
+=item B<template>
+
+Template name to download
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has to
implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
+=item B<-r or --registry>
+
+Custom registry url for downloading data. This might need privileged credentials which
can be specified by --username and --password parameters.
+
+=item B<-u or --username>
+
+Username for custom registry authentication
+
+=item B<-p or --password>
+
+Password for custom registry authentication
+
+=item B<-t or --template-dir>
+
+Custom directory for downloading template data
+
+=back
+
+=item B<create -s source -c uri -f format template>
+
+Create already downloaded template into image with given format.
+
+=over 5
+
+=item B<template>
+
+Template name to download.
+
+=item B<format>
+
+Image format e.g qcow2
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has to
implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
+=item B<-c or --connect>
+
+Driver parameter can be specified with only supported driver by libvirt-sandbox. These
are lxc:///, qemu:///session, qemu:///system.
+
+=back
+
+=item B<run -n name -N network -v volume -s source -c uri template command...>
+
+Run already built image. If B<command> is not specified, the default defined
+command for the image will be run.
+
+=over 6
+
+=item B<template>
+
+Template name to download.
+
+=item B<imagepath>
+
+Image path where template image will be stored.
+
+=item B<-n or --name>
+
+The sandbox guest name
+
+=item B<-N or --network>
+
+Network params will be passed directly to the virt-sandbox. More information about
network params, See C<virt-sandbox(8)>
+
+=item B<-v or --volume>
+
+Volume params are for binding host-paths to the guest. E.g -v /home:/home will map /home
directory from host to the guest.
+
+=item B<-c or --connect>
+
+Driver parameter can be specified with only supported driver by libvirt-sandbox. These
are lxc:///, qemu:///session, qemu:///system.
+
+=back
+
+=item B<delete name imagepath -s source >
+
+Delete downloaded template data and its built image.
+
+=over 3
+
+=item B<name>
+
+Template name to delete.
+
+=item B<imagepath>
+
+Image path where template data or image stays.
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has to
implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
+=back
+
+=back
+
+=head1 SEE ALSO
+
+C<virt-sandbox(8)>
+
+=head1 FILES
+
+Container content will be stored in subdirectories of
+/var/lib/libvirt/templates, by default.
+
+=head1 AUTHORS
+
+Daniel P. Berrange <dan(a)berrange.com>
+
+Eren Yagdiran <erenyagdiran(a)gmail.com>
+
+=head1 COPYRIGHT
+
+Copyright (C) 2013 Red Hat, Inc.
+Copyright (C) 2015 Universitat Politecnica de Catalunya.
+
+=head1 LICENSE
+
+virt-sandbox-image is distributed under the terms of the GNU LGPL v2+.
+This is free software; see the source for copying conditions.
+There is NO warranty; not even for MERCHANTABILITY or FITNESS
+FOR A PARTICULAR PURPOSE
diff --git a/libvirt-sandbox.spec.in b/libvirt-sandbox.spec.in
index 54fde55..b488cec 100644
--- a/libvirt-sandbox.spec.in
+++ b/libvirt-sandbox.spec.in
@@ -102,6 +102,7 @@ rm -rf $RPM_BUILD_ROOT
%{_libexecdir}/virt-sandbox-service-util
%{python_sitelib}/libvirt_sandbox
%{_mandir}/man1/virt-sandbox.1*
+%{_mandir}/man1/virt-sandbox-image.1*
%{_mandir}/man1/virt-sandbox-service.1*
%{_mandir}/man1/virt-sandbox-service-*.1*
--
2.4.3
7:12 a.m.
New subject: [libvirt] [PATCH sandbox v5 15/20] Image: man file for virt-sandbox-image
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
bin/Makefile.am | 5 ++
bin/virt-sandbox-image.pod | 169 +++++++++++++++++++++++++++++++++++++++++++++
libvirt-sandbox.spec.in | 1 +
3 files changed, 175 insertions(+)
create mode 100644 bin/virt-sandbox-image.pod
diff --git a/bin/Makefile.am b/bin/Makefile.am
index deedcf6..398e90c 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -20,6 +20,7 @@ POD_FILES = \
virt-sandbox-service-delete.pod \
virt-sandbox-service-reload.pod \
virt-sandbox-service-upgrade.pod \
+ virt-sandbox-image.pod \
$(NULL)
EXTRA_DIST = $(bin_SCRIPTS) $(POD_FILES) virt-sandbox-service-bash-completion.sh
virt-sandbox-service.logrotate
EXTRA_DIST += virt-sandbox-service-bash-completion.sh
@@ -34,6 +35,7 @@ man1_MANS = \
virt-sandbox-service-delete.1 \
virt-sandbox-service-reload.1 \
virt-sandbox-service-upgrade.1 \
+ virt-sandbox-image.1 \
$(NULL)
POD2MAN = pod2man -c "Virtualization Support" -r
"$(PACKAGE)-$(VERSION)"
@@ -65,6 +67,9 @@ virt-sandbox-service-reload.1: virt-sandbox-service-reload.pod
Makefile
virt-sandbox-service-upgrade.1: virt-sandbox-service-upgrade.pod Makefile
$(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
+virt-sandbox-image.1: virt-sandbox-image.pod Makefile
+ $(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
+
CLEANFILES = $(man1_MANS)
virt_sandbox_SOURCES = virt-sandbox.c
diff --git a/bin/virt-sandbox-image.pod b/bin/virt-sandbox-image.pod
new file mode 100644
index 0000000..85954b0
--- /dev/null
+++ b/bin/virt-sandbox-image.pod
@@ -0,0 +1,169 @@
+=head1 NAME
+
+virt-sandbox-image - Sandbox Container Image Tool
+
+=head1 SYNOPSIS
+
+ {download,create,run,delete}
+
+ commands:
+
+ download Download template data
+
+ create Create image from template data
+
+ run Run an already built image
+
+ delete Delete template data
+
+=head1 DESCRIPTION
+
+virt-sandbox-image.py is a sandbox container image tool developed in python.
+This tool can download,create,run and delete templates which are provided by
+different sources. This tool comes with Docker source by default. Other sources
+can be implemented by extending source class
The mention on the default source and possibility to add more may not
fit a man page... or we'll need to reword / remove it someday.
+=head1 OPTIONS
+
+=over 4
+
+=item B<download -s source -r registry -u username -p password -t template_directory
template>
+
Again here, do we really want to use the word "registry" sounding too
dockerish?
+Download a template by given name with a specified source.
+
+=over 6
+
+=item B<template>
+
+Template name to download
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has
to implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
Do we really want to include development hints in the man?
+=item B<-r or --registry>
+
+Custom registry url for downloading data. This might need privileged credentials which
can be specified by --username and --password parameters.
+
+=item B<-u or --username>
+
+Username for custom registry authentication
+
+=item B<-p or --password>
+
+Password for custom registry authentication
+
+=item B<-t or --template-dir>
+
+Custom directory for downloading template data
+
+=back
+
+=item B<create -s source -c uri -f format template>
+
+Create already downloaded template into image with given format.
+
+=over 5
+
+=item B<template>
+
+Template name to download.
+
+=item B<format>
+
+Image format e.g qcow2
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has
to implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
+=item B<-c or --connect>
+
+Driver parameter can be specified with only supported driver by libvirt-sandbox. These
are lxc:///, qemu:///session, qemu:///system.
+
+=back
+
+=item B<run -n name -N network -v volume -s source -c uri template command...>
+
+Run already built image. If B<command> is not specified, the default defined
+command for the image will be run.
+
+=over 6
+
+=item B<template>
+
+Template name to download.
+
+=item B<imagepath>
+
+Image path where template image will be stored.
+
+=item B<-n or --name>
+
+The sandbox guest name
+
+=item B<-N or --network>
+
+Network params will be passed directly to the virt-sandbox. More information about
network params, See C<virt-sandbox(8)>
+
+=item B<-v or --volume>
+
+Volume params are for binding host-paths to the guest. E.g -v /home:/home will map /home
directory from host to the guest.
+
+=item B<-c or --connect>
+
+Driver parameter can be specified with only supported driver by libvirt-sandbox. These
are lxc:///, qemu:///session, qemu:///system.
+
+=back
+
+=item B<delete name imagepath -s source >
+
+Delete downloaded template data and its built image.
+
+=over 3
+
+=item B<name>
+
+Template name to delete.
+
+=item B<imagepath>
+
+Image path where template data or image stays.
+
+=item B<-s or --source>
+
+Source parameter will try load source module under sources/ directory. Each source has
to implement Source.py base class and register itself with a proper name
+Default source is Docker.
+
+=back
+
+=back
+
+=head1 SEE ALSO
+
+C<virt-sandbox(8)>
+
+=head1 FILES
+
+Container content will be stored in subdirectories of
+/var/lib/libvirt/templates, by default.
+
+=head1 AUTHORS
+
+Daniel P. Berrange <dan(a)berrange.com>
+
+Eren Yagdiran <erenyagdiran(a)gmail.com>
+
+=head1 COPYRIGHT
+
+Copyright (C) 2013 Red Hat, Inc.
+Copyright (C) 2015 Universitat Politecnica de Catalunya.
+
+=head1 LICENSE
+
+virt-sandbox-image is distributed under the terms of the GNU LGPL v2+.
+This is free software; see the source for copying conditions.
+There is NO warranty; not even for MERCHANTABILITY or FITNESS
+FOR A PARTICULAR PURPOSE
diff --git a/libvirt-sandbox.spec.in b/libvirt-sandbox.spec.in
index 54fde55..b488cec 100644
--- a/libvirt-sandbox.spec.in
+++ b/libvirt-sandbox.spec.in
@@ -102,6 +102,7 @@ rm -rf $RPM_BUILD_ROOT
%{_libexecdir}/virt-sandbox-service-util
%{python_sitelib}/libvirt_sandbox
%{_mandir}/man1/virt-sandbox.1*
+%{_mandir}/man1/virt-sandbox-image.1*
%{_mandir}/man1/virt-sandbox-service.1*
%{_mandir}/man1/virt-sandbox-service-*.1*
ACK, but would be good to think about the raised questions before
pushing.
--
Cedric
7:16 a.m.
New subject: [libvirt] [PATCH sandbox v5 15/20] Image: man file for virt-sandbox-image
On Wed, Sep 09, 2015 at 02:12:28PM +0200, Cedric Bosdonnat wrote:
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
> From: Eren Yagdiran <erenyagdiran(a)gmail.com>
>
> Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
> ---
> bin/Makefile.am | 5 ++
> bin/virt-sandbox-image.pod | 169 +++++++++++++++++++++++++++++++++++++++++++++
> libvirt-sandbox.spec.in | 1 +
> 3 files changed, 175 insertions(+)
> create mode 100644 bin/virt-sandbox-image.pod
>
> diff --git a/bin/Makefile.am b/bin/Makefile.am
> index deedcf6..398e90c 100644
> --- a/bin/Makefile.am
> +++ b/bin/Makefile.am
> @@ -20,6 +20,7 @@ POD_FILES = \
> virt-sandbox-service-delete.pod \
> virt-sandbox-service-reload.pod \
> virt-sandbox-service-upgrade.pod \
> + virt-sandbox-image.pod \
> $(NULL)
> EXTRA_DIST = $(bin_SCRIPTS) $(POD_FILES) virt-sandbox-service-bash-completion.sh
virt-sandbox-service.logrotate
> EXTRA_DIST += virt-sandbox-service-bash-completion.sh
> @@ -34,6 +35,7 @@ man1_MANS = \
> virt-sandbox-service-delete.1 \
> virt-sandbox-service-reload.1 \
> virt-sandbox-service-upgrade.1 \
> + virt-sandbox-image.1 \
> $(NULL)
>
> POD2MAN = pod2man -c "Virtualization Support" -r
"$(PACKAGE)-$(VERSION)"
> @@ -65,6 +67,9 @@ virt-sandbox-service-reload.1: virt-sandbox-service-reload.pod
Makefile
> virt-sandbox-service-upgrade.1: virt-sandbox-service-upgrade.pod Makefile
> $(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
>
> +virt-sandbox-image.1: virt-sandbox-image.pod Makefile
> + $(AM_V_GEN)$(POD2MAN) $< $(srcdir)/$@
> +
> CLEANFILES = $(man1_MANS)
>
> virt_sandbox_SOURCES = virt-sandbox.c
> diff --git a/bin/virt-sandbox-image.pod b/bin/virt-sandbox-image.pod
> new file mode 100644
> index 0000000..85954b0
> --- /dev/null
> +++ b/bin/virt-sandbox-image.pod
> @@ -0,0 +1,169 @@
> +=head1 NAME
> +
> +virt-sandbox-image - Sandbox Container Image Tool
> +
> +=head1 SYNOPSIS
> +
> + {download,create,run,delete}
> +
> + commands:
> +
> + download Download template data
> +
> + create Create image from template data
> +
> + run Run an already built image
> +
> + delete Delete template data
> +
> +=head1 DESCRIPTION
> +
> +virt-sandbox-image.py is a sandbox container image tool developed in python.
> +This tool can download,create,run and delete templates which are provided by
> +different sources. This tool comes with Docker source by default. Other sources
> +can be implemented by extending source class
The mention on the default source and possibility to add more may not
fit a man page... or we'll need to reword / remove it someday.
> +=head1 OPTIONS
> +
> +=over 4
> +
> +=item B<download -s source -r registry -u username -p password -t
template_directory template>
> +
>
Again here, do we really want to use the word "registry" sounding too
dockerish?
>
> +Download a template by given name with a specified source.
> +
> +=over 6
> +
> +=item B<template>
> +
> +Template name to download
> +
> +=item B<-s or --source>
> +
> +Source parameter will try load source module under sources/ directory. Each source
has to implement Source.py base class and register itself with a proper name
> +Default source is Docker.
> +
Do we really want to include development hints in the man?
> +=item B<-r or --registry>
> +
> +Custom registry url for downloading data. This might need privileged credentials
which can be specified by --username and --password parameters.
> +
> +=item B<-u or --username>
> +
> +Username for custom registry authentication
> +
> +=item B<-p or --password>
> +
> +Password for custom registry authentication
> +
> +=item B<-t or --template-dir>
> +
> +Custom directory for downloading template data
> +
> +=back
> +
> +=item B<create -s source -c uri -f format template>
> +
> +Create already downloaded template into image with given format.
> +
> +=over 5
> +
> +=item B<template>
> +
> +Template name to download.
> +
> +=item B<format>
> +
> +Image format e.g qcow2
> +
> +=item B<-s or --source>
> +
> +Source parameter will try load source module under sources/ directory. Each source
has to implement Source.py base class and register itself with a proper name
> +Default source is Docker.
> +
> +=item B<-c or --connect>
> +
> +Driver parameter can be specified with only supported driver by libvirt-sandbox.
These are lxc:///, qemu:///session, qemu:///system.
> +
> +=back
> +
> +=item B<run -n name -N network -v volume -s source -c uri template
command...>
> +
> +Run already built image. If B<command> is not specified, the default defined
> +command for the image will be run.
> +
> +=over 6
> +
> +=item B<template>
> +
> +Template name to download.
> +
> +=item B<imagepath>
> +
> +Image path where template image will be stored.
> +
> +=item B<-n or --name>
> +
> +The sandbox guest name
> +
> +=item B<-N or --network>
> +
> +Network params will be passed directly to the virt-sandbox. More information about
network params, See C<virt-sandbox(8)>
> +
> +=item B<-v or --volume>
> +
> +Volume params are for binding host-paths to the guest. E.g -v /home:/home will map
/home directory from host to the guest.
> +
> +=item B<-c or --connect>
> +
> +Driver parameter can be specified with only supported driver by libvirt-sandbox.
These are lxc:///, qemu:///session, qemu:///system.
> +
> +=back
> +
> +=item B<delete name imagepath -s source >
> +
> +Delete downloaded template data and its built image.
> +
> +=over 3
> +
> +=item B<name>
> +
> +Template name to delete.
> +
> +=item B<imagepath>
> +
> +Image path where template data or image stays.
> +
> +=item B<-s or --source>
> +
> +Source parameter will try load source module under sources/ directory. Each source
has to implement Source.py base class and register itself with a proper name
> +Default source is Docker.
> +
> +=back
> +
> +=back
> +
> +=head1 SEE ALSO
> +
> +C<virt-sandbox(8)>
> +
> +=head1 FILES
> +
> +Container content will be stored in subdirectories of
> +/var/lib/libvirt/templates, by default.
> +
> +=head1 AUTHORS
> +
> +Daniel P. Berrange <dan(a)berrange.com>
> +
> +Eren Yagdiran <erenyagdiran(a)gmail.com>
> +
> +=head1 COPYRIGHT
> +
> +Copyright (C) 2013 Red Hat, Inc.
> +Copyright (C) 2015 Universitat Politecnica de Catalunya.
> +
> +=head1 LICENSE
> +
> +virt-sandbox-image is distributed under the terms of the GNU LGPL v2+.
> +This is free software; see the source for copying conditions.
> +There is NO warranty; not even for MERCHANTABILITY or FITNESS
> +FOR A PARTICULAR PURPOSE
> diff --git a/libvirt-sandbox.spec.in b/libvirt-sandbox.spec.in
> index 54fde55..b488cec 100644
> --- a/libvirt-sandbox.spec.in
> +++ b/libvirt-sandbox.spec.in
> @@ -102,6 +102,7 @@ rm -rf $RPM_BUILD_ROOT
> %{_libexecdir}/virt-sandbox-service-util
> %{python_sitelib}/libvirt_sandbox
> %{_mandir}/man1/virt-sandbox.1*
> +%{_mandir}/man1/virt-sandbox-image.1*
> %{_mandir}/man1/virt-sandbox-service.1*
> %{_mandir}/man1/virt-sandbox-service-*.1*
>
ACK, but would be good to think about the raised questions before
pushing.
Yeah, I think we could temporarily postpone this, as the man page could
do with a little more work
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 16/20] Add config for environment variables
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Add the config gobject to store custom environment variables.
This will allow creating custom environment variables on a sandbox
with a parameter formatted like --env key1=val1
Add testcase for custom environment variables
"make check" now includes testcase for environment variables
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/libvirt-sandbox-config.c | 171 ++++++++++++++++++++++++++++++-
libvirt-sandbox/libvirt-sandbox-config.h | 13 +++
libvirt-sandbox/libvirt-sandbox.sym | 9 ++
libvirt-sandbox/tests/test-config.c | 10 ++
4 files changed, 202 insertions(+), 1 deletion(-)
diff --git a/libvirt-sandbox/libvirt-sandbox-config.c
b/libvirt-sandbox/libvirt-sandbox-config.c
index 2506072..780d174 100644
--- a/libvirt-sandbox/libvirt-sandbox-config.c
+++ b/libvirt-sandbox/libvirt-sandbox-config.c
@@ -64,6 +64,7 @@ struct _GVirSandboxConfigPrivate
GList *networks;
GList *mounts;
GList *disks;
+ GHashTable* envs;
gchar *secLabel;
gboolean secDynamic;
@@ -276,6 +277,8 @@ static void gvir_sandbox_config_finalize(GObject *object)
g_list_foreach(priv->networks, (GFunc)g_object_unref, NULL);
g_list_free(priv->networks);
+ g_hash_table_destroy(priv->envs);
+
g_list_foreach(priv->disks, (GFunc)g_object_unref, NULL);
g_list_free(priv->disks);
@@ -483,6 +486,7 @@ static void gvir_sandbox_config_init(GVirSandboxConfig *config)
priv->arch = g_strdup(uts.machine);
priv->secDynamic = TRUE;
+ priv->envs = g_hash_table_new(g_str_hash, g_str_equal);
priv->uid = geteuid();
priv->gid = getegid();
priv->username = g_strdup(g_get_user_name());
@@ -1144,6 +1148,102 @@ gboolean gvir_sandbox_config_has_networks(GVirSandboxConfig
*config)
return priv->networks ? TRUE : FALSE;
}
+/**
+ * gvir_sandbox_config_add_env:
+ * @config: (transfer none): the sandbox config
+ * @key: (transfer none): the key for environment variable
+ * @value: (transfer none): the value for environment variable
+ *
+ * Adds a new environment variable to the sandbox
+ *
+ */
+void gvir_sandbox_config_add_env(GVirSandboxConfig *config,
+ gchar *k,
+ gchar *v)
+{
+ gchar * key = g_strdup(k);
+ gchar * value = g_strdup(v);
+ GVirSandboxConfigPrivate *priv = config->priv;
+ g_hash_table_insert(priv->envs,key,value);
+}
+
+/**
+ * gvir_sandbox_config_get_envs:
+ * @config: (transfer none): the sandbox config
+ *
+ * Retrieves the hashtable of custom environment variables in the sandbox
+ *
+ * Returns: (transfer full) (element-type gchar gchar): the hashtable of environment
variables
+ */
+GHashTable *gvir_sandbox_config_get_envs(GVirSandboxConfig *config)
+{
+ GVirSandboxConfigPrivate *priv = config->priv;
+ return priv->envs;
+}
+
+/**
+ * gvir_sandbox_config_add_env_strv:
+ * @config: (transfer none): the sandbox config
+ * @envs: (transfer none)(array zero-terminated=1): the list of environment variables
+ *
+ * Parses @envs whose elements are in the format KEY=VALUE
+ *
+ * --env KEY=VALUE
+ */
+gboolean gvir_sandbox_config_add_env_strv(GVirSandboxConfig *config,
+ gchar **envs,
+ GError **error)
+{
+ gsize i = 0;
+ while (envs && envs[i]) {
+ if (!gvir_sandbox_config_add_env_opts(config,
+ envs[i],
+ error))
+ return FALSE;
+ i++;
+ }
+ return TRUE;
+}
+
+/**
+ * gvir_sandbox_config_add_env_opts:
+ * @config: (transfer none): the sandbox config
+ * @env: (transfer none): the env config
+ *
+ * Parses @env in the format KEY=VALUE
+ * creating #GVirSandboxConfigEnv instances for each element. For
+ * example
+ *
+ * --env KEY=VALUE
+ */
+
+gboolean gvir_sandbox_config_add_env_opts(GVirSandboxConfig *config,
+ const char *opt,
+ GError **error)
+{
+ gchar *tmp = NULL;
+ gchar *key = NULL;
+ gchar *value = NULL;
+
+ if (!(tmp = g_strdup(opt)))
+ return FALSE;
+
+ key = strchr(tmp, '=');
+
+ if (!key) {
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ _("Wrong environment format on %s"), opt);
+ return FALSE;
+ }
+
+ *key = '\0';
+ value = key + 1;
+
+ gvir_sandbox_config_add_env(config, tmp, value);
+
+ g_free(tmp);
+ return TRUE;
+}
/**
* gvir_sandbox_config_add_disk:
@@ -1163,7 +1263,6 @@ void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
priv->disks = g_list_append(priv->disks, dsk);
}
-
/**
* gvir_sandbox_config_get_disks:
* @config: (transfer none): the sandbox config
@@ -1172,6 +1271,7 @@ void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
*
* Returns: (transfer full) (element-type GVirSandboxConfigMount): the list of disks
*/
+
GList *gvir_sandbox_config_get_disks(GVirSandboxConfig *config)
{
GVirSandboxConfigPrivate *priv = config->priv;
@@ -1949,6 +2049,39 @@ static GVirSandboxConfigMount
*gvir_sandbox_config_load_config_mount(GKeyFile *f
goto cleanup;
}
+static gboolean gvir_sandbox_config_load_config_env(GKeyFile *file,
+ guint i,
+ GError **error,
+ gchar **key,
+ gchar **value)
+{
+ gboolean ret = TRUE;
+ gchar *index = NULL;
+ GError *e = NULL;
+ index = g_strdup_printf("env.%u", i);
+ if ((*key = g_key_file_get_string(file, index, "key", &e)) == NULL) {
+ if (e->code == G_KEY_FILE_ERROR_GROUP_NOT_FOUND) {
+ g_error_free(e);
+ ret = FALSE;
+ goto cleanup;
+ }
+ g_error_free(e);
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ "%s", _("Missing environment key in config
file"));
+ ret = FALSE;
+ goto cleanup;
+ }
+ if ((*value = g_key_file_get_string(file, index, "value", NULL)) == NULL)
{
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ "%s", _("Missing environment value in config
file"));
+ ret = FALSE;
+ goto cleanup;
+ }
+
+ cleanup:
+ g_free(index);
+ return ret;
+}
static GVirSandboxConfigDisk *gvir_sandbox_config_load_config_disk(GKeyFile *file,
guint i,
@@ -2244,6 +2377,19 @@ static gboolean gvir_sandbox_config_load_config(GVirSandboxConfig
*config,
priv->mounts = g_list_append(priv->mounts, mount);
}
+ for (i = 0 ; i < 1024; i++) {
+ gchar *key = NULL;
+ gchar *value = NULL;
+ if (!(gvir_sandbox_config_load_config_env(file, i, error,&key,&value))
&& *error){
+ g_free(key);
+ g_free(value);
+ ret = FALSE;
+ goto cleanup;
+ }
+ if (key){
+ g_hash_table_insert(priv->envs,key,value);
+ }
+ }
for (i = 0 ; i < 1024 ; i++) {
GVirSandboxConfigDisk *disk;
@@ -2274,6 +2420,19 @@ static gboolean gvir_sandbox_config_load_config(GVirSandboxConfig
*config,
return ret;
}
+static void gvir_sandbox_config_save_config_env(gchar *key,
+ gchar *value,
+ GKeyFile *file,
+ guint i)
+{
+ gchar *index = NULL;
+ index = g_strdup_printf("env.%u", i);
+ g_key_file_set_string(file, index, "key",key);
+ g_key_file_set_string(file, index, "value",value);
+ g_free(index);
+ g_free(key);
+ g_free(value);
+}
static void gvir_sandbox_config_save_config_disk(GVirSandboxConfigDisk *config,
GKeyFile *file,
@@ -2484,6 +2643,16 @@ static void gvir_sandbox_config_save_config(GVirSandboxConfig
*config,
i++;
}
+
+ GHashTableIter iter;
+ gpointer key, value;
+ i = 0;
+ g_hash_table_iter_init (&iter, priv->envs);
+ while (g_hash_table_iter_next (&iter, &key, &value)){
+ gvir_sandbox_config_save_config_env(key,value,file,i);
+ i++;
+ }
+
i = 0;
tmp = priv->disks;
while (tmp) {
diff --git a/libvirt-sandbox/libvirt-sandbox-config.h
b/libvirt-sandbox/libvirt-sandbox-config.h
index 2c5f0a6..e5e53f7 100644
--- a/libvirt-sandbox/libvirt-sandbox-config.h
+++ b/libvirt-sandbox/libvirt-sandbox-config.h
@@ -122,6 +122,19 @@ gboolean gvir_sandbox_config_add_network_strv(GVirSandboxConfig
*config,
GError **error);
gboolean gvir_sandbox_config_has_networks(GVirSandboxConfig *config);
+void gvir_sandbox_config_add_env(GVirSandboxConfig *config,
+ gchar *key,
+ gchar *value);
+GHashTable *gvir_sandbox_config_get_envs(GVirSandboxConfig *config);
+gboolean gvir_sandbox_config_add_env_strv(GVirSandboxConfig *config,
+ gchar **envs,
+ GError **error);
+gboolean gvir_sandbox_config_add_env_opts(GVirSandboxConfig *config,
+ const char *env,
+ GError **error);
+gboolean gvir_sandbox_config_has_envs(GVirSandboxConfig *config);
+
+
void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
GVirSandboxConfigDisk *dsk);
GList *gvir_sandbox_config_get_disks(GVirSandboxConfig *config);
diff --git a/libvirt-sandbox/libvirt-sandbox.sym b/libvirt-sandbox/libvirt-sandbox.sym
index 6f8097a..b7c5921 100644
--- a/libvirt-sandbox/libvirt-sandbox.sym
+++ b/libvirt-sandbox/libvirt-sandbox.sym
@@ -217,3 +217,12 @@ LIBVIRT_SANDBOX_0.6.0 {
local:
*;
};
+
+LIBVIRT_SANDBOX_0.6.1 {
+ global:
+ gvir_sandbox_config_add_env;
+ gvir_sandbox_config_add_env_strv;
+ gvir_sandbox_config_add_env_opts;
+ gvir_sandbox_config_env_get_type;
+ gvir_sandbox_config_has_envs;
+} LIBVIRT_SANDBOX_0.6.0;
diff --git a/libvirt-sandbox/tests/test-config.c b/libvirt-sandbox/tests/test-config.c
index da05187..ac10bab 100644
--- a/libvirt-sandbox/tests/test-config.c
+++ b/libvirt-sandbox/tests/test-config.c
@@ -58,6 +58,13 @@ int main(int argc, char **argv)
"host-bind:/tmp=",
NULL
};
+
+ const gchar *envs[] = {
+ "key1=val1",
+ "key2=val2",
+ NULL
+ };
+
const gchar *disks[] = {
"file:dbdata=/tmp/img.blah,format=qcow2",
"file:cache=/tmp/img.qcow2",
@@ -103,6 +110,9 @@ int main(int argc, char **argv)
if (!gvir_sandbox_config_add_mount_strv(cfg1, (gchar**)mounts, &err))
goto cleanup;
+ if (!gvir_sandbox_config_add_env_strv(cfg1, (gchar**)envs, &err))
+ goto cleanup;
+
if (!gvir_sandbox_config_add_disk_strv(cfg1, (gchar**)disks, &err))
goto cleanup;
--
2.4.3
7:13 a.m.
New subject: [libvirt] [PATCH sandbox v5 16/20] Add config for environment variables
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Add the config gobject to store custom environment variables.
This will allow creating custom environment variables on a sandbox
with a parameter formatted like --env key1=val1
Add testcase for custom environment variables
"make check" now includes testcase for environment variables
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/libvirt-sandbox-config.c | 171 ++++++++++++++++++++++++++++++-
libvirt-sandbox/libvirt-sandbox-config.h | 13 +++
libvirt-sandbox/libvirt-sandbox.sym | 9 ++
libvirt-sandbox/tests/test-config.c | 10 ++
4 files changed, 202 insertions(+), 1 deletion(-)
diff --git a/libvirt-sandbox/libvirt-sandbox-config.c
b/libvirt-sandbox/libvirt-sandbox-config.c
index 2506072..780d174 100644
--- a/libvirt-sandbox/libvirt-sandbox-config.c
+++ b/libvirt-sandbox/libvirt-sandbox-config.c
@@ -64,6 +64,7 @@ struct _GVirSandboxConfigPrivate
GList *networks;
GList *mounts;
GList *disks;
+ GHashTable* envs;
gchar *secLabel;
gboolean secDynamic;
@@ -276,6 +277,8 @@ static void gvir_sandbox_config_finalize(GObject *object)
g_list_foreach(priv->networks, (GFunc)g_object_unref, NULL);
g_list_free(priv->networks);
+ g_hash_table_destroy(priv->envs);
+
g_list_foreach(priv->disks, (GFunc)g_object_unref, NULL);
g_list_free(priv->disks);
@@ -483,6 +486,7 @@ static void gvir_sandbox_config_init(GVirSandboxConfig *config)
priv->arch = g_strdup(uts.machine);
priv->secDynamic = TRUE;
+ priv->envs = g_hash_table_new(g_str_hash, g_str_equal);
priv->uid = geteuid();
priv->gid = getegid();
priv->username = g_strdup(g_get_user_name());
@@ -1144,6 +1148,102 @@ gboolean gvir_sandbox_config_has_networks(GVirSandboxConfig
*config)
return priv->networks ? TRUE : FALSE;
}
+/**
+ * gvir_sandbox_config_add_env:
+ * @config: (transfer none): the sandbox config
+ * @key: (transfer none): the key for environment variable
+ * @value: (transfer none): the value for environment variable
+ *
+ * Adds a new environment variable to the sandbox
+ *
+ */
+void gvir_sandbox_config_add_env(GVirSandboxConfig *config,
+ gchar *k,
+ gchar *v)
+{
+ gchar * key = g_strdup(k);
+ gchar * value = g_strdup(v);
+ GVirSandboxConfigPrivate *priv = config->priv;
+ g_hash_table_insert(priv->envs,key,value);
+}
+
+/**
+ * gvir_sandbox_config_get_envs:
+ * @config: (transfer none): the sandbox config
+ *
+ * Retrieves the hashtable of custom environment variables in the sandbox
+ *
+ * Returns: (transfer full) (element-type gchar gchar): the hashtable of environment
variables
+ */
+GHashTable *gvir_sandbox_config_get_envs(GVirSandboxConfig *config)
+{
+ GVirSandboxConfigPrivate *priv = config->priv;
+ return priv->envs;
+}
+
+/**
+ * gvir_sandbox_config_add_env_strv:
+ * @config: (transfer none): the sandbox config
+ * @envs: (transfer none)(array zero-terminated=1): the list of environment variables
+ *
+ * Parses @envs whose elements are in the format KEY=VALUE
+ *
+ * --env KEY=VALUE
+ */
+gboolean gvir_sandbox_config_add_env_strv(GVirSandboxConfig *config,
+ gchar **envs,
+ GError **error)
+{
+ gsize i = 0;
+ while (envs && envs[i]) {
+ if (!gvir_sandbox_config_add_env_opts(config,
+ envs[i],
+ error))
+ return FALSE;
+ i++;
+ }
+ return TRUE;
+}
+
+/**
+ * gvir_sandbox_config_add_env_opts:
+ * @config: (transfer none): the sandbox config
+ * @env: (transfer none): the env config
+ *
+ * Parses @env in the format KEY=VALUE
+ * creating #GVirSandboxConfigEnv instances for each element. For
+ * example
+ *
+ * --env KEY=VALUE
+ */
+
+gboolean gvir_sandbox_config_add_env_opts(GVirSandboxConfig *config,
+ const char *opt,
+ GError **error)
+{
+ gchar *tmp = NULL;
+ gchar *key = NULL;
+ gchar *value = NULL;
+
+ if (!(tmp = g_strdup(opt)))
+ return FALSE;
+
+ key = strchr(tmp, '=');
+
+ if (!key) {
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ _("Wrong environment format on %s"), opt);
+ return FALSE;
+ }
+
+ *key = '\0';
+ value = key + 1;
+
+ gvir_sandbox_config_add_env(config, tmp, value);
+
+ g_free(tmp);
+ return TRUE;
+}
/**
* gvir_sandbox_config_add_disk:
@@ -1163,7 +1263,6 @@ void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
priv->disks = g_list_append(priv->disks, dsk);
}
-
/**
* gvir_sandbox_config_get_disks:
* @config: (transfer none): the sandbox config
@@ -1172,6 +1271,7 @@ void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
*
* Returns: (transfer full) (element-type GVirSandboxConfigMount): the list of disks
*/
+
GList *gvir_sandbox_config_get_disks(GVirSandboxConfig *config)
{
GVirSandboxConfigPrivate *priv = config->priv;
@@ -1949,6 +2049,39 @@ static GVirSandboxConfigMount
*gvir_sandbox_config_load_config_mount(GKeyFile *f
goto cleanup;
}
+static gboolean gvir_sandbox_config_load_config_env(GKeyFile *file,
+ guint i,
+ GError **error,
+ gchar **key,
+ gchar **value)
+{
+ gboolean ret = TRUE;
+ gchar *index = NULL;
+ GError *e = NULL;
+ index = g_strdup_printf("env.%u", i);
+ if ((*key = g_key_file_get_string(file, index, "key", &e)) == NULL) {
+ if (e->code == G_KEY_FILE_ERROR_GROUP_NOT_FOUND) {
+ g_error_free(e);
+ ret = FALSE;
+ goto cleanup;
+ }
+ g_error_free(e);
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ "%s", _("Missing environment key in config
file"));
+ ret = FALSE;
+ goto cleanup;
+ }
+ if ((*value = g_key_file_get_string(file, index, "value", NULL)) == NULL)
{
+ g_set_error(error, GVIR_SANDBOX_CONFIG_ERROR, 0,
+ "%s", _("Missing environment value in config
file"));
+ ret = FALSE;
+ goto cleanup;
+ }
+
+ cleanup:
+ g_free(index);
+ return ret;
+}
static GVirSandboxConfigDisk *gvir_sandbox_config_load_config_disk(GKeyFile *file,
guint i,
@@ -2244,6 +2377,19 @@ static gboolean gvir_sandbox_config_load_config(GVirSandboxConfig
*config,
priv->mounts = g_list_append(priv->mounts, mount);
}
+ for (i = 0 ; i < 1024; i++) {
+ gchar *key = NULL;
+ gchar *value = NULL;
+ if (!(gvir_sandbox_config_load_config_env(file, i, error,&key,&value))
&& *error){
+ g_free(key);
+ g_free(value);
+ ret = FALSE;
+ goto cleanup;
+ }
+ if (key){
+ g_hash_table_insert(priv->envs,key,value);
+ }
+ }
for (i = 0 ; i < 1024 ; i++) {
GVirSandboxConfigDisk *disk;
@@ -2274,6 +2420,19 @@ static gboolean gvir_sandbox_config_load_config(GVirSandboxConfig
*config,
return ret;
}
+static void gvir_sandbox_config_save_config_env(gchar *key,
+ gchar *value,
+ GKeyFile *file,
+ guint i)
+{
+ gchar *index = NULL;
+ index = g_strdup_printf("env.%u", i);
+ g_key_file_set_string(file, index, "key",key);
+ g_key_file_set_string(file, index, "value",value);
+ g_free(index);
+ g_free(key);
+ g_free(value);
+}
static void gvir_sandbox_config_save_config_disk(GVirSandboxConfigDisk *config,
GKeyFile *file,
@@ -2484,6 +2643,16 @@ static void gvir_sandbox_config_save_config(GVirSandboxConfig
*config,
i++;
}
+
+ GHashTableIter iter;
+ gpointer key, value;
+ i = 0;
+ g_hash_table_iter_init (&iter, priv->envs);
+ while (g_hash_table_iter_next (&iter, &key, &value)){
+ gvir_sandbox_config_save_config_env(key,value,file,i);
+ i++;
+ }
+
i = 0;
tmp = priv->disks;
while (tmp) {
diff --git a/libvirt-sandbox/libvirt-sandbox-config.h
b/libvirt-sandbox/libvirt-sandbox-config.h
index 2c5f0a6..e5e53f7 100644
--- a/libvirt-sandbox/libvirt-sandbox-config.h
+++ b/libvirt-sandbox/libvirt-sandbox-config.h
@@ -122,6 +122,19 @@ gboolean gvir_sandbox_config_add_network_strv(GVirSandboxConfig
*config,
GError **error);
gboolean gvir_sandbox_config_has_networks(GVirSandboxConfig *config);
+void gvir_sandbox_config_add_env(GVirSandboxConfig *config,
+ gchar *key,
+ gchar *value);
+GHashTable *gvir_sandbox_config_get_envs(GVirSandboxConfig *config);
+gboolean gvir_sandbox_config_add_env_strv(GVirSandboxConfig *config,
+ gchar **envs,
+ GError **error);
+gboolean gvir_sandbox_config_add_env_opts(GVirSandboxConfig *config,
+ const char *env,
+ GError **error);
+gboolean gvir_sandbox_config_has_envs(GVirSandboxConfig *config);
+
+
void gvir_sandbox_config_add_disk(GVirSandboxConfig *config,
GVirSandboxConfigDisk *dsk);
GList *gvir_sandbox_config_get_disks(GVirSandboxConfig *config);
diff --git a/libvirt-sandbox/libvirt-sandbox.sym b/libvirt-sandbox/libvirt-sandbox.sym
index 6f8097a..b7c5921 100644
--- a/libvirt-sandbox/libvirt-sandbox.sym
+++ b/libvirt-sandbox/libvirt-sandbox.sym
@@ -217,3 +217,12 @@ LIBVIRT_SANDBOX_0.6.0 {
local:
*;
};
+
+LIBVIRT_SANDBOX_0.6.1 {
+ global:
+ gvir_sandbox_config_add_env;
+ gvir_sandbox_config_add_env_strv;
+ gvir_sandbox_config_add_env_opts;
+ gvir_sandbox_config_env_get_type;
+ gvir_sandbox_config_has_envs;
+} LIBVIRT_SANDBOX_0.6.0;
diff --git a/libvirt-sandbox/tests/test-config.c b/libvirt-sandbox/tests/test-config.c
index da05187..ac10bab 100644
--- a/libvirt-sandbox/tests/test-config.c
+++ b/libvirt-sandbox/tests/test-config.c
@@ -58,6 +58,13 @@ int main(int argc, char **argv)
"host-bind:/tmp=",
NULL
};
+
+ const gchar *envs[] = {
+ "key1=val1",
+ "key2=val2",
+ NULL
+ };
+
const gchar *disks[] = {
"file:dbdata=/tmp/img.blah,format=qcow2",
"file:cache=/tmp/img.qcow2",
@@ -103,6 +110,9 @@ int main(int argc, char **argv)
if (!gvir_sandbox_config_add_mount_strv(cfg1, (gchar**)mounts, &err))
goto cleanup;
+ if (!gvir_sandbox_config_add_env_strv(cfg1, (gchar**)envs, &err))
+ goto cleanup;
+
if (!gvir_sandbox_config_add_disk_strv(cfg1, (gchar**)disks, &err))
goto cleanup;
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 17/20] Add environment parameter to virt-sandbox
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Allow users to add custom environment variables to their sandbox.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
bin/virt-sandbox.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/bin/virt-sandbox.c b/bin/virt-sandbox.c
index 332e53e..4c400d5 100644
--- a/bin/virt-sandbox.c
+++ b/bin/virt-sandbox.c
@@ -80,6 +80,7 @@ int main(int argc, char **argv) {
GError *error = NULL;
gchar *name = NULL;
gchar **disks = NULL;
+ gchar **envs = NULL;
gchar **mounts = NULL;
gchar **includes = NULL;
gchar *includefile = NULL;
@@ -111,6 +112,8 @@ int main(int argc, char **argv) {
N_("root directory of the sandbox"), "DIR" },
{ "disk", ' ', 0, G_OPTION_ARG_STRING_ARRAY, &disks,
N_("add a disk in the guest"),
"TYPE:TAGNAME=SOURCE,format=FORMAT" },
+ { "env", 'e', 0, G_OPTION_ARG_STRING_ARRAY, &envs,
+ N_("add a environment variable for the sandbox"),
"KEY=VALUE" },
{ "mount", 'm', 0, G_OPTION_ARG_STRING_ARRAY, &mounts,
N_("mount a filesystem in the guest"), "TYPE:TARGET=SOURCE"
},
{ "include", 'i', 0, G_OPTION_ARG_STRING_ARRAY, &includes,
@@ -201,6 +204,13 @@ int main(int argc, char **argv) {
gvir_sandbox_config_set_username(cfg, "root");
}
+ if (envs &&
+ !gvir_sandbox_config_add_env_strv(cfg, envs, &error)) {
+ g_printerr(_("Unable to parse custom environment variables: %s\n"),
+ error && error->message ? error->message :
_("Unknown failure"));
+ goto cleanup;
+ }
+
if (disks &&
!gvir_sandbox_config_add_disk_strv(cfg, disks, &error)) {
g_printerr(_("Unable to parse disks: %s\n"),
@@ -350,6 +360,10 @@ inheriting the host's root filesystem.
NB. C<DIR> must contain a matching install of the libvirt-sandbox
package. This restriction may be lifted in a future version.
+=item B<--env key=value>
+
+Sets up a custom environment variable on a running sandbox.
+
=item B<--disk TYPE:TAGNAME=SOURCE,format=FORMAT>
Sets up a disk inside the sandbox by using B<SOURCE> with a symlink named as
B<TAGNAME>
--
2.4.3
7:14 a.m.
New subject: [libvirt] [PATCH sandbox v5 17/20] Add environment parameter to virt-sandbox
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Allow users to add custom environment variables to their sandbox.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
bin/virt-sandbox.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/bin/virt-sandbox.c b/bin/virt-sandbox.c
index 332e53e..4c400d5 100644
--- a/bin/virt-sandbox.c
+++ b/bin/virt-sandbox.c
@@ -80,6 +80,7 @@ int main(int argc, char **argv) {
GError *error = NULL;
gchar *name = NULL;
gchar **disks = NULL;
+ gchar **envs = NULL;
gchar **mounts = NULL;
gchar **includes = NULL;
gchar *includefile = NULL;
@@ -111,6 +112,8 @@ int main(int argc, char **argv) {
N_("root directory of the sandbox"), "DIR" },
{ "disk", ' ', 0, G_OPTION_ARG_STRING_ARRAY, &disks,
N_("add a disk in the guest"),
"TYPE:TAGNAME=SOURCE,format=FORMAT" },
+ { "env", 'e', 0, G_OPTION_ARG_STRING_ARRAY, &envs,
+ N_("add a environment variable for the sandbox"),
"KEY=VALUE" },
{ "mount", 'm', 0, G_OPTION_ARG_STRING_ARRAY, &mounts,
N_("mount a filesystem in the guest"),
"TYPE:TARGET=SOURCE" },
{ "include", 'i', 0, G_OPTION_ARG_STRING_ARRAY,
&includes,
@@ -201,6 +204,13 @@ int main(int argc, char **argv) {
gvir_sandbox_config_set_username(cfg, "root");
}
+ if (envs &&
+ !gvir_sandbox_config_add_env_strv(cfg, envs, &error)) {
+ g_printerr(_("Unable to parse custom environment variables: %s\n"),
+ error && error->message ? error->message :
_("Unknown failure"));
+ goto cleanup;
+ }
+
if (disks &&
!gvir_sandbox_config_add_disk_strv(cfg, disks, &error)) {
g_printerr(_("Unable to parse disks: %s\n"),
@@ -350,6 +360,10 @@ inheriting the host's root filesystem.
NB. C<DIR> must contain a matching install of the libvirt-sandbox
package. This restriction may be lifted in a future version.
+=item B<--env key=value>
+
+Sets up a custom environment variable on a running sandbox.
+
=item B<--disk TYPE:TAGNAME=SOURCE,format=FORMAT>
Sets up a disk inside the sandbox by using B<SOURCE> with a symlink named as
B<TAGNAME>
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 18/20] init-common: Exporting custom environment variables
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Common-init reads config file and exports custom environment
variables from config file and applies them to the running sandbox.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/libvirt-sandbox-init-common.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/libvirt-sandbox/libvirt-sandbox-init-common.c
b/libvirt-sandbox/libvirt-sandbox-init-common.c
index e3e14a3..42beadc 100644
--- a/libvirt-sandbox/libvirt-sandbox-init-common.c
+++ b/libvirt-sandbox/libvirt-sandbox-init-common.c
@@ -348,6 +348,21 @@ static gboolean setup_network(GVirSandboxConfig *config, GError
**error)
}
+static gboolean setup_custom_env(GVirSandboxConfig *config, GError **error)
+{
+ gboolean ret = FALSE;
+ GHashTableIter iter;
+ gpointer key, value;
+ g_hash_table_iter_init (&iter, gvir_sandbox_config_get_envs(config));
+ while (g_hash_table_iter_next (&iter, &key, &value)){
+ if(setenv(key,value,1)!=0)
+ goto cleanup;
+ }
+ ret = TRUE;
+ cleanup:
+ return ret;
+}
+
static int change_user(const gchar *user,
uid_t uid,
gid_t gid,
@@ -1436,6 +1451,9 @@ int main(int argc, char **argv) {
if (!setup_disk_tags())
exit(EXIT_FAILURE);
+ if (!setup_custom_env(config, &error))
+ goto error;
+
if (!setup_network(config, &error))
goto error;
--
2.4.3
7:14 a.m.
New subject: [libvirt] [PATCH sandbox v5 18/20] init-common: Exporting custom environment variables
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Common-init reads config file and exports custom environment
variables from config file and applies them to the running sandbox.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/libvirt-sandbox-init-common.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/libvirt-sandbox/libvirt-sandbox-init-common.c
b/libvirt-sandbox/libvirt-sandbox-init-common.c
index e3e14a3..42beadc 100644
--- a/libvirt-sandbox/libvirt-sandbox-init-common.c
+++ b/libvirt-sandbox/libvirt-sandbox-init-common.c
@@ -348,6 +348,21 @@ static gboolean setup_network(GVirSandboxConfig *config, GError
**error)
}
+static gboolean setup_custom_env(GVirSandboxConfig *config, GError **error)
+{
+ gboolean ret = FALSE;
+ GHashTableIter iter;
+ gpointer key, value;
+ g_hash_table_iter_init (&iter, gvir_sandbox_config_get_envs(config));
+ while (g_hash_table_iter_next (&iter, &key, &value)){
+ if(setenv(key,value,1)!=0)
+ goto cleanup;
+ }
+ ret = TRUE;
+ cleanup:
+ return ret;
+}
+
static int change_user(const gchar *user,
uid_t uid,
gid_t gid,
@@ -1436,6 +1451,9 @@ int main(int argc, char **argv) {
if (!setup_disk_tags())
exit(EXIT_FAILURE);
+ if (!setup_custom_env(config, &error))
+ goto error;
+
if (!setup_network(config, &error))
goto error;
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 19/20] Image: Add custom environment support
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Any custom key=value pair can be used as a custom environment variable
in virt-sandbox-image.
e.g virt-sandbox-image run ubuntu /var/lib/libvirt/templates -c lxc:/// -i /bin/bash -e
key1=val1
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 16 ++++++++++++++++
libvirt-sandbox/image/sources/DockerSource.py | 11 +++++++++++
libvirt-sandbox/image/sources/Source.py | 10 ++++++++++
3 files changed, 37 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 30e2558..c17b577 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -129,6 +129,19 @@ def run(args):
params.append('-N')
params.append(networkArgs)
+ allEnvs = source.get_env(args.template, args.template_dir)
+ envArgs = args.env
+ if envArgs is not None:
+ allEnvs = allEnvs + envArgs
+ for env in allEnvs:
+ envsplit = env.split("=")
+ envlen = len(envsplit)
+ if envlen == 2:
+ params.append("--env")
+ params.append(env)
+ else:
+ pass
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
@@ -222,6 +235,9 @@ def gen_run_args(subparser):
help=_("command arguments to run"))
parser.add_argument("-N","--network",
help=_("Network params for running template"))
+ parser.add_argument("-e","--env",action="append",
+ help=_("Environment params for running template"))
+
parser.set_defaults(func=run)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 31c1d80..4455198 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -38,6 +38,12 @@ class DockerConfParser():
return self.json_data['config']['Cmd']
def getEntrypoint(self):
return self.json_data['config']['Entrypoint']
+ def getEnvs(self):
+ lst = self.json_data['config']['Env']
+ if lst is not None and isinstance(lst,list):
+ return lst
+ else:
+ return []
class DockerSource(Source):
@@ -388,5 +394,10 @@ class DockerSource(Source):
else:
return entrypoint + cmd
+ def get_env(self, templatename, templatedir):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ return configParser.getEnvs()
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index a5d3844..8a21f90 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -95,3 +95,13 @@ class Source():
of a template.
"""
pass
+
+ @abstractmethod
+ def get_env(self,templatename, templatedir):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+
+ Get the dict of environment variables to set
+ """
+ pass
--
2.4.3
7:15 a.m.
New subject: [libvirt] [PATCH sandbox v5 19/20] Image: Add custom environment support
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Any custom key=value pair can be used as a custom environment variable
in virt-sandbox-image.
e.g virt-sandbox-image run ubuntu /var/lib/libvirt/templates -c lxc:/// -i /bin/bash -e
key1=val1
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 16 ++++++++++++++++
libvirt-sandbox/image/sources/DockerSource.py | 11 +++++++++++
libvirt-sandbox/image/sources/Source.py | 10 ++++++++++
3 files changed, 37 insertions(+)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index 30e2558..c17b577 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -129,6 +129,19 @@ def run(args):
params.append('-N')
params.append(networkArgs)
+ allEnvs = source.get_env(args.template, args.template_dir)
+ envArgs = args.env
+ if envArgs is not None:
+ allEnvs = allEnvs + envArgs
+ for env in allEnvs:
+ envsplit = env.split("=")
+ envlen = len(envsplit)
+ if envlen == 2:
+ params.append("--env")
+ params.append(env)
+ else:
+ pass
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
@@ -222,6 +235,9 @@ def gen_run_args(subparser):
help=_("command arguments to run"))
parser.add_argument("-N","--network",
help=_("Network params for running template"))
+ parser.add_argument("-e","--env",action="append",
+ help=_("Environment params for running template"))
+
parser.set_defaults(func=run)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 31c1d80..4455198 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -38,6 +38,12 @@ class DockerConfParser():
return self.json_data['config']['Cmd']
def getEntrypoint(self):
return self.json_data['config']['Entrypoint']
+ def getEnvs(self):
+ lst = self.json_data['config']['Env']
+ if lst is not None and isinstance(lst,list):
+ return lst
+ else:
+ return []
class DockerSource(Source):
@@ -388,5 +394,10 @@ class DockerSource(Source):
else:
return entrypoint + cmd
+ def get_env(self, templatename, templatedir):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ return configParser.getEnvs()
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index a5d3844..8a21f90 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -95,3 +95,13 @@ class Source():
of a template.
"""
pass
+
+ @abstractmethod
+ def get_env(self,templatename, templatedir):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+
+ Get the dict of environment variables to set
+ """
+ pass
ACK
--
Cedric
11:29 a.m.
New subject: [libvirt] [PATCH sandbox v5 20/20] Image: Add Volume Support
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Volumes let user to map host-paths into sandbox. Docker containers
need volumes for data persistence.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 24 +++++++++++++++++++++++-
libvirt-sandbox/image/sources/DockerSource.py | 13 +++++++++++++
libvirt-sandbox/image/sources/Source.py | 10 ++++++++++
3 files changed, 46 insertions(+), 1 deletion(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index c17b577..3f1ab0d 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -101,6 +101,7 @@ def create(args):
def run(args):
try:
+ global image_dir
if args.connect is not None:
check_connect(args.connect)
source = dynamic_source_loader(args.source)
@@ -142,6 +143,26 @@ def run(args):
else:
pass
+ allVolumes = source.get_volumes(args.template, args.template_dir)
+ volumeArgs = args.volume
+ if volumeArgs is not None:
+ allVolumes = allVolumes + volumeArgs
+ for volume in allVolumes:
+ volumeSplit = volume.split(":")
+ volumelen = len(volumeSplit)
+ if volumelen == 2:
+ hostPath = volumeSplit[0]
+ guestPath = volumeSplit[1]
+ elif volumelen == 1:
+ guestPath = volumeSplit[0]
+ hostPath = image_dir + guestPath
+ if not os.path.exists(hostPath):
+ os.makedirs(hostPath)
+ else:
+ pass
+ params.append("--mount")
+ params.append("host-bind:%s=%s" %(guestPath,hostPath))
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
@@ -237,7 +258,8 @@ def gen_run_args(subparser):
help=_("Network params for running template"))
parser.add_argument("-e","--env",action="append",
help=_("Environment params for running template"))
-
+ parser.add_argument("--volume",action="append",
+ help=_("Volume params for running template"))
parser.set_defaults(func=run)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 4455198..2c358fe 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -28,6 +28,7 @@ import traceback
import os
import subprocess
import shutil
+import collections
class DockerConfParser():
@@ -44,6 +45,13 @@ class DockerConfParser():
return lst
else:
return []
+ def getVolumes(self):
+ volumes = self.json_data['config']['Volumes']
+ volumelist = []
+ if isinstance(volumes,collections.Iterable):
+ for key,value in volumes.iteritems():
+ volumelist.append(key)
+ return volumelist
class DockerSource(Source):
@@ -399,5 +407,10 @@ class DockerSource(Source):
configParser = DockerConfParser(configfile)
return configParser.getEnvs()
+ def get_volumes(self, templatename, templatedir):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ return configParser.getVolumes()
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 8a21f90..f1dd3e7 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -105,3 +105,13 @@ class Source():
Get the dict of environment variables to set
"""
pass
+
+ @abstractmethod
+ def get_volumes(self,templatename, templatedir):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+
+ Get the list of volumes associated with the template
+ """
+ pass
--
2.4.3
7:08 a.m.
New subject: [libvirt] [PATCH sandbox v5 20/20] Image: Add Volume Support
On Tue, 2015-09-08 at 17:29 +0100, Daniel P. Berrange wrote:
From: Eren Yagdiran <erenyagdiran(a)gmail.com>
Volumes let user to map host-paths into sandbox. Docker containers
need volumes for data persistence.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
libvirt-sandbox/image/cli.py | 24 +++++++++++++++++++++++-
libvirt-sandbox/image/sources/DockerSource.py | 13 +++++++++++++
libvirt-sandbox/image/sources/Source.py | 10 ++++++++++
3 files changed, 46 insertions(+), 1 deletion(-)
diff --git a/libvirt-sandbox/image/cli.py b/libvirt-sandbox/image/cli.py
index c17b577..3f1ab0d 100755
--- a/libvirt-sandbox/image/cli.py
+++ b/libvirt-sandbox/image/cli.py
@@ -101,6 +101,7 @@ def create(args):
def run(args):
try:
+ global image_dir
if args.connect is not None:
check_connect(args.connect)
source = dynamic_source_loader(args.source)
@@ -142,6 +143,26 @@ def run(args):
else:
pass
+ allVolumes = source.get_volumes(args.template, args.template_dir)
+ volumeArgs = args.volume
+ if volumeArgs is not None:
+ allVolumes = allVolumes + volumeArgs
+ for volume in allVolumes:
+ volumeSplit = volume.split(":")
+ volumelen = len(volumeSplit)
+ if volumelen == 2:
+ hostPath = volumeSplit[0]
+ guestPath = volumeSplit[1]
+ elif volumelen == 1:
+ guestPath = volumeSplit[0]
+ hostPath = image_dir + guestPath
+ if not os.path.exists(hostPath):
+ os.makedirs(hostPath)
+ else:
+ pass
+ params.append("--mount")
+ params.append("host-bind:%s=%s" %(guestPath,hostPath))
+
cmd = cmd + params + ['--'] + commandToRun
subprocess.call(cmd)
os.unlink(diskfile)
On top of the questions mentioned in the other thread regarding volumes,
we also need to decide what to do for the data volumes cleanup. We may
want to keep them, but we surely also want to have a way to prune them.
--
Cedric
@@ -237,7 +258,8 @@ def gen_run_args(subparser):
help=_("Network params for running template"))
parser.add_argument("-e","--env",action="append",
help=_("Environment params for running template"))
-
+ parser.add_argument("--volume",action="append",
+ help=_("Volume params for running template"))
parser.set_defaults(func=run)
def main():
diff --git a/libvirt-sandbox/image/sources/DockerSource.py
b/libvirt-sandbox/image/sources/DockerSource.py
index 4455198..2c358fe 100644
--- a/libvirt-sandbox/image/sources/DockerSource.py
+++ b/libvirt-sandbox/image/sources/DockerSource.py
@@ -28,6 +28,7 @@ import traceback
import os
import subprocess
import shutil
+import collections
class DockerConfParser():
@@ -44,6 +45,13 @@ class DockerConfParser():
return lst
else:
return []
+ def getVolumes(self):
+ volumes = self.json_data['config']['Volumes']
+ volumelist = []
+ if isinstance(volumes,collections.Iterable):
+ for key,value in volumes.iteritems():
+ volumelist.append(key)
+ return volumelist
class DockerSource(Source):
@@ -399,5 +407,10 @@ class DockerSource(Source):
configParser = DockerConfParser(configfile)
return configParser.getEnvs()
+ def get_volumes(self, templatename, templatedir):
+ configfile, diskfile = self._get_template_data(templatename, templatedir)
+ configParser = DockerConfParser(configfile)
+ return configParser.getVolumes()
+
def debug(msg):
sys.stderr.write(msg)
diff --git a/libvirt-sandbox/image/sources/Source.py
b/libvirt-sandbox/image/sources/Source.py
index 8a21f90..f1dd3e7 100644
--- a/libvirt-sandbox/image/sources/Source.py
+++ b/libvirt-sandbox/image/sources/Source.py
@@ -105,3 +105,13 @@ class Source():
Get the dict of environment variables to set
"""
pass
+
+ @abstractmethod
+ def get_volumes(self,templatename, templatedir):
+ """
+ :param templatename: name of the template image to download
+ :param templatedir: local directory path in which to find template
+
+ Get the list of volumes associated with the template
+ """
+ pass
4:05 a.m.
On Tue, Sep 08, 2015 at 12:55:04PM -0400, Cole Robinson wrote:
On 09/08/2015 12:29 PM, Daniel P. Berrange wrote:
>
> Daniel P Berrange (1):
> Add virt-sandbox-image
>
> Daniel P. Berrange (1):
> Rename 'name' to 'template' to disambiguate
Two different .gitconfig ?
The first patch was created by Eren on my behalf, hence the slight
difference in author. I'll adjust the author name when I push.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:45 a.m.
I have now pushed all these patches except the volume one and
the man page one. Eren, thank you very much for all the time
you invested in working on this feature. It is great to finally
have this ability in libvirt-sandbox !
Regards,
Daniel
On Tue, Sep 08, 2015 at 05:29:31PM +0100, Daniel P. Berrange wrote:
This is an update of Eren's v4 patch series to provide
Docker image support. In this v5 I have incorporated
fixes for all of the feedback I gave Eren against the
v4, so I think this series is ready for merging now.
One thing I still want to look at separately is how
applicable this design is to other container formats,
in particular the 'appc' specification
https://github.com/appc/spec/blob/master/spec/discovery.md
Before we make a release containing the docker impl, I
want to be confident we've not done anything silly
which will cause us compat problems if we want to
extend to cover appc later.
Daniel P Berrange (1):
Add virt-sandbox-image
Daniel P. Berrange (1):
Rename 'name' to 'template' to disambiguate
Eren Yagdiran (18):
Fix docker authentication handling
Image: Add Hooking Mechanism
Image: virt-sandbox-image default dir constants
Image: Add download function
Image: Refactor create function
Image: Add delete function
Image: Add get_command function to Source
Image: Add run args
Image: Add check_connect function
Image: Add get_disk function to Source
Image: Add run function
Image: Add network support
Image: man file for virt-sandbox-image
Add config for environment variables
Add environment parameter to virt-sandbox
init-common: Exporting custom environment variables
Image: Add custom environment support
Image: Add Volume Support
.gitignore | 1 +
bin/Makefile.am | 8 +-
bin/virt-sandbox-image | 8 +
bin/virt-sandbox-image.pod | 169 +++++++++++
bin/virt-sandbox.c | 14 +
configure.ac | 3 +
libvirt-sandbox.spec.in | 3 +
libvirt-sandbox/Makefile.am | 2 +-
libvirt-sandbox/image/Makefile.am | 10 +
libvirt-sandbox/image/__init__.py | 0
libvirt-sandbox/image/cli.py | 293 ++++++++++++++++++
libvirt-sandbox/image/sources/DockerSource.py | 416 ++++++++++++++++++++++++++
libvirt-sandbox/image/sources/Makefile.am | 9 +
libvirt-sandbox/image/sources/Source.py | 117 ++++++++
libvirt-sandbox/image/sources/__init__.py | 0
libvirt-sandbox/libvirt-sandbox-config.c | 171 ++++++++++-
libvirt-sandbox/libvirt-sandbox-config.h | 13 +
libvirt-sandbox/libvirt-sandbox-init-common.c | 18 ++
libvirt-sandbox/libvirt-sandbox.sym | 9 +
libvirt-sandbox/tests/test-config.c | 10 +
po/POTFILES.in | 1 +
21 files changed, 1272 insertions(+), 3 deletions(-)
create mode 100644 bin/virt-sandbox-image
create mode 100644 bin/virt-sandbox-image.pod
create mode 100644 libvirt-sandbox/image/Makefile.am
create mode 100644 libvirt-sandbox/image/__init__.py
create mode 100755 libvirt-sandbox/image/cli.py
create mode 100644 libvirt-sandbox/image/sources/DockerSource.py
create mode 100644 libvirt-sandbox/image/sources/Makefile.am
create mode 100644 libvirt-sandbox/image/sources/Source.py
create mode 100644 libvirt-sandbox/image/sources/__init__.py
--
2.4.3
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3397
days inactive
3399
days old
48 comments
3 participants
participants (3)
-
Cedric Bosdonnat -
Cole Robinson -
Daniel P. Berrange