- Devel - libvirt List Archives

[libvirt] How to prevent libvirt from adding iptables rules?
by Mariano Absatz 15 Apr '09

15 Apr '09

Hi, I'm new to libvirt but not a complete neophite. I'm using libvirt and kvm in ubuntu with "vmbuilder". I'm creating a couple of VMs inside a host that is directly connected to internet with a public routeable address. Since I only have one public address, I won't use bridging. I'm using shorewall (www.shorewall.net) to configure my iptables rules. I intend to use DNAT to route specific ports in the host to one or other VM. With standard masquerading, I give the VMs access to the outside world. At first I used the 'default' network (with a different rfc1918 network)... everything was kinda working until I rebooted the host... at that point I lost connectivity between the outside world and the VMs. From inside the host I had no trouble connecting to the VMs. If I restarted shorewall (which actually cleans all iptables rules and regenerate them according to its configuration) everything works fine. After sending a report and some debugging in the shorewall mailing list, it was clear that libvirt was adding rules to iptables. After reading a bit (http://libvirt.org/formatnetwork.html#examplesPrivate) I created a new network called "isolated". I stopped default (and disabled its autostart), and defined and started isolated. This is the content of isolated.xml: <network> <name>isolated</name> <uuid>51cffbcc-88f5-4edc-a81c-1765c1045691</uuid> <bridge name='virbr%d' stp='on' forwardDelay='0' /> <ip address='10.3.14.1' netmask='255.255.255.0'> <dhcp> <range start='10.3.14.128' end='10.3.14.254' /> </dhcp> </ip> </network> I modified my VMs to use isolated rather than default, but rules keep being added to iptables when libvirt-bin is started. Is there a way to convince libvirt not to add these rules? Feel free to ask for any data that I didn't send here. TIA. -- Mariano Absatz - "El Baby" el.baby(a)gmail.com www.clueless.com.ar -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- An expert is a person who has made all the mistakes that can be made in a very narrow field. Niels Bohr Danish physicist (1885 - 1962) -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- * TagZilla 0.066 * http://tagzilla.mozdev.org

5 11

[libvirt] Libvirt PHP binding
by Radek Hladik 15 Apr '09

15 Apr '09

Hi, I am trying to develop PHP binding for Libvirt API. As discussed previously on the list the best way is to create a ZEND extension. There is a great tutorial on the net ( http://devzone.zend.com/tag/Extension ) and PHP provides a lot of macros and functions to make extension writing easier. I've implemented functions for connecting, listing domains, getting info about domains and node, dumping domain XML, some basic domain life-cycle actions and very basic error reporting. Before moving to other functions I need to solve a few issues :-) The biggest issue is that PHP used from webserver can (and usually does) run multithread. One thread can process more than one request and the threads can run in parallel. PHP/ZEND provides "tools" to solve this. It provides "request-global" variables, resources (containers for objects like file handles or connection pointers) and much more. In single threaded environment (like calling from command line) all these fallback to usual behavior. In addition ZEND has its own memory manager to be able to better manage the memory on per request basis (functions are called emalloc(), efree(),...). One can still use malloc() and free() but this is bypassing the ZEND memory manager. Free()ing emalloced memory or vice versa leads to crash. So one thing is that every memory allocated by libvirt (using malloc()) I need to copy to memory allocated using emalloc(). Also every memory, that would be free()ed by libvirt I need to malloc() and copy from emmaloc()ed memory. The first one is done by simple macro: #define RECREATE_STRING_WITH_E(str_out, str_in) \ str_out = estrndup(str_in, strlen(str_in)); \ free(str_in); The second case I encountered only on authentication callback and I use simple strndup() to copy it for libvirt. When running from command line everything seems to work fine. This is of course singlethreaded and potential resource leaks need not to cause a problem. However when running from webserver, the result is much more interesting. When connecting to qemu:///system (via local socket) using <? libvirt_connect($uri,true); ?> I can crash the libvirt daemon after cca 10 page reloads - sometimes with message Apr 13 15:32:44 kvmtest kernel: libvirtd[8263]: segfault at 4 ip 00000039d7223fc0 sp 00007fa6fbc29a88 error 6 in libdbus-1.so.3.4.0[39d7200000+3c000] in system log, sometimes without any message. (When running the same script from command line it worked for 1000 executions without noticing a problem). When connecting to qemu+tcp:///system using credentials (explained later): <? libvirt_connect($uri,true,Array(VIR_CRED_AUTHNAME=>"fred",VIR_CRED_PASSPHRASE=>"fred")); ?> It works but httpd processes open a lot pipes and after a few hours with page refreshing every 10 sec I even ran into error: [Mon Apr 13 02:40:26 2009] [error] [client 10.38.25.152] PHP Warning: libvirt_connect() unable to make pipe: Too many open files Next issue I am not sure about is callbacks. I need them mainly for authentication. As PHP is intended to run the whole script non-interactively at most times, I've created this solution. When calling the libvirt_connect() PHP function you can provide list of credentials in form of an array Array(VIR_CRED_AUTHNAME=>"fred",VIR_CRED_PASSPHRASE=>"fred"). This PHP array is parsed to C array and this prepopulated array is passed to my authentication callback function. That function receives the requested credential, looks it up in the provided array and returns it. I think this suits more the PHP nature but in future I may provide real callback solution. Little trouble is that libvirt free()s the values returned by callback so I need to copy them to malloc()ed memory. I am not sure about multithread safety of callback functions but I think that if the function only obtains the parameters via cbdata and operates only with them then it should be safe. However sometimes I get error: [Mon Apr 13 14:19:48 2009] [error] [client 10.38.25.152] PHP Warning: libvirt_connect(): Failed to collect auth credentials and I need to restart http daemon for a few times to make it work again. The last trouble is with error callback, where I am not sure whether it is thread safe. I need to call PHP function for reporting error, storing the error in "global" variable for next use and the PHP may even terminate the whole processing of the request... And if you are still reading then I can point you to http://phplibvirt.cybersales.cz/ where you can download the source code and browse the documentation - you can find there list of implemented functions and brief instructions how to install the extension. But be warned it really can crash your libvirt and maybe apache! For completeness: I use Fedora 10 with some rawhide updates: httpd-2.2.11-6.x86_64 php-5.2.9-1.fc11.x86_64 php-devel-5.2.9-1.fc11.x86_64 libvirt-0.6.0-2.fc11.x86_64 qemu-0.9.1-12.fc11.x86_64 libvirt-devel-0.6.0-2.fc11.x86_64 php-cli-5.2.9-1.fc11.x86_64 kvm-83-5.fc10.x86_64 php-common-5.2.9-1.fc11.x86_64 Radek

3 8

[libvirt] [Patch] Fix vcpupin to inactive domains on Xend3.0.3.
by Takahashi Tomohiro 15 Apr '09

15 Apr '09

Hi, I made the patch that corrected the following problem. It is that when I execute the "virsh vcpupin" command to inactive domains on Xend3.0.3, Libvirt outputs the following Segmentation fault. # virsh vcpupin guest_dom 0 0,1,2 Segmentation fault Signed-off-by: Tomohiro Takahashi <takatom(a)jp.fujitsu.com> Thanks, Tomohiro Takahashi --- xm_internal.c.org 2009-04-07 21:50:17.000000000 +0900 +++ xm_internal.c 2009-04-13 20:53:22.000000000 +0900 @@ -1700,7 +1700,8 @@ int xenXMDomainPinVcpu(virDomainPtr doma ret = 0; cleanup: - VIR_FREE(mapstr); + if( *mapstr != '\0' ) + VIR_FREE(mapstr); VIR_FREE(cpuset); xenUnifiedUnlock(priv); return (ret);

2 3

[libvirt] [PATCH] lxc: Add lxcGetHostname()
by Dan Smith 15 Apr '09

15 Apr '09

This patch adds the getHostname method to the lxc driver structure (using the qemu driver's generic code). Apparently virsh started using that method during console attachment recently. Without this implementation, the virsh refused to attach to lxc consoles. Signed-off-by: Dan Smith <danms(a)us.ibm.com> -- Dan Smith IBM Linux Technology Center email: danms(a)us.ibm.com Index: src/lxc_driver.c =================================================================== RCS file: /data/cvs/libvirt/src/lxc_driver.c,v retrieving revision 1.65 diff -u -r1.65 lxc_driver.c --- src/lxc_driver.c 31 Mar 2009 15:47:17 -0000 1.65 +++ src/lxc_driver.c 14 Apr 2009 18:22:45 -0000 @@ -1404,6 +1404,20 @@ return ret; } +static char *lxcGetHostname (virConnectPtr conn) +{ + char *result; + + result = virGetHostname(); + if (result == NULL) { + virReportSystemError (conn, errno, + "%s", _("failed to determine host name")); + return NULL; + } + /* Caller frees this string. */ + return result; +} + /* Function Tables */ static virDriver lxcDriver = { VIR_DRV_LXC, /* the number virDrvNo */ @@ -1413,7 +1427,7 @@ NULL, /* supports_feature */ NULL, /* type */ lxcVersion, /* version */ - NULL, /* getHostname */ + lxcGetHostname, /* getHostname */ NULL, /* getMaxVcpus */ NULL, /* nodeGetInfo */ NULL, /* getCapabilities */

3 2

[libvirt] vcpuinfo returns wrong CPU value for kvm
by Gerrit Slomma 15 Apr '09

15 Apr '09

vcpuinfo returns a wrong value for kvm with libvirt 0.6.2: virsh # nodeinfo CPU-Modell: x86_64 CPU(s): 2 CPU-Frequenz: 2333 MHz CPU-Socket(s): 1 Kern(e) pro Socket: 2 Thread(s) pro Kern: 1 NUMA-Zelle(n): 1 Speichergrösse: 3062956 kB virsh # dominfo rr019v3 Id: 2 Name: rr019v3 UUID: ff1ba599-5801-fca9-ab33-8962c2dfa46c OS Typ: hvm Status: laufend CPU(s): 2 CPU-Zeit: 46,5s Max Speicher: 1572864 kB Verwendeter Speicher: 1572864 kB Automatischer Start: deaktiviert As default a kvm-vm has a affinity on all cpus provided by the node virsh # vcpuinfo rr019v3 VCPU: 0 CPU: 0 Status: laufend CPU-Affinität: yy VCPU: 1 CPU: 0 Status: laufend CPU-Affinität: yy When i set the affinity of the vCPUs to dedicated CPUs of the node vcpuinfo returns CPU 0 as the assigned CPU. virsh # vcpupin rr019v3 0 0 virsh # vcpupin rr019v3 1 1 virsh # vcpuinfo rr019v3 VCPU: 0 CPU: 0 Status: laufend CPU-Affinität: y- VCPU: 1 CPU: 0 Status: laufend CPU-Affinität: -y Furthermore even when pinning all - in my case both - vCPUs of the domain to the second CPU of the node virsh reports the first cpu as the assigned cpu. virsh # vcpupin rr019v3 0 1 virsh # vcpuinfo rr019v3 VCPU: 0 CPU: 0 Status: laufend CPU-Affinität: -y VCPU: 1 CPU: 0 Status: laufend CPU-Affinität: -y Watching at the code in virsh.c i am a little bit on my wits end where virVcpuInfoPtr is defined as i am no c-programmer - only some java on my side. On my xen-machine with libvirt 0.3.3 the CPU is shown correctly though: virsh vcpuinfo rr010v2 VCPU: 0 CPU: 0 Status: blockiert CPU-Zeit: 550,4s CPU-Affinität: y--- VCPU: 1 CPU: 1 Status: blockiert CPU-Zeit: 27,9s CPU-Affinität: -y-- VCPU: 2 CPU: 2 Status: blockiert CPU-Zeit: 31,2s CPU-Affinität: --y- VCPU: 3 CPU: 3 Status: blockiert CPU-Zeit: 49,1s CPU-Affinität: ---y Is this only working with the xen-driver?

2 1

[libvirt] [PATCH] lxc: stop rootless containers from messing with system mounts
by Serge E. Hallyn 15 Apr '09

15 Apr '09

If a container has no root, liblxc remounts /proc. If the system had marked / as MS_SHARED, then even though the container is in a new mounts namespace, the mount event is propagated back to the host mounts namespace, overwriting /proc. After that, for instance, ps will no longer show system processes. A Fedora 11 default install has / MS_SHARED. Make sure that root is not MS_SHARED before remounting /proc. I'm making it MS_SLAVE so that the container will receive mount events from the host, but not vice versa. Signed-off-by: Serge Hallyn <serue(a)us.ibm.com> --- src/lxc_container.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/src/lxc_container.c b/src/lxc_container.c index d3959f6..8addd23 100644 --- a/src/lxc_container.c +++ b/src/lxc_container.c @@ -273,7 +273,11 @@ static int lxcContainerChildMountSort(const void *a, const void *b) #endif #ifndef MS_PRIVATE -#define MS_PRIVATE 1<<18 +#define MS_PRIVATE (1<<18) +#endif + +#ifndef MS_SLAVE +#define MS_SLAVE (1<<19) #endif static int lxcContainerPivotRoot(virDomainFSDefPtr root) @@ -558,6 +562,11 @@ static int lxcContainerSetupExtraMounts(virDomainDefPtr vmDef) { int i; + if (mount("", "/", NULL, MS_SLAVE|MS_REC, NULL) < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to make / slave")); + return -1; + } for (i = 0 ; i < vmDef->nfss ; i++) { // XXX fix to support other mount types if (vmDef->fss[i]->type != VIR_DOMAIN_FS_TYPE_MOUNT) -- 1.6.2

2 2

[libvirt] libvirt arch detection on x86_64 host
by Gerry Reno 15 Apr '09

15 Apr '09

I have a 64-bit host that is running a 32-bit OS (Fedora 10). # cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 9850 Quad-Core Processor stepping : 3 cpu MHz : 2511.416 cache size : 512 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs bogomips : 5022.83 clflush size : 64 power management: ts ttp tm stc 100mhzsteps hwpstate processor : 1 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 9850 Quad-Core Processor stepping : 3 cpu MHz : 2511.416 cache size : 512 KB physical id : 0 siblings : 4 core id : 1 cpu cores : 4 apicid : 1 initial apicid : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs bogomips : 5023.06 clflush size : 64 power management: ts ttp tm stc 100mhzsteps hwpstate processor : 2 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 9850 Quad-Core Processor stepping : 3 cpu MHz : 2511.416 cache size : 512 KB physical id : 0 siblings : 4 core id : 3 cpu cores : 4 apicid : 3 initial apicid : 3 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs bogomips : 5023.06 clflush size : 64 power management: ts ttp tm stc 100mhzsteps hwpstate processor : 3 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 9850 Quad-Core Processor stepping : 3 cpu MHz : 2511.416 cache size : 512 KB physical id : 0 siblings : 4 core id : 2 cpu cores : 4 apicid : 2 initial apicid : 2 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs bogomips : 5023.06 clflush size : 64 power management: ts ttp tm stc 100mhzsteps hwpstate # virsh nodeinfo CPU model: i686 CPU(s): 4 CPU frequency: 2511 MHz CPU socket(s): 1 Core(s) per socket: 4 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 4147340 kB # virsh capabilities <capabilities> <host> <cpu> <arch>i686</arch> </cpu> </host> <guest> <os_type>hvm</os_type> <arch name='i686'> <wordsize>32</wordsize> <emulator>/usr/bin/qemu</emulator> <machine>pc</machine> <machine>isapc</machine> <domain type='qemu'> </domain> <domain type='kvm'> <emulator>/usr/bin/qemu-kvm</emulator> </domain> </arch> <features> <pae/> <nonpae/> <acpi default='on' toggle='yes'/> <apic default='on' toggle='no'/> </features> </guest> <guest> <os_type>hvm</os_type> <arch name='x86_64'> <wordsize>64</wordsize> <emulator>/usr/bin/qemu-system-x86_64</emulator> <machine>pc</machine> <machine>isapc</machine> <domain type='qemu'> </domain> </arch> <features> <acpi default='on' toggle='yes'/> <apic default='on' toggle='no'/> </features> </guest> <guest> <os_type>hvm</os_type> <arch name='mips'> <wordsize>32</wordsize> <emulator>/usr/bin/qemu-system-mips</emulator> <machine>mips</machine> <domain type='qemu'> </domain> </arch> </guest> <guest> <os_type>hvm</os_type> <arch name='mipsel'> <wordsize>32</wordsize> <emulator>/usr/bin/qemu-system-mipsel</emulator> <machine>mips</machine> <domain type='qemu'> </domain> </arch> </guest> <guest> <os_type>hvm</os_type> <arch name='sparc'> <wordsize>32</wordsize> <emulator>/usr/bin/qemu-system-sparc</emulator> <machine>sun4m</machine> <domain type='qemu'> </domain> </arch> </guest> <guest> <os_type>hvm</os_type> <arch name='ppc'> <wordsize>32</wordsize> <emulator>/usr/bin/qemu-system-ppc</emulator> <machine>g3bw</machine> <machine>mac99</machine> <machine>prep</machine> <domain type='qemu'> </domain> </arch> </guest> </capabilities> Shouldn't the host arch have been detected and identified as x86_64? Regards, Gerry

3 3

[libvirt] test driver for interface config
by Laine Stump 15 Apr '09

15 Apr '09

I'm now trying to write the test driver for the interface config driver that I'm putting into libvirt. For comparison I'm using the network driver. My understanding so far is that it calls the functions in the real driver to construct network objects and manipulate them, then just doesn't call the code that makes the changes to the system config. My driver is a bit different, in that I don't need any object management - I just pass everything through to the netcf library, and return the results to the caller; libvirt is just being used as an RPC conduit with some minimal wrapping. What I'm wondering now is exactly what the test driver should be testing - should the test driver also call netcf and return the results, or should it be a minimal interface driver in its own right that, for example, comes up with no interfaces, and allows adding only one? The former (call netcf) is problematic in that many netcf functions make changes to the real live network config (which I don't think is a very nice thing to have in test code, but maybe I'm being over-cautious) and netcf doesn't have provisions for a test mode where changes made to the config aren't committed. In the latter case it seems that not a lot will be tested (just the client side API, and libvirt's RPC - all parameter checking and XML parsing will be done by netcf), although maybe that's all anyone is looking for. I would hesitate to make the driver any more complicated than just allowing a single interface, or do any parameter validation that's normally done by netcf, since then much of the testing would be of code that was only used in the test itself - not very productive. Any advice?

5 19

[libvirt] [PATCH 2/2] VirtualBox support
by Pritesh Kothari 14 Apr '09

14 Apr '09

Hi All, I have attached a patch which when applied on the HEAD as of today would allow virtualbox support in libvirt. It takes cares of all the stuff mentioned on the list earlier. Still if I have missed anything, please do tell me. The patch works very well with the VirtualBox OSE version and the 2.2 release. [PATCH 1/2] contains diff of files already in libvirt. [PATCH 2/2] contains new files needed for VirtualBox support. Regards, Pritesh

3 5

[libvirt] RFC: Import / export of Domain XML from/to native "formats"
by Daniel P. Berrange 14 Apr '09

14 Apr '09

Some of the frequently asked questions I see floating around various user forums / mailing lists wrt libvirt XML, relate the import or export of existing configurations from/to libvirt. eg - What is the libvirt XML matching the QEMU args 'qemu -foo -bar -wiz' - What are the QEMU arguments used by libvirt for XML config XYZ - What is the libvirt XML matching the Xen file XYZ in /etc/xen - What is the Xen config file corresponding to the libvirt XML XYZ When those questions arise, the best we do is point the user at wiki page or XML format docs, sometimes give them some examples, etc It occurred to me that we could do much better than this. We already have code in libvirt that knows how todo - libvirt XML -> QEMU args - libvirt XML -> Xen config file - Xen config file -> libvirt XML - Xen SEXPR -> libvirt XML - libvirt XML -> Xen SEXPR it is not possible to access this functionality directly though - it is used internally via things like the DumpXML / DefineXML commands. It would be helpful to be able to use these conversion functions directly in many cases, to be able to do conversions, without having to actually define a new VM. I did a proof of concept creating some tools - virt-xml-from-qemu-args - virt-xml-to-qemu-args - virt-xml-from-xen-config - virt-xml-to-xen-config - virt-xml-from-xen-sxpr - virt-xml-to-xen-sxpr that just linked directly to the internal conversion APIs. This is a bit of a hack though, and I think it might be preferable to have a formal public API for invoking the conversions, instead of restricting them to a set of command line tools. What I think we could do is: - char *virConnectDomainXMLExport(const char *nativeFormat, const char *xml) Export 'xml' doc from libvirt XML to a config in 'nativeFormat', returning the native config data - char *virConnectDomainXMLImport(const char *nativeFormat, const char *config) Import 'config' data in 'nativeFormat', to libvirt XML, returning the libvirt XML doc The 'nativeFormat' would be a hypervisor specific format. The QEMU driver would initially provide a format called 'qemu-argv'. The Xen driver would provide 2 formats 'xen-xm' and 'xen-sxpr'. Later the QEMU driver might also provide 'qemu-conf' as & when QEMU gets native config file format. Most of the hard code for these functions already exists. The only one missing is the QEMU ARGV -> libvirt XML conversion, but its not too hard to get a basic impl up & running. In virsh, we might expose virsh domxml-export [XMLFILE [CONFIGFILE]] virsh domxml-import [CONFIGFILE [XMLFILE]] Both these would read/write from/to STDIN/STDOUT, if their optional filename args were ommitted. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 1