On 04/07/2009 11:16 AM, Cole Robinson wrote: Thanks, I hadn't considered that end of testing. It definitely sounds useful. I guess I need to make something more general purpose. A netcf test mode might help, but it may not be useful enough - for example it wouldn't work if the test client was one that didn't support netcf - when connecting to a test driver, it's always on the local machine, correct?

On Tue, 2009-04-07 at 16:50 +0100, Daniel P. Berrange wrote: For the time being though, netcf not being available also means that the virInterface calls won't do anything with any other driver, since there is no other driver impl. It makes more sense to me to use netcf during testing until there is a different implementation of the virInterface API - at that point that impl and the test driver will also be able to share a lot of code, so that we don't have lots of test-only code. David

On Tue, Apr 07, 2009 at 10:39:22AM -0400, Laine Stump wrote: No, this isn't quite right. The test driver is a driver in its own right - it doesn't call into any of the 'real' network driver. Both the 'test' and 'real' network drivers share the same XML parsing & formatting routines and internal object structures from network_conf.h/.c It shouldn't really be any different - the same general pattern should work - interface_conf.{c,h} - generic XML parsing and formatting APIs - netcf_driver.c - the reusable netcf interface driver impl - test.c - the complete test driver impl - vmware_driver.c - the complete vmware driver impl So, the netcf_driver.c would provide impl of the interface APIs for any driver which doesn't provide an impl itself. eg QEMU, Xen. Other drivers like 'test' implement the full set of APIs directly themselves. I also expect any VMWare, Hyper-V, Xen-API driver to implement the interface APIs directly. All drivers would use the generic XML parsing/formatting APIs to construct an in-memory representation of the objects being managed. Even though its doing a passthrough to netcf library, the netcf driver would still need to parse the XML in accordance with libvirt defined schema. It just happens that netcf uses the same XML format, so it can then convert it back into XML and pass it on through to netcf. The test driver wouldn't call anything in netcf. It is intended to be an entirely self-contained in-memory impl of the APIs with all the semantics required by the public APIs. This is exactly why netcf would not be called from the test driver :-) The test driver is designed to provide a way to do various unit test tasks - libvirt unit testing of the internal driver API framework - libvirt unit testing of the remote driver - libvirt unit testing of the libvirtd daemon - libvirt unit testing of the virsh program - Unit testing of end-user applications (eg virt-manager) Ultimately the test driver exposes as much functionality of the APIs as is possible, so that applications using it can exercise as much of their functionality as possible. So it'd certainly need to support multiple interfaces (with all the various configs - briding, bonding, vlans, etc) Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

On Tue, 2009-04-07 at 16:35 +0100, Daniel P. Berrange wrote: While that is still the case (libvirt interafce XML == netcf XML), it makes more sense to exercise as much of netcf as possible in the tests. The 'root' argument to ncf_init makes that possible, and if there are other things needed, we should add it to netcf, and exercise as many as the same code paths as non-test uses will as possible. Given that the libvirt API is just pass-through for now, that would test much less of the 'live' code than we can and should exercise. David

Daniel P. Berrange wrote: I suppose it depends on the definition of 'live' here, since currently with the test driver we do go through the same XML parsing routines that the 'live' drivers use (that's how I interpreted David's intentions anyways). Obviously we don't want to be mucking with actual network interfaces, but if we can use netcf to parse XML, that saves code duplication, and tests relevant 'live' code. Just for my own clarification, is the interface driver _actually_ going to duplicate the XML parsing/building of netcf? I assumed we were just going to do a wholesale passthrough to netcf in that case. That sounds like the best thing to do to just get this up and running. If we need to in the future, we could always just fork off our own subset of the netcf format and implement our own parser. But it doesn't sound like it gains us much doing it up front. If we do rely on netcf for xml parsing, I think it would also be a reasonable requirement that netcf be installed to use the test driver interface commands. - Cole

On 04/09/2009 11:26 AM, Daniel P. Berrange wrote: With all this new information (for me anyway ;-), I think we can all agree that we will, in the end, need to be able to parse the interface XML in libvirt, and should have a test driver that is fully functional with no external dependencies (see Cole and Dan's previous mails). On the other hand, there's a very short time until the release that we want minimal netcf functionality in, and an impaired newbie is doing the work. So, in the interest of getting the most useful stuff into the code in the shortest time, here is a proposed schedule of what to do when. All of these things will be done before the entire task is considered complete, but some will be finished 2 releases from now rather than for the next release. Each of the steps will be committable in their own right, thus allowing maximum use before release: 1) implement a live interface driver that is simply a passthrough to netcf, no parsing/formatting of XML, just a conduit. This will give us something that can demonstrate the new functionality and be useful. 2) implement a test driver that uses netcf in "alternate root" mode (with a flag during init to turn off the iptables tweaking and disable if_up and if_down) (this may or may not be ready for the next release). 3) implement XML parsing/formatting functions for interfaces in libvirt. (this is a functional no-op, so wouldn't necessarily need to be committed until step (4), although at this point someone could use these functions to implement a VirtualBox interface driver, for example). 4) modify the test driver to keep its own list of interfaces, and call the parsing/formatting code in (3) rather than netcf. 5) modify the live driver from (1) to do the parse/format as XML is going to/from netcf, and add-in things that netcf doesn't do (eg, interface state, DHCP addresses, etc). (we might want to switch the order of 4 and 5) This is not nearly as nice as presenting the entire thing all at once in one giant comprehensive set of patches, but makes it possible to get a working (albeit not ideal) "configure physical host interfaces" feature into the next libvirt release, and doesn't produce too much throwaway code. Does this sound acceptable?

On 04/10/2009 06:47 AM, Daniel P. Berrange wrote: Yes. I completely agree. I'm just saying that can't be done by the end of next week (isn't that when the freeze for the next release will be?), so how about we allow for something incremental that will get the motor running, and in the following few weeks update it to make it "right". (If not, then I'll do everything to completion before submitting any patchsets, and it will necessarily be a later date before there's usable functionality in the code) And this is exactly why I think it would be acceptable, for a very short time, to omit the parse/reformat - it is an identity transform. True, in the long run it may not be, but long before the two schemas diverge, we will be finished doing it the right way. But it would be trivial to write - it would literally be the real driver (already written at that point), with a different init call. And of course, all the plumbing will stay, so it won't be 100% throwaway Well, in your plan it *is* still necessary, it's just that it's an addon to my step 1 ;-)

On Tue, Apr 14, 2009 at 08:42:06AM +0100, Daniel P. Berrange wrote: David Lutterkort also doesn't feel netcf is ready to just drop into RHEL 5.4 since it has had near 0 testing... and the network management API isn't much good without netcf. So we need to keep pushing ahead to get this ready for rhel 6 but I don't see a whole lot of point in trying to rush it into 5.4. --Hugh

On Fri, 2009-04-10 at 11:47 +0100, Daniel P. Berrange wrote: Here though you're going excactly in the opposite direction: you will have two different validators that are in danger of diverging, when you could have just one. You don't even need to reformat - just pass the original string on. David