[Libvirt] SSH with certificates
Hello everybody, We are trying to use libvirt with qemu over ssh and our goal is to have authentication done by certificates. Therefore I created a keypair on the client and send the public key to the server. Std. SSH connections work without an password prompt as expected (ssh <ip> -l user), but if I try "sudo virsh -c qemu+ssh://user@<ip>/system" it prompts for a password (Beside that password prompt it works as expected). Is there anything else I have to do beside registering the public key at the server? Thanks for your help and best regards, Christian
On Wed, May 06, 2009 at 03:33:47PM +0200, Christian Weyermann wrote:
Hello everybody,
We are trying to use libvirt with qemu over ssh and our goal is to have authentication done by certificates. Therefore I created a keypair on the client and send the public key to the server. Std. SSH connections work without an password prompt as expected (ssh <ip> -l user), but if I try "sudo virsh -c qemu+ssh://user@<ip>/system" it prompts for a password (Beside that password prompt it works as expected).
Is there anything else I have to do beside registering the public key at the server?
Do you really mean x509 certificates, or are you actually talking about RSA/DSA public keys ? I wasn't aware that any SSH did x509 certs for authentication. Why are you using 'sudo' for this ? It seems rather pointless to switch to root to run virsh, since it is connecting to a remote host and you can trivially do that as your normal user. Using 'sudo' will almost certainly stop ssh connecting to the 'ssh-agent' in your session. Also note, you can explicitly prevent all password prompts from ssh by add '?no_tty=1' to the end of the URI There should never be any need to use sudo for virsh in any reasonable modern Linux desktop. libvirt will use PolicyKit to authenticate securely as non-root for local connections, and remote connections should just be run as the normal user Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange schrieb:
On Wed, May 06, 2009 at 03:33:47PM +0200, Christian Weyermann wrote:
Hello everybody,
We are trying to use libvirt with qemu over ssh and our goal is to have authentication done by certificates. Therefore I created a keypair on the client and send the public key to the server. Std. SSH connections work without an password prompt as expected (ssh <ip> -l user), but if I try "sudo virsh -c qemu+ssh://user@<ip>/system" it prompts for a password (Beside that password prompt it works as expected).
Is there anything else I have to do beside registering the public key at the server?
Do you really mean x509 certificates, or are you actually talking about RSA/DSA public keys ? I wasn't aware that any SSH did x509 certs for authentication.
Why are you using 'sudo' for this ? It seems rather pointless to switch to root to run virsh, since it is connecting to a remote host and you can trivially do that as your normal user. Using 'sudo' will almost certainly stop ssh connecting to the 'ssh-agent' in your session.
Also note, you can explicitly prevent all password prompts from ssh by add '?no_tty=1' to the end of the URI
There should never be any need to use sudo for virsh in any reasonable modern Linux desktop. libvirt will use PolicyKit to authenticate securely as non-root for local connections, and remote connections should just be run as the normal user
Regards, Daniel
Thanks, for your help The problem has been the sudo. Soo stupid from me... If I do sudo ssh will look in the home directoy of root and, of course, there is no ssh key there. So he prompts me for authentication.
Hello everybody, I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so? Thanks and best regards, Christian
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
Daniel
Do you think, there might be a chance reaching this goal anyway, using VNC-Kerberos Auth via SASL, as the virt-viewer supports SASL? Thanks
On Mon, Jun 08, 2009 at 02:00:58PM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
Do you think, there might be a chance reaching this goal anyway, using VNC-Kerberos Auth via SASL, as the virt-viewer supports SASL?
No, afraid that won't help you. The key issue is that there is no way to specify authorization data on a per-VM basis. So if you authenticate successfully you have access. We need to add a way to check the authenticated username against an access control list of some form. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 02:00:58PM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
Do you think, there might be a chance reaching this goal anyway, using VNC-Kerberos Auth via SASL, as the virt-viewer supports SASL?
No, afraid that won't help you. The key issue is that there is no way to specify authorization data on a per-VM basis. So if you authenticate successfully you have access. We need to add a way to check the authenticated username against an access control list of some form. Do you have any idea when this issue will be tackled?
Best Regards, Chris
On Wed, Jun 10, 2009 at 01:36:42PM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 02:00:58PM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
Do you think, there might be a chance reaching this goal anyway, using VNC-Kerberos Auth via SASL, as the virt-viewer supports SASL?
No, afraid that won't help you. The key issue is that there is no way to specify authorization data on a per-VM basis. So if you authenticate successfully you have access. We need to add a way to check the authenticated username against an access control list of some form. Do you have any idea when this issue will be tackled?
It is on our wish list for Real Soon Now, but we haven't identified anyone to actually do the work yet... patches welcome :)... --Hugh
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it? -jim
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is,
Daniel P. Berrange schrieb: that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method. However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL. Kind Regards, Chris
On Thu, Jun 11, 2009 at 11:10:47AM +0200, Christian Weyermann wrote:
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is,
Daniel P. Berrange schrieb: that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method.
However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL.
GTK-VNC builds on Windows, and so does libvirt. So the intent was that we'd be able to have virt-viewer working on Windows using those two. Oh, when I say Windows, i mean Mingw32 Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 11:10:47AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME
Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is, that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method.
However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL.
GTK-VNC builds on Windows, and so does libvirt. So the intent was that we'd be able to have virt-viewer working on Windows using those two. Oh, when I say Windows, i mean Mingw32
Ok, so the other part of the post is correct? So what do you think about the effort for implementing this feature?
On Thu, Jun 11, 2009 at 11:45:43AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 11:10:47AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
> Hello everybody, > > I encountered the following problem. I want my users to only be able to > connect to their own virtual machines via VNC. Is there any way to do so? > > The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME
Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is, that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method.
However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL.
GTK-VNC builds on Windows, and so does libvirt. So the intent was that we'd be able to have virt-viewer working on Windows using those two. Oh, when I say Windows, i mean Mingw32
Ok, so the other part of the post is correct? So what do you think about the effort for implementing this feature?
Well I've had the demo program from GTK-VNC working sucessfully under Wine, and had virsh successfully working under Wine. So I see no reason why we virt-viewer should be troublesome to get working. I hope it'll just be a lot of silly small bugfixes/portability fixes, rather than any large fundamental problem. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Thu, Jun 11, 2009 at 11:45:43AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 11:10:47AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
> On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote: > > > >> Hello everybody, >> >> I encountered the following problem. I want my users to only be able to >> connect to their own virtual machines via VNC. Is there any way to do so? >> >> >> > The VNC authentication setup is currently being done per-host, so there > is no way to define ACLs per-(user,vm) tuple as you describe. > > > What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME
Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is, that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method.
However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL.
GTK-VNC builds on Windows, and so does libvirt. So the intent was that we'd be able to have virt-viewer working on Windows using those two. Oh, when I say Windows, i mean Mingw32
Ok, so the other part of the post is correct? So what do you think about the effort for implementing this feature?
Well I've had the demo program from GTK-VNC working sucessfully under Wine, and had virsh successfully working under Wine. So I see no reason why we virt-viewer should be troublesome to get working. I hope it'll just be a lot of silly small bugfixes/portability fixes, rather than any large fundamental problem. Sorry I didn`t express myself clearly. I meant the VNC per-VM auth, but
Daniel P. Berrange schrieb: that is useful information, too ;-)
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 11:45:43AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 11:10:47AM +0200, Christian Weyermann wrote:
Daniel P. Berrange schrieb:
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
> On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote: > > > >> Hello everybody, >> >> I encountered the following problem. I want my users to only be able to >> connect to their own virtual machines via VNC. Is there any way to do so? >> >> >> > The VNC authentication setup is currently being done per-host, so there > is no way to define ACLs per-(user,vm) tuple as you describe. > > > What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. If you want to have plain passwords, then its better to use the new SASL authentication method, with its Digest-MD5 plugin. That is still not top-grade security, but it is better then VNC password and allows configuration of arbitrary Username+pasword pairs.. At which point we just need ACLs against the usernames. SASL also provide Kerberos auth, where we can do an ACL against the Kerberos principle name. And VeNCrypt provides TLS+x509 certificates which you can either layer SASL over again, or require client x509 certs and do an ACL against the client CNAME
Ok, so let me sumarize: It is possible to define username+password pairs via SASL. SASL can also sync with Kerberos. So the only problem left is, that there is no way to assign a specific username to a VM. So, what we need is a plugin, where we have an username and a virtual machine as input and we need to refuse the connection, if this pair is not valid. The VNC Server is part of libvirt, so the perfect method to add this functionallity would be the VNC Servers authenticate or start method.
However, a Windows user is still not able to connect as there is no windows vnc client capable of doing SASL.
GTK-VNC builds on Windows, and so does libvirt. So the intent was that we'd be able to have virt-viewer working on Windows using those two. Oh, when I say Windows, i mean Mingw32
Ok, so the other part of the post is correct? So what do you think about the effort for implementing this feature?
Well I've had the demo program from GTK-VNC working sucessfully under Wine, and had virsh successfully working under Wine. So I see no reason why we virt-viewer should be troublesome to get working. I hope it'll just be a lot of silly small bugfixes/portability fixes, rather than any large fundamental problem.
Ok, so I have just one question left. What do you think about the effort of implementing the "per VM" authentication feature? This would include VNC and "normal" libvirt access?
On Thu, Jun 11, 2009 at 04:05:39AM -0400, Jim Paris wrote:
Daniel P. Berrange wrote:
On Mon, Jun 08, 2009 at 11:35:00AM +0200, Christian Weyermann wrote:
Hello everybody,
I encountered the following problem. I want my users to only be able to connect to their own virtual machines via VNC. Is there any way to do so?
The VNC authentication setup is currently being done per-host, so there is no way to define ACLs per-(user,vm) tuple as you describe.
What about the VNC password? That's per-VM, isn't it?
With KVM/QEMU, you can set a VNC password per VM. But I think it is either/or though; you can use VNC with passwords (no encryption), or use VNC with TLS, which is encrypted, but anyone with a valid certificate can connect (to any VM). Someone correct me if I'm wrong on that. -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
Daniel P. Berrange wrote:
What about the VNC password? That's per-VM, isn't it?
That is true by I don't really consider VNC password to be useful. It is utterly insecure. ...
Garry Dolley wrote:
With KVM/QEMU, you can set a VNC password per VM.
But I think it is either/or though; you can use VNC with passwords (no encryption), or use VNC with TLS, which is encrypted, but anyone with a valid certificate can connect (to any VM).
Ok, makes sense. In my own case, where VNC only listens on 127.0.0.1 and all remote connections are tunneled through SSH anyway, I think that plaintext passwords and the lack of VNC encryption would be OK. Thanks for the clearifications. -jim
participants (5)
-
Christian Weyermann -
Daniel P. Berrange -
Garry Dolley -
Hugh O. Brock -
Jim Paris