Well, if the domain is not running then I get this exception with libvirt 0.6.3 and libvirt-java 0.4.2: org.libvirt.LibvirtException: invalid argument in cannot pin vcpus on an inactive domain at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.Connect.processError(Unknown Source) at org.libvirt.Domain.processError(Unknown Source) at org.libvirt.Domain.getVcpusInfo(Unknown Source) at TestMultiThreadingLibvirtWithError.run(TestMultiThreadingLibvirtWithError.java:20) at java.lang.Thread.run(Thread.java:636) The error message is misleading and current git version gives this exception when the domain is not running: org.libvirt.LibvirtException: Requested operation is not valid: cannot list vcpu pinning for an inactive domain at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.Connect.processError(Unknown Source) at org.libvirt.Domain.processError(Unknown Source) at org.libvirt.Domain.getVcpusInfo(Unknown Source) at TestMultiThreadingLibvirtWithError.run(TestMultiThreadingLibvirtWithError.java:20) at java.lang.Thread.run(Thread.java:636) And that's perfectly valid, as the complete vCPU information is only available when the domain is running, therefore the exception is expected when you call getVcpusInfo on a non-running domain. Also this has nothing to do with thread-safety. If this isn't the exception you're referring to, then please show a stacktrace of the exception you're seeing. Matthias 2010/9/3 Ravi Pawar <ravifc@gmail.com>:

thank you for the reply and time you spend. your guest domain should not be running and then run my program without any change.you will see an exception where you dont expect it to occur. libvirt : 0.6.3 and libvirt-java: 0.4.2 i am using. Please let me know if i miss to provide you any other details.

On Fri, Sep 3, 2010 at 3:29 AM, Matthias Bolte <matthias.bolte@googlemail.com> wrote:

...

2010/9/2 Ravi Pawar <ravifc@gmail.com>:

...

please check the java file attached they throws the exception in case of passing. please let me know if i am missing something.

Can you explain in more detail what one should do to reproduce the problem you seem to see with the attached code? Also you didn't mention which versions of libvirt and libvirt-java you're using.

When I run you example code (with a guest name that exists in my setup and libvirt and libvirt-java form current git head) I don't see any exceptions. Even when I add e.printStackTrace() to the catch block of TestMultiThreadingLibvirtWithError I don't see exceptions.

$ java -classpath ./target/libvirt-0.4.6.jar:./ravi:/usr/share/java/jna.jar Test start class 0 start class 1 start class 2 start class 3 start class 4 start class 5 start class 6 start class 7 start class 8 start class 9

So this works for me.

Matthias

-- Thanks & Regards

Ravi Pawar

Ah, okay, now I understand what you want to say. You have two threads A and B. When A runs on its own then it's just fine, but when you add B (that triggers an exception on purpose and ignores it) then A picks it up and reports it. This is the point where thread safety comes to mind. libvirt stores errors in thread-local-storage. It uses pthread_key_create/pthread_{get,set}specific for this (or TlsAlloc/Tls{Get,Set}Value on Windows). I think what's happening here is that all your threads in Java share the same thread-local-storage. Therefore, thread A can pickup the error triggered by thread B, that should not happen. I'm not sure how to fix that. Matthias 2010/9/3 Ravi Pawar <ravifc@gmail.com>:

here is the stacktrace:

org.libvirt.LibvirtException: invalid argument in cannot pin vcpus on an inactive domain         at org.libvirt.ErrorHandler.processError(Unknown Source)         at org.libvirt.Connect.processError(Unknown Source)         at org.libvirt.Connect.storagePoolLookupByName(Unknown Source)         at TestMultiThreadingLibvirt.run(TestMultiThreadingLibvirt.java:25)         at java.lang.Thread.run(Thread.java:619) libvir: QEMU error : invalid argument in cannot pin vcpus on an inactive domain

from your stacktrace i could find out that you have printed the stacktrace in TestMultiThreadingLibvirtWithError  class. please do check the code i have pasted below for the second thread.

public class TestMultiThreadingLibvirtWithError implements Runnable {         Connect m_conn = null;         public TestMultiThreadingLibvirtWithError(Connect conn)         {                 m_conn = conn;

        }         @Override         public void run() {

                        try {                                 Domain d = m_conn.domainLookupByName("afzaltestmachine");                                 VcpuInfo[] vcpuInfo = d.getVcpusInfo();

                        }                         catch (Exception e)                         {                            //we are explicitly eating up the exception here and we know Exception is going to occur here.

                        }

        }

}

please let me know if my you have more queries to my problem.

On Fri, Sep 3, 2010 at 2:37 PM, Matthias Bolte <matthias.bolte@googlemail.com> wrote:

...

Well, if the domain is not running then I get this exception with libvirt 0.6.3 and libvirt-java 0.4.2:

org.libvirt.LibvirtException: invalid argument in cannot pin vcpus on an inactive domain        at org.libvirt.ErrorHandler.processError(Unknown Source)        at org.libvirt.Connect.processError(Unknown Source)        at org.libvirt.Domain.processError(Unknown Source)        at org.libvirt.Domain.getVcpusInfo(Unknown Source)        at TestMultiThreadingLibvirtWithError.run(TestMultiThreadingLibvirtWithError.java:20)        at java.lang.Thread.run(Thread.java:636)

The error message is misleading and current git version gives this exception when the domain is not running:

org.libvirt.LibvirtException: Requested operation is not valid: cannot list vcpu pinning for an inactive domain        at org.libvirt.ErrorHandler.processError(Unknown Source)        at org.libvirt.Connect.processError(Unknown Source)        at org.libvirt.Domain.processError(Unknown Source)        at org.libvirt.Domain.getVcpusInfo(Unknown Source)        at TestMultiThreadingLibvirtWithError.run(TestMultiThreadingLibvirtWithError.java:20)        at java.lang.Thread.run(Thread.java:636)

And that's perfectly valid, as the complete vCPU information is only available when the domain is running, therefore the exception is expected when you call getVcpusInfo on a non-running domain. Also this has nothing to do with thread-safety.

If this isn't the exception you're referring to, then please show a stacktrace of the exception you're seeing.

Matthias

2010/9/3 Ravi Pawar <ravifc@gmail.com>:

...

thank you for the reply and time you spend. your guest domain should not be running and then run my program without any change.you will see an exception where you dont expect it to occur. libvirt : 0.6.3 and libvirt-java: 0.4.2 i am using. Please let me know if i miss to provide you any other details.

On Fri, Sep 3, 2010 at 3:29 AM, Matthias Bolte <matthias.bolte@googlemail.com> wrote:

...

2010/9/2 Ravi Pawar <ravifc@gmail.com>:

...

please check the java file attached they throws the exception in case of passing. please let me know if i am missing something.

Can you explain in more detail what one should do to reproduce the problem you seem to see with the attached code? Also you didn't mention which versions of libvirt and libvirt-java you're using.

When I run you example code (with a guest name that exists in my setup and libvirt and libvirt-java form current git head) I don't see any exceptions. Even when I add e.printStackTrace() to the catch block of TestMultiThreadingLibvirtWithError I don't see exceptions.

$ java -classpath ./target/libvirt-0.4.6.jar:./ravi:/usr/share/java/jna.jar Test start class 0 start class 1 start class 2 start class 3 start class 4 start class 5 start class 6 start class 7 start class 8 start class 9

So this works for me.

Matthias

-- Thanks & Regards

Ravi Pawar

-- Thanks & Regards

Ravi Pawar

2010/9/3 Daniel P. Berrange <berrange@redhat.com>:

On Fri, Sep 03, 2010 at 01:02:09PM +0200, Matthias Bolte wrote:

...

Ah, okay, now I understand what you want to say.

You have two threads A and B. When A runs on its own then it's just fine, but when you add B (that triggers an exception on purpose and ignores it) then A picks it up and reports it. This is the point where thread safety comes to mind.

libvirt stores errors in thread-local-storage. It uses pthread_key_create/pthread_{get,set}specific for this (or TlsAlloc/Tls{Get,Set}Value on Windows).

I think what's happening here is that all your threads in Java share the same thread-local-storage. Therefore, thread A can pickup the error triggered by thread B, that should not happen.

That could only happen if the JVM was using green threads instead of native threads, but no serious JVM still uses green threads these days.

Well, yes. That's what I thought too and that's why it took me a while until I understood what he meant. But I can reproduce it his test program and with latest libvirt and libvirt-java and the default JVM here on Ubuntu 10.04: $ java -version java version "1.6.0_18" OpenJDK Runtime Environment (IcedTea6 1.8.1) (6b18-1.8.1-0ubuntu1) OpenJDK 64-Bit Server VM (build 16.0-b13, mixed mode) I get stacktraces printed from TestMultiThreadingLibvirt for exceptions that are actually triggered in TestMultiThreadingLibvirtWithError in another thread. Matthias

2010/9/3 Daniel P. Berrange <berrange@redhat.com>:

On Fri, Sep 03, 2010 at 01:02:09PM +0200, Matthias Bolte wrote:

...

Ah, okay, now I understand what you want to say.

You have two threads A and B. When A runs on its own then it's just fine, but when you add B (that triggers an exception on purpose and ignores it) then A picks it up and reports it. This is the point where thread safety comes to mind.

libvirt stores errors in thread-local-storage. It uses pthread_key_create/pthread_{get,set}specific for this (or TlsAlloc/Tls{Get,Set}Value on Windows).

I think what's happening here is that all your threads in Java share the same thread-local-storage. Therefore, thread A can pickup the error triggered by thread B, that should not happen.

I'm not sure how to fix that.

Looking at the java code, I believe the problem is that the java bindings are *not* using the threadsafe error APIs:

In Connect.java

   /**     * call the error handling logic. Should be called after every libvirt call     *     * @throws LibvirtException     */    protected void processError() throws LibvirtException {        ErrorHandler.processError(libvirt, VCP);    }

Which calls into

   public static void processError(Libvirt libvirt, ConnectionPointer conn) throws LibvirtException {        virError vError = new virError();        int errorCode = libvirt.virConnCopyLastError(conn, vError);

And virConnCopyLastError is *not* threadsafe:

/**  * virConnCopyLastError:  * @conn: pointer to the hypervisor connection  * @to: target to receive the copy  *  * Copy the content of the last error caught on that connection  *  * This method is not protected against access from multiple  * threads. In a multi-threaded application, always use the  * global virGetLastError() API which is backed by thread  * local storage.

Regards, Daniel

Hm that might explain it, I'll check if that's the cause. Matthias