Hi, I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ? cheers, Arnaud Champion

Hi Matthias, I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :

Now, I have made a little code to connect to an esx hypervisor like this : ... some init stuff...           // Define the virConnectAuth struct           virConnectAuth auth = new virConnectAuth           {               cbdata = authDataPtr,               cb = AuthCallback,               credtypes = credtypesPtr,               ncredtype = (uint)credtypes.Length           };           IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0); I have the call back method "AuthCallback" defined like this :       private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata)       {           AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData));           int offset = 0;           int credIndex = 0;           while (credIndex < ncred)           {               IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);               virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential));               offset += Marshal.SizeOf(cred);               switch(cred.type)               {                   case virConnectCredentialType.VIR_CRED_AUTHNAME:                       cred.result = authData.user_name;                       cred.resultlen = (uint)authData.user_name.Length;                       break;                   case virConnectCredentialType.VIR_CRED_PASSPHRASE:                       cred.result = authData.password;                       cred.resultlen = (uint) authData.password.Length;                       break;                   default:                       return -1;               }               Marshal.StructureToPtr(cred, currentCred, true);               credIndex++;           }           return 0;       } When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase. But when I run my code in a debugger of Visual Studio I have these messages : HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 ) but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.

Anyway, thanks ofr these informations, this can help. Best regards, Arnaud -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion(a)devatom.fr&gt; Cc: <libvir-list(a)redhat.com&gt; Subject: Re: [libvirt] alignment of data fields when compiling with mingwin > 2010/10/15  &lt;arnaud.champion(a)devatom.fr&gt;: >> >> Hi, >> >> I'm currently working on libvirt csharp bindings, and I have some trouble >> with virConnectOpenAuth marshaling. >> I need to know what is the alignment of data fields in structure when >> compiling with mingwin. >> Anyone know that ? >> >> cheers, >> >> Arnaud Champion >> > > Why do you need to know the alignment? Do you need to build or access > members of the virConnectCredential or virConnectAuth structs by > offset in memory? > > The Wikipedia article about data structure alignment might be helpful > when you really need to care about alignment. > > If you actually need the offset of the members in those structs you > can just use the offsetof macro and let the compiler tell you: > > #include <stdio.h> > #include <stddef.h> > #include <libvirt/libvirt.h> > > void main(void) > { >   printf("virConnectCredential\n"); >   printf("  type      %2u\n", offsetof(virConnectCredential, type)); >   printf("  prompt    %2u\n", offsetof(virConnectCredential, prompt)); >   printf("  challenge %2u\n", offsetof(virConnectCredential, challenge)); >   printf("  defresult %2u\n", offsetof(virConnectCredential, defresult)); >   printf("  result    %2u\n", offsetof(virConnectCredential, result)); >   printf("  resultlen %2u\n", offsetof(virConnectCredential, resultlen)); >   printf("\n"); >   printf("virConnectAuth\n"); >   printf("  credtype  %2u\n", offsetof(virConnectAuth, credtype)); >   printf("  ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); >   printf("  cb        %2u\n", offsetof(virConnectAuth, cb)); >   printf("  cbdata    %2u\n", offsetof(virConnectAuth, cbdata)); > } > > > Output on x86: > > virConnectCredential >  type       0 >  prompt     4 >  challenge  8 >  defresult 12 >  result    16 >  resultlen 20 > > virConnectAuth >  credtype   0 >  ncredtype  4 >  cb         8 >  cbdata    12 > > > Output on x86_64: > > virConnectCredential >  type       0 >  prompt     8 >  challenge 16 >  defresult 24 >  result    32 >  resultlen 40 > > virConnectAuth >  credtype   0 >  ncredtype  8 >  cb        16 >  cbdata    24 > > [1] > http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_o... > > Matthias

You are perfectly right, I have to find the way to avoid C# memory freeing ! You're a master ;) Arnaud -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; Sent: Sunday, October 17, 2010 3:21 PM To: <arnaud.champion(a)devatom.fr&gt; Cc: <libvir-list(a)redhat.com&gt; Subject: Re: [libvirt] alignment of data fields when compiling with mingwin > 2010/10/17  &lt;arnaud.champion(a)devatom.fr&gt;: >> >> Hi Matthias, >> >> I'm working on the virConnectOpenAuth method C# binding. And in fact, It >> partially work. Let me explain you : >> > > >> Now, I have made a little code to connect to an esx hypervisor like this >> : >> >> ... some init stuff... >> >>          // Define the virConnectAuth struct >>          virConnectAuth auth = new virConnectAuth >>          { >>              cbdata = authDataPtr, >>              cb = AuthCallback, >>              credtypes = credtypesPtr, >>              ncredtype = (uint)credtypes.Length >>          }; >> >>          IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, >> 0); >> >> I have the call back method "AuthCallback" defined like this : >> >>      private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) >>      { >> >>          AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, >> typeof (AuthData)); >>          int offset = 0; >>          int credIndex = 0; >> >>          while (credIndex < ncred) >>          { >>              IntPtr currentCred = new IntPtr(creds.ToInt32() + offset); >> >>              virConnectCredential cred = (virConnectCredential) >> Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); >>              offset += Marshal.SizeOf(cred); >>              switch(cred.type) >>              { >>                  case virConnectCredentialType.VIR_CRED_AUTHNAME: >>                      cred.result = authData.user_name; >>                      cred.resultlen = (uint)authData.user_name.Length; >>                      break; >>                  case virConnectCredentialType.VIR_CRED_PASSPHRASE: >>                      cred.result = authData.password; >>                      cred.resultlen = (uint) authData.password.Length; >>                      break; >>                  default: >>                      return -1; >>              } >>              Marshal.StructureToPtr(cred, currentCred, true); >> >>              credIndex++; >>          } >>          return 0; >>      } >> >> When I test this code, all seems to work well, I mean the "AuthCallback" >> method is called twice, once for authname and once for passphrase. >> >> But when I run my code in a debugger of Visual Studio I have these >> messages >> : >> >> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >> RtlFreeHeap( 00700000, 00EE6D78 ) >> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >> RtlFreeHeap( 00700000, 00EE6A80 ) >> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >> RtlFreeHeap( 00700000, 00EE6D78 ) >> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >> RtlFreeHeap( 00700000, 00EE6A80 ) >> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >> RtlFreeHeap( 00EE0000, 007DE480 ) >> >> but the connection is done, and I am able to list domains of the >> hypervisor >> for example. The real problem is that this program only work inside the >> the >> visual studio debugger, when I launch the executable without debugger, it >> fails. >> So I am searching why this doesn't work, I want to check the bindings. >> And I >> also want to try to fill the "result" member of the virConnectCredential >> structure directly to understanding what can be the problem. > > > Okay, you see an invalid free call either because free is called on an > initial invalid pointer of because free is called twice on the same > pointer (on the second call the pointer is invalid). > > When you look at the libvirt codebase in src/util/authhelper.c in > virRequestUsername you'll see that the cred.result is returned to the > driver directly without making a copy of it. The driver is then > responsible for freeing this memory. So the ownership of the memory > cred.result points to is transferred from your callback to the driver. > > I think you see a double-free here because libvirt and C# are both > trying to free the same memory, but C# should not do this. I'm not > familiar with C#, so I'm just guessing here based on what you have > described. This cannot be a problem with the alignment or struct > layout, otherwise authentication would not work. > > You can test if it's double-free by applying the attached patch to > libvirt. This patch stops the virRequestUsername and > virRequestPassword functions from transferring ownership of the > cred.result by copying the string. This creates a memory leak in the > normal case, but should stop the double-free in case C# is freeing > cred.result here. > > Matthias > > >> Anyway, thanks ofr these informations, this can help. >> >> Best regards, >> >> Arnaud >> >> -------------------------------------------------- >> From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; >> Sent: Sunday, October 17, 2010 12:04 PM >> To: <arnaud.champion(a)devatom.fr&gt; >> Cc: <libvir-list(a)redhat.com&gt; >> Subject: Re: [libvirt] alignment of data fields when compiling with >> mingwin >> >>> 2010/10/15  &lt;arnaud.champion(a)devatom.fr&gt;: >>>> >>>> Hi, >>>> >>>> I'm currently working on libvirt csharp bindings, and I have some >>>> trouble >>>> with virConnectOpenAuth marshaling. >>>> I need to know what is the alignment of data fields in structure when >>>> compiling with mingwin. >>>> Anyone know that ? >>>> >>>> cheers, >>>> >>>> Arnaud Champion >>>> >>> >>> Why do you need to know the alignment? Do you need to build or access >>> members of the virConnectCredential or virConnectAuth structs by >>> offset in memory? >>> >>> The Wikipedia article about data structure alignment might be helpful >>> when you really need to care about alignment. >>> >>> If you actually need the offset of the members in those structs you >>> can just use the offsetof macro and let the compiler tell you: >>> >>> #include <stdio.h> >>> #include <stddef.h> >>> #include <libvirt/libvirt.h> >>> >>> void main(void) >>> { >>>  printf("virConnectCredential\n"); >>>  printf("  type      %2u\n", offsetof(virConnectCredential, type)); >>>  printf("  prompt    %2u\n", offsetof(virConnectCredential, prompt)); >>>  printf("  challenge %2u\n", offsetof(virConnectCredential, challenge)); >>>  printf("  defresult %2u\n", offsetof(virConnectCredential, defresult)); >>>  printf("  result    %2u\n", offsetof(virConnectCredential, result)); >>>  printf("  resultlen %2u\n", offsetof(virConnectCredential, resultlen)); >>>  printf("\n"); >>>  printf("virConnectAuth\n"); >>>  printf("  credtype  %2u\n", offsetof(virConnectAuth, credtype)); >>>  printf("  ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); >>>  printf("  cb        %2u\n", offsetof(virConnectAuth, cb)); >>>  printf("  cbdata    %2u\n", offsetof(virConnectAuth, cbdata)); >>> } >>> >>> >>> Output on x86: >>> >>> virConnectCredential >>>  type       0 >>>  prompt     4 >>>  challenge  8 >>>  defresult 12 >>>  result    16 >>>  resultlen 20 >>> >>> virConnectAuth >>>  credtype   0 >>>  ncredtype  4 >>>  cb         8 >>>  cbdata    12 >>> >>> >>> Output on x86_64: >>> >>> virConnectCredential >>>  type       0 >>>  prompt     8 >>>  challenge 16 >>>  defresult 24 >>>  result    32 >>>  resultlen 40 >>> >>> virConnectAuth >>>  credtype   0 >>>  ncredtype  8 >>>  cb        16 >>>  cbdata    24 >>> >>> [1] >>> >>> http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_o... >>> >>> Matthias >> >> >> >

I will try these ways, in my search I have also seen some explainations that said to PInvoke char* as IntPtr instead of string in structures, then use Marshal.PtrToStringAnsi/Marshal.StringToHGlobalAnsi but for now it fails. I will try with the strdup PInvoke. -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; Sent: Sunday, October 17, 2010 6:58 PM To: <arnaud.champion(a)devatom.fr&gt; Cc: <libvir-list(a)redhat.com&gt; Subject: Re: [libvirt] alignment of data fields when compiling with mingwin > A bit of googling turns this function up > > Marshal.StringToHGlobalAnsi > Marshal.StringToHGlobalAuto > Marshal.StringToHGlobalUni > > they create an unmanaged copy a managed string. > > But those might not help, because the docs say that the returned > IntPtr must be freed using Marshal.FreeHGlobal. All this methods use > the GlobalAlloc function from the WinAPI to allocate memory. Maybe > GlobalAlloc is not compatible with the "libc" free function that > libvirt will use to free the memory. > > A bit more googling told me how to import functions from DLLs in C#. > This one imports strdup: > > [DllImport("msvcrt.dll"), CLSCompliant(false)] > public static extern IntPtr _strdup([Const] IntPtr strSource); > > Maybe you can do something like this: > > string result_original = ...; > IntPtr result_global = StringToHGlobalAuto(result_original); > IntPtr result_duplicated = _strdup(result_global); > FreeHGlobal(result_global); > > Then pass result_duplicated to libvirt. > > I'm not sure if that works at all. Maybe there is a simpler/better > solution for this. > > Matthias > > 2010/10/17 <arnaud.champion(a)devatom.fr&gt;: >> You are perfectly right, I have to find the way to avoid C# memory >> freeing ! >> You're a master ;) >> >> Arnaud >> >> -------------------------------------------------- >> From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; >> Sent: Sunday, October 17, 2010 3:21 PM >> To: <arnaud.champion(a)devatom.fr&gt; >> Cc: <libvir-list(a)redhat.com&gt; >> Subject: Re: [libvirt] alignment of data fields when compiling with >> mingwin >> >>> 2010/10/17 <arnaud.champion(a)devatom.fr&gt;: >>>> >>>> Hi Matthias, >>>> >>>> I'm working on the virConnectOpenAuth method C# binding. And in fact, >>>> It >>>> partially work. Let me explain you : >>>> >>> >>> >>>> Now, I have made a little code to connect to an esx hypervisor like >>>> this >>>> : >>>> >>>> ... some init stuff... >>>> >>>> // Define the virConnectAuth struct >>>> virConnectAuth auth = new virConnectAuth >>>> { >>>> cbdata = authDataPtr, >>>> cb = AuthCallback, >>>> credtypes = credtypesPtr, >>>> ncredtype = (uint)credtypes.Length >>>> }; >>>> >>>> IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref >>>> auth, >>>> 0); >>>> >>>> I have the call back method "AuthCallback" defined like this : >>>> >>>> private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) >>>> { >>>> >>>> AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, >>>> typeof (AuthData)); >>>> int offset = 0; >>>> int credIndex = 0; >>>> >>>> while (credIndex < ncred) >>>> { >>>> IntPtr currentCred = new IntPtr(creds.ToInt32() + >>>> offset); >>>> >>>> virConnectCredential cred = (virConnectCredential) >>>> Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); >>>> offset += Marshal.SizeOf(cred); >>>> switch(cred.type) >>>> { >>>> case virConnectCredentialType.VIR_CRED_AUTHNAME: >>>> cred.result = authData.user_name; >>>> cred.resultlen = (uint)authData.user_name.Length; >>>> break; >>>> case virConnectCredentialType.VIR_CRED_PASSPHRASE: >>>> cred.result = authData.password; >>>> cred.resultlen = (uint) authData.password.Length; >>>> break; >>>> default: >>>> return -1; >>>> } >>>> Marshal.StructureToPtr(cred, currentCred, true); >>>> >>>> credIndex++; >>>> } >>>> return 0; >>>> } >>>> >>>> When I test this code, all seems to work well, I mean the >>>> "AuthCallback" >>>> method is called twice, once for authname and once for passphrase. >>>> >>>> But when I run my code in a debugger of Visual Studio I have these >>>> messages >>>> : >>>> >>>> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >>>> RtlFreeHeap( 00700000, 00EE6D78 ) >>>> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >>>> RtlFreeHeap( 00700000, 00EE6A80 ) >>>> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >>>> RtlFreeHeap( 00700000, 00EE6D78 ) >>>> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >>>> RtlFreeHeap( 00700000, 00EE6A80 ) >>>> HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to >>>> RtlFreeHeap( 00EE0000, 007DE480 ) >>>> >>>> but the connection is done, and I am able to list domains of the >>>> hypervisor >>>> for example. The real problem is that this program only work inside >>>> the >>>> the >>>> visual studio debugger, when I launch the executable without debugger, >>>> it >>>> fails. >>>> So I am searching why this doesn't work, I want to check the bindings. >>>> And I >>>> also want to try to fill the "result" member of the >>>> virConnectCredential >>>> structure directly to understanding what can be the problem. >>> >>> >>> Okay, you see an invalid free call either because free is called on an >>> initial invalid pointer of because free is called twice on the same >>> pointer (on the second call the pointer is invalid). >>> >>> When you look at the libvirt codebase in src/util/authhelper.c in >>> virRequestUsername you'll see that the cred.result is returned to the >>> driver directly without making a copy of it. The driver is then >>> responsible for freeing this memory. So the ownership of the memory >>> cred.result points to is transferred from your callback to the driver. >>> >>> I think you see a double-free here because libvirt and C# are both >>> trying to free the same memory, but C# should not do this. I'm not >>> familiar with C#, so I'm just guessing here based on what you have >>> described. This cannot be a problem with the alignment or struct >>> layout, otherwise authentication would not work. >>> >>> You can test if it's double-free by applying the attached patch to >>> libvirt. This patch stops the virRequestUsername and >>> virRequestPassword functions from transferring ownership of the >>> cred.result by copying the string. This creates a memory leak in the >>> normal case, but should stop the double-free in case C# is freeing >>> cred.result here. >>> >>> Matthias >>> >>> >>>> Anyway, thanks ofr these informations, this can help. >>>> >>>> Best regards, >>>> >>>> Arnaud >>>> >>>> -------------------------------------------------- >>>> From: "Matthias Bolte" <matthias.bolte(a)googlemail.com&gt; >>>> Sent: Sunday, October 17, 2010 12:04 PM >>>> To: <arnaud.champion(a)devatom.fr&gt; >>>> Cc: <libvir-list(a)redhat.com&gt; >>>> Subject: Re: [libvirt] alignment of data fields when compiling with >>>> mingwin >>>> >>>>> 2010/10/15 <arnaud.champion(a)devatom.fr&gt;: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm currently working on libvirt csharp bindings, and I have some >>>>>> trouble >>>>>> with virConnectOpenAuth marshaling. >>>>>> I need to know what is the alignment of data fields in structure >>>>>> when >>>>>> compiling with mingwin. >>>>>> Anyone know that ? >>>>>> >>>>>> cheers, >>>>>> >>>>>> Arnaud Champion >>>>>> >>>>> >>>>> Why do you need to know the alignment? Do you need to build or access >>>>> members of the virConnectCredential or virConnectAuth structs by >>>>> offset in memory? >>>>> >>>>> The Wikipedia article about data structure alignment might be helpful >>>>> when you really need to care about alignment. >>>>> >>>>> If you actually need the offset of the members in those structs you >>>>> can just use the offsetof macro and let the compiler tell you: >>>>> >>>>> #include <stdio.h> >>>>> #include <stddef.h> >>>>> #include <libvirt/libvirt.h> >>>>> >>>>> void main(void) >>>>> { >>>>> printf("virConnectCredential\n"); >>>>> printf(" type %2u\n", offsetof(virConnectCredential, type)); >>>>> printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); >>>>> printf(" challenge %2u\n", offsetof(virConnectCredential, >>>>> challenge)); >>>>> printf(" defresult %2u\n", offsetof(virConnectCredential, >>>>> defresult)); >>>>> printf(" result %2u\n", offsetof(virConnectCredential, result)); >>>>> printf(" resultlen %2u\n", offsetof(virConnectCredential, >>>>> resultlen)); >>>>> printf("\n"); >>>>> printf("virConnectAuth\n"); >>>>> printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); >>>>> printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); >>>>> printf(" cb %2u\n", offsetof(virConnectAuth, cb)); >>>>> printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); >>>>> } >>>>> >>>>> >>>>> Output on x86: >>>>> >>>>> virConnectCredential >>>>> type 0 >>>>> prompt 4 >>>>> challenge 8 >>>>> defresult 12 >>>>> result 16 >>>>> resultlen 20 >>>>> >>>>> virConnectAuth >>>>> credtype 0 >>>>> ncredtype 4 >>>>> cb 8 >>>>> cbdata 12 >>>>> >>>>> >>>>> Output on x86_64: >>>>> >>>>> virConnectCredential >>>>> type 0 >>>>> prompt 8 >>>>> challenge 16 >>>>> defresult 24 >>>>> result 32 >>>>> resultlen 40 >>>>> >>>>> virConnectAuth >>>>> credtype 0 >>>>> ncredtype 8 >>>>> cb 16 >>>>> cbdata 24 >>>>> >>>>> [1] >>>>> >>>>> http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_o... >>>>> >>>>> Matthias >>>> >>>> >>>> >>> >> >> > -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list