[libvirt] alignment of data fields when compiling with mingwin
?Hi, I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ? cheers, Arnaud Champion
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory? The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment. If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you: #include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h> void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); } Output on x86: virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20 virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12 Output on x86_64: virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40 virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24 [1] http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C... Matthias
Hi Matthias, I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you : I have defined the virConnectAuth struct by this binding : ///<summary> /// Structure to handle connection authentication ///</summary> [StructLayout(LayoutKind.Sequential)] public struct virConnectAuth { /// <summary> /// List of supported virConnectCredentialType values, should be a IntPtr to an int array or to a virConnectCredentialType array /// </summary> public IntPtr credtypes; ///<summary> /// Number of virConnectCredentialType in credtypes ///</summary> [MarshalAs(UnmanagedType.U4)] public uint ncredtype; ///<summary> /// Callback used to collect credentials, a virConnectAuthCallback delegate in bindings ///</summary> [MarshalAs(UnmanagedType.FunctionPtr)] public virConnectAuthCallback cb; ///<summary> /// Data transported with callback, should be a IntPtr on what you want ///</summary> public IntPtr cbdata; } And I have defined the virConnectCredential struct by this binding : ///<summary> /// Credential structure ///</summary> [StructLayout(LayoutKind.Sequential)] public struct virConnectCredential { ///<summary> /// One of virConnectCredentialType constants ///</summary> [MarshalAs(UnmanagedType.I4)] public virConnectCredentialType type; ///<summary> /// Prompt to show to user ///</summary> [MarshalAs(UnmanagedType.LPStr)] public string prompt; ///<summary> /// Additional challenge to show ///</summary> [MarshalAs(UnmanagedType.LPStr)] public string challenge; ///<summary> /// Optional default result ///</summary> [MarshalAs(UnmanagedType.LPStr)] public string defresult; ///<summary> /// Result to be filled with user response (or defresult). An IntPtr to a marshalled allocated string ///</summary> [MarshalAs(UnmanagedType.LPStr)] public string result; ///<summary> /// Length of the result ///</summary> [MarshalAs(UnmanagedType.U4)] public uint resultlen; } Now, I have made a little code to connect to an esx hypervisor like this : ... some init stuff... // Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length }; IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0); I have the call back method "AuthCallback" defined like this : private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) { AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0; while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset); virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true); credIndex++; } return 0; } When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase. But when I run my code in a debugger of Visual Studio I have these messages : HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 ) but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem. Anyway, thanks ofr these informations, this can help. Best regards, Arnaud -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1] http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
2010/10/17 <arnaud.champion@devatom.fr>:
Hi Matthias,
I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :
Now, I have made a little code to connect to an esx hypervisor like this :
... some init stuff...
// Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length };
IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0);
I have the call back method "AuthCallback" defined like this :
private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) {
AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0;
while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);
virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true);
credIndex++; } return 0; }
When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase.
But when I run my code in a debugger of Visual Studio I have these messages :
HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 )
but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.
Okay, you see an invalid free call either because free is called on an initial invalid pointer of because free is called twice on the same pointer (on the second call the pointer is invalid). When you look at the libvirt codebase in src/util/authhelper.c in virRequestUsername you'll see that the cred.result is returned to the driver directly without making a copy of it. The driver is then responsible for freeing this memory. So the ownership of the memory cred.result points to is transferred from your callback to the driver. I think you see a double-free here because libvirt and C# are both trying to free the same memory, but C# should not do this. I'm not familiar with C#, so I'm just guessing here based on what you have described. This cannot be a problem with the alignment or struct layout, otherwise authentication would not work. You can test if it's double-free by applying the attached patch to libvirt. This patch stops the virRequestUsername and virRequestPassword functions from transferring ownership of the cred.result by copying the string. This creates a memory leak in the normal case, but should stop the double-free in case C# is freeing cred.result here. Matthias
Anyway, thanks ofr these informations, this can help.
Best regards,
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1] http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
You are perfectly right, I have to find the way to avoid C# memory freeing ! You're a master ;) Arnaud -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 3:21 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/17 <arnaud.champion@devatom.fr>:
Hi Matthias,
I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :
Now, I have made a little code to connect to an esx hypervisor like this :
... some init stuff...
// Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length };
IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0);
I have the call back method "AuthCallback" defined like this :
private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) {
AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0;
while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);
virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true);
credIndex++; } return 0; }
When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase.
But when I run my code in a debugger of Visual Studio I have these messages :
HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 )
but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.
Okay, you see an invalid free call either because free is called on an initial invalid pointer of because free is called twice on the same pointer (on the second call the pointer is invalid).
When you look at the libvirt codebase in src/util/authhelper.c in virRequestUsername you'll see that the cred.result is returned to the driver directly without making a copy of it. The driver is then responsible for freeing this memory. So the ownership of the memory cred.result points to is transferred from your callback to the driver.
I think you see a double-free here because libvirt and C# are both trying to free the same memory, but C# should not do this. I'm not familiar with C#, so I'm just guessing here based on what you have described. This cannot be a problem with the alignment or struct layout, otherwise authentication would not work.
You can test if it's double-free by applying the attached patch to libvirt. This patch stops the virRequestUsername and virRequestPassword functions from transferring ownership of the cred.result by copying the string. This creates a memory leak in the normal case, but should stop the double-free in case C# is freeing cred.result here.
Matthias
Anyway, thanks ofr these informations, this can help.
Best regards,
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1] http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
A bit of googling turns this function up Marshal.StringToHGlobalAnsi Marshal.StringToHGlobalAuto Marshal.StringToHGlobalUni they create an unmanaged copy a managed string. But those might not help, because the docs say that the returned IntPtr must be freed using Marshal.FreeHGlobal. All this methods use the GlobalAlloc function from the WinAPI to allocate memory. Maybe GlobalAlloc is not compatible with the "libc" free function that libvirt will use to free the memory. A bit more googling told me how to import functions from DLLs in C#. This one imports strdup: [DllImport("msvcrt.dll"), CLSCompliant(false)] public static extern IntPtr _strdup([Const] IntPtr strSource); Maybe you can do something like this: string result_original = ...; IntPtr result_global = StringToHGlobalAuto(result_original); IntPtr result_duplicated = _strdup(result_global); FreeHGlobal(result_global); Then pass result_duplicated to libvirt. I'm not sure if that works at all. Maybe there is a simpler/better solution for this. Matthias 2010/10/17 <arnaud.champion@devatom.fr>:
You are perfectly right, I have to find the way to avoid C# memory freeing ! You're a master ;)
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 3:21 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/17 <arnaud.champion@devatom.fr>:
Hi Matthias,
I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :
Now, I have made a little code to connect to an esx hypervisor like this :
... some init stuff...
// Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length };
IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0);
I have the call back method "AuthCallback" defined like this :
private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) {
AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0;
while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);
virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true);
credIndex++; } return 0; }
When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase.
But when I run my code in a debugger of Visual Studio I have these messages :
HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 )
but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.
Okay, you see an invalid free call either because free is called on an initial invalid pointer of because free is called twice on the same pointer (on the second call the pointer is invalid).
When you look at the libvirt codebase in src/util/authhelper.c in virRequestUsername you'll see that the cred.result is returned to the driver directly without making a copy of it. The driver is then responsible for freeing this memory. So the ownership of the memory cred.result points to is transferred from your callback to the driver.
I think you see a double-free here because libvirt and C# are both trying to free the same memory, but C# should not do this. I'm not familiar with C#, so I'm just guessing here based on what you have described. This cannot be a problem with the alignment or struct layout, otherwise authentication would not work.
You can test if it's double-free by applying the attached patch to libvirt. This patch stops the virRequestUsername and virRequestPassword functions from transferring ownership of the cred.result by copying the string. This creates a memory leak in the normal case, but should stop the double-free in case C# is freeing cred.result here.
Matthias
Anyway, thanks ofr these informations, this can help.
Best regards,
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1]
http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
I will try these ways, in my search I have also seen some explainations that said to PInvoke char* as IntPtr instead of string in structures, then use Marshal.PtrToStringAnsi/Marshal.StringToHGlobalAnsi but for now it fails. I will try with the strdup PInvoke. -------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 6:58 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
A bit of googling turns this function up
Marshal.StringToHGlobalAnsi Marshal.StringToHGlobalAuto Marshal.StringToHGlobalUni
they create an unmanaged copy a managed string.
But those might not help, because the docs say that the returned IntPtr must be freed using Marshal.FreeHGlobal. All this methods use the GlobalAlloc function from the WinAPI to allocate memory. Maybe GlobalAlloc is not compatible with the "libc" free function that libvirt will use to free the memory.
A bit more googling told me how to import functions from DLLs in C#. This one imports strdup:
[DllImport("msvcrt.dll"), CLSCompliant(false)] public static extern IntPtr _strdup([Const] IntPtr strSource);
Maybe you can do something like this:
string result_original = ...; IntPtr result_global = StringToHGlobalAuto(result_original); IntPtr result_duplicated = _strdup(result_global); FreeHGlobal(result_global);
Then pass result_duplicated to libvirt.
I'm not sure if that works at all. Maybe there is a simpler/better solution for this.
Matthias
2010/10/17 <arnaud.champion@devatom.fr>:
You are perfectly right, I have to find the way to avoid C# memory freeing ! You're a master ;)
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 3:21 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/17 <arnaud.champion@devatom.fr>:
Hi Matthias,
I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :
Now, I have made a little code to connect to an esx hypervisor like this :
... some init stuff...
// Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length };
IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0);
I have the call back method "AuthCallback" defined like this :
private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) {
AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0;
while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);
virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true);
credIndex++; } return 0; }
When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase.
But when I run my code in a debugger of Visual Studio I have these messages :
HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 )
but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.
Okay, you see an invalid free call either because free is called on an initial invalid pointer of because free is called twice on the same pointer (on the second call the pointer is invalid).
When you look at the libvirt codebase in src/util/authhelper.c in virRequestUsername you'll see that the cred.result is returned to the driver directly without making a copy of it. The driver is then responsible for freeing this memory. So the ownership of the memory cred.result points to is transferred from your callback to the driver.
I think you see a double-free here because libvirt and C# are both trying to free the same memory, but C# should not do this. I'm not familiar with C#, so I'm just guessing here based on what you have described. This cannot be a problem with the alignment or struct layout, otherwise authentication would not work.
You can test if it's double-free by applying the attached patch to libvirt. This patch stops the virRequestUsername and virRequestPassword functions from transferring ownership of the cred.result by copying the string. This creates a memory leak in the normal case, but should stop the double-free in case C# is freeing cred.result here.
Matthias
Anyway, thanks ofr these informations, this can help.
Best regards,
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>:
Hi,
I'm currently working on libvirt csharp bindings, and I have some trouble with virConnectOpenAuth marshaling. I need to know what is the alignment of data fields in structure when compiling with mingwin. Anyone know that ?
cheers,
Arnaud Champion
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1]
http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
Wow, we got it ! In fact, you got it... By pinning strdup function, it works. No more heap problem !!!!! I'm now able to use the virConnectOpenAuth function, and that's great. Thanks a lot for your help. Arnaud -------------------------------------------------- From: <arnaud.champion@devatom.fr> Sent: Sunday, October 17, 2010 8:39 PM To: "Matthias Bolte" <matthias.bolte@googlemail.com> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
I will try these ways, in my search I have also seen some explainations that said to PInvoke char* as IntPtr instead of string in structures, then use Marshal.PtrToStringAnsi/Marshal.StringToHGlobalAnsi but for now it fails. I will try with the strdup PInvoke.
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 6:58 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
A bit of googling turns this function up
Marshal.StringToHGlobalAnsi Marshal.StringToHGlobalAuto Marshal.StringToHGlobalUni
they create an unmanaged copy a managed string.
But those might not help, because the docs say that the returned IntPtr must be freed using Marshal.FreeHGlobal. All this methods use the GlobalAlloc function from the WinAPI to allocate memory. Maybe GlobalAlloc is not compatible with the "libc" free function that libvirt will use to free the memory.
A bit more googling told me how to import functions from DLLs in C#. This one imports strdup:
[DllImport("msvcrt.dll"), CLSCompliant(false)] public static extern IntPtr _strdup([Const] IntPtr strSource);
Maybe you can do something like this:
string result_original = ...; IntPtr result_global = StringToHGlobalAuto(result_original); IntPtr result_duplicated = _strdup(result_global); FreeHGlobal(result_global);
Then pass result_duplicated to libvirt.
I'm not sure if that works at all. Maybe there is a simpler/better solution for this.
Matthias
2010/10/17 <arnaud.champion@devatom.fr>:
You are perfectly right, I have to find the way to avoid C# memory freeing ! You're a master ;)
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 3:21 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/17 <arnaud.champion@devatom.fr>:
Hi Matthias,
I'm working on the virConnectOpenAuth method C# binding. And in fact, It partially work. Let me explain you :
Now, I have made a little code to connect to an esx hypervisor like this :
... some init stuff...
// Define the virConnectAuth struct virConnectAuth auth = new virConnectAuth { cbdata = authDataPtr, cb = AuthCallback, credtypes = credtypesPtr, ncredtype = (uint)credtypes.Length };
IntPtr conn = libVirt.virConnectOpenAuth(tbURI.Text, ref auth, 0);
I have the call back method "AuthCallback" defined like this :
private int AuthCallback(IntPtr creds, uint ncred, IntPtr cbdata) {
AuthData authData = (AuthData) Marshal.PtrToStructure(cbdata, typeof (AuthData)); int offset = 0; int credIndex = 0;
while (credIndex < ncred) { IntPtr currentCred = new IntPtr(creds.ToInt32() + offset);
virConnectCredential cred = (virConnectCredential) Marshal.PtrToStructure(currentCred, typeof (virConnectCredential)); offset += Marshal.SizeOf(cred); switch(cred.type) { case virConnectCredentialType.VIR_CRED_AUTHNAME: cred.result = authData.user_name; cred.resultlen = (uint)authData.user_name.Length; break; case virConnectCredentialType.VIR_CRED_PASSPHRASE: cred.result = authData.password; cred.resultlen = (uint) authData.password.Length; break; default: return -1; } Marshal.StructureToPtr(cred, currentCred, true);
credIndex++; } return 0; }
When I test this code, all seems to work well, I mean the "AuthCallback" method is called twice, once for authname and once for passphrase.
But when I run my code in a debugger of Visual Studio I have these messages :
HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6D78 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00700000, 00EE6A80 ) HEAP[Win32_virConnectOpenAuth.exe]: Invalid address specified to RtlFreeHeap( 00EE0000, 007DE480 )
but the connection is done, and I am able to list domains of the hypervisor for example. The real problem is that this program only work inside the the visual studio debugger, when I launch the executable without debugger, it fails. So I am searching why this doesn't work, I want to check the bindings. And I also want to try to fill the "result" member of the virConnectCredential structure directly to understanding what can be the problem.
Okay, you see an invalid free call either because free is called on an initial invalid pointer of because free is called twice on the same pointer (on the second call the pointer is invalid).
When you look at the libvirt codebase in src/util/authhelper.c in virRequestUsername you'll see that the cred.result is returned to the driver directly without making a copy of it. The driver is then responsible for freeing this memory. So the ownership of the memory cred.result points to is transferred from your callback to the driver.
I think you see a double-free here because libvirt and C# are both trying to free the same memory, but C# should not do this. I'm not familiar with C#, so I'm just guessing here based on what you have described. This cannot be a problem with the alignment or struct layout, otherwise authentication would not work.
You can test if it's double-free by applying the attached patch to libvirt. This patch stops the virRequestUsername and virRequestPassword functions from transferring ownership of the cred.result by copying the string. This creates a memory leak in the normal case, but should stop the double-free in case C# is freeing cred.result here.
Matthias
Anyway, thanks ofr these informations, this can help.
Best regards,
Arnaud
-------------------------------------------------- From: "Matthias Bolte" <matthias.bolte@googlemail.com> Sent: Sunday, October 17, 2010 12:04 PM To: <arnaud.champion@devatom.fr> Cc: <libvir-list@redhat.com> Subject: Re: [libvirt] alignment of data fields when compiling with mingwin
2010/10/15 <arnaud.champion@devatom.fr>: > > Hi, > > I'm currently working on libvirt csharp bindings, and I have some > trouble > with virConnectOpenAuth marshaling. > I need to know what is the alignment of data fields in structure > when > compiling with mingwin. > Anyone know that ? > > cheers, > > Arnaud Champion >
Why do you need to know the alignment? Do you need to build or access members of the virConnectCredential or virConnectAuth structs by offset in memory?
The Wikipedia article about data structure alignment might be helpful when you really need to care about alignment.
If you actually need the offset of the members in those structs you can just use the offsetof macro and let the compiler tell you:
#include <stdio.h> #include <stddef.h> #include <libvirt/libvirt.h>
void main(void) { printf("virConnectCredential\n"); printf(" type %2u\n", offsetof(virConnectCredential, type)); printf(" prompt %2u\n", offsetof(virConnectCredential, prompt)); printf(" challenge %2u\n", offsetof(virConnectCredential, challenge)); printf(" defresult %2u\n", offsetof(virConnectCredential, defresult)); printf(" result %2u\n", offsetof(virConnectCredential, result)); printf(" resultlen %2u\n", offsetof(virConnectCredential, resultlen)); printf("\n"); printf("virConnectAuth\n"); printf(" credtype %2u\n", offsetof(virConnectAuth, credtype)); printf(" ncredtype %2u\n", offsetof(virConnectAuth, ncredtype)); printf(" cb %2u\n", offsetof(virConnectAuth, cb)); printf(" cbdata %2u\n", offsetof(virConnectAuth, cbdata)); }
Output on x86:
virConnectCredential type 0 prompt 4 challenge 8 defresult 12 result 16 resultlen 20
virConnectAuth credtype 0 ncredtype 4 cb 8 cbdata 12
Output on x86_64:
virConnectCredential type 0 prompt 8 challenge 16 defresult 24 result 32 resultlen 40
virConnectAuth credtype 0 ncredtype 8 cb 16 cbdata 24
[1]
http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C...
Matthias
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (2)
-
arnaud.champion＠devatom.fr -
Matthias Bolte