Libvirt Rust bindings could use some work
Hello, I was reading the Rust bindings for libvirt and I noticed that they are not very idiomatic. A couple of examples: 1. It is conventional to have a *-sys crate containing only the C interface and the linking configuration. The virt crate would then depend on this libvirt-sys crate to implement a safe and ergonomic wrapper. 2. The API heavily uses integers to indicate flags. Sometimes those integers are type aliased and sometimes they are not. In Rust, this can be much nicer, using enums to represent only valid flags. The same applies to some return values. 3. Since libvirt is thread-safe, the Rust types could be Send and Sync, so that they can also be used in a multithreaded context. Special care needs to be taken with translating virGetLastError to Rust Results, but it shouldn't be a problem. For a nice example of a Rust wrapper around a C library, see: https://github.com/rust-lang/git2-rs I am willing to send some patches to fix some of these problems, but this will almost certainly lead to breaking API changes, and I am not sure what your opinion is on that. Finally, while I am not a legal expert, I believe that licensing a Rust library as LGPL does not make much sense. Rust does not have a stable ABI so everything is statically linked. This means that adding virt to a Cargo.toml file of a non-GPL application is already in violation of the license. Thanks, Wim
On Fri, Feb 18, 2022 at 10:28:29AM +0100, Wim de With wrote:
Hello,
I was reading the Rust bindings for libvirt and I noticed that they are not very idiomatic. A couple of examples:
1. It is conventional to have a *-sys crate containing only the C interface and the linking configuration. The virt crate would then depend on this libvirt-sys crate to implement a safe and ergonomic wrapper. 2. The API heavily uses integers to indicate flags. Sometimes those integers are type aliased and sometimes they are not. In Rust, this can be much nicer, using enums to represent only valid flags. The same applies to some return values. 3. Since libvirt is thread-safe, the Rust types could be Send and Sync, so that they can also be used in a multithreaded context. Special care needs to be taken with translating virGetLastError to Rust Results, but it shouldn't be a problem.
For a nice example of a Rust wrapper around a C library, see: https://github.com/rust-lang/git2-rs
I am willing to send some patches to fix some of these problems, but this will almost certainly lead to breaking API changes, and I am not sure what your opinion is on that.
We are aware of the issues with the current API of libvirt-rs, but unfortunately nobody has been able to dedicate time to addressing them. Any contributions towards that goal that you or anyone else could make would certainly be greatly appreciated! Breaking the API should be fine I think. The current version number is 0.2.11, so there shouldn't be any expectation in terms of API stability. Note that libvirt-python is mostly autogenerated, and there is an ongoing effort to make the same happen for libvirt-go-module. Ideally libvirt-rust would also follow this trend and end up with very little manually-written code in it.
Finally, while I am not a legal expert, I believe that licensing a Rust library as LGPL does not make much sense. Rust does not have a stable ABI so everything is statically linked. This means that adding virt to a Cargo.toml file of a non-GPL application is already in violation of the license.
Licensing is a pretty fun minefield :) For example, even though libvirt-rs itself would be statically linked into any application that uses it, by virtue of it being a wrapper around the C library you'd still end up dynamically linking against that. I tend to agree that language bindings should follow the rest of the language ecosystem in terms of licensing, and for libvirt-rs specifically that would probably mean MIT. Perhaps we should consider looking into relicensing the project? -- Andrea Bolognani / Red Hat / Virtualization
On a Monday in 2022, Andrea Bolognani wrote:
On Fri, Feb 18, 2022 at 10:28:29AM +0100, Wim de With wrote: Licensing is a pretty fun minefield :)
For example, even though libvirt-rs itself would be statically linked into any application that uses it, by virtue of it being a wrapper around the C library you'd still end up dynamically linking against that.
I tend to agree that language bindings should follow the rest of the language ecosystem in terms of licensing, and for libvirt-rs specifically that would probably mean MIT. Perhaps we should consider looking into relicensing the project?
Alternatively, stay away from "ecosystems" that are not compatible with copyleft. Jano
On Mon, Feb 21, 2022 at 04:30:28PM +0100, Ján Tomko wrote:
On a Monday in 2022, Andrea Bolognani wrote:
On Fri, Feb 18, 2022 at 10:28:29AM +0100, Wim de With wrote: Licensing is a pretty fun minefield :)
For example, even though libvirt-rs itself would be statically linked into any application that uses it, by virtue of it being a wrapper around the C library you'd still end up dynamically linking against that.
I tend to agree that language bindings should follow the rest of the language ecosystem in terms of licensing, and for libvirt-rs specifically that would probably mean MIT. Perhaps we should consider looking into relicensing the project?
Alternatively, stay away from "ecosystems" that are not compatible with copyleft.
libvirt itself is LGPL. Proprietary applications can link against it. It's also where all the actual logic lives. Language bindings are supposed to be "uninteresting" in that for the most part they just convert inputs and outputs between the formats understood by the two runtimes, so using a fairly liberal license for them doesn't seem like a big deal to me. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Feb 21, 2022 at 10:21:32AM -0500, Andrea Bolognani wrote:
On Fri, Feb 18, 2022 at 10:28:29AM +0100, Wim de With wrote:
Hello,
I was reading the Rust bindings for libvirt and I noticed that they are not very idiomatic. A couple of examples:
1. It is conventional to have a *-sys crate containing only the C interface and the linking configuration. The virt crate would then depend on this libvirt-sys crate to implement a safe and ergonomic wrapper. 2. The API heavily uses integers to indicate flags. Sometimes those integers are type aliased and sometimes they are not. In Rust, this can be much nicer, using enums to represent only valid flags. The same applies to some return values. 3. Since libvirt is thread-safe, the Rust types could be Send and Sync, so that they can also be used in a multithreaded context. Special care needs to be taken with translating virGetLastError to Rust Results, but it shouldn't be a problem.
For a nice example of a Rust wrapper around a C library, see: https://github.com/rust-lang/git2-rs
I am willing to send some patches to fix some of these problems, but this will almost certainly lead to breaking API changes, and I am not sure what your opinion is on that.
We are aware of the issues with the current API of libvirt-rs, but unfortunately nobody has been able to dedicate time to addressing them. Any contributions towards that goal that you or anyone else could make would certainly be greatly appreciated!
Breaking the API should be fine I think. The current version number is 0.2.11, so there shouldn't be any expectation in terms of API stability.
Note that libvirt-python is mostly autogenerated, and there is an ongoing effort to make the same happen for libvirt-go-module. Ideally libvirt-rust would also follow this trend and end up with very little manually-written code in it.
Note that the auto-generating part in Go only applies to the very low level shim in CGo, which was originally only needed to ensure serialization for accessing thread locals errors, and now is in progress of being repurposed as a dlopen layer. The actual application facing APIs remain entirely hand written today. This lets us expose APIs in a way that looks quite different from the C where we have virTypedParameter APIs, and also where we have event callbacks.
Finally, while I am not a legal expert, I believe that licensing a Rust library as LGPL does not make much sense. Rust does not have a stable ABI so everything is statically linked. This means that adding virt to a Cargo.toml file of a non-GPL application is already in violation of the license.
Yes, that is an unfortunate artifact of static linking.
Licensing is a pretty fun minefield :)
For example, even though libvirt-rs itself would be statically linked into any application that uses it, by virtue of it being a wrapper around the C library you'd still end up dynamically linking against that.
I tend to agree that language bindings should follow the rest of the language ecosystem in terms of licensing, and for libvirt-rs specifically that would probably mean MIT. Perhaps we should consider looking into relicensing the project?
Since the intent of libvirt using LGPL was explicitly to allow non-GPL compatible applications to use libvirt, it is a mistake to be using a license that breaks this ability for Rust. In Golang we've used MIT license For Rust we should aim for whatever is most appropriate - MIT or BSD or Apache - I'm not familiar with which is dominant in the Rust world. The challenge will be getting copyright holder agreement. Sahid has written the vast majority of the code, but has changed employers a few times - most code was while he was at Red Hat, but some was at Canonical. The further complication is that French legal rules often mean an employee retains copyright. To be safe we would need to cover all possiblities and get explicit agreement from Sahid, Canonical and Red Hat. I can give approval on behalf of Red Hat. Then there's a small number of other contributor who would need to be found, or their contribution removed & re-written. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Since the intent of libvirt using LGPL was explicitly to allow non-GPL compatible applications to use libvirt, it is a mistake to be using a license that breaks this ability for Rust.
In Golang we've used MIT license
For Rust we should aim for whatever is most appropriate - MIT or BSD or Apache - I'm not familiar with which is dominant in the Rust world.
Apache and MIT or even dual licensing of both are the most common. The official API guidelines recommend dual licensing under Apache and MIT.
The challenge will be getting copyright holder agreement. Sahid has written the vast majority of the code, but has changed employers a few times - most code was while he was at Red Hat, but some was at Canonical. The further complication is that French legal rules often mean an employee retains copyright. To be safe we would need to cover all possiblities and get explicit agreement from Sahid, Canonical and Red Hat. I can give approval on behalf of Red Hat.
Then there's a small number of other contributor who would need to be found, or their contribution removed & re-written.
This may sound overly ambitious but what about reimplementing libvirt-rust from scratch? I've already seen so many problems with it that I am pretty sure that incremental changes will end up replacing the entire library anyway. I can at least make a start and license my attempt under a permissive license. I'm not sure what the legal implications of that are but I'm just throwing the idea out there.
On Mon, Feb 21, 2022 at 11:53 AM Wim de With <wf@dewith.io> wrote:
Since the intent of libvirt using LGPL was explicitly to allow non-GPL compatible applications to use libvirt, it is a mistake to be using a license that breaks this ability for Rust.
In Golang we've used MIT license
For Rust we should aim for whatever is most appropriate - MIT or BSD or Apache - I'm not familiar with which is dominant in the Rust world.
Apache and MIT or even dual licensing of both are the most common. The official API guidelines recommend dual licensing under Apache and MIT.
I would prefer we didn't repeat that dumb advice in libvirt-rs. Just go with Apache-2.0 if we want a permissively licensed crate. I would suggest MPL-2.0 for libvirt-rs, though. That allows proprietary linking but maintains that each file that makes up the crate *must* remain FOSS, and is compatible with GNU licenses. It's static-link compatible copyleft, basically. -- 真実はいつも一つ！/ Always, there's only one truth!
On Mon, Feb 21, 2022 at 10:21:32AM -0500, Andrea Bolognani wrote:
We are aware of the issues with the current API of libvirt-rs, but unfortunately nobody has been able to dedicate time to addressing them. Any contributions towards that goal that you or anyone else could make would certainly be greatly appreciated!
Breaking the API should be fine I think. The current version number is 0.2.11, so there shouldn't be any expectation in terms of API stability.
I realized that GitLab is nowadays used for communication so I made more specific issues there. Do you prefer merge requests there or patches on the mailing list?
Note that libvirt-python is mostly autogenerated, and there is an ongoing effort to make the same happen for libvirt-go-module. Ideally libvirt-rust would also follow this trend and end up with very little manually-written code in it.
I've done some experiments here with bindgen, and it is pretty straightforward to generate FFI bindings using it. The problem is that generating safe idiomatic Rust wrappers on top of these bindings is non-trivial. Still, most wrapper functions will probably follow the same pattern, so it may be useful to investigate if macros could be used to make it easier to add new functions.
From looking at the Git diff from v2.5.0 to v8.0.0, the libvirt API doesn't seem to change that often. Almost all changes consist of adding new enum variants and some new functions. Am I correct in assuming that an application that was written for v2.5.0 would still work with v8.0.0?
To me it seems the best course of action would be to pick some minimum version of libvirt to support, and make sure that the Rust API wraps all functions there. From that point, adding functions and enum variants introduced in later versions of libvirt can be incremental changes and will (hopefully) be fairly easy to add. Rust has the concept of features to enable conditional compiling of those functions and enum variants only when the underlying libvirt supports them. Wim
On Mon, Feb 21, 2022 at 05:28:24PM +0100, Wim de With wrote:
On Mon, Feb 21, 2022 at 10:21:32AM -0500, Andrea Bolognani wrote:
We are aware of the issues with the current API of libvirt-rs, but unfortunately nobody has been able to dedicate time to addressing them. Any contributions towards that goal that you or anyone else could make would certainly be greatly appreciated!
Breaking the API should be fine I think. The current version number is 0.2.11, so there shouldn't be any expectation in terms of API stability.
I realized that GitLab is nowadays used for communication so I made more specific issues there. Do you prefer merge requests there or patches on the mailing list?
Merge requests exclusively for anything other than libvirt.git
Note that libvirt-python is mostly autogenerated, and there is an ongoing effort to make the same happen for libvirt-go-module. Ideally libvirt-rust would also follow this trend and end up with very little manually-written code in it.
I've done some experiments here with bindgen, and it is pretty straightforward to generate FFI bindings using it. The problem is that generating safe idiomatic Rust wrappers on top of these bindings is non-trivial. Still, most wrapper functions will probably follow the same pattern, so it may be useful to investigate if macros could be used to make it easier to add new functions.
From looking at the Git diff from v2.5.0 to v8.0.0, the libvirt API doesn't seem to change that often. Almost all changes consist of adding new enum variants and some new functions. Am I correct in assuming that an application that was written for v2.5.0 would still work with v8.0.0?
Correct, libvirt never knowingly breaks API compatibility. The API is strictly append-only.
To me it seems the best course of action would be to pick some minimum version of libvirt to support, and make sure that the Rust API wraps all functions there. From that point, adding functions and enum variants introduced in later versions of libvirt can be incremental changes and will (hopefully) be fairly easy to add. Rust has the concept of features to enable conditional compiling of those functions and enum variants only when the underlying libvirt supports them.
Conditional compilation is what we do in Go / Python bindings too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (5)
-
Andrea Bolognani -
Daniel P. Berrangé -
Ján Tomko -
Neal Gompa -
Wim de With