On 10/10/2012 05:49 PM, Eric Blake wrote:

On 10/10/2012 08:20 AM, Corey Bryant wrote:

...

The first call to add an fd to an fd set was previously not allowed to choose the fd set ID. The ID was generated as the first available and ensuing calls could add more fds by specifying the fd set ID. This change allows users to choose the fd set ID on the first call.

Unfortunately, it now allows the user to choose arbitrary integer set ids with large gaps, where previously, the user could only influence set ids by populating all intermediate ids. That is, before this patch, a user would have to create 1000000 sets to have an id of 1000000 (if they didn't run out of memory first on all the earlier sets), but now they can have an id that large with just one set. Or, taken further, previously, a user request of -9223372036854775808 would likely fail (if not, how beefy is your machine?), but now it can succeed and cause confusion because of integer wraparound. Arbitrary set ids is not necessarily bad, but I think you need to add bounds-checking on the user's requested fdset_id to make sure it is positive.

I agree. I'll add some set ID bounds checking in v3. -- Regards, Corey Bryant

On 10/10/2012 08:20 AM, Corey Bryant wrote:

qmp_add_fd() gets an fd that was received over a socket with SCM_RIGHTS and adds it to an fd set. This patch adds support that will enable adding an fd that was inherited on the command line to an fd set.

This patch also prevents removal of an fd from an fd set during initialization. This allows the fd to remain in the fd set after probing of the image file.

You're mixing code motion and semantic change in one patch (keyword "also" in your commit message; should the semantic change be moved to a separate patch (possibly squashed into 1/3)?

Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com> ---

- if (has_fdset_id) { - QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { - if (mon_fdset->id == fdset_id) { - break; - } - } - }

Since you maintain mon_fdsets in sorted id order, should you optimize this loop to abort the QLIST_FOREACH early if fdset_id is less than mon_fdset->id? [I only noticed this because of code motion, so it is a pre-existing condition and therefore a separate patch, or another thing to squash into 1/3] -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

Am 10.10.2012 16:20, schrieb Corey Bryant:

qmp_add_fd() gets an fd that was received over a socket with SCM_RIGHTS and adds it to an fd set. This patch adds support that will enable adding an fd that was inherited on the command line to an fd set.

This patch also prevents removal of an fd from an fd set during initialization. This allows the fd to remain in the fd set after probing of the image file.

"This patch also..." usually means that it should be split in two patches. Though in this case I'd vote for immediately dropping the second patch again: This makes the probing work with file descriptors using a hack for a certain situation (namely qemu startup) and leaves other cases (like hotplug) broken. I think it's kind of acceptable to leave it broken for both cases in this patch series, you can work around it by passing the 'format' option. If we do fix it, however, we should fix it consistently for all use cases. The best approach for this is probably to avoid opening the image file twice for probing; instead, we should first open the BlockDriverState for raw-posix, use that for probing and then put a second qcow2 or whatever BlockDriverState on top of the very same BDS without reopening it. The rest is code motion, except for: - if (mon_fdset->id == 0) { + if (!mon_fdset_cur) { It's not nice to hide such changes in code motion patches, but the change is obviously not wrong. The commit message should be changed to tell that this is (mostly) code motion. This doesn't really change anything semantically (except for the probing part). Kevin

Am 11.10.2012 17:04, schrieb Corey Bryant:

On 10/11/2012 07:25 AM, Kevin Wolf wrote:

...

Am 10.10.2012 16:20, schrieb Corey Bryant:

...

qmp_add_fd() gets an fd that was received over a socket with SCM_RIGHTS and adds it to an fd set. This patch adds support that will enable adding an fd that was inherited on the command line to an fd set.

This patch also prevents removal of an fd from an fd set during initialization. This allows the fd to remain in the fd set after probing of the image file.

"This patch also..." usually means that it should be split in two patches. Though in this case I'd vote for immediately dropping the second patch again: This makes the probing work with file descriptors using a hack for a certain situation (namely qemu startup) and leaves other cases (like hotplug) broken.

I don't think hotplug is broken. In that case the fd will only be removed from the fd set if the following is true:

(mon_fdset_fd->removed || (QLIST_EMPTY(&mon_fdset->dup_fds) && mon_refcount == 0))

We can ignore the removed part for now. What's important here is that if there are no dup_fd references and there is at least one monitor connected, an fd will *not* be removed.

Ah yes, that's the part I missed. Then your approach of special-casing the command line is probably okay, though I'd still want to change the probing mechanism to avoid the reopen. Seems I need to find a better excuse to make someone do it. Meh. ;-) Kevin

On 10/10/2012 08:20 AM, Corey Bryant wrote:

This option can be used for passing file descriptors on the command line. It mirrors the existing add-fd QMP command which allows an fd to be passed to QEMU via SCM_RIGHTS and added to an fd set.

This can be combined with commands such as -drive to link file descriptors in an fd set to a drive:

qemu-kvm -add-fd fd=4,set=2,opaque="rdwr:/path/to/file" -add-fd fd=5,set=2,opaque="rdonly:/path/to/file" -drive file=/dev/fdset/2,index=0,media=disk

This example adds fds 4 and 5, and the accompanying opaque strings to the fd set with ID=2. qemu_open() already knows how to handle a filename of this format. qemu_open() searches the corresponding fd set for an fd and when it finds a match, QEMU goes on to use a dup of that fd just like it would have used an fd that it opened itself.

Missing some argument validation. Earlier, I complained about set validation, now I'm going to complain about fd validation.

+static int parse_add_fd(QemuOpts *opts, void *opaque) +{ + int fd; + int64_t fdset_id; + const char *fd_opaque = NULL; + + fd = qemu_opt_get_number(opts, "fd", -1); + fdset_id = qemu_opt_get_number(opts, "set", -1); + fd_opaque = qemu_opt_get(opts, "opaque");

If I call 'qemu -add-fd fd=-2,set=1', that had better fail because fd -2 does not exist. Likewise if I call 'qemu -add-fd fd=10000000,set=1' (here, picking an fd that I know can't be opened). More subtly, 'qemu -add-fd fd=4,set=1 4<&-' should fail, since fd 4 was not inherited down to the qemu process. Fuzzier is whether 'qemu -add-fd fd=0,set=1' makes sense, as that would then make stdin be competing for multiple uses; this would be a situation that the monitor command can't duplicate, so you have introduced new ways to possibly break things from the command line. I'm thinking it's safest to reject fd of 0, 1, or 2 for now, and only relax it later IF we can prove that it would be both useful and safe. So it sounds like you need something like fcntl(fd,F_GETFL) to see that an the fd was actually inherited, as well as validating that fd > STDERR_FILENO. Another missing validation check is for duplicate use. With the monitor command, you ALWAYS have a unique fd (thanks to SCM_RIGHTS). But with the command line, I can type 'qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2'. Oops - I've now corrupted your set layout, unless you validate that every fd requested in -add-fd does not already reside in any existing set. Thinking aloud: On the other hand, being able to pass in one fd to multiple sets MIGHT be useful; in the SCM_RIGHTS monitor command case, I can pass the same fd from the management perspective into multiple sets, even though in qemu's perspective, there will be multiple fds created (one per call). Perhaps instead of directly adding the inherited fd to a set, and having to then sweep all sets to check for duplicates, it might make sense to add dup(fd) to a set, so that if I call: qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2 -add-fd fd=5,set=2 what REALLY happens is that qemu adds dup(4)==6 to set 1, dup(4)==7 to set 2, and dup(5)==8 to set 3. Then, after all ALL -add-fd have been processed, qemu then does another pass through them calling close(4) and close(5) (to avoid holding the original fds open indefinitely if the corresponding sets are discarded). Hmm, notice that if you dup() before adding to a set, then that the dup() resolves my first complaint of validating that the inherited fd exists; it also changes the problem of dealing with fd 0 (it would now be valid to add fd 0 to any number of sets; but a final cleanup loop had better be careful to not call close(0) unintentionally). Personally, though, I'm not sure the complexity of using dup() buys us anything, so I'm happy with the simpler solution of using fd as-is, coupled with a check for no reuse of an fd already in a set. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On 10/11/2012 08:45 AM, Corey Bryant wrote:

...

Another missing validation check is for duplicate use. With the monitor command, you ALWAYS have a unique fd (thanks to SCM_RIGHTS). But with the command line, I can type 'qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2'. Oops - I've now corrupted your set layout, unless you validate that every fd requested in -add-fd does not already reside in any existing set.

I don't see this validation check for duplicate use of fd's being necessary. Like you say below, in the QMP add-fd case we can add the same fd multiple times. So we should be able to add the same fd multiple times via the command line. The only difference between QMP and command line in this case is that the QMP fd is a dup and therefore a different number and the command line fd will be the same fd. I'd prefer to leave this alone unless there's a compelling reason to block adding of the same fd.

There is a compelling reason to prevent duplicates among your sets: qemu_close(). Suppose I add fd 4 into set 1 and 2, and then discard set 2 via monitor commands. Then, when qemu_close() drops the last reference to set 2, it steps through and calls close() on all fds in that set, including fd 4. Oops - now set 1 is invalid, because it is tracking a closed fd. And worse, if qemu then does something else to open a new fd, it will get fd 4 again, and now set 1 will be tracking the WRONG fd. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On 10/10/2012 04:31 PM, Eric Blake wrote:

Another missing validation check is for duplicate use. With the monitor command, you ALWAYS have a unique fd (thanks to SCM_RIGHTS). But with the command line, I can type 'qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2'. Oops - I've now corrupted your set layout, unless you validate that every fd requested in -add-fd does not already reside in any existing set.

Thinking aloud: On the other hand, being able to pass in one fd to multiple sets MIGHT be useful; in the SCM_RIGHTS monitor command case, I can pass the same fd from the management perspective into multiple sets, even though in qemu's perspective, there will be multiple fds created (one per call). Perhaps instead of directly adding the inherited fd to a set, and having to then sweep all sets to check for duplicates, it might make sense to add dup(fd) to a set, so that if I call:

qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2 -add-fd fd=5,set=2

what REALLY happens is that qemu adds dup(4)==6 to set 1, dup(4)==7 to set 2, and dup(5)==8 to set 3. Then, after all ALL -add-fd have been processed, qemu then does another pass through them calling close(4) and close(5) (to avoid holding the original fds open indefinitely if the corresponding sets are discarded).

Another idea: a hybrid approach - the _first_ -add-fd 4 directly adds 4 to the set, all other -add-fd 4 end up adding dup(4) instead (well, fcntl(F_DUPFD_CLOEXEC), but you get the picture). That is, do the duplicate scanning, and if there is no duplicate, use the fd directly; if there IS a duplicate, then put a unique fd number as a copy into the remaining sets. That way, you don't have to do a final close() sweep across the -add-fd arguments passed on the command line, and you still don't have to worry about duplicated fds across multiple sets causing mayhem in qemu_close(). -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On 10/11/2012 12:04 PM, Eric Blake wrote:

On 10/10/2012 04:31 PM, Eric Blake wrote:

...

Another missing validation check is for duplicate use. With the monitor command, you ALWAYS have a unique fd (thanks to SCM_RIGHTS). But with the command line, I can type 'qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2'. Oops - I've now corrupted your set layout, unless you validate that every fd requested in -add-fd does not already reside in any existing set.

Thinking aloud: On the other hand, being able to pass in one fd to multiple sets MIGHT be useful; in the SCM_RIGHTS monitor command case, I can pass the same fd from the management perspective into multiple sets, even though in qemu's perspective, there will be multiple fds created (one per call). Perhaps instead of directly adding the inherited fd to a set, and having to then sweep all sets to check for duplicates, it might make sense to add dup(fd) to a set, so that if I call:

qemu -add-fd fd=4,set=1 -add-fd fd=4,set=2 -add-fd fd=5,set=2

what REALLY happens is that qemu adds dup(4)==6 to set 1, dup(4)==7 to set 2, and dup(5)==8 to set 3. Then, after all ALL -add-fd have been processed, qemu then does another pass through them calling close(4) and close(5) (to avoid holding the original fds open indefinitely if the corresponding sets are discarded).

Another idea: a hybrid approach - the _first_ -add-fd 4 directly adds 4 to the set, all other -add-fd 4 end up adding dup(4) instead (well, fcntl(F_DUPFD_CLOEXEC), but you get the picture). That is, do the duplicate scanning, and if there is no duplicate, use the fd directly; if there IS a duplicate, then put a unique fd number as a copy into the remaining sets. That way, you don't have to do a final close() sweep across the -add-fd arguments passed on the command line, and you still don't have to worry about duplicated fds across multiple sets causing mayhem in qemu_close().

This would simplify the code, but I wonder if it would be confusing to users when they call query-fdsets and only see a single fd 4. It may make more sense to dup all fds that are passed with -add-fd, and then it basically works the same as the QMP add-fd via SCM_RIGHTS. On a somewhat related note, one major difference between the QMP add-fd and command line -add-fd, is that -add-fd doesn't return the fd that was added. So opaque strings will be even more important when passing fds on the command line. -- Regards, Corey Bryant