10:34 a.m.
I'd imagine most developers on this list have their own workflows to
track/apply/test patches from the mailing list. I normally just save
patches in Mutt's Maildir and apply them manually to a Git branch. I
recently began using the 'patches'[1] tool, that some of you on this
list must be familiar with, to test arbitrary QEMU patches from the
mailing list, and found it relatively easy large patch series from
mailing list.
I'm writing this to check if the libvirt upstream community finds it
desirable to setup such a patches database.
What is 'patches'?
------------------
Quoting the README[2]:
"patches is a patch tracking system. It consists of two parts: a set
of commands that build a database of patches from a mailing list and
then a set of commands that can search that database. It supports
the following features:
- Tracking patch status by determining which patches are already
committed, have newer versions posted, are RFC, etc.
- Applying patches or pull requests with a single command.
- Searching for patches using a rich query language."
A quick workflow to test patches from mailing list
---------------------------------------------------
FWIW, this is my rudimentary use-case.
Clone the repo, and fetch the patches database (may take a few minutes):
$ git clone https://github.com/stefanha/patches.git
$ cd patches
$ ./patches fetch http://wiki.qemu.org/patches/patches.json
And, suppose, I want to test Dan's LUKS encryption support (26) patch series, I
go find the message ID:
$ ./patches list | grep -B5 "Support LUKS encryption"
Message-id: 1456747261-22032-1-git-send-email-berrange(a)redhat.com
From: Daniel P. Berrange <berrange(a)redhat.com>
Date: 2016-02-29
Tags: v4
[0/26] Support LUKS encryption in block devices
Trivially apply it (assuming you have a test branch):
$ ./patches apply --git-dir ~/src/qemu/ \
id:1456747261-22032-1-git-send-email-berrange@redhat.com
Applying: crypto: add cryptographic random byte source
Applying: crypto: add support for PBKDF2 algorithm
Applying: crypto: add support for generating initialization vectors
Applying: crypto: add support for anti-forensic split algorithm
Applying: crypto: skip testing of unsupported cipher algorithms
[...]
Test away!
Rinse, repeat:
"The `fetch` command should be run whenever you want to refresh the
patch database; it will download any new patches that have arrived
since the last time you did a fetch."
Here's[3] the documentation to setup the patches database.
[1] https://github.com/stefanha/patches/tree/stefanha-tweaks
[2] https://github.com/stefanha/patches/blob/stefanha-tweaks/README.md
[3] https://github.com/stefanha/patches/blob/stefanha-tweaks/README-server.md
--
/kashyap
11:09 a.m.
On 03/23/2016 11:34 AM, Kashyap Chamarthy wrote:
I'd imagine most developers on this list have their own workflows
to
track/apply/test patches from the mailing list. I normally just save
patches in Mutt's Maildir and apply them manually to a Git branch. I
recently began using the 'patches'[1] tool, that some of you on this
list must be familiar with, to test arbitrary QEMU patches from the
mailing list, and found it relatively easy large patch series from
mailing list.
I'm writing this to check if the libvirt upstream community finds it
desirable to setup such a patches database.
I haven't played much with 'patches' so I can't give a first hand opinion.
But
if it's low effort to set up and maintain seems like a no-brainer to generate
and publish the metadata so libvirt devs can start playing with it.
Although your details don't cover it, one of the major benefits of 'patches'
is that it also scoops up metadata like Reviewed-by:, and gives an easy way to
get those tags into commit messages. Good for giving prolific reviewers credit
(and shaming those that don't do enough reviews :) ).
The corollary is that you can search for unreviewed series, which IMO is
desperately needed for libvir-list; the review time is pretty bad and I can't
begin to count the number of drive-by patches we've received on list over the
years that go completely unresponded. And I say this knowing full well I
probably have the worst review-to-patches-authored ratio of all libvirt
committers... but I don't live on libvir-list and often go 3-6 months at a
time where I don't have any libvirt patches, and usually during those times I
just skim-and-archive the list contents. If we had a programmatic tool to show
me the unreviewed patches, I could run that on occasion and pitch in much more
easily, at least reviewing bits that don't need a lot of context. We could
even have an informal policy like 'theres XXX number of unreviewed patches on
the list... no @redhat people post patches until we review the backlog'
What if a patch series is NACKd or abandoned, is there a way to get it out of
the patches metadata?
- Cole
11:38 a.m.
On Wed, Mar 23, 2016 at 12:09:29PM -0400, Cole Robinson wrote:
On 03/23/2016 11:34 AM, Kashyap Chamarthy wrote:
> I'd imagine most developers on this list have their own workflows to
> track/apply/test patches from the mailing list. I normally just save
> patches in Mutt's Maildir and apply them manually to a Git branch. I
> recently began using the 'patches'[1] tool, that some of you on this
> list must be familiar with, to test arbitrary QEMU patches from the
> mailing list, and found it relatively easy large patch series from
> mailing list.
s/easy/easy to test/
> I'm writing this to check if the libvirt upstream community
finds it
> desirable to setup such a patches database.
>
I haven't played much with 'patches' so I can't give a first hand
opinion. But if it's low effort to set up and maintain seems like a
no-brainer to generate and publish the metadata so libvirt devs can
start playing with it.
I haven't set up the database myself, I'm afraid. From my quick look at
patches/patchlib/scan.py, it uses a `notmuch` database (an .xz
conpressed file), and I just need to have a config file. I use
`notmuch`[*] for email indexing sometimes, so I can try to play with
this and setup a database.
I was just a bit hesitant to start as I'll be away on leave for 2 weeks
starting this weekend. Still, I'll try to see if I can squeeze this
task in before I leave, since I brought it up.
Although your details don't cover it, one of the major benefits
of
'patches' is that it also scoops up metadata like Reviewed-by:, and
gives an easy way to get those tags into commit messages. Good for
giving prolific reviewers credit (and shaming those that don't do
enough reviews :) ).
:-) Understood. You can see the kind of prefixes it supports in the
README.md.
The corollary is that you can search for unreviewed series, which
IMO
is desperately needed for libvir-list; the review time is pretty bad
and I can't begin to count the number of drive-by patches we've
received on list over the years that go completely unresponded. And I
say this knowing full well I probably have the worst
review-to-patches-authored ratio of all libvirt committers... but I
don't live on libvir-list and often go 3-6 months at a time where I
don't have any libvirt patches, and usually during those times I just
skim-and-archive the list contents. If we had a programmatic tool to
show me the unreviewed patches,
It shows something closer to unreviewed patches, looking at the terms
supported by its query language (using `notmuch`; see the "Search
Language" section in README.md):
[...]
- `status:rfc` show RFC postings
- `status:committed` show committed series
- `status:applied` show series with a "Thanks, applied" reply
- `status:unapplied` short hand for `not (status:broken or status:obsolete or
status:pull-request or status:rfc or status:committed or status:applied)`
[...]
- `status:reviewed` show series where every patch has at least one Reviewed-by
- `reviewed-by:ADDRESS` show series if a patch has a Reviewed-by by `ADDRESS`
I could run that on occasion and pitch
in much more easily, at least reviewing bits that don't need a lot of
context. We could even have an informal policy like 'theres XXX number
of unreviewed patches on the list... no @redhat people post patches
until we review the backlog'
What if a patch series is NACKd or abandoned, is there a way to get it
out of the patches metadata?
Seems like NACKed patches are displayed (I don't find any search term
for "abandoned") can be fished out of the repo:
[...] # I snipped out some of prefixes
- `nacked-by:ADDRESS` show series if a patch has a Nacked-by by `ADDRESS`
- `acked-by:ADDRESS` show series if a patch has a Acked-by by `ADDRESS`
[*] https://notmuchmail.org/
--
/kashyap
11:40 a.m.
On Wed, Mar 23, 2016 at 04:34:26PM +0100, Kashyap Chamarthy wrote:
I'd imagine most developers on this list have their own workflows
to
track/apply/test patches from the mailing list. I normally just save
patches in Mutt's Maildir and apply them manually to a Git branch. I
recently began using the 'patches'[1] tool, that some of you on this
list must be familiar with, to test arbitrary QEMU patches from the
mailing list, and found it relatively easy large patch series from
mailing list.
I'm writing this to check if the libvirt upstream community finds it
desirable to setup such a patches database.
It would probably be useful. I'll see if I can setup a cron job on
the libvirt.org server to publish it and we can give it a try.
If it works out, we might want to get a bit more formal in giving
Reviewed-by lines, etc, rather than the generic ACK
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
1 p.m.
On Wed, Mar 23, 2016 at 04:40:48PM +0000, Daniel P. Berrange wrote:
On Wed, Mar 23, 2016 at 04:34:26PM +0100, Kashyap Chamarthy wrote:
> I'd imagine most developers on this list have their own workflows to
> track/apply/test patches from the mailing list. I normally just save
> patches in Mutt's Maildir and apply them manually to a Git branch. I
> recently began using the 'patches'[1] tool, that some of you on this
> list must be familiar with, to test arbitrary QEMU patches from the
> mailing list, and found it relatively easy large patch series from
> mailing list.
>
> I'm writing this to check if the libvirt upstream community finds it
> desirable to setup such a patches database.
It would probably be useful. I'll see if I can setup a cron job on
the libvirt.org server to publish it and we can give it a try.
That'd be nice, if you have time. Indeed, the intention was to
just try, informally.
If it works out, we might want to get a bit more formal in giving
Reviewed-by lines, etc, rather than the generic ACK
Yep.
--
/kashyap
1:46 p.m.
On Wed, Mar 23, 2016 at 04:34:26PM +0100, Kashyap Chamarthy wrote:
[snip]
So things turn out to be a bit more complicated for libvirt than for
QEMU, because we have one mailing list accepting patches for 10+
different git repositories. Patches has no built-in support for this
model, so you have to create a separate configuration file for each
repo, so that it stores the patches json file separately for each
and then pass that config file when fetching
git clone git://github.com/stefanha/patches.git
REPOS="libvirt libvirt-designer libvirt-glib libvirt-java
libvirt-perl libvirt-php libvirt-python libvirt-sandbox
libvirt-snmp libvirt-tck"
for repo in $REPOS
do
cat > $HOME/.patchesrc-$repo <<EOF
[options]
patches_dir=$HOME/.patches/$repo
[fetch]
url=http://libvirt.org/patches/$repo/patches.json
EOF
cat > $HOME/bin/patchesrc-$repo <<EOF
#!/bin/sh
exec $HOME/patches/patches --config $HOME/.patchesrc-$repo \$@
EOF
chmod +x $HOME/bin/patchesrc-$repo
done
Now, assuming $HOME/bin is in your $PATH, you can run
$ patches-libvirt fetch
$ patches-libvirt list
Or to look at other repos, eg sandbox
$ patches-libvirt-sandbox fetch
$ patches-libvirt-sandbox list
patchesrc-libvirt-sandbox list
Message-id: 20160323074705.GA4974(a)bogon.m.sigxcpu.org
From: Guido Günther <agx(a)sigxcpu.org>
Date: 2016-03-23
[1/1] Ignore generated file
$ patches-libvirt-sandbox apply id:20160323074705.GA4974@bogon.m.sigxcpu.org
nb assumes your $CWD is the libvirt-sandbox git checkout
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3:31 a.m.
On Wed, Mar 23, 2016 at 06:46:53PM +0000, Daniel P. Berrange wrote:
On Wed, Mar 23, 2016 at 04:34:26PM +0100, Kashyap Chamarthy wrote:
[snip]
So things turn out to be a bit more complicated for libvirt than for
QEMU, because we have one mailing list accepting patches for 10+
different git repositories.
Ah-ha, good point.
Patches has no built-in support for this
model, so you have to create a separate configuration file for each
repo, so that it stores the patches json file separately for each
and then pass that config file when fetching
git clone git://github.com/stefanha/patches.git
REPOS="libvirt libvirt-designer libvirt-glib libvirt-java
libvirt-perl libvirt-php libvirt-python libvirt-sandbox
libvirt-snmp libvirt-tck"
for repo in $REPOS
do
cat > $HOME/.patchesrc-$repo <<EOF
[options]
patches_dir=$HOME/.patches/$repo
[fetch]
url=http://libvirt.org/patches/$repo/patches.json
EOF
Yeah, this nicely solves it currently, thanks for clearly spelling it
out.
How often are the JSON databases updated? Once daily at least, I
presume.
cat > $HOME/bin/patchesrc-$repo <<EOF
#!/bin/sh
exec $HOME/patches/patches --config $HOME/.patchesrc-$repo \$@
EOF
chmod +x $HOME/bin/patchesrc-$repo
done
Now, assuming $HOME/bin is in your $PATH, you can run
$ patches-libvirt fetch
$ patches-libvirt list
Or to look at other repos, eg sandbox
$ patches-libvirt-sandbox fetch
$ patches-libvirt-sandbox list
patchesrc-libvirt-sandbox list
Message-id: 20160323074705.GA4974(a)bogon.m.sigxcpu.org
From: Guido Günther <agx(a)sigxcpu.org>
Date: 2016-03-23
[1/1] Ignore generated file
$ patches-libvirt-sandbox apply id:20160323074705.GA4974@bogon.m.sigxcpu.org
nb assumes your $CWD is the libvirt-sandbox git checkout
Excellent, I tested with just the libvirt repo, works perfectly well.
Thanks for the quick action!
--
/kashyap
5:38 a.m.
On Thu, Mar 24, 2016 at 09:31:44AM +0100, Kashyap Chamarthy wrote:
On Wed, Mar 23, 2016 at 06:46:53PM +0000, Daniel P. Berrange wrote:
> On Wed, Mar 23, 2016 at 04:34:26PM +0100, Kashyap Chamarthy wrote:
> [snip]
>
> So things turn out to be a bit more complicated for libvirt than for
> QEMU, because we have one mailing list accepting patches for 10+
> different git repositories.
Ah-ha, good point.
> Patches has no built-in support for this
> model, so you have to create a separate configuration file for each
> repo, so that it stores the patches json file separately for each
> and then pass that config file when fetching
>
> git clone git://github.com/stefanha/patches.git
>
> REPOS="libvirt libvirt-designer libvirt-glib libvirt-java
> libvirt-perl libvirt-php libvirt-python libvirt-sandbox
> libvirt-snmp libvirt-tck"
> for repo in $REPOS
> do
> cat > $HOME/.patchesrc-$repo <<EOF
> [options]
> patches_dir=$HOME/.patches/$repo
>
> [fetch]
> url=http://libvirt.org/patches/$repo/patches.json
> EOF
Yeah, this nicely solves it currently, thanks for clearly spelling it
out.
How often are the JSON databases updated? Once daily at least, I
presume.
Once an hour
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3146
days inactive
3152
days old
7 comments
3 participants
participants (3)
-
Cole Robinson -
Daniel P. Berrange -
Kashyap Chamarthy