[petsc-dev] private branch of petsc-dev (fwd)

[Jed Brown]

On Fri, Feb 8, 2013 at 7:48 PM, Satish Balay <balay at mcs.anl.gov> wrote:

>
> > > - *never* sync with petsc-dev until this work is done.
> > >
> > > - merge with petsc-dev when its done. [either merge - or rebase it
> over]
> > >
> > > The *never* sync with petsc-dev is a drawback for this.
> >
> > You could just `hg pull --rebase`? Unless you mean don't push to
> > petsc-dev, then yes, don't push to petsc-dev until the feature is done.
>
> 'hg pull --rebase' would be great for a single developer of the
> feature.  But with 2 developers commiting/communicating commits - one
> would have to 'strip out the old changes' - as you put it - either
> manually or with mercurial tools - and then propogate this to remote
> repo aswell.  So its easy to make mistakes in the workflow?
>

A feature like MINRESQLP should be able to go a very long time before
conflicting. It's usually considered bad practice to constantly merge
(because most merge commits don't convey useful content), but it's also
useful to know when something will automerge successfully and sometimes we
want to test with all known patches applied. In git, I use a personal
testing branch for this (jed/test in the discussion below). I explain my
git workflow below. It's mildly advanced, but hopefully explains how to
achieve

*1.* Many people can work independently with minimal interruptions.
*2.* Every commit that eventually appears in a release (to be tagged from
'master') is semantically significant and preserves the integrity and
testing performed the first time it is committed with intent to publish.
(You can rework locally as much as you want, but the SHA1 is indelible once
you decide that a commit is complete.) This implies zero content-free
merges and zero rebases for topic branches.
*3.* Every commit can be reviewed before it graduates to 'master'.
*4.* The process can be applied recursively and scales naturally to an
arbitrary number of contributors working on an arbitrary number of projects.

$ cd the-repo && git checkout master # 'master' is the default branch
(master) $ git pull --ff-only

Comments:
1. The --ff-only option tells 'pull' to abort if I have any changes in
'master' that are not already upstream (not fast-forward). I use it here
only for explicitness.
2. Since I put __git_ps1 in my bash prompt, I see the current branch (and
sometimes other useful info).

I always keep my 'master' branch clean. I use it to test the upstream
version of the code and to apply isolated bug fixes. Those look like the
following (though I usually commit from emacs using magit)

(master) $ emacs broken-file.c makefile
(master) $ make runex42_3 DIFF=diffupdate # my script to refresh output
(master) $ git add output/ex42_3.out # new file
(master) $ git commit -am'Fix bug X introduced in <SHA1>, add runex42_3'
(master) $ git push

If there is a conflict here, I just

(master) $ git pull --rebase
(master) $ git push

because this spot bug-fix doesn't have any useful history that I can
maintain. Now let's consider a more interesting feature

(master) $ git checkout -b jed/feature1 # create a branch and check it out
(jed/feature1) $ emacs file-to-modify.c && git commit -am'implement first
part of feature1'

and another

(jed/feature1) $ git checkout -b jed/fetaure2 master # base this on
'master' because it is independent of feature1
(jed/feature2) $ emacs file-to-modify.c # oops, same file, and some changes
conflict
(jed/feature2) $ git commit -am'first part of feature2, I might not know
that it conflicts with feature1'

Now I want to see what's happening upstream:

$ git checkout master
(master) $ git pull

and to find out how my patches will appear if merged into upstream:

(master) $ git checkout -b jed/test
(jed/test) $ git merge jed/feature1 # applies cleanly, cool
(jed/test) $ git merge jed/feature2 # conflicts
Auto-merging file-to-modify.c
CONFLICT (content): Merge conflict in file-to-modify.c
*Recorded preimage for 'file-to-modify.c'*
Automatic merge failed; fix conflicts and then commit the result.

(I'll explain the bold part later.) I will resolve this merge and commit
the result:

(jed/test) $ git commit -a -m 'test resolution'
*Recorded resolution for 'file-to-modify.c'.*
[detached HEAD 2fe010b] test resolution

Then test in the 'jed/test' branch. Maybe I build other applications that
will be users of feature1 and feature2. Now I go back to working on
feature1:

$ git checkout jed/feature1
(jed/feature1) $ emacs && make all alltests
(jed/feature1) $ git commit -am'finish implementing feature1, add tests,
etc'

Now I'm ready to publish feature1, so I

(master) $ git pull
(master) $ git merge jed/feature1 && make alltests
(master) $ git push

Now I work on feature2 a bit more and integrate all the latest

(jed/test) $ git merge master # get upstream stuff
(jed/test) $ git merge jed/feature2 # get new feature2 stuff

then go back to feature2 and finish up

(jed/feature2) $ emacs && make all alltests
(jed/feature2) $ git commit -am'finish implementing feature2, add tests,
etc'

and now it's time to publish

(master) $ git merge jed/feature2
Auto-merging file-to-modify.c
CONFLICT (content): Merge conflict in file-to-modify.c
*Resolved 'file-to-modify.c' using previous resolution.*
Automatic merge failed; fix conflicts and then commit the result.

*Explanation of the bold:*
So what happened here? We resolved that conflict in jed/test, but jed/test
is messy and may have lots of incomplete stuff in it. We never merge
jed/test anywhere and we might throw it away at any time. However, since we
resolved this conflict before (even though it was in a different branch
that we won't use now), git can REuse the REcorded REsolution. Git tells me
there was a merge conflict and doesn't auto-commit the merge (I need to run
'git commit' to complete the process), but I don't have to manually resolve
this time. (The fact that Hg has no equivalent functionality perplexes me,
but explains why Hg projects with many contributors are always cluttered
with meaningless merges.)

I find this mechanism super convenient, but it's disabled by default,
presumably because it can feel like magic. Enable it with

$ git config rerere.enabled true

or globally with

$ git config --global rerere.enabled true

As a user, the effect is that I only ever need to resolve conflicts once.
Git remembers that resolution and will reuse it anywhere that I merge
similar content. There is technical documentation for the feature, but I
used it for years before reading these:

http://git-scm.com/2010/03/08/rerere.html
http://www.kernel.org/pub/software/scm/git/docs/git-rerere.html
http://gitster.livejournal.com/41795.html


This "topic branch" workflow lets me have long-lived feature branches that
have linear history (no merges from upstream into the feature branch), but
in which the combined state is frequently tested and the eventual merge to
'master' is painless.

If something shows up in 'master' that I need to complete my work on
feature3, then I make an explicit merge

(jed/feature3) $ git merge master # explain what functionality I'm
obtaining in commit message

This merge is semantically important and unavoidable so there is no reason
to try to avoid it. We're only trying to avoid the meaningless merges "some
other stuff happened, don't know if it affects me so I'll merge it".

Note that it's fine to have many people working on the same topic branch.
When you push to a topic branch, you are "publishing" to _those_ people. If
that group is sizable, you'll still have a personal branch that you use to
manage and test your patches before publishing to that topic branch. (Just
apply the workflow above recursively, with 'master' replaced by
'topic-branch-with-several-people'.)

*Throw-away integration branches*
If many people are working concurrently, it's sometimes useful to have a
snapshot of what a merge would look like with everyone's latest (still
incomplete) work. In the kernel/git workflow, that is provided by a branch
they call 'pu' (Proposed Updates). It is blown away and rebuilt
periodically, but then we can easily check whether our new stuff conflicts
with anything anyone else is working on by running

(jed/feature4) $ git checkout -b jed/test-pu # create temporary branch at
same state as jed/feature4
(jed/test-pu) $ git merge origin/pu
test things out, go back to work, eventually discard jed/test-pu
$ git branch -D jed/test-pu

If we discover some bad conflict, this experiment might prompt a design
discussion with the people working on a component that we thought was
independent of our work.


> > Erase the command `hg branch` from your vocabulary.
>

I wish the Hg developers would admit they made a mistake and erase 'hg
branch' from Hg's vocabulary. Users that really want that ridiculous thing
can add it to their hgrc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130209/5797235a/attachment.html>