[petsc-dev] git worktree

[Scott Kruger]

Relatively recently, I learned about the git worktree feature and
attached my write-up of how I use it in petsc.   I have no idea whether
the response will be:

   This has been around since 2015 at least, and you're just now
   finding out about it?  LOL!

or:

  I can't believe I never heard about it either!


Since Patrick recently talked about shallow clones with git on slack, I
suspect it's the latter (and I didn't hear about this feature from petsc
dev's which is where I typically gain all my git knowledge).  Basically,
if you have more than one clone of petsc on your drive, you'll be
interested in the worktree feature.

The reason why the write-up is a bit long boils down the fact that we
have the `/` in our branch names.  It makes things a bit more
complicated compared to my other projects (but is nice for the directory
structure).  I have not scripted away the complexity either -- I haven't
reached that level of annoyance.

The reason why I just don't have the rst file as an MR, is because the
way I have it point to an existing branch seems cumbersome.  Perhaps a
git guru knows an easier way with some type of detached state or faster
way of getting the HEAD to point to the right sha in one go.  I'd be
very interested if someone knows a better method.

Scott


-- 
Scott Kruger
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
Boulder, CO 80303                Fax:   (303) 448-7756
-------------- next part --------------


Working on multiple branches simultaneously
===========================================

Our goal is to have a parallel structure of directories each with a different
branch.

Let's start off with the basic structure::

    - ptroot
           |- petsc  (main)


The petsc directory is the directory that comes from `git clone` and we 
have main as a general branch.  

The simplest example is to do a quick bugfix in a separate worktree::

    git worktree add ../petsc-bugfix

The output of this is::

    Preparing worktree (new branch 'petsc-bugfix')
    Updating files: 100% (9829/9829), done.
    HEAD is now at ...

The directory is now this::

    - ptroot
           |- petsc  (main)
           |
           |- petsc-bugfix  (petsc-bugfix)

This is like a separate clone, but is more lightweight because it does not copy
over the `.git` directory (it has a `.git` file instead) and has advantages
because typing `git branch` shows information on all of the worktree's::

    * main
    + petsc-bugfix

where the `*` denotes the branch of the directory we are in and `+` denotes
other worktree branches (this appears to be a feature in newer versions of git).


The naming convention of a git branch in petsc is `developer/branch-name`; e.g.,
`scott/test-fix-reporting`.  The slash will introduce some wrinkles into the
normal worktree usage.   Let's try this::

    git worktree add ../scott/test-fix-reporting

We now have::

    - ptroot
           |- petsc  (main)
           |
           |- petsc-bugfix  (petsc-bugfix)
           |
           |- scott
                  |
                  |- test-fix-reporting (test-fix-reporting)


which isn't *exactly* what we wanted.  Instead, we  use the `-b` flag to use the
right branch name::

   git worktree add -b 'scott/test-fix-reporting' ../scott/test-fix-reporting
   cd ../scott/test-fix-reporting
   git branch --set-upstream-to=origin/scott/test-fix-reporting scott/test-fix-reporting

The last 2 steps were to avoid using the `--set-upstream` to the first `git
push`.  Those two steps are not strictly necessary.

(Aside:  `git worktree add` can take a 3rd argument to give the branch name and
many tutorials use that; however that doesn't work with `/` in the name.  The
documentation itself says that the argument is `commit-ish`.  The `-b` argument
is needed for the PETSc naming convention.)

We now have::

    - ptroot
           |- petsc  (main)
           |
           |- petsc-bugfix  (petsc-bugfix)
           |
           |- scott
                  |
                  |- test-fix-reporting (scott/test-fix-reporting)

which is what we wanted as `git branch` shows (again, assuming a newer version
of git)::

    > git branch
    + main
    + petsc-bugfix
    + scott/test-fix-reporting

This provides a nicely organized structure.  

Tracking an existing remote branch
===================================

The above shows a worktree based on performing the equivalent of 
a `git  checkout -b` to start with a new branch.   Here, we show how to follow
an existing remote branch.  
For reasons given by the Aside above, our naming scheme makes this a bit more
complicated.  Here is what I have working::

   # Get version that matches remote branch
   git checkout barry/feature-pintogpu

   # Need to create worktree with different name at the remote branch to avoid conflicts
   git checkout -b temp                                                    

   # About to create a branch with the same name so delete
   git branch -D barry/feature-pintogpu

   # Local branch name matches worktree
   git worktree add -b'barry/feature-pintogpu' ../barry/feature-pintogpu

   # Cleanup
   git checkout main; git branch -D temp

   # Point the worktree to match the remote branch and pull for testing
   cd ../barry/feature-pintogpu/
   git branch --set-upstream-to=origin/barry/feature-pintogpu barry/feature-pintogpu
   git pull



Cleaning up
============

Using worktrees is much lighter weight than separate clones and it's easy to end
up with additional directories that you want to clean up.  The easiest is to do
the inverse of the add command::

    git worktree remove ../scott/test-fix-reporting


Other features
===============

The above commands and workflow handle most of what you need about worktrees,
but running `git worktree --help` shows additional commands.  Most of them are
obvious:  you can move a working directory, you can clean up (`prune`) some of the
worktree information if you delete a directory without using the `remove` command,
etc.  



Comments on other workflows
================================

It is useful to have a good organization so that one can distinguish between the
original clone (which has a `.git` subdirectory) and the worktrees (which has a
`.git` file which you can examine).   Other tutorials have the worktrees as
subdirectories of the original clone.  This seems confusing as it is not as
obvious in how git operations (such as `git grep`) will work.

Another common workflow is to have a single directory that has rolling fixes on
a given topic.  This workflow is valid and totally works as one can change
directories in a worktree.  The main problem however is this:

    *no branch may be checked at the same time*
    
This means that if I have a directory of `../scott/test` that always has my work
on test harness development, the main problem I will have is getting that
directory up to `main`.  The fix is to have a branch that can be used to track
`main`:  `test-main` created in my `petsc` directory.  The workflow would then
be::

    cd ../scott/test
    git checkout test-main
    git rebase main
    git checkout -b scott/test-very-specific-fix

Different workflows are possible of course, but this gives a flavor of the basic
steps needed to create your own and fits within the common petsc labelling
scheme.