[petsc-dev] git worktree

[Scott Kruger]

With later versions of git, `git branch` shows branches are also setup
as worktree's which I personally like quite a bit.   Also, thinking
through the worktree workflow has forced me to improve my own workflow
in terms of directory naming discipline (but that is probably just me
who had far too many clones around).

The lightweight clone is simpler, but I think that can be fixed.

Scott



On 2021-05-18 17:17, Jed Brown did write:
> I don't follow the advantage over lightweight clones, such as: 
> 
> $ git clone --branch=release --reference petsc gitlab:petsc/petsc petsc-release
> Cloning into 'petsc-release'...
> remote: Enumerating objects: 261, done.
> remote: Counting objects: 100% (261/261), done.
> remote: Compressing objects: 100% (52/52), done.
> remote: Total 474 (delta 213), reused 239 (delta 207), pack-reused 213
> Receiving objects: 100% (474/474), 430.41 KiB | 1.96 MiB/s, done.
> Resolving deltas: 100% (296/296), completed with 110 local objects.
> Updating files: 100% (9721/9721), done.
> $ cd petsc-release/
> release= ~/petsc-release$ du -hs .git
> 2.3M    .git
> 
> Scott Kruger <kruger at txcorp.com> writes:
> 
> > Relatively recently, I learned about the git worktree feature and
> > attached my write-up of how I use it in petsc.   I have no idea whether
> > the response will be:
> >
> >    This has been around since 2015 at least, and you're just now
> >    finding out about it?  LOL!
> >
> > or:
> >
> >   I can't believe I never heard about it either!
> >
> >
> > Since Patrick recently talked about shallow clones with git on slack, I
> > suspect it's the latter (and I didn't hear about this feature from petsc
> > dev's which is where I typically gain all my git knowledge).  Basically,
> > if you have more than one clone of petsc on your drive, you'll be
> > interested in the worktree feature.
> >
> > The reason why the write-up is a bit long boils down the fact that we
> > have the `/` in our branch names.  It makes things a bit more
> > complicated compared to my other projects (but is nice for the directory
> > structure).  I have not scripted away the complexity either -- I haven't
> > reached that level of annoyance.
> >
> > The reason why I just don't have the rst file as an MR, is because the
> > way I have it point to an existing branch seems cumbersome.  Perhaps a
> > git guru knows an easier way with some type of detached state or faster
> > way of getting the HEAD to point to the right sha in one go.  I'd be
> > very interested if someone knows a better method.
> >
> > Scott
> >
> >
> > -- 
> > Scott Kruger
> > Tech-X Corporation               kruger at txcorp.com
> > 5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
> > Boulder, CO 80303                Fax:   (303) 448-7756
> >
> >
> > Working on multiple branches simultaneously
> > ===========================================
> >
> > Our goal is to have a parallel structure of directories each with a different
> > branch.
> >
> > Let's start off with the basic structure::
> >
> >     - ptroot
> >            |- petsc  (main)
> >
> >
> > The petsc directory is the directory that comes from `git clone` and we 
> > have main as a general branch.  
> >
> > The simplest example is to do a quick bugfix in a separate worktree::
> >
> >     git worktree add ../petsc-bugfix
> >
> > The output of this is::
> >
> >     Preparing worktree (new branch 'petsc-bugfix')
> >     Updating files: 100% (9829/9829), done.
> >     HEAD is now at ...
> >
> > The directory is now this::
> >
> >     - ptroot
> >            |- petsc  (main)
> >            |
> >            |- petsc-bugfix  (petsc-bugfix)
> >
> > This is like a separate clone, but is more lightweight because it does not copy
> > over the `.git` directory (it has a `.git` file instead) and has advantages
> > because typing `git branch` shows information on all of the worktree's::
> >
> >     * main
> >     + petsc-bugfix
> >
> > where the `*` denotes the branch of the directory we are in and `+` denotes
> > other worktree branches (this appears to be a feature in newer versions of git).
> >
> >
> > The naming convention of a git branch in petsc is `developer/branch-name`; e.g.,
> > `scott/test-fix-reporting`.  The slash will introduce some wrinkles into the
> > normal worktree usage.   Let's try this::
> >
> >     git worktree add ../scott/test-fix-reporting
> >
> > We now have::
> >
> >     - ptroot
> >            |- petsc  (main)
> >            |
> >            |- petsc-bugfix  (petsc-bugfix)
> >            |
> >            |- scott
> >                   |
> >                   |- test-fix-reporting (test-fix-reporting)
> >
> >
> > which isn't *exactly* what we wanted.  Instead, we  use the `-b` flag to use the
> > right branch name::
> >
> >    git worktree add -b 'scott/test-fix-reporting' ../scott/test-fix-reporting
> >    cd ../scott/test-fix-reporting
> >    git branch --set-upstream-to=origin/scott/test-fix-reporting scott/test-fix-reporting
> >
> > The last 2 steps were to avoid using the `--set-upstream` to the first `git
> > push`.  Those two steps are not strictly necessary.
> >
> > (Aside:  `git worktree add` can take a 3rd argument to give the branch name and
> > many tutorials use that; however that doesn't work with `/` in the name.  The
> > documentation itself says that the argument is `commit-ish`.  The `-b` argument
> > is needed for the PETSc naming convention.)
> >
> > We now have::
> >
> >     - ptroot
> >            |- petsc  (main)
> >            |
> >            |- petsc-bugfix  (petsc-bugfix)
> >            |
> >            |- scott
> >                   |
> >                   |- test-fix-reporting (scott/test-fix-reporting)
> >
> > which is what we wanted as `git branch` shows (again, assuming a newer version
> > of git)::
> >
> >     > git branch
> >     + main
> >     + petsc-bugfix
> >     + scott/test-fix-reporting
> >
> > This provides a nicely organized structure.  
> >
> > Tracking an existing remote branch
> > ===================================
> >
> > The above shows a worktree based on performing the equivalent of 
> > a `git  checkout -b` to start with a new branch.   Here, we show how to follow
> > an existing remote branch.  
> > For reasons given by the Aside above, our naming scheme makes this a bit more
> > complicated.  Here is what I have working::
> >
> >    # Get version that matches remote branch
> >    git checkout barry/feature-pintogpu
> >
> >    # Need to create worktree with different name at the remote branch to avoid conflicts
> >    git checkout -b temp                                                    
> >
> >    # About to create a branch with the same name so delete
> >    git branch -D barry/feature-pintogpu
> >
> >    # Local branch name matches worktree
> >    git worktree add -b'barry/feature-pintogpu' ../barry/feature-pintogpu
> >
> >    # Cleanup
> >    git checkout main; git branch -D temp
> >
> >    # Point the worktree to match the remote branch and pull for testing
> >    cd ../barry/feature-pintogpu/
> >    git branch --set-upstream-to=origin/barry/feature-pintogpu barry/feature-pintogpu
> >    git pull
> >
> >
> >
> > Cleaning up
> > ============
> >
> > Using worktrees is much lighter weight than separate clones and it's easy to end
> > up with additional directories that you want to clean up.  The easiest is to do
> > the inverse of the add command::
> >
> >     git worktree remove ../scott/test-fix-reporting
> >
> >
> > Other features
> > ===============
> >
> > The above commands and workflow handle most of what you need about worktrees,
> > but running `git worktree --help` shows additional commands.  Most of them are
> > obvious:  you can move a working directory, you can clean up (`prune`) some of the
> > worktree information if you delete a directory without using the `remove` command,
> > etc.  
> >
> >
> >
> > Comments on other workflows
> > ================================
> >
> > It is useful to have a good organization so that one can distinguish between the
> > original clone (which has a `.git` subdirectory) and the worktrees (which has a
> > `.git` file which you can examine).   Other tutorials have the worktrees as
> > subdirectories of the original clone.  This seems confusing as it is not as
> > obvious in how git operations (such as `git grep`) will work.
> >
> > Another common workflow is to have a single directory that has rolling fixes on
> > a given topic.  This workflow is valid and totally works as one can change
> > directories in a worktree.  The main problem however is this:
> >
> >     *no branch may be checked at the same time*
> >     
> > This means that if I have a directory of `../scott/test` that always has my work
> > on test harness development, the main problem I will have is getting that
> > directory up to `main`.  The fix is to have a branch that can be used to track
> > `main`:  `test-main` created in my `petsc` directory.  The workflow would then
> > be::
> >
> >     cd ../scott/test
> >     git checkout test-main
> >     git rebase main
> >     git checkout -b scott/test-very-specific-fix
> >
> > Different workflows are possible of course, but this gives a flavor of the basic
> > steps needed to create your own and fits within the common petsc labelling
> > scheme.

-- 
Scott Kruger
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
Boulder, CO 80303                Fax:   (303) 448-7756