[petsc-dev] git worktree

[Jed Brown]

I don't follow the advantage over lightweight clones, such as: 

$ git clone --branch=release --reference petsc gitlab:petsc/petsc petsc-release
Cloning into 'petsc-release'...
remote: Enumerating objects: 261, done.
remote: Counting objects: 100% (261/261), done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 474 (delta 213), reused 239 (delta 207), pack-reused 213
Receiving objects: 100% (474/474), 430.41 KiB | 1.96 MiB/s, done.
Resolving deltas: 100% (296/296), completed with 110 local objects.
Updating files: 100% (9721/9721), done.
$ cd petsc-release/
release= ~/petsc-release$ du -hs .git
2.3M    .git

Scott Kruger <kruger at txcorp.com> writes:

> Relatively recently, I learned about the git worktree feature and
> attached my write-up of how I use it in petsc.   I have no idea whether
> the response will be:
>
>    This has been around since 2015 at least, and you're just now
>    finding out about it?  LOL!
>
> or:
>
>   I can't believe I never heard about it either!
>
>
> Since Patrick recently talked about shallow clones with git on slack, I
> suspect it's the latter (and I didn't hear about this feature from petsc
> dev's which is where I typically gain all my git knowledge).  Basically,
> if you have more than one clone of petsc on your drive, you'll be
> interested in the worktree feature.
>
> The reason why the write-up is a bit long boils down the fact that we
> have the `/` in our branch names.  It makes things a bit more
> complicated compared to my other projects (but is nice for the directory
> structure).  I have not scripted away the complexity either -- I haven't
> reached that level of annoyance.
>
> The reason why I just don't have the rst file as an MR, is because the
> way I have it point to an existing branch seems cumbersome.  Perhaps a
> git guru knows an easier way with some type of detached state or faster
> way of getting the HEAD to point to the right sha in one go.  I'd be
> very interested if someone knows a better method.
>
> Scott
>
>
> -- 
> Scott Kruger
> Tech-X Corporation               kruger at txcorp.com
> 5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
> Boulder, CO 80303                Fax:   (303) 448-7756
>
>
> Working on multiple branches simultaneously
> ===========================================
>
> Our goal is to have a parallel structure of directories each with a different
> branch.
>
> Let's start off with the basic structure::
>
>     - ptroot
>            |- petsc  (main)
>
>
> The petsc directory is the directory that comes from `git clone` and we 
> have main as a general branch.  
>
> The simplest example is to do a quick bugfix in a separate worktree::
>
>     git worktree add ../petsc-bugfix
>
> The output of this is::
>
>     Preparing worktree (new branch 'petsc-bugfix')
>     Updating files: 100% (9829/9829), done.
>     HEAD is now at ...
>
> The directory is now this::
>
>     - ptroot
>            |- petsc  (main)
>            |
>            |- petsc-bugfix  (petsc-bugfix)
>
> This is like a separate clone, but is more lightweight because it does not copy
> over the `.git` directory (it has a `.git` file instead) and has advantages
> because typing `git branch` shows information on all of the worktree's::
>
>     * main
>     + petsc-bugfix
>
> where the `*` denotes the branch of the directory we are in and `+` denotes
> other worktree branches (this appears to be a feature in newer versions of git).
>
>
> The naming convention of a git branch in petsc is `developer/branch-name`; e.g.,
> `scott/test-fix-reporting`.  The slash will introduce some wrinkles into the
> normal worktree usage.   Let's try this::
>
>     git worktree add ../scott/test-fix-reporting
>
> We now have::
>
>     - ptroot
>            |- petsc  (main)
>            |
>            |- petsc-bugfix  (petsc-bugfix)
>            |
>            |- scott
>                   |
>                   |- test-fix-reporting (test-fix-reporting)
>
>
> which isn't *exactly* what we wanted.  Instead, we  use the `-b` flag to use the
> right branch name::
>
>    git worktree add -b 'scott/test-fix-reporting' ../scott/test-fix-reporting
>    cd ../scott/test-fix-reporting
>    git branch --set-upstream-to=origin/scott/test-fix-reporting scott/test-fix-reporting
>
> The last 2 steps were to avoid using the `--set-upstream` to the first `git
> push`.  Those two steps are not strictly necessary.
>
> (Aside:  `git worktree add` can take a 3rd argument to give the branch name and
> many tutorials use that; however that doesn't work with `/` in the name.  The
> documentation itself says that the argument is `commit-ish`.  The `-b` argument
> is needed for the PETSc naming convention.)
>
> We now have::
>
>     - ptroot
>            |- petsc  (main)
>            |
>            |- petsc-bugfix  (petsc-bugfix)
>            |
>            |- scott
>                   |
>                   |- test-fix-reporting (scott/test-fix-reporting)
>
> which is what we wanted as `git branch` shows (again, assuming a newer version
> of git)::
>
>     > git branch
>     + main
>     + petsc-bugfix
>     + scott/test-fix-reporting
>
> This provides a nicely organized structure.  
>
> Tracking an existing remote branch
> ===================================
>
> The above shows a worktree based on performing the equivalent of 
> a `git  checkout -b` to start with a new branch.   Here, we show how to follow
> an existing remote branch.  
> For reasons given by the Aside above, our naming scheme makes this a bit more
> complicated.  Here is what I have working::
>
>    # Get version that matches remote branch
>    git checkout barry/feature-pintogpu
>
>    # Need to create worktree with different name at the remote branch to avoid conflicts
>    git checkout -b temp                                                    
>
>    # About to create a branch with the same name so delete
>    git branch -D barry/feature-pintogpu
>
>    # Local branch name matches worktree
>    git worktree add -b'barry/feature-pintogpu' ../barry/feature-pintogpu
>
>    # Cleanup
>    git checkout main; git branch -D temp
>
>    # Point the worktree to match the remote branch and pull for testing
>    cd ../barry/feature-pintogpu/
>    git branch --set-upstream-to=origin/barry/feature-pintogpu barry/feature-pintogpu
>    git pull
>
>
>
> Cleaning up
> ============
>
> Using worktrees is much lighter weight than separate clones and it's easy to end
> up with additional directories that you want to clean up.  The easiest is to do
> the inverse of the add command::
>
>     git worktree remove ../scott/test-fix-reporting
>
>
> Other features
> ===============
>
> The above commands and workflow handle most of what you need about worktrees,
> but running `git worktree --help` shows additional commands.  Most of them are
> obvious:  you can move a working directory, you can clean up (`prune`) some of the
> worktree information if you delete a directory without using the `remove` command,
> etc.  
>
>
>
> Comments on other workflows
> ================================
>
> It is useful to have a good organization so that one can distinguish between the
> original clone (which has a `.git` subdirectory) and the worktrees (which has a
> `.git` file which you can examine).   Other tutorials have the worktrees as
> subdirectories of the original clone.  This seems confusing as it is not as
> obvious in how git operations (such as `git grep`) will work.
>
> Another common workflow is to have a single directory that has rolling fixes on
> a given topic.  This workflow is valid and totally works as one can change
> directories in a worktree.  The main problem however is this:
>
>     *no branch may be checked at the same time*
>     
> This means that if I have a directory of `../scott/test` that always has my work
> on test harness development, the main problem I will have is getting that
> directory up to `main`.  The fix is to have a branch that can be used to track
> `main`:  `test-main` created in my `petsc` directory.  The workflow would then
> be::
>
>     cd ../scott/test
>     git checkout test-main
>     git rebase main
>     git checkout -b scott/test-very-specific-fix
>
> Different workflows are possible of course, but this gives a flavor of the basic
> steps needed to create your own and fits within the common petsc labelling
> scheme.