<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 21, 2013 at 11:18 AM, Sean Farley <span dir="ltr"><<a href="mailto:sean.michael.farley@gmail.com" target="_blank">sean.michael.farley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im">On Mon, Jan 21, 2013 at 11:03 AM, Jed Brown <<a href="mailto:jedbrown@mcs.anl.gov">jedbrown@mcs.anl.gov</a>> wrote:<br>
><br>
> On Mon, Jan 21, 2013 at 10:53 AM, Sean Farley<br>
> <<a href="mailto:sean.michael.farley@gmail.com">sean.michael.farley@gmail.com</a>> wrote:<br>
>><br>
>> Well … did you try this with the equivalent mercurial feature:<br>
>> largefiles?<br>
><br>
><br>
> Nope, feel free. Most of the speedup is independent of the large files<br>
> (which only change the git repo size from 78MB to 50MB).<br>
<br>
</div>Righto.</blockquote><div><br></div><div style>Here's the clone without any "git-fat" business (18 seconds, versus 12 seconds with git-fat):</div><div><br></div><div><div>$ time git clone git@bitbucket.org:jedbrown/petsc-git</div>
<div>Cloning into 'petsc-git'...</div><div>remote: Counting objects: 300368, done.</div><div>remote: Compressing objects: 100% (66014/66014), done.</div><div>remote: Total 300368 (delta 233578), reused 300368 (delta 233578)</div>
<div>Receiving objects: 100% (300368/300368), 68.18 MiB | 10.22 MiB/s, done.</div><div>Resolving deltas: 100% (233578/233578), done.</div><div>18.067 real 16.042 user 2.080 sys 100.30 cpu</div></div><div><div>$ du -hs petsc-git/.git</div>
<div>77M petsc-git/.git</div></div><div><br></div><div><br></div><div style>Does hg largefiles have a way to version the patterns that it should match? It looks like the mailing list advice is to have a script that you ask users to run that will modify their .hg/hgrc. Is there a way to make lfconvert use a list of files rather than a single threshold size?<br>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im"></div><div class="im">
>> Also, what files did<br>
>> you deem were "fat"<br>
><br>
> A smattering of powerpoint slides, pdfs, random binaries, and a few very<br>
> large log files. Note that this was a performance experiment and don't care<br>
> about which files. In practice, I'd suggest managing fewer (or even none).<br>
<br>
</div>Huh? Do you mean not even worrying about separating binary files?</blockquote></div><br>PETSc does not actually have many large files. There were a few binaries accidentally checked in, but it just doesn't make that big of a difference. If we switched to git, we might just delete that very worst offenders. The second to last column has the file sizes for everything over 1MB:<br>
<br><div class="gmail_extra"><font face="courier new, monospace">petsc-git$ git fat find 1000000</font></div><div class="gmail_extra"><font face="courier new, monospace">src/dm/mesh/examples/tutorials/ex_coarsen_3 filter=fat -text # 9415934 3</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/dm/mesh/examples/tutorials/ex_interpolate_1 filter=fat -text # 9038252 1</font></div><div class="gmail_extra"><font face="courier new, monospace">zope/var/data.fs.hg filter=fat -text # 8615555 13</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/snes/examples/tutorials/phi filter=fat -text # 8388616 1</font></div><div class="gmail_extra"><font face="courier new, monospace">zope/var/hgdata.fs filter=fat -text # 6842744 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/dm/impls/mesh/examples/tutorials/illinois/<a href="http://il.ps">il.ps</a> filter=fat -text # 6578178 1</font></div><div class="gmail_extra"><font face="courier new, monospace">src/dm/mesh/examples/tutorials/illinois/<a href="http://il.ps">il.ps</a> filter=fat -text # 6578178 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">zope/var/Data.fs filter=fat -text # 5501166 11</font></div><div class="gmail_extra"><font face="courier new, monospace">src/contrib/blopex/driver_fiedler/L-matrix.petsc filter=fat -text # 4182032 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/docs/website/tops/images/topsanimation.gif filter=fat -text # 3137151 1</font></div><div class="gmail_extra"><font face="courier new, monospace">src/tops/images/topsanimation.gif filter=fat -text # 3137151 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">docs/website/docs/tutorials/nersc01/nersc01.pdf filter=fat -text # 1251497 1</font></div><div class="gmail_extra"><font face="courier new, monospace">docs/webpage/docs/tutorials/nersc01/nersc01.pdf filter=fat -text # 1251497 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/docs/website/documentation/tutorials/nersc01/nersc01.pdf filter=fat -text # 1251497 1</font></div><div class="gmail_extra"><font face="courier new, monospace">docs/website/documentation/tutorials/nersc01/nersc01.pdf filter=fat -text # 1251497 1</font></div>
<div class="gmail_extra"><font face="courier new, monospace">src/ts/examples/tutorials/output/ex12_1.out filter=fat -text # 1219820 1</font></div><div class="gmail_extra"><font face="courier new, monospace">src/contrib/blopex/driver_fiedler/DL-matrix.petsc filter=fat -text # 1048592 1</font></div>
<div class="gmail_extra"><br></div>FWIW, I ran hg lfconvert --size 1 (treat all files >1MB as "large") and .hg/store is still 150MB (though .hg/largefiles ballooned to 220MB).</div></div>