On Thu, Jul 26, 2012 at 1:52 PM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div class="im">On Thu, Jul 26, 2012 at 1:30 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Thu, Jul 26, 2012 at 12:21 PM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br></div><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div class="gmail_quote">Builder does some of this, but I think not all, and I don't think it's as convenient. The differences are partly syntactic. First, to add a test, you have to add lines to a single global file (currently 1500 lines). By creating the dictionary globally, we can't do consistency checks when the test is declared, instead you only do it later once you have lost line number information. You also can't build in the source directory, or anything similar, which makes it hard for a user to split an example out into their own toy project. Since you mess around with stdout, we are losing warning messages (including color) and lose some Emacs M-x compile convenience. (this can be fixed easily enough).</div>
</div></blockquote><div><br></div></div><div>1) I am fine with changing where the options come from, meaning if we want to also check for an import somewhere. Personally, I think</div><div> its easier to maintain a central database than a distributed database. I would only suggest a distributed database for scalability, which</div>
<div> we do not need.</div><div><br></div><div>2) I take your point about users adding tests (extensibility), which is why I would be fine with checking for an import</div><div><br></div><div>3) I have no idea what you mean about consistency checks</div>
</div></blockquote><div><br></div></div><div>Suppose the user misspells a file name or dict key, or they forget a mandatory key. With a dict, all you can do is spit out the entry that you think is corrupt and the user is on their own to figure out where the declaration is in the file. With my model, you immediately get the file and line number where the invalid record was created. With separate files, we can also get more context-sensitive behavior to avoid repeating ourselves so much.</div>
</div></blockquote><div><br></div><div>You can tell from the primary key what it is. If the primary key is wrong, it does not help you in your scenario as it could be</div><div>misspelled, or intended to be put in another directory. This is dubious at best.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">
<div><br></div><div>4) You can run builder on source anywhere. This is how I check the crazy things people send us.</div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div class="gmail_quote">Note that my suggestion does not _prevent_ using a global script, but it localizes the declaration and would make it possible to run locally.</div></div><div class="gmail_quote">
<br></div><div class="gmail_quote">On Thu, Jul 26, 2012 at 11:37 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>./test.py build ex19</div></blockquote></div></div>
</div></blockquote><div><br></div><div><div>$ config/builder2.py buildExample src/snes/examples/tutorials/ex19.c</div><div>Namespace(files=['src/snes/examples/tutorials/ex19.c'], func=<function buildExample at 0x1fa56e0>)</div>
<div>Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']</div><div>ERROR IN LINK ****************************** /usr/bin/ld: cannot find -lpetsc collect2: error: ld returned 1 exit status </div>
</div><div><br></div><div>(maybe it doesn't work --with-single-library=0)</div></div></blockquote><div><br></div></div><div>It does not. I don't see a reason to support this. If its does, it is accidental.</div></div>
</blockquote><div><br></div></div><div>It's still a supported option in PETSc. I use it in my default PETSC_ARCH because (a) link is one second faster and (b) it keeps us honest about dependencies.</div></div></blockquote>
<div><br></div><div>We should have a discussion about whether we want to support this after 3.3.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">
<div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div><div>$ PETSC_ARCH=mpich-singlelib config/builder2.py buildExample src/snes/examples/tutorials/ex19.c</div>
<div>Namespace(files=['src/snes/examples/tutorials/ex19.c'], func=<function buildExample at 0x25dd6e0>)</div>
<div>Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']</div></div><div><br></div><div>(hmm, where is the executable anyway)</div><div><div><br></div><div>$ ls -ltr | tail -3</div><div>lrwxrwxrwx 1 jed users 35 Jul 26 11:57 make.log.bkp -> /home/jed/petsc/mpich/conf/make.log</div>
<div>lrwxrwxrwx 1 jed users 45 Jul 26 11:58 make.log -> /home/jed/petsc/mpich-singlelib/conf/make.log</div><div>-rw-r--r-- 1 jed users 5271714 Jul 26 11:58 RDict.log</div></div><div><br></div><div>(oh cool, it overwrote my library build with the make.log from this example)</div>
</div></blockquote><div><br></div></div><div>It shouldn't. It should move make.log to make.log.bkp first. There must be a bug.</div></div></blockquote><div><br></div></div><div>So the second time I run it, the original make.log is gone. Not better.</div>
</div></blockquote><div><br></div><div>No no. Your original make log is in PETSC_ARCH/conf/make.log. It should never be corrupted. You</div><div>remove the link at make.log.bkp.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div>$ cat make.log</div><div> Loaded configure to cache: size 298004</div><div>Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']</div><div> Compiling C files ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']</div>
<div> Pushing language C</div><div>/opt/mpich2/bin/mpicc -c -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/petsc/include -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/usr/elemental-mpich/include -I/usr/include -I/opt/mpich2/include -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -MMD -D__INSDIR__= /home/jed/petsc/src/snes/examples/tutorials/ex19.c</div>
<div>sh: /opt/mpich2/bin/mpicc -c -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/petsc/include -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/usr/elemental-mpich/include -I/usr/include -I/opt/mpich2/include -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -MMD -D__INSDIR__= /home/jed/petsc/src/snes/examples/tutorials/ex19.c</div>
<div>Executing: /opt/mpich2/bin/mpicc -c -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/petsc/include -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/usr/elemental-mpich/include -I/usr/include -I/opt/mpich2/include -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -MMD -D__INSDIR__= /home/jed/petsc/src/snes/examples/tutorials/ex19.c</div>
<div>sh: </div><div> Popping language C</div><div> Moving ex19.o to /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o</div><div> Moving ex19.d to /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.d</div>
<div>Linking object ['/home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o'] into /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19</div><div> Pushing language C</div><div> Pushing language C</div>
<div> Popping language C</div><div> Pushing language CUDA</div><div> Popping language CUDA</div><div> Pushing language Cxx</div>
<div> Popping language Cxx</div><div> Pushing language FC</div><div> Popping language FC</div><div> Pushing language C</div>
<div> Popping language C</div><div>sh: /opt/mpich2/bin/mpicc -o /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19 -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o -L/home/jed/petsc/mpich-singlelib/lib -lpetsc -Wl,-rpath,/home/jed/petsc/mpich-singlelib/lib -L/home/jed/petsc/mpich-singlelib/lib -ltriangle -lX11 -lchaco -lparmetis -lmetis -Wl,-rpath,/home/jed/usr/elemental-mpich/lib -L/home/jed/usr/elemental-mpich/lib -lelemental -lplcg -lpmrrr -lamspub -lml -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -lmpichcxx -lstdc++ -lpthread -lblacs -lHYPRE -lmpichcxx -lstdc++ -Wl,-rpath,/usr/lib -L/usr/lib -lcholmod -lamd -lcolamd -lcamd -lccolamd -lumfpack -lcholmod -lcamd -lccolamd -lcolamd -lamd -llapack -lblas -lm -L/usr/local/cuda/lib -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -ldl -Wl,-rpath,/opt/mpich2/lib -lmpich -lopa -lmpl -lpthread -lrt -lgcc_s -ldl </div>
<div>Executing: /opt/mpich2/bin/mpicc -o /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19 -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o -L/home/jed/petsc/mpich-singlelib/lib -lpetsc -Wl,-rpath,/home/jed/petsc/mpich-singlelib/lib -L/home/jed/petsc/mpich-singlelib/lib -ltriangle -lX11 -lchaco -lparmetis -lmetis -Wl,-rpath,/home/jed/usr/elemental-mpich/lib -L/home/jed/usr/elemental-mpich/lib -lelemental -lplcg -lpmrrr -lamspub -lml -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -lmpichcxx -lstdc++ -lpthread -lblacs -lHYPRE -lmpichcxx -lstdc++ -Wl,-rpath,/usr/lib -L/usr/lib -lcholmod -lamd -lcolamd -lcamd -lccolamd -lumfpack -lcholmod -lcamd -lccolamd -lcolamd -lamd -llapack -lblas -lm -L/usr/local/cuda/lib -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -ldl -Wl,-rpath,/opt/mpich2/lib -lmpich -lopa -lmpl -lpthread -lrt -lgcc_s -ldl </div>
<div>sh: </div><div> Popping language C</div></div><div><br></div><div>(the above output loses warnings)</div></div></blockquote><div><br></div></div></div><div>Bug.</div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div>$ mpich-singlelib/lib/ex19-obj/ex19 -happy-running</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br></div><div>./test.py check ex19_fas</div></blockquote>
</div></div></div></blockquote><div><br></div><div><div>$ PETSC_ARCH=mpich-singlelib config/builder2.py check --testnum 5 src/snes/examples/tutorials/ex19.c</div><div>Namespace(args=[], files=['src/snes/examples/tutorials/ex19.c'], func=<function check at 0xfe17d0>, retain=False, testnum=5)</div>
<div>All tests pass</div></div><div><br></div><div>(edit file, insert syntax error, re-run)</div><div><br></div><div>[same output as above]</div><div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br></div><div>./test.py check ex19_\* # run all the ex19 tests</div>
<div><br></div><div>./test.py check -n 16 --if '(X || Fortran) && !Complex'<br></div></blockquote></div></div></div></blockquote><div><br></div></div><div>How?</div></div></blockquote><div><br></div><div>
<br></div></div><div>Internally it knows all these predicates like num processes, Fortran and complex.</div></div></blockquote><div><br></div></div></div><div>It knows Fortran only in that the source file is Fortran. How does it know whether an example supports complex or whether a given run uses SuperLU, etc?</div>
</div></blockquote><div><br></div><div>Same way we do in the makefiles now. It has an attribute "complex", "superlu", etc.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div> There is no command line syntax for it. I did</div>
<div>not think there was for make either. </div>
</div></blockquote><div><br></div></div><div>It's manual through TESTEXAMPLES_SUPERLU, etc.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">
<div>You can do that from the run parameters at the op of builder.py.</div></div></blockquote><div><br></div></div><div>I don't think so. I think you have to add another dict key to your example declarations.</div></div>
</blockquote></div><br>Some have these.<div><br></div><div> Matt<br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div>