[petsc-dev] Introducing new "test harness" to PETSc
Scott Kruger
kruger at txcorp.com
Wed Jan 18 14:33:44 CST 2017
On 1/18/17 1:10 PM, Matthew Knepley wrote:
> On Wed, Jan 18, 2017 at 2:05 PM, Scott Kruger <kruger at txcorp.com
> <mailto:kruger at txcorp.com>> wrote:
>
>
>
> This does not work
>
> knepley/fix-configure-pragmatic *+$:/PETSc3/petsc/petsc-dev$ make -f
> gmakefile test search=plex%ex1_2
> make -f gmakefile test search=plex%ex1_2
> # No tests run
>
>
>
> If you replace test with "echo_test" as a target it will echo all of
> the targets. This is useful for debugging.
>
> make -f gmakefile echo_test search=plex%
>
> and then make sure you see it.
>
> Then do an echo_test to see the match.
>
> Here is an example with the current version:
> --------------------------------------------------
> > make -f gmakefile echo_test search=sys%
> sys_tests-runex19 sys_tests-runex10 sys_tests-runex11
> sys_tests-runex12 sys_tests-runex14 sys_tests-runex31
> sys_tests-runex31_2 sys_tests-runex20_2 sys_tests-runex20_3
> sys_tutorials-runex9_2 ...
>
> > make -f gmakefile echo_test search=sys%ex9_2
> sys_tutorials-runex9_2
>
>
> Okay, the problem seems to be that you can only use one % in the pattern
Yes, it's a limit of gmake syntax. We also have the searchin argument.
In gmakefile itself:
# Tests can be generated by searching
# Percent is a wildcard (only one allowed):
# make -f gmakefile test search=sys%ex2
# To match internal substrings (matches *ex2*):
# make -f gmakefile test searchin=ex2
# For args:
# make -f gmakefile test argsearch=cuda
However, the searchin does not take wildcard.
For what you want: dm%plex%
perhaps this could work:
--------------------------------------------------
--- a/gmakefile
+++ b/gmakefile
@@ -323,6 +323,10 @@ $(alltest.d) : ;
# make -f gmakefile test argsearch=cuda
ifdef search
TESTTARGETS := $(filter $(search),$(alltesttargets))
+ ifdef searchin
+ TESTTARGETS2 := $(foreach v,$(TESTTARGETS),$(if $(findstring
$(searchin),$(v)),$(v)))
+ TESTTARGETS := $(TESTTARGETS2)
+ endif
else ifdef searchin
--------------------------------------------------
Such that:
--------------------------------------------------
> make -f gmakefile echo_test search=sys% searchin=ex2
sys_tests-runex20_2 sys_tests-runex20_3 sys_tests-runex24
sys_tests-runex28 sys_tests-runex25 sys_tests-runex21 sys_tests-runex20
sys_tests-runex23 sys_tests-runex22 sys_tutorials-runex2 sys_tests-runex2
--------------------------------------------------
This still isn't as nice as true regex searching, but it probably gets
one a lot of they way there.
It may also be possible to use shell for true regex.
Looking at that now.
Scott
>
> knepley/fix-configure-pragmatic *+$:/PETSc3/petsc/petsc-dev$ make -f
> gmakefile echo_test search=dm%
> make -f gmakefile echo_test search=dm%
> dm_impls_plex_tests-runex1_2 dm_impls_plex_tests-runex1_3
> dm_impls_plex_tests-runex1_0 dm_impls_plex_tests-runex1_1
> dm_impls_plex_tests-runex1_6 dm_impls_plex_tests-runex1_7
> dm_impls_plex_tests-runex1_4 dm_impls_plex_tests-runex1_5
> dm_impls_plex_tests-runex1_8 dm_impls_plex_tests-runex1_9
> dm_impls_plex_tests-runex1_cgns_1 dm_impls_plex_tests-runex1_cgns_0
> dm_impls_plex_tests-runex1_refine_overlap_1
> dm_impls_plex_tests-runex1_refine_overlap_0
> dm_impls_plex_tests-runex1_med_1 dm_impls_plex_tests-runex1_med_0
> dm_impls_plex_tests-runex1_med_3 dm_impls_plex_tests-runex1_med_2
> dm_impls_plex_tests-runex1_gmsh_3 dm_impls_plex_tests-runex1_gmsh_2
> dm_impls_plex_tests-runex1_gmsh_1 dm_impls_plex_tests-runex1_gmsh_0
> dm_impls_plex_tests-runex1_gmsh_6 dm_impls_plex_tests-runex1_gmsh_5
> dm_impls_plex_tests-runex1_gmsh_4 dm_impls_plex_tests-runex1_fluent_2
> dm_impls_plex_tests-runex1_fluent_3 dm_impls_plex_tests-runex1_fluent_0
> dm_impls_plex_tests-runex1_fluent_1
> dm_impls_plex_tests-runex1_part_simple_1
> dm_impls_plex_tests-runex1_part_simple_0
>
> knepley/fix-configure-pragmatic *+$:/PETSc3/petsc/petsc-dev$ make -f
> gmakefile echo_test search=dm%plex%
> make -f gmakefile echo_test search=dm%plex%
>
> I do not think this is going to be flexible enough.
>
> Matt
>
>
> --------------------------------------------------
>
>
>
> > 3) How do I give extra arguments with the make
> interface, rather than
> > test_harness?
>
> That is, run some modification of the test and still do the
> diff, but
> don't change the reference output? I have used
> PETSC_OPTIONS for that.
>
> make .... PETSC_OPTIONS=-more_magic
>
>
> I'm not sure I 100% understand what is being asked (perhaps it's my
> cold medicine), but I think TESTFLAGS does what you want; e.g.,
>
> --------------------------------------------------
> > make -f gmakefile test search=sys%ex9_2 TESTFLAGS="-h"
> TEST sys_tutorials-runex9_2
> Usage:
> arch-osx-xsdk-opt/tests/src/sys/examples/tutorials/runex9_2.sh [options]
>
> OPTIONS
> -a <args> ......... Override default arguments
> -c <cleanup> ...... Cleanup (remove generated files)
> -e <args> ......... Add extra arguments to default
> -h ................ help: print this message
> -n <integer> ...... Override the number of processors to use
> -o <output file> .. Override default output file to diff with
> -t <testname> ..... Override test name
> -v ................ Verbose: Print commands
> # No tests run
> --------------------------------------------------
>
>
> > Once I have all the functionality of my Python stuff, I will
> throw it away.
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
>
> --
> Tech-X Corporation kruger at txcorp.com
> <mailto:kruger at txcorp.com>
> 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841
> <tel:%28720%29%20974-1841>
> Boulder, CO 80303 Fax: (303) 448-7756
> <tel:%28303%29%20448-7756>
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
--
Tech-X Corporation kruger at txcorp.com
5621 Arapahoe Ave, Suite A Phone: (720) 974-1841
Boulder, CO 80303 Fax: (303) 448-7756
More information about the petsc-dev
mailing list