[petsc-dev] git status to test list

Jacob Faibussowitsch jacob.fai at gmail.com
Mon Jul 6 14:57:20 CDT 2020 

>  Now stage 2 has a failed job, will no more jobs in stage 3 be started? 
Honestly, I’m not sure. According to these stale MR's https://gitlab.com/gitlab-org/gitlab/-/issues/30476 <https://gitlab.com/gitlab-org/gitlab/-/issues/30476> , https://gitlab.com/gitlab-org/gitlab/-/issues/23605 <https://gitlab.com/gitlab-org/gitlab/-/issues/23605> this question has been raised before. As per https://gitlab.com/gitlab-org/gitlab/-/issues/30476#note_215843706 <https://gitlab.com/gitlab-org/gitlab/-/issues/30476#note_215843706> It seems like the only way to cull the whole pipeline on failure is to add a “after_script” to every job which conditionally kills the pipeline if the job fails.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Jul 6, 2020, at 2:47 PM, Barry Smith <bsmith at petsc.dev> wrote:
> 
> 
>   This sounds like a good idea but is confusing to me.
>  
>   We use stages so if anything fails in stage 2 we don't waste compute time on anything in stage 3 or higher. 
> 
>   If one uses this needs how does that affect this? Will it run everything to the end despite failures in earlier stages or will it stop any more higher stage jobs from starting if there is a failure on a stage? So for example stage 2 is running and because of needs job x of stage 3 starts. Now stage 2 has a failed job, will no more jobs in stage 3 be started? 
> 
> Ideally if anything fails in stage 2 it kills the jobs started in stage 3 but I guess it doesn't do that.
> 
>   Barry
> 
> 
>> On Jul 6, 2020, at 1:15 PM, Jacob Faibussowitsch <jacob.fai at gmail.com <mailto:jacob.fai at gmail.com>> wrote:
>> 
>>> If we did this in the pipeline they could be significantly faster
>> We might also consider not locking _all_ stage n+1 jobs behind completion of stage n. Gitlab supports this via the “needs” command https://gitlab.com/help/ci/yaml/README.md#needs <https://gitlab.com/help/ci/yaml/README.md#needs> which allows you to run jobs immediately after their prerequisite jobs have finished regardless of stage. For example, I find that I am quite often waiting on stage 2 freebsd jobs to start in order to advance to stage 3, even after every other stage 2 job has run its course. 
>> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>>> On Jul 6, 2020, at 12:47 PM, Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>>> 
>>> 
>>>   We should be more clever than checking just examples but also check source code. If, for example, only KSP is changed then tests could skip the sys/vec/mat levels saving time.
>>> 
>>>   If we did this in the pipeline they could be significantly faster; all the time saved not testing sys to mat for Matt's changes :-)
>>> 
>>>   Is there a fallacy here?
>>> 
>>>   Barry
>>> 
>>>   Need to check changes in include files as well, of course.
>>> 
>>> 
>>>> On Jul 6, 2020, at 12:13 PM, Jacob Faibussowitsch <jacob.fai at gmail.com <mailto:jacob.fai at gmail.com>> wrote:
>>>> 
>>>> While were on this topic, I always parse git diff —name-only $(git merge-base —fork-point master) when testing. This gives you all the files that are different between your branch and master. You can switch it out for maint if needs be.
>>>> 
>>>> Best regards,
>>>> 
>>>> Jacob Faibussowitsch
>>>> (Jacob Fai - booss - oh - vitch)
>>>> Cell: (312) 694-3391
>>>> 
>>>>> On Jul 6, 2020, at 12:00 PM, Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>>>>> 
>>>>> 
>>>>> I couldn't see how Jed can avoid doing the filename hacking since alltesttargets are the hacked filenames but probably I am missing something. 
>>>>> 
>>>>>  Anyways this patch works for me based on Jed's email.
>>>>> 
>>>>>  Barry
>>>>> <since.patch>
>>>>> 
>>>>>> On Jul 6, 2020, at 9:40 AM, Jed Brown <jed at jedbrown.org <mailto:jed at jedbrown.org>> wrote:
>>>>>> 
>>>>>> It'd be possible to hack the file names, but I'd rather not duplicate that logic.
>>>>>> 
>>>>>> My inclination would be to filter similar to argsearch, which requires
>>>>>> adding the source files to testfiles.  Then you could use
>>>>>> 
>>>>>> make test since=@
>>>>>> 
>>>>>> ifdef since
>>>>>> gitchanged := $(shell git diff --name-only $(since) src/)
>>>>>> endif
>>>>>> 
>>>>>> Pierre Jolivet <pierre.jolivet at enseeiht.fr <mailto:pierre.jolivet at enseeiht.fr>> writes:
>>>>>> 
>>>>>>> Hello,
>>>>>>> To git/regexp enthusiasts and/or test harness people: is there some script/command lying around to convert a list of modified examples (according to git) into a list understandable by the test system?
>>>>>>> Thanks for your help,
>>>>>>> Pierre
>>>>>>> 
>>>>>>> PS: here is such a list
>>>>>>> $ git status -uno
>>>>>>> On branch master
>>>>>>> Your branch is up to date with 'origin/master'.
>>>>>>> 
>>>>>>> Changes not staged for commit:
>>>>>>> (use "git add <file>..." to update what will be committed)
>>>>>>> (use "git checkout -- <file>..." to discard changes in working directory)
>>>>>>> 
>>>>>>> 	modified:   config/BuildSystem/config/types.py
>>>>>>> 	modified:   include/petscsys.h
>>>>>>> 	modified:   src/dm/label/tutorials/ex1f90.F90
>>>>>>> 	modified:   src/ksp/ksp/tests/ex16f.F90
>>>>>>> 	modified:   src/ksp/ksp/tutorials/ex5f.F90
>>>>>>> 	modified:   src/ksp/ksp/tutorials/ex75f.F90
>>>>>>> 	modified:   src/ksp/ksp/tutorials/ex76f.F90
>>>>>>> 	modified:   src/ksp/ksp/tutorials/ex77f.F90
>>>>>>> 	modified:   src/ksp/ksp/tutorials/ex7f.F90
>>>>>>> 	modified:   src/mat/tests/ex196f90.F90
>>>>>>> 	modified:   src/sys/tests/ex13f.F90
>>>>>>> 	modified:   src/sys/tests/ex47f.F90
>>>>>>> 	modified:   src/sys/tutorials/ex17f.F90
>>>>>>> 	modified:   src/sys/tutorials/ex2f.F90
>>>>>>> 	modified:   src/vec/vec/tutorials/ex11f90.F90
>>>>>>> 	modified:   src/vec/vec/tutorials/ex18f.F90
>>>>>>> 	modified:   src/vec/vec/tutorials/ex1f.F90
>>>>>>> 
>>>>>>> no changes added to commit (use "git add" and/or "git commit -a")
>>>>> 
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20200706/a3bd5ba8/attachment.html>

More information about the petsc-dev mailing list