[petsc-dev] alternatives to alt files

[Jed Brown]

Hapla Vaclav via petsc-dev <petsc-dev at mcs.anl.gov> writes:

> I'm not sure you about this nomenclature. Let me take DMPlexInterpolate as an example:
> So (1) comparing the actual view of the interpolated mesh to the snapshot stored in the output file would be an integration test, whereas (2) calling a set of DMPlexCheck* functions, which would fail if something is wrong in the produced topology, would be a unit test?
>
> I wouldn't use the term integration testing here because here I would test exactly the same level of functionality, whereas integration testing should mean one level up, i.e. tests how parts of the system work together<https://codeutopia.net/blog/2015/04/11/what-are-unit-testing-integration-testing-and-functional-testing/>. Or do you mean those checks could be run as separate tests while the diff test is indivisible? But these checks could be for example easily turned into functions returning booleans, and that test would print these booleans and compare with the output file. I think this wouldn't change it from a unit test to an integration test.
>
> What PETSc diff tests use could be IMHO called snapshot/golden master/characterization testing [approach], applied to both unit and integration testing [levels].
> https://sqa.stackexchange.com/questions/29696/what-is-snapshot-testing
> https://lists.boost.org/boost-users/2013/04/78334.php
>
> I really don't want to be knit-picking here; I just to want to clearly understand what is the intention.

It's spelled "nit-picking", but you're entirely correct.  We're being
sloppy/non-specific to call it "integration", but the theme is
consistent in that a "unit" tests smaller scope than "integration".
It's kinda tedious to set up fixtures to unit test everything so we
often have integration (or snapshot/...) tests for things that could be
unit testable.  That unfortunately means that test failures often
require debugging to isolate which component is failing.