[petsc-dev] Chaco from --download-chaco vs --download-trilinos (Re: broken stuff in next for many days)

[Satish Balay]

Ah - yes. I did check the difference in petsc code wrt
PETSC_HAVE_CHACO_INT_ASSIGNMENT - but didn't check the details inside
trilinos/chaco. So the available standalone chaco tarball is version
2.2 but the version included inside seacas is 3.0.0 [with a switch
from short to int]

We have these hierarchy of packages - and need to figure out a better
[install] interface so the same version is used irrespective of the
mode of install?

Right now - --download-exodusii gets and installs seacas [with only
exodusii enabled]

So we have:

--download-ml --download-chaco --download-zoltan [separate packages]

--download-exodusii [installs seacas - and enables only exodusii]

--download-trilinos [installs trilinos which includes seacas(exodusii,chaco), ml - zoltan]

since seacas is distributed separately and includes exodusii and chaco - have:

--download-seacas=1 [with options exodusii=1,chaco=1 by default [or disabled?]

--download-seacas=1
--download-seacas=1 --download-seacas-exodusii=0 [only install chaco]
--download-seacas-chaco=0 [only install exodusii]

[but zoltan, ml are not distributed separately?]

Or should we always use trilinos? - if one wants any of the above
packages [a huge download]? and enable or disable selected
sub-components as desired?

Another similar package is SuiteSparse. At some point we had install
for only sub-components(cholmod,umfpack) - and then merged them all
into --download-suitesparse.

Satish

On Mon, 23 Apr 2018, Vaclav Hapla wrote:

> I can confirm --download-chaco vs --download-trilinos give different chaco partitioning - I can reproduce on my machine. So I made some investigation.
> 
> PETSC_HAVE_CHACO_INT_ASSIGNMENT is set only for --download-trilinos. 
> This macro makes difference only at 4 places:
> 
> At src/mat/partition/impls/chaco/chaco.c:8 and src/dm/impls/plex/plexpartition.c:1192:
> #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
> #include <chaco.h>
> #else
> /* Older versions of Chaco do not have an include file */
> PETSC_EXTERN int interface(int nvtxs, int *start, int *adjacency, int *vwgts,
>                      float *ewgts, float *x, float *y, float *z, char *outassignname,
>                      char *outfilename, short *assignment, int architecture, int ndims_tot,
>                      int mesh_dims[3], double *goal, int global_method, int local_method,
>                      int rqi_flag, int vmax, int ndims, double eigtol, long seed);
> #endif
> 
> At src/mat/partition/impls/chaco/chaco.c:68 and src/dm/impls/plex/plexpartition.c:1227:
> #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
>   int                   *assignment;
> #else
>   short                 *assignment;  
> #endif
> 
> So it seems PETSc is not responsible for the different outputs.
> 
> In Trilinos, Chaco is shipped within the Seacas package. The notice at http://gsjaardema.github.io/seacas/#chaco <http://gsjaardema.github.io/seacas/#chaco> says
> "The short *assignment argument to the interface function has been changed to int *assignment to permit decompositions with more than 32,768 processors."
> They also say
> "There are also now a CHACO_VERSION_MAJOR, CHACO_VERSION_MINOR, CHACO_VERSION_PATCH defines in chaco.h."
> Looking there, the version is 3.0.0.
> 
> PETSc's configure --download-chaco gets this one: http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz <http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz>
> It actually includes a git repo. There we can see it's the original author's version + some updates by Barry and Satish, mainly to avoid some name clashes.
> The original version is probably this one: https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz <https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz>
> Looking at the contents of the tarball, these sources data back to 2000.
> I tried comparing these two version but there are two many changes in formatting to identify crucial changes easily. But definitely these two versions differ and can give different results.
> 
> The main intent of my tests in src/dm/impls/plex/examples/tutorials/ex5.c is testing HDF5 I/O with different formats and I just needed to somehow distribute the sequential mesh read from exodus file. So I will replace chaco by simple here. But still it's a question whether we want to do something about the confusing situation with Chaco.
> 
> Vaclav
> 
> > 22. 4. 2018 v 17:56, Satish Balay <balay at mcs.anl.gov>:
> > 
> > On Sun, 22 Apr 2018, Smith, Barry F. wrote:
> > 
> >>> It would be good to figure out exactly where the difference comes form
> >>> [wrt chaco from trilinos vs separate install of chaco - or some other
> >>> interaction from trilinos]
> >> 
> >>   Different random numbers being generated?
> > 
> > Its trilinos vs non-trilinos build of packages - so I don't think its 'different random number generation' issue.
> > 
> > Trilinos install does:
> > 
> >    if self.libraries.check(self.dlib, "interface"):
> >      self.addDefine('HAVE_CHACO',1)
> >      self.addDefine('HAVE_CHACO_INT_ASSIGNMENT',1)
> >    if self.libraries.check(self.dlib, "ML_Set_PrintLevel"):
> >      self.addDefine('HAVE_ML',1)
> >    if self.libraries.check(self.dlib, "Zoltan_LB_Partition"):
> >      self.addDefine('HAVE_ZOLTAN',1)
> >    if self.libraries.check(self.dlib, "ex_close"):
> >      self.addDefine('HAVE_EXODUSII',1)
> > 
> > 
> > For one there is different chaco code in petsc based on this flag
> > HAVE_CHACO_INT_ASSIGNMENT. Its not clear to me if this is causing the
> > difference.
> > 
> > --download-chaco is using 'Chaco-2.2-p2.tar.gz'. This matches the
> > latest chaco tarball from their website [Chaco-2.2.tar.gz]. So I do
> > not know if chaco packaged by trilinos has changes that cause this
> > difference [clearly it added some change that resulted in us using the
> > flag HAVE_CHACO_INT_ASSIGNMENT]
> > 
> > And then there is also exodus.. --download-exodus uses the latest
> > snapshot from https://github.com/gsjaardema/seacas/
> > 
> > Satish
> 
>