[petsc-dev] Chaco from --download-chaco vs --download-trilinos (Re: broken stuff in next for many days)

[Vaclav Hapla]

I can confirm --download-chaco vs --download-trilinos give different chaco partitioning - I can reproduce on my machine. So I made some investigation.

PETSC_HAVE_CHACO_INT_ASSIGNMENT is set only for --download-trilinos. 
This macro makes difference only at 4 places:

At src/mat/partition/impls/chaco/chaco.c:8 and src/dm/impls/plex/plexpartition.c:1192:
#if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
#include <chaco.h>
#else
/* Older versions of Chaco do not have an include file */
PETSC_EXTERN int interface(int nvtxs, int *start, int *adjacency, int *vwgts,
                     float *ewgts, float *x, float *y, float *z, char *outassignname,
                     char *outfilename, short *assignment, int architecture, int ndims_tot,
                     int mesh_dims[3], double *goal, int global_method, int local_method,
                     int rqi_flag, int vmax, int ndims, double eigtol, long seed);
#endif

At src/mat/partition/impls/chaco/chaco.c:68 and src/dm/impls/plex/plexpartition.c:1227:
#if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
  int                   *assignment;
#else
  short                 *assignment;  
#endif

So it seems PETSc is not responsible for the different outputs.

In Trilinos, Chaco is shipped within the Seacas package. The notice at http://gsjaardema.github.io/seacas/#chaco <http://gsjaardema.github.io/seacas/#chaco> says
"The short *assignment argument to the interface function has been changed to int *assignment to permit decompositions with more than 32,768 processors."
They also say
"There are also now a CHACO_VERSION_MAJOR, CHACO_VERSION_MINOR, CHACO_VERSION_PATCH defines in chaco.h."
Looking there, the version is 3.0.0.

PETSc's configure --download-chaco gets this one: http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz <http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz>
It actually includes a git repo. There we can see it's the original author's version + some updates by Barry and Satish, mainly to avoid some name clashes.
The original version is probably this one: https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz <https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz>
Looking at the contents of the tarball, these sources data back to 2000.
I tried comparing these two version but there are two many changes in formatting to identify crucial changes easily. But definitely these two versions differ and can give different results.

The main intent of my tests in src/dm/impls/plex/examples/tutorials/ex5.c is testing HDF5 I/O with different formats and I just needed to somehow distribute the sequential mesh read from exodus file. So I will replace chaco by simple here. But still it's a question whether we want to do something about the confusing situation with Chaco.

Vaclav

> 22. 4. 2018 v 17:56, Satish Balay <balay at mcs.anl.gov>:
> 
> On Sun, 22 Apr 2018, Smith, Barry F. wrote:
> 
>>> It would be good to figure out exactly where the difference comes form
>>> [wrt chaco from trilinos vs separate install of chaco - or some other
>>> interaction from trilinos]
>> 
>>   Different random numbers being generated?
> 
> Its trilinos vs non-trilinos build of packages - so I don't think its 'different random number generation' issue.
> 
> Trilinos install does:
> 
>    if self.libraries.check(self.dlib, "interface"):
>      self.addDefine('HAVE_CHACO',1)
>      self.addDefine('HAVE_CHACO_INT_ASSIGNMENT',1)
>    if self.libraries.check(self.dlib, "ML_Set_PrintLevel"):
>      self.addDefine('HAVE_ML',1)
>    if self.libraries.check(self.dlib, "Zoltan_LB_Partition"):
>      self.addDefine('HAVE_ZOLTAN',1)
>    if self.libraries.check(self.dlib, "ex_close"):
>      self.addDefine('HAVE_EXODUSII',1)
> 
> 
> For one there is different chaco code in petsc based on this flag
> HAVE_CHACO_INT_ASSIGNMENT. Its not clear to me if this is causing the
> difference.
> 
> --download-chaco is using 'Chaco-2.2-p2.tar.gz'. This matches the
> latest chaco tarball from their website [Chaco-2.2.tar.gz]. So I do
> not know if chaco packaged by trilinos has changes that cause this
> difference [clearly it added some change that resulted in us using the
> flag HAVE_CHACO_INT_ASSIGNMENT]
> 
> And then there is also exodus.. --download-exodus uses the latest
> snapshot from https://github.com/gsjaardema/seacas/
> 
> Satish

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180423/785c35b2/attachment.html>