[petsc-dev] Chaco from --download-chaco vs --download-trilinos (Re: broken stuff in next for many days)

[Smith, Barry F.]

   Let's just have the alt file for the chaco diffs and leave everything else the way it. I can't see the effort to change builds stuff worth it in this case.

   Barry


> On Apr 23, 2018, at 2:13 PM, Balay, Satish <balay at mcs.anl.gov> wrote:
> 
> I was thinking more in terms of: our download options should match
> source distribution model of externalpackages [but not necessarily
> build all components]
> 
> I see chaco is primarily part of seacas and not distributed separately.
> [and then seacas is redistributed in trilinos]
> 
> So how about:
> 
> --download-seacas [build chaco and exodusii]
> 
> And remove the separate package support for exodusii and chaco [i.e
> remove --download-exodusii --download-chaco]
> 
> [and then somehow make the seacas build via trilinos match the standalone build]
> 
> I see zoltan is distributed separately, in seacas and also in trilinos
> [so current --download-zoltan is still ok?]
> 
> ML is the odd one out - I guess we'll have to extract/repackage the
> latest version from trilinos - and use it with --download-ml.
> 
> Satish
> 
> On Mon, 23 Apr 2018, Smith, Barry F. wrote:
> 
>> 
>>  Making people install all of Trilinos for just Chaco or just ML is just plain mean and we are not mean.
>> 
>>   Suitesparse is at least not huge and is well contained so just because we install all of it doesn't mean we should require the same route for Trilinos.
>> 
>>   Barry
>> 
>> 
>> 
>>> On Apr 23, 2018, at 10:11 AM, Balay, Satish <balay at mcs.anl.gov> wrote:
>>> 
>>> Ah - yes. I did check the difference in petsc code wrt
>>> PETSC_HAVE_CHACO_INT_ASSIGNMENT - but didn't check the details inside
>>> trilinos/chaco. So the available standalone chaco tarball is version
>>> 2.2 but the version included inside seacas is 3.0.0 [with a switch
>>> from short to int]
>>> 
>>> We have these hierarchy of packages - and need to figure out a better
>>> [install] interface so the same version is used irrespective of the
>>> mode of install?
>>> 
>>> Right now - --download-exodusii gets and installs seacas [with only
>>> exodusii enabled]
>>> 
>>> So we have:
>>> 
>>> --download-ml --download-chaco --download-zoltan [separate packages]
>>> 
>>> --download-exodusii [installs seacas - and enables only exodusii]
>>> 
>>> --download-trilinos [installs trilinos which includes seacas(exodusii,chaco), ml - zoltan]
>>> 
>>> since seacas is distributed separately and includes exodusii and chaco - have:
>>> 
>>> --download-seacas=1 [with options exodusii=1,chaco=1 by default [or disabled?]
>>> 
>>> --download-seacas=1
>>> --download-seacas=1 --download-seacas-exodusii=0 [only install chaco]
>>> --download-seacas-chaco=0 [only install exodusii]
>>> 
>>> [but zoltan, ml are not distributed separately?]
>>> 
>>> Or should we always use trilinos? - if one wants any of the above
>>> packages [a huge download]? and enable or disable selected
>>> sub-components as desired?
>>> 
>>> Another similar package is SuiteSparse. At some point we had install
>>> for only sub-components(cholmod,umfpack) - and then merged them all
>>> into --download-suitesparse.
>>> 
>>> Satish
>>> 
>>> On Mon, 23 Apr 2018, Vaclav Hapla wrote:
>>> 
>>>> I can confirm --download-chaco vs --download-trilinos give different chaco partitioning - I can reproduce on my machine. So I made some investigation.
>>>> 
>>>> PETSC_HAVE_CHACO_INT_ASSIGNMENT is set only for --download-trilinos. 
>>>> This macro makes difference only at 4 places:
>>>> 
>>>> At src/mat/partition/impls/chaco/chaco.c:8 and src/dm/impls/plex/plexpartition.c:1192:
>>>> #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
>>>> #include <chaco.h>
>>>> #else
>>>> /* Older versions of Chaco do not have an include file */
>>>> PETSC_EXTERN int interface(int nvtxs, int *start, int *adjacency, int *vwgts,
>>>>                    float *ewgts, float *x, float *y, float *z, char *outassignname,
>>>>                    char *outfilename, short *assignment, int architecture, int ndims_tot,
>>>>                    int mesh_dims[3], double *goal, int global_method, int local_method,
>>>>                    int rqi_flag, int vmax, int ndims, double eigtol, long seed);
>>>> #endif
>>>> 
>>>> At src/mat/partition/impls/chaco/chaco.c:68 and src/dm/impls/plex/plexpartition.c:1227:
>>>> #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT)
>>>> int                   *assignment;
>>>> #else
>>>> short                 *assignment;  
>>>> #endif
>>>> 
>>>> So it seems PETSc is not responsible for the different outputs.
>>>> 
>>>> In Trilinos, Chaco is shipped within the Seacas package. The notice at http://gsjaardema.github.io/seacas/#chaco <http://gsjaardema.github.io/seacas/#chaco> says
>>>> "The short *assignment argument to the interface function has been changed to int *assignment to permit decompositions with more than 32,768 processors."
>>>> They also say
>>>> "There are also now a CHACO_VERSION_MAJOR, CHACO_VERSION_MINOR, CHACO_VERSION_PATCH defines in chaco.h."
>>>> Looking there, the version is 3.0.0.
>>>> 
>>>> PETSc's configure --download-chaco gets this one: http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz <http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz>
>>>> It actually includes a git repo. There we can see it's the original author's version + some updates by Barry and Satish, mainly to avoid some name clashes.
>>>> The original version is probably this one: https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz <https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz>
>>>> Looking at the contents of the tarball, these sources data back to 2000.
>>>> I tried comparing these two version but there are two many changes in formatting to identify crucial changes easily. But definitely these two versions differ and can give different results.
>>>> 
>>>> The main intent of my tests in src/dm/impls/plex/examples/tutorials/ex5.c is testing HDF5 I/O with different formats and I just needed to somehow distribute the sequential mesh read from exodus file. So I will replace chaco by simple here. But still it's a question whether we want to do something about the confusing situation with Chaco.
>>>> 
>>>> Vaclav
>>>> 
>>>>> 22. 4. 2018 v 17:56, Satish Balay <balay at mcs.anl.gov>:
>>>>> 
>>>>> On Sun, 22 Apr 2018, Smith, Barry F. wrote:
>>>>> 
>>>>>>> It would be good to figure out exactly where the difference comes form
>>>>>>> [wrt chaco from trilinos vs separate install of chaco - or some other
>>>>>>> interaction from trilinos]
>>>>>> 
>>>>>> Different random numbers being generated?
>>>>> 
>>>>> Its trilinos vs non-trilinos build of packages - so I don't think its 'different random number generation' issue.
>>>>> 
>>>>> Trilinos install does:
>>>>> 
>>>>>  if self.libraries.check(self.dlib, "interface"):
>>>>>    self.addDefine('HAVE_CHACO',1)
>>>>>    self.addDefine('HAVE_CHACO_INT_ASSIGNMENT',1)
>>>>>  if self.libraries.check(self.dlib, "ML_Set_PrintLevel"):
>>>>>    self.addDefine('HAVE_ML',1)
>>>>>  if self.libraries.check(self.dlib, "Zoltan_LB_Partition"):
>>>>>    self.addDefine('HAVE_ZOLTAN',1)
>>>>>  if self.libraries.check(self.dlib, "ex_close"):
>>>>>    self.addDefine('HAVE_EXODUSII',1)
>>>>> 
>>>>> 
>>>>> For one there is different chaco code in petsc based on this flag
>>>>> HAVE_CHACO_INT_ASSIGNMENT. Its not clear to me if this is causing the
>>>>> difference.
>>>>> 
>>>>> --download-chaco is using 'Chaco-2.2-p2.tar.gz'. This matches the
>>>>> latest chaco tarball from their website [Chaco-2.2.tar.gz]. So I do
>>>>> not know if chaco packaged by trilinos has changes that cause this
>>>>> difference [clearly it added some change that resulted in us using the
>>>>> flag HAVE_CHACO_INT_ASSIGNMENT]
>>>>> 
>>>>> And then there is also exodus.. --download-exodus uses the latest
>>>>> snapshot from https://github.com/gsjaardema/seacas/
>>>>> 
>>>>> Satish
>>>> 
>>>> 
>>> 
>> 
>> 
>