itaps-parallel VisIt Linked with Parallel FMDB

Wed Nov 19 10:53:41 CST 2008

Hi Onkar and all,

Late last night I was able to read the mesh from your parallel test case
into VisIt using 4 processors and FMDB's implementation of iMesh/iMeshP.
I am now trying Ting's larger dataset.

Some things I've discovered (don't know if its correct behavior or
not)...

     A. iMeshP_getVtxCoordIndex on the root set always returns nothing
        regardless of entity type and entity topology type passed into
        it.
     B. iMesh_getVtxCoordIndex on root set using entity type of
        iBase_VERTEX and entity topology type of iMesh_POINT and
        neighbor entity type of iBase_VERTEX returns what I think of as
        the global node ids of the nodes of the mesh.
     C. iMesh_getVtxCoordIndex on root set using entity type of
        iBase_REGION entity topolgy type of iMesh_ALL_TOPOLOGIES and
        neighbor entity type of iBase_VERTEX returns connectivity lists
        IN TERMS OF global node ids. 
     D. I needed to call iMesh_getVtxCoordIndex twice, first to get the
        global node ids and second to get the connectivity lists in
        terms of the global node ids and then convert those to local
        node ids to return a mesh piece on a given processor.
     E. I tried to open and read Ting's 8 processor dataset using
        'mpirun -np 8' on Tim's mesh machine. MPI is usually able to
        start more tasks than there are processors and works fine for
        really small parallel jobs in this mode. But I got messages from
        my code indicating that it started only 4 tasks.

For some reason, I cannot get to mesh machine right now.

Mark

On Thu, 2008-11-13 at 17:40 -0500, Onkar Sahni wrote:
> Thats good... keep us updated and in case if you need help in debugging
> (and also if errors are coming from FMDB). I guess we would not be able to
> resolve this before SC08 but if we do I will be happy to load a mesh in
> VisIt and just project it and show that it loads.
> 
> - Onkar
> 
> > Hi Onkar,
> >
> > Some good news. I've managed to link VisIt with parallel FMDB vi iMeshP
> > and open a multi-part sms file. However, when I go to plot it in VisIt,
> > it is segv'ing somewhere in VisIt. So, I still have some work to do.
> > However, up until now, it has been a major chore to get everything to
> > link and operate correctly. VisIt's shared libraries have made it
> > particularly difficult.
> >
> > Mark
> >
> > On Wed, 2008-11-12 at 16:33, Onkar Sahni wrote:
> >> I think my point is being missed, here is my last attempt... if in case
> >> of
> >> MOAB "AC_CONFIG_HEADERS is called to produce MBCN_FCDefs.h" why not one
> >> can produce "ITAPS_FCDefs.h" in a central place and one central
> >> iMesh_protos.h includes ITAPS_FCDefs.h that is generated during as an
> >> install/build process (all ITAPS header files can live in one location,
> >> for example, /usr/local/itaps/headers).
> >>
> >> In summary, user/app. would do following basic steps:
> >>
> >> * Download itaps-headers (say in /<path-itaps-headers>/)
> >> * Run configure to generate ITAPS_FCDefs.h in /<path-itaps-headers>/ (an
> >> app./user would do this step with platform and compiler combination of
> >> their choice)
> >>
> >> * Download any implementation/service (such as FMDB/MOAB/GRUMMP or
> >> iZoltan)
> >> * Run configure and compile it (point to /<path-itaps-headers>/ to get
> >> header files from central place)
> >>
> >> * Write app. code like MHD solver
> >> * Build it and link with appropriate libs. (ofcourse fortran name
> >> mangling
> >> has to be consistent while compling app. code with ITAPS_FCDefs.h
> >> generated by app.)
> >>
> >> Is this not possible?
> >>
> >> - Onkar
> >>
> >> > On Wed, 2008-11-12 at 16:53 -0500, Onkar Sahni wrote:
> >> >> > The FC_FUNC macro winds up getting defined by something; Autoconf's
> >> >> configure script does some work to define it. Or, like Onkar
> >> suggests,
> >> >> you can define some logic ahead of time to set whatever FC_FUNC does
> >> >> using logic like so...
> >> >> >
> >> >> > #ifdef AIX
> >> >> > #define FC_FUNC(lowFunc,UPfunc) _lowFunc
> >> >> > #elif GFORTRAN
> >> >> > #define FC_FUNC(lowFunc,UPfunc) UPfunc
> >> >> > #elif ...
> >> >> > ..
> >> >> > .
> >> >>
> >> >>   So here is my question (basically what I was trying to ask in
> >> >> tele-con)
> >> >> which may or may not be possible:
> >> >>
> >> >>   Can "#define FC_FUNC............" be generated in a central place
> >> >> during
> >> >> the process to install and build itaps softwares (before building any
> >> >
> >> > I don't think it can because its definition depends on the name
> >> mangling
> >> > properties of the fortran compiler being used at time of
> >> (configuration)
> >> > compilation. Yes, the number of possible definitions for FC_FUNC is
> >> > finite, but know which it should be set to for a given compiler (and
> >> > what to use in the way of a detecting that) varies from compiler to
> >> > compiler. The BEST you can achieve is to identify what FC_FUNC should
> >> be
> >> > identified for common cases or cases that you know of so that it
> >> doesn't
> >> > need to be re-computed with each configure/compile.
> >> >
> >> >> implementation/library or service). This would be an extra configure
> >> >> step in say /usr/local/itaps/headers (central location where all
> >> headers
> >> >> sit along with probably configure stuff) and this step would just
> >> >> generate a new header file say "itaps_fc_func.h".
> >> >
> >> >
> >> >> Say if one uses gfortran then the header file generated in "central"
> >> >> place
> >> >> (as an extra step using configure in "central" place) will contain
> >> >> following lines and every service and implementation simply use it.
> >> >>
> >> >>   #ifndef _ITAPS_FC_FUNC_H_
> >> >>   #define _ITAPS_FC_FUNC_H_
> >> >>
> >> >>   /* following line is for specific platform, compiler... combination
> >> */
> >> >>   #define FC_FUNC(lowFunc,UPfunc) UPfunc
> >> >>
> >> >>   #endif
> >> >>
> >> >> - Onkar
> >> >>
> >> >> >
> >> >> > The only issue I see with this approach is that you need to have
> >> >> defined
> >> >> ahead of time all the possible #ifdef #elif cases above and whenever
> >> you
> >> >> go to a new machine, then you'll have to re-visit this logic. So, I
> >> like
> >> >> the approach where Autconf's configure is used to set FC_FUNC.
> >> >> >
> >> >> > Finally, whatever FC_FUNC is set to, it MUST match the actual name
> >> >> mangling used by whatever fortran compiler is being used to compile
> >> the
> >> >> code. So, it is not just a matter of everyone agreeing on an
> >> >> > implementation for FC_FUNC. FC_FUNC must also address whatever name
> >> >> mangling is being used by a given fortran compiler on a given
> >> platform
> >> >> when the code is getting compiled. For that reason, I think the best
> >> >> approach is the Autoconf approach.
> >> >> >
> >> >> > Mark
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Mark C. Miller, Lawrence Livermore National Laboratory
> >> >> > email: mailto:miller86 at llnl.gov
> >> >> > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
> >> >> > (Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> >> > --
> >> > Mark C. Miller, Lawrence Livermore National Laboratory
> >> > email: mailto:miller86 at llnl.gov
> >> > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
> >> > (Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
> >> >
> >> >
> > --
> > Mark C. Miller, Lawrence Livermore National Laboratory
> > email: mailto:miller86 at llnl.gov
> > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
> > (Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
> >
> >
> 
> 
-- 
Mark C. Miller, Lawrence Livermore National Laboratory
email: mailto:miller86 at llnl.gov
(M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
(Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)