[MPICH] Problem when enabling ADIOI_MPE_LOGGING

[Anthony Chan]

Just to be clear, ADIOI_MPE_LOGGING only enables client side logging.
Also I would think that a typical MPICH2 build with ADIOI_MPE_LOGGING 
enabled will cause compilation error (so not sure what you did).

In order to get ADIOI_MPE_LOGGING to work correctly, you need to build
mpich2 *TWICE* with second build dependent on the first one.  The main 
reason is that MPICH2 builds MPE after ROMIO/ADIO.  In order words, 
ADIOI_MPE_LOGGING can't be enabled without a valid set of MPE logging 
header files. So here are the steps that I would do,

1, untar a new copy of mpich2, say mpich2-1.0.6p1, into directory
    <mpich2>

2, build a standard version of mpich2 with all the options that you
    normally need using VPATH build.  You don't want to do inpath build
    here because we need clean mpich2 source tree for the 2nd build, i.e.

    mkdir <build_std>
    cd <build_std>
    <mpich2>/configure CC=... CXX=... F77=... F90=... \
                       --prefix=<install_std>
    make
    make install

    Notice we have the 1st installation of MPICH2 in <install_std>.

3, build another version of mpich2 with an extra CFLAGS that enables
    ADIOI_MPE_LOGGING and points to a valid set of MPE logging include
    files, i.e. -I<install_std>/include.

    mkdir <build_adiolog>
    cd <build_adiolog>
    <mpich2>/configure CC=... CXX=... F77=... F90=...   \
           --prefix=<install_adiolog>                   \
           --with-file-system=pvfs2+pvfs+ufs+nfs        \
           --with-pvfs2=<pvfs-install-dir>              \
           CFLAGS="-DADIOI_MPE_LOGGING -I<install_std>/include"
    make
    make install

    The 2nd installation of MPICH2 will be in <install_adiolog>.

4, now you can compile your MPIO+PVFS application with

    <install_adiolog>/bin/mpicc -mpe=mpilog -c foo.c

    and link as

    <install_adiolog>/bin/mpicc -mpe=mpilog -o <exe> *.o

    mpiXX wrappers from <install_adiolog> must be used with -mpe=mpilog
    for linking, because libmpich.a in <install_adiolog> has references
    to MPE symbols.

5, Now run the <exe> as usual.  This will generate clog2 file with
    {read,write,seek,...} nested inside MPIO states.  Then invoke
    jumpshot on the clog2, and follows the GUI.

ADIOI_MPE_LOGGING isn't meant for general users, so no big effort
has been spent to make this easy. Let me know if this works for you
though.

A.Chan

On Fri, 30 Nov 2007, Christina Patrick wrote:

> Hi Everybody,
>
> I want to monitor the time it takes to complete collective I/O in my
> application. However, I also want the breakup of the times the
> collective call spends in calling the PVFS functions as well as the
> communication. I saw some ready made logging present in the files
> ad_pvfs2_write.c. The calls PVFS_sys_write were logged by MPE_Log
> event() as below (one excerpt of the code):
>
> -------------------------------
> #ifdef ADIOI_MPE_LOGGING
>                    MPE_Log_event( ADIOI_MPE_write_a, 0, NULL );
> #endif
>                    err_flag = PVFS_sys_write(pvfs_fs->object_ref, file_req,
>                                              file_offsets, PVFS_BOTTOM,
>                                              mem_req,
>                                              &(pvfs_fs->credentials),
>                                              &resp_io);
> #ifdef ADIOI_MPE_LOGGING
>                    MPE_Log_event( ADIOI_MPE_write_b, 0, NULL );
> #endif
> -------------------------------
>
> I modified the makefiles to add the debug flag ADIOI_MPE_LOGGING and
> enabled the logging. However, when I run my application and generate
> the clog and slog files, I cannot see these events using jumpshot. I
> logged the event ID's when they are created in ad_init.c and they are
> as below from a sample run:
>
> ADIOI_MPE_write_a     = 604 ADIOI_MPE_write_b     = 605
>
> What am I missing? Is there something that I need to do additionally?
> How exactly should I profile my application and which tool should I
> use to get the breakup of these times.
>
> I have never profiled MPI before. I would really appreciate any help
> and suggestions that you could offer.
>
> Thanks and Regards,
> Christina.
>
>