[MOAB-dev] write_file freezes on Cetus

[Grindeanu, Iulian R.]

Anton,
To get a better feel of what is going on, please run with DEBUG_IO=1 for writing. 

so you will need to recompile the example to write with these options:
about line 219 in the file MOABsource/examples/ReduceExchangeTags.cpp

...
// Write out to output file to visualize reduction/exchange of tag data
  mbi->write_file("test.h5m", "H5M", "PARALLEL=WRITE_PART;DEBUG_IO=1;");
...

Or you can even try with DEBUG_IO=3, and send us the output. do not give 30 minutes, give the minimum (on vesta is 5), it should take less than a minute to write ( or hang up )

Thanks,
Iulian
________________________________________
From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on behalf of Rob Latham [robl at mcs.anl.gov]
Sent: Tuesday, April 08, 2014 10:39 AM
To: moab-dev at mcs.anl.gov
Subject: Re: [MOAB-dev] write_file freezes on Cetus

On 04/07/2014 07:45 PM, Grindeanu, Iulian R. wrote:
> Hi Anton,
> It worked for me on Vesta, even with your file, so it must be something with your build
> this is how I configure moab on vesta, mira should be very similar.
> You do not need mbcslam, or zoltan for your test (or pnetcdf)

he did say cetus, which is running a slightly older driver.

I'd like to see a stack trace of the hang, in case it's actually just
doing i/o really really slowly.

==rob

>
> ../moab/configure CC=mpixlc CFLAGS=-qarch=qp CXX=mpixlcxx CXXFLAGS="-qarch=qp -qpic=large -qmaxmem=-1" FC=mpixlf90 FCFLAGS=-qarch=qp --prefix=/home/grindean/lib/moab --enable-optimize --disable-debug --with-mpi --build=ppc64-redhat-linux --host=powerpc64-bgq-linux --with-zlib=/soft/libraries/alcf/current/xl/ZLIB --with-hdf5=/soft/libraries/hdf5/current/cnk-xl/current --disable-shared --enable-static QTDIR=/usr/lib64/qt-3.3 build_alias=ppc64-redhat-linux host_alias=powerpc64-bgq-linux --with-pnetcdf=/soft/libraries/pnetcdf/current/cnk-xl/V1R2M0-20131217  --enable-mbcslam --with-zoltan=/home/grindean/3rdparty/lib/Zoltan_trilinos --with-netcdf=/soft/libraries/netcdf/4.3.0-f4.2/cnk-xl/current
>
> All needed libraries are under /soft
>
> Iulian
>
>
> ________________________________________
> From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on behalf of kanaev at ibrae.ac.ru [kanaev at ibrae.ac.ru]
> Sent: Sunday, April 06, 2014 10:57 AM
> To: Vijay S. Mahadevan
> Cc: MOAB dev
> Subject: Re: [MOAB-dev] write_file freezes on Cetus
>
> Hi Vijay,
>
> The problem is still there and really seems to be about my mesh file. I've
> tried to run ReduceExchangeTags with 64bricks_1khex.h5m
>
> qsub -n 4 -t 00:30:00 -A ThermHydraX ReduceExchangeTags 64bricks_1khex.h5m
> USERTAG 100
>
>   and it works but then i tried it with my file
>
> qsub -n 4 -t 00:30:00 -A ThermHydraX ReduceExchangeTags karmanp4.h5m
> USERTAG 100
>
> and it freezes on writing test.h5m
>
> i've also tried
>
> qsub -n 2 -t 00:30:00 -A ThermHydraX ReduceExchangeTags karmanp4.h5m
> USERTAG 100
>
> and it worked
>
> What can be the problem? I've attached the mesh file i used - i made mesh
> in Cubit, converted it to h5m with mbconvert and parted it with mbpart for
> 4 processors. My MOAB version is 4.7.0 and so are the tools mentioned
> above. MOAB version installed on Mira is older.
> Where can those bin tools from MOAB build be found on Mira filesystem?
> Should i build a new version in my home folder?
>
> Thanks,
> Anton
>
> On Fri, 4 Apr 2014 13:56:50 -0500, "Vijay S. Mahadevan" <vijay.m at gmail.com>
> wrote:
>>> I run [akanaev at cetuslac1 ~]$ qsub -n 4 -t 01:00:00 -A ThermHydraX
>>> ReduceExchangeTags karmanp4.h5m USERTAG 100
>>>
>>> It makes 7600 Bytes file test.h5m and freezes like this forever
>>
>> Can you verify if it works on other combinations of processors (2, 8
>> etc) ? If the example still fails, please send over the input mesh
>> file so that we can find out the actual issue.
>>
>> Vijay
>>
>> On Fri, Apr 4, 2014 at 1:34 PM,  <kanaev at ibrae.ac.ru> wrote:
>>> Hello MOAB-dev,
>>>
>>>
>>>
>>> I've tried to run one of your examples ReduceExchangeTags.cpp on Cetus
>>>
>>> Makefile i used:
>>>
>>>
>>>
>>> ZLIB_DIR=/soft/libraries/alcf/current/xl/ZLIB
>>>
>>> HDF5_DIR=/soft/libraries/hdf5/current/cnk-xl/current
>>>
>>> MOAB_DIR=/soft/applications/moab/4.6.0/cnk-xl/current
>>>
>>>
>>>
>>> LDADD = -lMOAB -liMesh -lhdf5
>>>
>>> LIBS =-lz
>>>
>>>
>>>
>>> CXX = mpixlcxx
>>>
>>> CXXFLAGS = -O0
>>>
>>>
>>>
>>> LDFLAGS =  -L$(ZLIB_DIR)/lib  -L$(HDF5_DIR)/lib -L$(MOAB_DIR)/lib
>>>
>>>
>>>
>>> INCLUDES=-I$(MOAB_DIR)/include
>>>
>>>
>>>
>>> CXXCOMPILE = $(CXX) $(CXXFLAGS) $(INCLUDES)
>>>
>>>
>>>
>>> ReduceExchangeTags: ReduceExchangeTags.o
>>>
>>> @rm -f ReduceExchangeTags
>>>
>>> $(CXXCOMPILE) ReduceExchangeTags.o -o ReduceExchangeTags $(LDFLAGS)
>> $(LDADD)
>>> $(LIBS)
>>>
>>>
>>>
>>> ReduceExchangeTags.o: ReduceExchangeTags.cpp
>>>
>>> $(CXXCOMPILE) -c ReduceExchangeTags.cpp
>>>
>>> clean:
>>>
>>> rm -rf *o ReduceExchangeTags
>>>
>>>
>>>
>>> I run [akanaev at cetuslac1 ~]$ qsub -n 4 -t 01:00:00 -A ThermHydraX
>>> ReduceExchangeTags karmanp4.h5m USERTAG 100
>>>
>>> It makes 7600 Bytes file test.h5m and freezes like this forever
>>>
>>> Please check what's wrong
>>>
>>>
>>>
>>> Anton

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA