[MOAB-dev] write_file freezes on Cetus

[Vijay S. Mahadevan]

Anton, I wasn't able to replicate the issue on my local machine. I
haven't checked it on Mira yet.

> Where can those bin tools from MOAB build be found on Mira filesystem?
> Should i build a new version in my home folder?

Yes, this is recommended. You can point to the system optimized HDF5
and NetCDF installations and then configure/build the moab sources
from the repo on your home folder. It is easier to maintain, upgrade
etc this way.

Here's the output from the runs on my local machine with moab-dev and
your input file:

[ mahadevan at Tachyon:~/Desktop/kanaev ] mpiexec -n 1
/Users/mahadevan/source/moab/examples/ReduceExchangeTags karmanp4.h5m
USERTAG 100
********** reduce_exchange_tags **********

 Input Parameters -
   Filenames: karmanp4.h5m
   Tag: Name=USERTAG Value=100

-Creating tag handle USERTAG...
-Exchanging tags between processors
-Reducing tags between processors
[0]: Number of Partitioned entities: 13900

********** reduce_exchange_tags DONE! **********
[ mahadevan at Tachyon:~/Desktop/kanaev ] mpiexec -n 2
/Users/mahadevan/source/moab/examples/ReduceExchangeTags karmanp4.h5m
USERTAG 100
********** reduce_exchange_tags **********

 Input Parameters -
   Filenames: karmanp4.h5m
   Tag: Name=USERTAG Value=100

-Creating tag handle USERTAG...
-Exchanging tags between processors
-Reducing tags between processors
[0]: Number of Partitioned entities: 6949
[1]: Number of Partitioned entities: 6951

********** reduce_exchange_tags DONE! **********
[ mahadevan at Tachyon:~/Desktop/kanaev ] mpiexec -n 4
/Users/mahadevan/source/moab/examples/ReduceExchangeTags karmanp4.h5m
USERTAG 100
********** reduce_exchange_tags **********

 Input Parameters -
   Filenames: karmanp4.h5m
   Tag: Name=USERTAG Value=100

-Creating tag handle USERTAG...
-Exchanging tags between processors
[3]: Number of Partitioned entities: 3478
-Reducing tags between processors
[0]: Number of Partitioned entities: 3492
[2]: Number of Partitioned entities: 3473
[1]: Number of Partitioned entities: 3457

********** reduce_exchange_tags DONE! **********

We will look at Mira's installation this coming week. Iulian, can you
talk to systems and install a recent version that is accessible for
everyone there ?

Vijay

On Sun, Apr 6, 2014 at 10:57 AM,  <kanaev at ibrae.ac.ru> wrote:
> Hi Vijay,
>
> The problem is still there and really seems to be about my mesh file. I've
> tried to run ReduceExchangeTags with 64bricks_1khex.h5m
>
> qsub -n 4 -t 00:30:00 -A ThermHydraX ReduceExchangeTags 64bricks_1khex.h5m
> USERTAG 100
>
>  and it works but then i tried it with my file
>
> qsub -n 4 -t 00:30:00 -A ThermHydraX ReduceExchangeTags karmanp4.h5m
> USERTAG 100
>
> and it freezes on writing test.h5m
>
> i've also tried
>
> qsub -n 2 -t 00:30:00 -A ThermHydraX ReduceExchangeTags karmanp4.h5m
> USERTAG 100
>
> and it worked
>
> What can be the problem? I've attached the mesh file i used - i made mesh
> in Cubit, converted it to h5m with mbconvert and parted it with mbpart for
> 4 processors. My MOAB version is 4.7.0 and so are the tools mentioned
> above. MOAB version installed on Mira is older.
> Where can those bin tools from MOAB build be found on Mira filesystem?
> Should i build a new version in my home folder?
>
> Thanks,
> Anton
>
> On Fri, 4 Apr 2014 13:56:50 -0500, "Vijay S. Mahadevan" <vijay.m at gmail.com>
> wrote:
>>> I run [akanaev at cetuslac1 ~]$ qsub -n 4 -t 01:00:00 -A ThermHydraX
>>> ReduceExchangeTags karmanp4.h5m USERTAG 100
>>>
>>> It makes 7600 Bytes file test.h5m and freezes like this forever
>>
>> Can you verify if it works on other combinations of processors (2, 8
>> etc) ? If the example still fails, please send over the input mesh
>> file so that we can find out the actual issue.
>>
>> Vijay
>>
>> On Fri, Apr 4, 2014 at 1:34 PM,  <kanaev at ibrae.ac.ru> wrote:
>>> Hello MOAB-dev,
>>>
>>>
>>>
>>> I've tried to run one of your examples ReduceExchangeTags.cpp on Cetus
>>>
>>> Makefile i used:
>>>
>>>
>>>
>>> ZLIB_DIR=/soft/libraries/alcf/current/xl/ZLIB
>>>
>>> HDF5_DIR=/soft/libraries/hdf5/current/cnk-xl/current
>>>
>>> MOAB_DIR=/soft/applications/moab/4.6.0/cnk-xl/current
>>>
>>>
>>>
>>> LDADD = -lMOAB -liMesh -lhdf5
>>>
>>> LIBS =-lz
>>>
>>>
>>>
>>> CXX = mpixlcxx
>>>
>>> CXXFLAGS = -O0
>>>
>>>
>>>
>>> LDFLAGS =  -L$(ZLIB_DIR)/lib  -L$(HDF5_DIR)/lib -L$(MOAB_DIR)/lib
>>>
>>>
>>>
>>> INCLUDES=-I$(MOAB_DIR)/include
>>>
>>>
>>>
>>> CXXCOMPILE = $(CXX) $(CXXFLAGS) $(INCLUDES)
>>>
>>>
>>>
>>> ReduceExchangeTags: ReduceExchangeTags.o
>>>
>>> @rm -f ReduceExchangeTags
>>>
>>> $(CXXCOMPILE) ReduceExchangeTags.o -o ReduceExchangeTags $(LDFLAGS)
>> $(LDADD)
>>> $(LIBS)
>>>
>>>
>>>
>>> ReduceExchangeTags.o: ReduceExchangeTags.cpp
>>>
>>> $(CXXCOMPILE) -c ReduceExchangeTags.cpp
>>>
>>> clean:
>>>
>>> rm -rf *o ReduceExchangeTags
>>>
>>>
>>>
>>> I run [akanaev at cetuslac1 ~]$ qsub -n 4 -t 01:00:00 -A ThermHydraX
>>> ReduceExchangeTags karmanp4.h5m USERTAG 100
>>>
>>> It makes 7600 Bytes file test.h5m and freezes like this forever
>>>
>>> Please check what's wrong
>>>
>>>
>>>
>>> Anton