[MOAB-dev] [Nek5000-users] MOAB partitioning
Popov, Emilian L.
popove at ornl.gov
Wed Nov 11 14:50:57 CST 2015
Do I also need parallel metis and netcdf?
Thanks,
-emilian
-----Original Message-----
From: Grindeanu, Iulian R. [mailto:iulian at mcs.anl.gov]
Sent: Wednesday, November 11, 2015 3:39 PM
To: Popov, Emilian L.; Vijay S. Mahadevan
Cc: moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
mbpart should not crash if you compile with serial config.
But you will not be able to use nek moab in parallel with that version of moab, so it makes sense to build parallel moab ________________________________________
From: Popov, Emilian L. [popove at ornl.gov]
Sent: Wednesday, November 11, 2015 2:21 PM
To: Grindeanu, Iulian R.; Vijay S. Mahadevan
Cc: moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
Is this the problem? the machine is a cluster and it is configures for multiple users, but let me see.
-emilian
-----Original Message-----
From: Grindeanu, Iulian R. [mailto:iulian at mcs.anl.gov]
Sent: Wednesday, November 11, 2015 2:45 PM
To: Popov, Emilian L.; Vijay S. Mahadevan
Cc: moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
It is not parallel, indeed
...
57 Features:
58 ---------
59 Parallel HDF5: no
can you find a parallel hdf5 ?
Can you build one?
on our machines, we use something like this to configure/build hdf5, after downloading the source
#!/bin/bash
export SOURCE=/homes/fathom/3rdparty/hdf5/hdf5-1.8.12
export MPICH=/homes/fathom/3rdparty/mpich/3.1/gnu
export ZLIB=/home/fathom/3rdparty/zlib/1.2.8/gcc
export MYINSTALL=/homes/fathom/3rdparty/hdf5/1.8.12/mpich-3.1/gnu
$SOURCE/configure --with-zlib=$ZLIB\
--enable-parallel \
--prefix=$MYINSTALL \
CXX=$MPICH/bin/mpicxx \
F77=$MPICH/bin/mpif77 \
FC=$MPICH/bin/mpif90 \
F90=$MPICH/bin/mpif90 \
CC=$MPICH/bin/mpicc \
--enable-fortran --enable-fortran2003 --enable-shared 2>&1 | tee configure.log
make 2>&1 | tee make.log &
make install 2>&1 | tee make.install &
________________________________________
From: Popov, Emilian L. [popove at ornl.gov]
Sent: Wednesday, November 11, 2015 1:23 PM
To: Grindeanu, Iulian R.; Vijay S. Mahadevan
Cc: moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
No I didn't mbconvert, but the result is the same
Here is the hdf5 settings file, I'd say it is not parallel, but please look.
Thanks,
-emilian
-----Original Message-----
From: Grindeanu, Iulian R. [mailto:iulian at mcs.anl.gov]
Sent: Wednesday, November 11, 2015 1:05 PM
To: Popov, Emilian L.; Vijay S. Mahadevan
Cc: moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
Hi,
DId you mbconvert again?
You should be able to use mbpart directly on the original exodus file Still, you did not compile/configure with mpi. You will not run anything related to moab in parallel.
Is your hdf5 build with parallel option?
You can look probably at
/opt/hdf5-1.8.13-gcc/lib/libhdf5.settings
search for Parallel HDF5, and for C Compiler
If parallel, I would use the same mpi implementation
(so configure moab with --with-mpi=, pointing to the same mpi as the one used for hdf5 configuration)
Iulian
________________________________________
From: Popov, Emilian L. [popove at ornl.gov]
Sent: Wednesday, November 11, 2015 11:38 AM
To: Vijay S. Mahadevan
Cc: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
Vijay,
I recompiled with hdf5 but still get the same segmentation fault. Here are the log files from configuration/compilation.
Thanks,
-emilian
-----Original Message-----
From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
Sent: Wednesday, November 11, 2015 11:46 AM
To: Popov, Emilian L.
Cc: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
> I can do that, is there any other option if I want to run Nek in parallel?
You need to compile, HDF5, MOAB and NEK with MPI. Also configure MOAB with NetCDF if you want to convert exodus files to h5m.
Vijay
On Wed, Nov 11, 2015 at 10:38 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
> I can do that, is there any other option if I want to run Nek in parallel?
>
> Thanks,
> -emilian
>
>
> -----Original Message-----
> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
> Sent: Wednesday, November 11, 2015 11:32 AM
> To: Popov, Emilian L.
> Cc: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>
> Emilian,
>
> You need to configure MOAB with HDF5 if you want to use the h5m file format. You should be able to apply mbpart directly to the exodus file however I think, but linkage to NEK with parallel I/O is only supported currently with h5m.
>
> Can you reconfigure MOAB with HDF5 ?
>
> Vijay
>
> On Wed, Nov 11, 2015 at 10:25 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
>> I converted it with 'mbconvert from an exodus file
>>
>> Thanks,
>> -emilian
>>
>>
>> -----Original Message-----
>> From: Grindeanu, Iulian R. [mailto:iulian at mcs.anl.gov]
>> Sent: Wednesday, November 11, 2015 11:22 AM
>> To: Popov, Emilian L.; Vijay S. Mahadevan
>> Cc: moab-dev at mcs.anl.gov
>> Subject: RE: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>
>> It is not an h5m file, it is exodus
>> so I assume the h5m moab writer failed, and used exodus writer
>>
>> how did you get this file?
>> if you change the extension to exo,
>> (pipe150cm.exo)
>> it will work
>>
>> with something like
>> mbpart -m ML_KWAY 32 pipe150cm.exo out.h5m
>>
>> Still, how did you obtain it?
>>
>> Thanks,
>> Iulian
>>
>> ________________________________________
>> From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on
>> behalf of Popov, Emilian L. [popove at ornl.gov]
>> Sent: Wednesday, November 11, 2015 10:12 AM
>> To: Vijay S. Mahadevan
>> Cc: moab-dev at mcs.anl.gov
>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>
>> The check on Moab build passes with no error messages: I don't know how to run Metis programs, but run one of them with help option and it displays the help, so I suppose I can run them.
>>
>> Here is the mesh file I use:
>>
>> Thanks,
>> -emilian
>>
>>
>> -----Original Message-----
>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>> Sent: Wednesday, November 11, 2015 10:39 AM
>> To: Popov, Emilian L.
>> Cc: moab-dev at mcs.anl.gov
>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>
>> Did make check pass on your MOAB build ? Can you run the programs in metis bin directory ?
>>
>> If the answer is yes to both questions above, can you send us the mesh file (hopefully not too large) that needs to be partitioned so that we can see if this is replicable ?
>>
>> Vijay
>>
>>
>> On Wed, Nov 11, 2015 at 9:32 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>> Vijay and all:
>>>
>>> When I run: mbpart -m ML_KWAY 32 <file> the code exits with a segmentation fault. Any thoughts?
>>>
>>> Thanks,
>>> -emilian
>>>
>>>
>>> -----Original Message-----
>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>> Sent: Monday, November 09, 2015 10:27 AM
>>> To: Popov, Emilian L.
>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>
>>> You can run "make check" to verify the build worked fine. Then run "make install", go to build/examples directory, use the makefile to compile an example. Update MOAB_DIR as needed in the makefile.
>>>
>>> Let us know if you run into any other issues.
>>>
>>> Vijay
>>>
>>> On Mon, Nov 9, 2015 at 9:21 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>> Yes, that was the problem, all looks OK now, appreciate the help. Moab seems to install fine as well. Is there any way I can test the Moab install?
>>>>
>>>> Thanks,
>>>> -emilian
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>>> Sent: Monday, November 09, 2015 10:11 AM
>>>> To: Popov, Emilian L.; moab-dev at mcs.anl.gov
>>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>>
>>>> You need to install metis (make install). This will install the header/libs under the prefix directory if you specified it during configuration.
>>>>
>>>> MOAB will NOT work with Metis source directory. Let me know if your installation went correctly.
>>>>
>>>> Vijay
>>>>
>>>> On Mon, Nov 9, 2015 at 9:06 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>>> Hi Vijay,
>>>>> I rebuilt metis, and it should be fine now. The library 'libmetis.a' is in ..../metis-5.1.0/build/Linux-x86_64/libmetis, there is no /lib directory.
>>>>> Let me know pls whether I need to change the Moab configure script in one way or other.
>>>>>
>>>>> Thanks,
>>>>> -emilian
>>>>>
>>>>> -----Original Message-----
>>>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>>>> Sent: Thursday, November 05, 2015 5:42 PM
>>>>> To: Popov, Emilian L.
>>>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>>>
>>>>> Hmm, I would say that your installation is screwed up for metis. Are you sure you configured Metis with a prefix ? MOAB can only work with the installed metis directory and not directly with the sources since these keep changing every version.
>>>>>
>>>>> When you install Metis correctly, you should see something like this:
>>>>>
>>>>> [ vijaysm at tachyon ~ ]> find /usr/software/metis-5.1.0/
>>>>> /usr/software/metis-5.1.0/ /usr/software/metis-5.1.0/include
>>>>> /usr/software/metis-5.1.0/include/metis.h
>>>>> /usr/software/metis-5.1.0/bin
>>>>> /usr/software/metis-5.1.0/bin/cmpfillin
>>>>> /usr/software/metis-5.1.0/bin/ndmetis
>>>>> /usr/software/metis-5.1.0/bin/graphchk
>>>>> /usr/software/metis-5.1.0/bin/m2gmetis
>>>>> /usr/software/metis-5.1.0/bin/gpmetis
>>>>> /usr/software/metis-5.1.0/bin/mpmetis
>>>>> /usr/software/metis-5.1.0/lib
>>>>> /usr/software/metis-5.1.0/lib/libmetis.a
>>>>>
>>>>> Vijay
>>>>>
>>>>> On Thu, Nov 5, 2015 at 4:34 PM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>>>> I'm not sure whether we look for a dynamic/static library or for bunch of files, but in /home/owp/METIS/metis-5.1.0 there is a folder 'libmetis' that contains several C files and headers.
>>>>>>
>>>>>> Thanks,
>>>>>> -emilian
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>>>>> Sent: Thursday, November 05, 2015 5:03 PM
>>>>>> To: Popov, Emilian L.
>>>>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>>>>
>>>>>> From your config.log
>>>>>>
>>>>>> METIS_INC_FLAGS='-I/home/owp/METIS/metis-5.1.0/include'
>>>>>> METIS_LIBS='-L/home/owp/METIS/metis-5.1.0 -lmetis'
>>>>>>
>>>>>> Is the metis library not in /home/owp/METIS/metis-5.1.0 ? Is it in /home/owp/METIS/metis-5.1.0/lib ? Please verify and let me know. It configures fine on my local build and on our nightlies.
>>>>>>
>>>>>> Vijay
>>>>>>
>>>>>> On Thu, Nov 5, 2015 at 2:51 PM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>>>>> See the end of make.log: from the patched version
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -emilian
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>>>>>> Sent: Thursday, November 05, 2015 3:33 PM
>>>>>>> To: Popov, Emilian L.; moab-dev at mcs.anl.gov
>>>>>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>>>>>
>>>>>>> If you are still interested in a patched tarball, its available
>>>>>>> at
>>>>>>>
>>>>>>> ftp://ftp.mcs.anl.gov/pub/fathom/moab-4.8.2p1.tar.gz
>>>>>>>
>>>>>>> Let me know if this fixes your issues. Thanks.
>>>>>>>
>>>>>>> Vijay
>>>>>>>
>>>>>>> On Thu, Nov 5, 2015 at 2:27 PM, Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>>>>>>> Ah I see. I recently merge a PR to fix precisely this issue.
>>>>>>>>
>>>>>>>> https://bitbucket.org/fathomteam/moab/pull-requests/157/fix-met
>>>>>>>> i
>>>>>>>> s
>>>>>>>> -
>>>>>>>> p
>>>>>>>> a
>>>>>>>> r
>>>>>>>> t
>>>>>>>> itioner-for-serial-builds/diff
>>>>>>>>
>>>>>>>> I can backport the fix to moab-4.8.2 if you prefer that. Or you
>>>>>>>> could directly use MOAB master from Bitbucket
>>>>>>>> (https://bitbucket.org/fathomteam/moab/overview).
>>>>>>>>
>>>>>>>> Vijay
>>>>>>>>
>>>>>>>> On Thu, Nov 5, 2015 at 2:00 PM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>>>>>>> Apparently I've sent you the config log instead of make log. Here it is, complains form 'mpi.h'
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -emilian
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Vijay S. Mahadevan [mailto:vijay.m at gmail.com]
>>>>>>>>> Sent: Thursday, November 05, 2015 12:49 PM
>>>>>>>>> To: Popov, Emilian L.
>>>>>>>>> Cc: moab-dev at mcs.anl.gov
>>>>>>>>> Subject: Re: [MOAB-dev] [Nek5000-users] MOAB partitioning
>>>>>>>>>
>>>>>>>>> Emilian, I don't see what is the problem anymore. MOAB successfully configured according to your logs and mbpart will be built with Metis support. From your output:
>>>>>>>>>
>>>>>>>>> METIS_INC_FLAGS='-I/home/owp/METIS/metis-5.1.0/include'
>>>>>>>>> METIS_LIBS='-L/home/owp/METIS/metis-5.1.0 -lmetis'
>>>>>>>>> #define HAVE_METIS 1
>>>>>>>>>
>>>>>>>>> So I don't see any issues here. Send us the error output from make ?
>>>>>>>>>
>>>>>>>>> Vijay
>>>>>>>>>
>>>>>>>>> On Thu, Nov 5, 2015 at 11:39 AM, Popov, Emilian L. <popove at ornl.gov> wrote:
>>>>>>>>>> Hello Vijay,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Is there any resolution on the below.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> -emilian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From: nek5000-users-bounces at lists.mcs.anl.gov
>>>>>>>>>> [mailto:nek5000-users-bounces at lists.mcs.anl.gov] On Behalf Of
>>>>>>>>>> nek5000-users at lists.mcs.anl.gov
>>>>>>>>>> Sent: Tuesday, October 27, 2015 2:59 PM
>>>>>>>>>> To: nek5000-users at lists.mcs.anl.gov; moab-dev at mcs.anl.gov
>>>>>>>>>>
>>>>>>>>>> Subject: Re: [Nek5000-users] MOAB partitioning
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Emilian,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Let's fix the MOAB related issues first on the MOAB mailing list.
>>>>>>>>>> Please send us your config.log file so that we can provide
>>>>>>>>>> pointers to fix the configuration problems. Once this is done, we can revisit NEK-MOAB.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Vijay
>>>>>>>>>>
>>>>>>>>>> On Monday, October 26, 2015, <nek5000-users at lists.mcs.anl.gov> wrote:
>>>>>>>>>>
>>>>>>>>>> Vijay,
>>>>>>>>>>
>>>>>>>>>> I downloaded moab-4.8.2 and configured and then ran 'make',
>>>>>>>>>> but am getting errors from missing 'mpi.h' and metis.h,
>>>>>>>>>> perhaps the configuration is not OK?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> -emilian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From: nek5000-users-bounces at lists.mcs.anl.gov
>>>>>>>>>> [mailto:nek5000-users-bounces at lists.mcs.anl.gov] On Behalf Of
>>>>>>>>>> nek5000-users at lists.mcs.anl.gov
>>>>>>>>>> Sent: Friday, October 23, 2015 12:24 PM
>>>>>>>>>> To: nek5000-users at lists.mcs.anl.gov
>>>>>>>>>> Subject: Re: [Nek5000-users] MOAB partitioning
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What version of MOAB are you using ? It should be available
>>>>>>>>>> with the last MOAB 4.8.2 release [1] I think. Let us know if you have problems.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Vijay
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] http://ftp.mcs.anl.gov/pub/fathom/moab-4.8.2.tar.gz
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Friday, October 23, 2015, <nek5000-users at lists.mcs.anl.gov> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks Vijay, however the option '-with-metis' does not seem
>>>>>>>>>> to exist, please advice.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -emilian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From: nek5000-users-bounces at lists.mcs.anl.gov
>>>>>>>>>> [mailto:nek5000-users-bounces at lists.mcs.anl.gov] On Behalf Of
>>>>>>>>>> nek5000-users at lists.mcs.anl.gov
>>>>>>>>>> Sent: Friday, October 23, 2015 11:35 AM
>>>>>>>>>> To: nek5000-users at lists.mcs.anl.gov
>>>>>>>>>> Subject: Re: [Nek5000-users] MOAB partitioning
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Emilian,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If you want the partitioner tools, you need to configure MOAB
>>>>>>>>>> with either metis (--with-metis=DIR) or with zoltan and
>>>>>>>>>> optionally parmetis. Let us know if you have questions. The
>>>>>>>>>> metis installation route is extremely straightforward and you should not encounter any issues.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Vijay
>>>>>>>>>>
>>>>>>>>>> On Friday, October 23, 2015, <nek5000-users at lists.mcs.anl.gov> wrote:
>>>>>>>>>>
>>>>>>>>>> Dear Nek/Moab users,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I built Moab, but in the bin folder the 'mbpart' code is
>>>>>>>>>> missing and I need it to partition an .h5m mesh file for Nek.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Any suggestions?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> -emilian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
More information about the moab-dev
mailing list