<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000'>Hi Carlos and Junior,<br>Thank you again for the cgns reader/writer.<br>I think I am close to integrate it, at least in 462.<br>After I changed the writer hpp file (), it built, but I see some issues when I am testing.<br>Could be related to my changes, probably I should initialize those members in a constructor.<br><br>So this test (reader) that I am doing seems fine:<br>../../../build46/tools/mbsize 2d_naca0012.cgns<br>File 2d_naca0012.cgns:<br><br>number of nodes = 3555<br>number of elems = 6944<br>number of parts = 3<br> type count total minimum average rms maximum std.dev.<br>------- ----- ------- -------------------- -------------------- -------------------- -------------------- --------------------<br> Edge 166 65 0.0061783 0.39047 0.71774 2.0409 0.60223<br> Tri 6944 3.1e+02 2.2192e-05 0.045078 0.13326 1.4755 0.1254<br>1D Side 20832 3.9e+03 0.0061783 0.18654 0.33264 2.0409 0.27542<br> Vertex 3555<br><br>But this (reader and writer) has issues. Again, it could be my changes, to make it compile.<br><br> ../../../build46/tools/mbconvert 3d_wingbody_mixed_JR.cgns test.cgns<br>*** Warning:The file being read is more recent that the CGNS library used ***<br><br>number of nodes = 285905<br>number of elems = 333820<br>number of parts = 1<br>Read "3d_wingbody_mixed_JR.cgns"<br>THE CGNS CONVERSION ONLY WORKS FOR ENTITIES CITED BELOW:<br> -MBVERTEX<br> -MBEDGE<br> -MBTRI<br> -MBQUAD<br> -MBTET<br> -MBPYRAMID<br> -MBHEX<br><br>The Number of Vertex is 285905.<br>The Number of Edges is 0.<br>The Number of Faces is 29782.<br>The Number of Cells is 333820.<br><br>test.cgns file is a 3-D mesh.<br><br>The mesh has 5 Tags.<br>Tag name= MATERIAL_SET<br> Number of MBEDGE = 0<br> Number of MBTRI = 0<br> Number of MBQUAD = 0<br> Number of MBTET = 0<br> Number of MBPYRAMID = 0<br> Number of MBPRISM = 0<br> Number of MBHEX = 0<br> Total number of Edges = 0<br> Total number of Faces = 0<br> Total number of Cells = 0<br> Sets[0].CGNSType= 0<br> Sets[0].IdSet = 0<br><br>Tag name= NEUMANN_SET<br> Number of MBEDGE = 0<br> Number of MBTRI = 0<br> Number of MBQUAD = 0<br> Number of MBTET = 0<br> Number of MBPYRAMID = 0<br> Number of MBPRISM = 0<br> Number of MBHEX = 0<br> Total number of Edges = 0<br> Total number of Faces = 0<br> Total number of Cells = 0<br> Sets[1].CGNSType= 0<br> Sets[1].IdSet = 0<br><br>Tag name= DIRICHLET_SET<br> Number of MBEDGE = 0<br> Number of MBTRI = 0<br> Number of MBQUAD = 0<br> Number of MBTET = 0<br> Number of MBPYRAMID = 0<br> Number of MBPRISM = 0<br> Number of MBHEX = 0<br> Total number of Edges = 0<br> Total number of Faces = 0<br> Total number of Cells = 0<br> Sets[2].CGNSType= 0<br> Sets[2].IdSet = 0<br><br>Tag name= GEOM_DIMENSION<br> Number of MBEDGE = 0<br> Number of MBTRI = 0<br> Number of MBQUAD = 0<br> Number of MBTET = 0<br> Number of MBPYRAMID = 0<br> Number of MBPRISM = 0<br> Number of MBHEX = 0<br> Total number of Edges = 0<br> Total number of Faces = 0<br> Total number of Cells = 0<br> Sets[3].CGNSType= 0<br> Sets[3].IdSet = 0<br><br>Tag name= GLOBAL_ID<br> Number of MBEDGE = 0<br> Number of MBTRI = 0<br> Number of MBQUAD = 29782<br> Number of MBTET = 72460<br> Number of MBPYRAMID = 3836<br> Number of MBPRISM = 0<br> Number of MBHEX = 257524<br> Total number of Edges = 0<br> Total number of Faces = 29782<br> Total number of Cells = 333820<br> Sets[4].CGNSType= 20<br> Sets[4].IdSet = 4<br><br>BeginSetsIndex = 1 EndSetsIndex = 0<br>Issue on writing connectivity -- 3-D<br>Invalid element range defined for section 'MATERIAL_SET'<br><br><br><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><style>p { margin: 0; }</style><div style="font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000">Hi Carlos,<br>I rebuilt cgns static, and it configured fine; <br>I have though some build errors; <br> CXX WriteSLAC.lo<br> CXX ReadNC.lo<br> CXX ReadGCRM.lo<br> CXX ReadCGNS.lo<br>../../../moab46/src/io/ReadCGNS.cpp:59:11: warning: unused parameter 'file_set' [-Wunused-parameter]<br>../../../moab46/src/io/ReadCGNS.cpp:551:11: warning: unused parameter 'element_type' [-Wunused-parameter]<br>../../../moab46/src/io/ReadCGNS.cpp:551:11: warning: unused parameter 'set_type' [-Wunused-parameter]<br> CXX WriteCGNS.lo<br>In file included from ../../../moab46/src/io/WriteCGNS.cpp:1:0:<br>../../../moab46/src/io/WriteCGNS.hpp:65:23: error: ISO C++ forbids initialization of member 'IdSet' [-fpermissive]<br>../../../moab46/src/io/WriteCGNS.hpp:65:23: error: making 'IdSet' static [-fpermissive]<br>../../../moab46/src/io/WriteCGNS.hpp:65:23: error: ISO C++ forbids in-class initialization of non-const static member 'IdSet'<br>../../../moab46/src/io/WriteCGNS.hpp:66:22: error: ISO C++ forbids initialization of member 'NbEdges' [-fpermissive]<br>../../../moab46/src/io/WriteCGNS.hpp:66:22: error: making 'NbEdges' static [-fpermissive]<br>../../../moab46/src/io/WriteCGNS.hpp:66:22: error: ISO C++ forbids in-class initialization of non-const static member 'NbEdges'<br>...<br><br>do you need to initialize these in the struct? I would prefer not to use "-fpermissive"<br><br>// A structure to store Set information.<br> struct SetStruct {<br> std::string TagName; // Tag name<br> cgsize_t IdSet = -1; // Id of the Set<br> cgsize_t NbEdges=0; // Number of Edges in the Set<br> cgsize_t NbFaces=0; // Number of Faces in the Set<br> cgsize_t NbCells=0; // Number of Cells in the Set <br> // vector with the number of entities in the Sets<br> // 0-MBEDGE | 1-MBTRI | 2-MBQUAD | 3-MBTET | 4-MBPYRAMID | 5-MBPRISM | 6-MBHEX<br> std::vector<cgsize_t> NbEntities;<br> ElementType_t CGNSType;<br> };<br><br><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><div dir="ltr"><div><div><div><div>Iulian,<br><br></div>I had a similar issue in the past. I recall that I edited the cgns.m4 but not exactly which part, probably the section below to link directly with hdf5_hl, instead of plain hdf5.<br>
<br>I use Arch linux but not the system libraries. I compile hdf5 and cgns in a local directory and link those.<br><br>I will take a look into the m4 file and see if I can reproduce the problem.<br><br><div>Regards,<br></div>
<div><br></div>Carlos<br><br><br><span style="font-family:courier new,monospace"> # Check if cgns is usable by itself<br> AC_CHECK_LIB( [cgns], [cg_open], [CGNS_LIBS="-lcgns"], [<br> # Check if cgns is usable with HDF5<br>
unset ac_cv_lib_cgns<br> unset ac_cv_lib_cgns_cg_open<br> # If we haven't already looked for HDF5 libraries, again now incase<br> # they're in the CGNS lib directory.<br> FATHOM_DETECT_HDF5_LIBS<br>
LDFLAGS="$LDFLAGS $HDF5_LDFLAGS"<br> AC_CHECK_LIB( [cgns], [cg_open], [CGNS_LIBS="-lcgns -lhdf5_hl"], [<br> # Try one more time with HDF5 and libcurl<br> unset ac_cv_lib_cgns<br> unset ac_cv_lib_cgns_cg_open<br>
AC_CHECK_LIB( [cgns], [cg_open], [CGNS_LIBS="-lcgns -lhdf5_hl -lcurl"], <br> [HAVE_CGNS=no], [-lhdf5_hl $HDF5_LIBS -lcurl] )],<br> [-lhdf5_hl $HDF5_LIBS] )],<br> )</span><br><br></div></div>
<br></div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Nov 19, 2013 at 3:42 PM, Iulian Grindeanu <span dir="ltr"><<a href="mailto:iulian@mcs.anl.gov" target="_blank">iulian@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif">Thanks Carlos,<br>I have some issues configuring moab with cgns; <br>
I did not rebuild hdf5, we are using version 1.8.8; cgns seems to be built without problems (I used ccmake)<br>But something is wrong with configuring with cgns.<br>Do you have any suggestions (besides rebuilding hdf5, cgns, ...)<br>
<br>the symbols needed from hdf5 seem to be there, so I can't really explain the error. Do you use ubuntu 12 or ubuntu 10? Or something else?<br><br><br>/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib> nm libhdf5.so | grep H5T_NATIVE_SCHAR_g<br>
00000000004cc170 D H5T_NATIVE_SCHAR_g<br>jun/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib> nm libhdf5.so | grep H5Tget_native_type<br>00000000002424f0 T H5Tget_native_type<br><br>...<br>configure:32106: result: no<br>
configure:32115: checking for cg_open in -lcgns<br>configure:32148: /homes/fathom/3rdparty/mpich2/mpich2-1.5/gcc/bin/mpicc -o conftest -Wall -pipe -pedantic -Wno-long-long -Wextra -Wcast-align -Wpointer-arith -Wformat -Wformat-security -Wshadow -Wunused-parameter -g -I/homes/fathom/3rdparty/cgns/include -DVALGRIND -DUNORDERED_MAP_NS=std::tr1 -DHAVE_UNORDERED_MAP=tr1/unordered_map -DHAVE_UNORDERED_SET=tr1/unordered_set -L/homes/fathom/3rdparty/cgns/lib -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib -L/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib -L/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib -L/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib -L/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib -L/homes/fathom/3rdparty/hdf5-1.8.8-par-mpich2.1.5-gcc/lib conftest.c -lcgns <span style="background-color:rgb(204,0,0)">-lhdf5_hl -lhdf5 </span> -lcurl -lm -L/homes/fathom/3rdparty/mpich2/mpich2-1.5/gcc/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../.. -lmpich -lopa -lmpl -lrt -lpthread -lgfortran -lm -lquadmath -L/homes/fathom/3rdparty/mpich2/mpich2-1.5/gcc/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../.. -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread -lgfortran -lm -lquadmath >&5<br>
/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `<span style="background-color:rgb(255,0,0)">H5T_NATIVE_SCHAR_g</span>'<br>/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5Tget_native_type'<br>
/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5Pset_link_creation_order'<br>/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5P_CLS_DATASET_CREATE_g'<br>/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5Sget_simple_extent_npoints'<br>
/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5T_IEEE_F64LE_g'<br>/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5P_CLS_FILE_CREATE_g'<br>/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5Tcopy'<br>
/homes/fathom/3rdparty/cgns/lib/libcgns.so: undefined reference to `H5Fopen'<br>...<br><br><hr><div><div class="h5"><blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff">
<div dir="ltr"><div>Iulian, here are some data on the supporting libs:<br><br>=========<br></div><div>HDF5 v1.8.11<br>=========<br></div><div><br></div><div>I compile the parallel version, but do not use the parallel IO capabilities yet in CGNS.<br>
</div><div><br><span style="font-family:courier new,monospace">./configure \<br> CC=%s \<br> CXX=%s \<br> FC=%s \<br> F9X=%s \<br> RUNPARALLEL=%s \<br> OMPI_MCA_disable_memory_allocator=1 \<br> --prefix=%s \<br> --enable-largefile \<br>
--enable-unsupported \<br> --enable-shared \<br> --disable-static \<br> --enable-production=yes \<br> --with-pthread=%s,%s \<br> --with-zlib=%s,%s \<br> --with-default-api-version=v18 \<br> --enable-parallel=yes \<br>
--enable-cxx \<br> --disable-sharedlib-rpath"</span><br><br></div><div>hdf5_config.log attached<br></div><div><br>=========<br>CGNS v3.2.1 (Beta)<br>=========<br><br></div><div>We have not considered the parallel IO capabilities as of now. There is no clear definition yet how to handle MIXED type element meshes, for which you can not specify a fixed record lenght to retrieve with hdf5. Although the HDF5 library I linked is mpi-enabled.<br>
</div><div>Junior at some point tested with version 3.1.4 release 2 and it worked fine, AFAIK.<br><br></div><div>The library is transitioning to CMake, I've used:<br><br><span style="font-family:courier new,monospace">cmake \<br>
-DCGNS_BUILD_SHARED:BOOL=ON \<br> -DCMAKE_BUILD_TYPE:STRING=Release \<br> -DCMAKE_INSTALL_PREFIX:PATH=%s \<br> -DCMAKE_SKIP_RPATH:BOOL=ON \<br> -DCGNS_BUILD_CGNSTOOLS:BOOL=OFF \<br> -DCGNS_ENABLE_64BIT:BOOL=ON \<br>
-DCGNS_ENABLE_FORTRAN:BOOL=OFF \<br> -DCGNS_ENABLE_PARALLEL:BOOL=ON \<br> -DCGNS_ENABLE_HDF5:BOOL=ON \<br> -DHDF5_INCLUDE_PATH:PATH=%s \<br> -DHDF5_LIBRARY:FILEPATH=%s \<br> -DHDF5_NEED_MPI:BOOL=ON \<br> -DHDF5_NEED_ZLIB:BOOL=ON \<br>
-DZLIB_LIBRARY:FILEPATH=%s \<br> -DCGNS_ENABLE_SCOPING:BOOL=OFF \<br> -DCGNS_ENABLE_TESTS:BOOL=OFF \<br> -DMPIEXEC:FILEPATH=%s \<br> -DMPI_C_COMPILER:FILEPATH=%s \<br> -DMPI_C_INCLUDE_PATH:PATH=\"%s\"</span><br>
</div><div><br>
<p style="margin:0px;text-indent:0px">cgns321_CMakeCache.txt is attached.</p><br>=========<br>MOAB v4.6.2<br>=========<br><br></div><div>The patch is against the tar file from the website (<a href="http://ftp.mcs.anl.gov/pub/fathom/moab-4.6.2.tar.gz" target="_blank">http://ftp.mcs.anl.gov/pub/fathom/moab-4.6.2.tar.gz</a>). There is no need from our side to keep it on 462. Feel free to include it in a latter release. It was only convenient for me to keep the patch against a fixed release. For this, I packaged in the previous email a tar.gz with the modified and new sources and sample cgns files.<br>
</div><div>The reader and writer are serial only. We tested against homogeneous and heterogeneous meshes up to 20million elements and it worked. You can read a CGNS file, convert it to H5M and then convert it back to CGNS.<br>
<br></div><div>The modifications to compile moab is just an --with-cgns=DIR flag.<br></div><div><br></div><div>These build use shared libs for now, as I am working on my workstation. Latter I will experiment with static objects for clusters.<br>
</div><div><br>If you need more details, let us know.<br><br>Regards,<br>Carlos<br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Nov 19, 2013 at 1:30 PM, Iulian Grindeanu <span dir="ltr"><<a href="mailto:iulian@mcs.anl.gov" target="_blank">iulian@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif">Thank you Carlos and Junior, for your help.<br>
I will need to build cgns library; you mentioned that you tested 3.2 beta, too; I will probably stick with 3.1.4, (release 2?). <br> Also, what version of hdf5 library do you use? Should it work for any version greater than 1.8? <br>
I assume you build statically? Can I also get your config.log files for cgns and moab?<br><br>I assume that you are not using any of the parallel io capability, because that seems to be available only in beta version. <br>
So everything is read/written in serial, isn't it?<br>I am not sure if we will port the cgns reader / writer to 4.6.2 also. Is your patch against 4.6.2 tar file or against "Version4.6" branch? There should not be much difference. <br>
<br>Tim, what do you suggest? In a way, it should be easier to update 4.6.2. But I don't want to merge to the master after that, there are too many changes. This patch should work out of the box for 4.6.2, after I build cgns.<br>
<br>Best Regards,<br>Iulian<br><br><hr><div><div><blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff">
<div dir="ltr"><div><div>Iulian,<br><br></div>Sorry for the delay in getting back to you... I have tested the new version and it worked OK for my mesh sample. I will test it latter for other cases.<br><br>Also, I am sending you the CGNS reader and writer along with input cgns mesh examples. We have worked on top of version 462, so things may look a little out of place from the repository.<br>
<br></div><div>The attached patch modifies some autotools files and new files (apply with patch -p0 -i moab462_cgns.patch). Moreover, the tar.gz contains sample meshes (simple airfoil mesh in 2D and a mixed-mesh discretization of a 3D wingbody geometry).<br>
<br></div><div>We have tested the CGNS capability with the latest library vesion 3.1.4 and 3.2-beta. Furthermore, we sticked with the base CGNS use-case, that is, single base/zone. We will also implement latter capability to write out user variables.<br>
<br></div><div>Please, feel free to improve the files and if you have any questions, let us know. <a href="mailto:carbrevi@gmail.com" target="_blank">carbrevi@gmail.com</a> and <a href="mailto:junior.hmg@gmail.com" target="_blank">junior.hmg@gmail.com</a><br>
<br>Regards,<br><br>Carlos<br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Nov 16, 2013 at 3:57 AM, Iulian Grindeanu <span dir="ltr"><<a href="mailto:iulian@mcs.anl.gov" target="_blank">iulian@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif">Hi Carlos,<br>That ticket is now closed, Tim fixed the bug.<br>
Please let us know if it works for you.<br><br>Thanks for your patience,<br>Iulian<br><hr><div><div><blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff">
<div dir="ltr"><div><div>Iulian,<br><br></div>thanks for checking this out. I will be following the trac ticket. The mesh I used is attached, partitioned for 8 procs (all tri). I noticed that are much simpler sample meshes you created to investigate this issue. Anyway, you can use the airfoil mesh if you find it useful.<br>
<br>I use a separate code to compute elements adjacency and then pass those to metis. To write the partitions into h5m I followed the code form mbzoltan tool. The partitioning code is correct (element-based) since I have used this code previously with other applications...<br>
<br></div>I am from Brazil indeed, utc -3h. I'm a PhD candidate from Technological Institute of Aeronautics in Sao Jose dos Campos.<br><div><div><br></div><div>Now I am building/adapting parts of my research code (high-order unstructured CFD) to use MOAB. I will stick with 462 for this, working with non-mixed meshes for this transition period. In the near feature I will also look into the structured meshes capability of the library, as well as the high-order elements implementation.<br>
<br></div><div>The CGNS reader/writer is in the works too and I will submit those as well. My co-worker Junior, cc'ed here, is dealing with the writer part and we are almos done.<br></div><div><br></div><div>Regards,<br>
Carlos Breviglieri<br></div><div><br><br></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Oct 12, 2013 at 5:40 PM, Iulian Grindeanu <span dir="ltr"><<a href="mailto:iulian@mcs.anl.gov" target="_blank">iulian@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-size:12pt;font-family:times new roman,new york,times,serif">Hi Carlos,<br>I am seeing a problem with ghosting, thank you for pointing it out. Danqing and I messed up with that code :( to fix some other issues we were seeing. <br>
<a href="http://trac.mcs.anl.gov/projects/ITAPS/ticket/284" target="_blank">http://trac.mcs.anl.gov/projects/ITAPS/ticket/284</a><br><br>It may take a while to fix it properly.<br>Thanks again,<br>Iulian<br><br><hr><div>
<div>
<blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff"><div style="font-size:12pt;font-family:times new roman,new york,times,serif">
Hmmmm,<br>Can you send me the 2d_naca0012.h5m with your partition? <br>If you do ghosting after reading, did you get the same results?<br>The elements/processors should not change ownership after ghosting, but maybe they do. <br>
It is indeed a work in progress, you may have found another issue :(<br>Not being able to write is probably the biggest problem.<br> <br>On what time zone do you work? :) Also, where are you from? The name looks Brazilian, Portuguese, or Italian ? Or Spanish? <br>
<br>Thanks,<br>Iulian<br><br><hr><blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff">
<div dir="ltr"><div><div><div><div><div><div>Hi Iulian,<br><br></div>I have just ran some tests with the latest clone of the master repo (saturday morning). Here are my findings with MOAB 470pre:<br><br></div>Now I am able to read partitined mixed meshes without errors. However, if I plot the owned entitied for a given proc, even for non-mixed meshes, they are different from the ones obtained by the partitioner, see below<br>
<br></div>Mesh distribution for 2d_naca0012.h5m (all tri mesh) from partitioner, over 8 procs:<br>proc[0] has 866 elements<br>proc[1] has 863 elements<br>proc[2] has 866 elements<br>proc[3] has 869 elements<br>proc[4] has 872 elements<br>
proc[5] has 869 elements<br>proc[6] has 862 elements<br>proc[7] has 877 elements<br><br></div>Mesh distribution seen by MOAB 470pre ("PARALLEL=READ_PART;PARTITION=PARALLEL_PARTITION;PARALLEL_RESOLVE_SHARED_ENTS;PARALLEL_GHOSTS=2.0.1")<br>
owned_entities[5], size = 864<br>owned_entities[6], size = 853<br>owned_entities[7], size = 871<br>owned_entities[1], size = 859<br>owned_entities[3], size = 860<br>owned_entities[4], size = 867<br>owned_entities[0], size = 866<br>
owned_entities[2], size = 861<br><br></div>besides procs 0, all other report wrong number of entities. Code to compute such distribution is below (based on example/HelloParMOAB.cpp). The mixed mesh (2d_naca0012_mixed.h5m) now is read (does not crash) but the distribution is off as well. With MOAB 462 the distribution is OK for homogeneous meshes.<br>
<br></div>Moreover, with MOAB 470pre, no output is written to disk with PARALLEL=WRITE_PART, regardless of the mesh type. Using PARALLEL=NONE works, but only one part of the domain is written, as expected.<br><div><br></div>
<div>I understand that this is a work in progress. If you need more information, let me know.<br></div><div><br>Regards,<br><br>Carlos Breviglieri<br><br><br> read_options = "PARALLEL=READ_PART;PARTITION=PARALLEL_PARTITION;PARALLEL_RESOLVE_SHARED_ENTS;PARALLEL_GHOSTS=2.0.1";<br>
<br> moab::Interface* mb = new Core;<br><br> // Create root sets for each mesh. Then pass these<br> // to the load_file functions to be populated.<br> EntityHandle rootset, partnset;<br> mb->create_meshset(MESHSET_SET, rootset);<br>
mb->create_meshset(MESHSET_SET, partnset);<br><br> // Create the parallel communicator object with the partition handle associated with MOAB<br> ParallelComm *pcomm = ParallelComm::get_pcomm(mb, partnset, &myComm);<br>
<br> // Load the file from disk with given options<br> mb->load_file(meshFile.c_str(), &rootset, read_options.c_str());<br><br> // Get all entities of dimension = dim<br> Range elemRange, owned_entities;<br>
int dim = 2;<br> mb->get_entities_by_dimension(rootset, dim, elemRange, false);<br><br> pcomm->filter_pstatus(elemRange, // pass entities that we want to filter<br> PSTATUS_NOT_OWNED, // status we are looking for<br>
PSTATUS_NOT, // operation applied ; so it will return owned entities (!not_owned = owned)<br> -1, // this means all processors<br> &owned_entities);<br>
<br> std::vector<int> procID(owned_entities.size(), myRank);<br><br> std::cout << "owned_entities[" << myRank << "], size = " << owned_entities.size() << std::endl;<br>
<br> Tag procID_tag;<br> mb->tag_get_handle("PROC_ID", 1, MB_TYPE_INTEGER, procID_tag, MB_TAG_CREAT | MB_TAG_DENSE, &procID[0]);<br><br> mb->tag_set_data(procID_tag, owned_entities, &procID[0]);<br>
<br> // WRITE_PART writes all partitions to a single output file (only h5m format supports parallel IO at the moment).<br> // One can use the mbconvert tool to convert the output to other formats.<br> mb->write_file(outputFile.c_str(), "H5M", "PARALLEL=WRITE_PART");<br>
<br><br><div><br><br><br><div><div><br><div><div><div><br><br></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 10, 2013 at 10:37 AM, Tim Tautges <span dir="ltr"><<a href="mailto:tautges@mcs.anl.gov" target="_blank">tautges@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Yeah, too complicated to backport, and latest works for Carlos anyway.<br>
<br>
- tim<br>
<br>
On 10/09/2013 09:41 PM, Iulian Grindeanu wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
------------------------------<u></u>------------------------------<u></u>------------------------------<u></u>------------------------------<div><div><br>
<br>
Both worked for me, with the current code (4.7.0pre)<br>
<br>
I don't get your error :(<br>
What version are you using? I will try 4.6.2, but it should be fine there too :(<br>
<br>
<br>
OK, I got an error with more quads, on 4.6.2 maybe I mixed them up when I saved :(<br>
mpiexec -np 2 /home/iulian/source/MOAB46/<u></u>tools/mbconvert -O PARALLEL=READ_PART -O PARTITION=PARALLEL_PARTITION -O<br>
PARALLEL_RESOLVE_SHARED_ENTS -O PARALLEL_GHOSTS=2.0.1 -o PARALLEL=WRITE_PART<br>
/home/iulian/tmp/2d_naca0012_<u></u>mixed2.h5m 2.h5m<br>
Leaked HDF5 object handle in function at ../../../moab46source/src/io/<u></u>ReadHDF5.cpp:1523<br>
Open at entrance: 1<br>
Open at exit: 2<br>
Leaked HDF5 object handle in function at ../../../moab46source/src/io/<u></u>ReadHDF5.cpp:827<br>
Open at entrance: 1<br>
Open at exit: 2<br>
Failed to load "/home/iulian/tmp/2d_naca0012_<u></u>mixed2.h5m".<br>
Error code: MB_INDEX_OUT_OF_RANGE (1)<br>
Error message: Failed in step PARALLEL READ PART<br>
Cannot close file with open handles: 0 file, 1 data, 0 group, 0 type, 0 attr<br>
<br>
<br>
I will look into it.<br>
<br>
Hi Carlos,<br>
It looks like it is a bug in 4.6.2.<br>
I don't know if it will be fixed, there are some important changes in ghosting in current version.<br>
So for Version4.6 branch, the model with 17 quads works fine if you don't do ghosting:<br>
<br>
iulian@T520-iuli:~/source/<u></u>MOAB46$ mpiexec -np 2 /home/iulian/source/MOAB46/<u></u>tools/mbconvert -O PARALLEL=READ_PART -O<br>
PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o PARALLEL=WRITE_PART<br>
/home/iulian/tmp/2d_naca0012_<u></u>mixed_invert.h5m 2.h5m<br>
Read "/home/iulian/tmp/2d_naca0012_<u></u>mixed_invert.h5m"<br>
Wrote "2.h5m"<br>
<br>
I would recommend upgrading to current version.<br>
That code is pretty complicated, and I am not sure if we will backport changes to Version4.6 branch.<br>
<br>
Tim, what do you suggest? Should I try to backport some changes in ParallelComm? I know you are working on that code.<br>
<br>
Thanks,<br>
Iulian<br>
<br>
<br>
<br>
<br>
<br>
</div></div></blockquote><div><div>
<br>
-- <br>
==============================<u></u>==============================<u></u>====<br>
"You will keep in perfect peace him whose mind is<br>
steadfast, because he trusts in you." Isaiah 26:3<br>
<br>
Tim Tautges Argonne National Laboratory<br>
(<a href="mailto:tautges@mcs.anl.gov" target="_blank">tautges@mcs.anl.gov</a>) (telecommuting from UW-Madison)<br>
phone (gvoice): <a href="tel:%28608%29%20354-1459" target="_blank">(608) 354-1459</a> 1500 Engineering Dr.<br>
fax: <a href="tel:%28608%29%20263-4499" target="_blank">(608) 263-4499</a> Madison, WI 53706<br>
<br>
</div></div></blockquote></div><br></div>
</blockquote><br></div></blockquote><br></div></div></div></div></blockquote></div><br></div>
</blockquote><br></div></div></div></div></blockquote></div><br></div>
</blockquote><br></div></div></div></div></blockquote></div><br></div>
</blockquote><br></div></div></div></div></blockquote></div><br></div>
</blockquote><br></div></blockquote><br></div></body></html>