[petsc-users] Can PETSc support parallelly import HDF5 mesh file?

[程奔]

Hello,

Recently I create unstructure mesh from Gmsh and its mesh format is msh file. However the mesh file contain around 100 million nodes, so when I use DMPlexCreateFromFile 
it only perform on a single CPU process thus out of memory. 


I have sent email to report this situation to your guys and you told me that I can use h5 mesh file. Thanks for your advise so I try to generate h5 file.


In order to load the very large msh, I first use a single computer node(64 CPU process) that with large CPU memory. I just use one CPU process to perform the code to generate h5 mesh file:




-----------------------------------------------------------------------------------------------

//Set up or input the mesh

ierr = DMPlexCreateFromFile(PETSC_COMM_WORLD, "mesh.msh","hjtest" ,PETSC_TRUE, &dm_nodes);CHKERRQ(ierr);


//Distribute the mesh into different processers
DM dm_nodes_Dist; //the dm object using to contain the distributed mesh temporarily 
PetscPartitioner part; //for distributing the mesh
ierr = DMPlexGetPartitioner(dm_nodes, &part);CHKERRQ(ierr); //Get the partitioner of the mesh we just created
ierr = PetscPartitionerSetType(part,PETSCPARTITIONERPARMETIS);CHKERRQ(ierr); //Set the partitioner from the commond line option
ierr = DMPlexDistribute(dm_nodes,0,NULL,&dm_nodes_Dist); CHKERRQ(ierr); //distribute the mesh into different processers
if (dm_nodes_Dist) {DMDestroy(&dm_nodes); dm_nodes = dm_nodes_Dist;} //delete the origin mesh and use the distributed one



PetscViewerHDF5Open(PETSC_COMM_WORLD, "mesh.h5", FILE_MODE_WRITE, &viewer);
DMView(dm_nodes, viewer);
PetscViewerDestroy(&viewer);


DMDestroy(&dm_nodes);
-----------------------------------------------------------------------------------------------




I get the large h5 mesh, then I want to use another computer(equiped with 20 computer node 1280 CPU process that have not so large CPU memory) to perform the parallel computation. So I perform the code that use 20 computer node 1280 CPU process:




*************************************************************************************************

ierr = PetscViewerHDF5Open(PETSC_COMM_WORLD, "mesh.h5", FILE_MODE_READ, &viewer); CHKERRQ(ierr);

ierr = PetscViewerSetFormat(viewer, PETSC_VIEWER_HDF5); CHKERRQ(ierr); 



ierr = DMPlexCreate(PETSC_COMM_WORLD, &dm_nodes); CHKERRQ(ierr);
ierr = DMSetType(dm_nodes, DMPLEX); CHKERRQ(ierr);




PetscPartitioner part;
ierr = DMPlexGetPartitioner(dm_nodes, &part); CHKERRQ(ierr);
ierr = PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); CHKERRQ(ierr);


ierr = DMLoad(dm_nodes, viewer); CHKERRQ(ierr);
ierr = PetscViewerDestroy(&viewer); CHKERRQ(ierr);
PetscPrintf(PETSC_COMM_WORLD,"# Loaded mesh from HDF5\n");

*************************************************************************************************








I don't know if in this way, the PETSc can load h5 mesh file parallelly so that it will not only perform on a single CPU process thus out of memory. I try several times but it seems that it always load the h5 file on the one CPU process, I don't know if it have some mistakes or something.




And I use the version of PETSc is 3.21.6 and the configure is attached, I do download the hdf5.




So I write this email to ask for the help.




Looking forward to your reply!


sinserely,
Cheng.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250529/55f381f1/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: configure.log
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250529/55f381f1/attachment-0001.ksh>