<div dir="ltr">Ok, the problem is that the default algorithm for detecting the connected components of the interface finds a lot of disconnected dofs.<div>What discretization is this? Nedelec elements? Can you try using <span class="gmail-s1" style="color:rgb(0,0,0);font-family:Menlo;font-size:21px;font-variant-ligatures:no-common-ligatures">-pc_</span><span class="gmail-s2" style="font-family:Menlo;font-size:21px;font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)"><b>bddc_use_lo</b></span><span class="gmail-s1" style="color:rgb(0,0,0);font-family:Menlo;font-size:21px;font-variant-ligatures:no-common-ligatures">cal_mat_graph 0?</span></div><div>Also, you are using -pc_bddc_monolithic, but you only have one field. That flag aggregates different fields, but you only have one. </div><div>Note that with Nedelec elements, you need a special change of basis for BDDC to work, see e.g. <a href="https://urldefense.us/v3/__https://www.osti.gov/servlets/purl/1377770__;!!G_uCfscf7eWS!Z94Qs8Q7RYEdhbAbvkaNorzlyoN4UH_ttW0EmR6d-NKweo4S35ELp-_Y60aJAAE1vzgZpof2VQYVxX9Xm1kM2vwiioZBFlo$">https://www.osti.gov/servlets/purl/1377770</a></div>
</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno mer 14 ago 2024 alle ore 05:15 neil liu <<a href="mailto:liufield@gmail.com">liufield@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi, Stefano, <div><br></div><div>Please see the attached for the smaller case(successful with BDDC).</div><div>and the Error_largerMesh shows the error with the large mesh using petsc debug mode. </div><div><br></div><div>Thanks a lot,</div><div><br></div><div>Xiaodong</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 13, 2024 at 5:47 PM Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">can you run the same options and add "-ksp_view -pc_bddc_check_level 1" for the smaller case? Also, can you send the full stack trace of the out-of-memory error using a debug version of PETSc?<div>A note aside: you should not need pc_bddc_use_vertices (which is on by default)</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno mar 13 ago 2024 alle ore 23:17 neil liu <<a href="mailto:liufield@gmail.com" target="_blank">liufield@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Dear Petsc developers,<div><br></div><div>I am testing PCBDDC for my vector based FEM solver(complex system). It can work well on a coarse mesh(tetrahedra cell #: 6,108; dof # : 39,596). Then I tried a finer mesh (tetrahedra cell #: 32,036; dof # : 206,362). It seems ASM can work well with </div><div><br></div><div><div>petsc-3.21.1/petsc/arch-linux-c-opt/bin/mpirun -n 4 ./app -pc_type asm -ksp_converged_reason -ksp_monitor -ksp_gmres_restart 100 -ksp_rtol 1e-4 -pc_asm_overalp 4 -sub_pc_type ilu -malloc_view<br></div><div><br></div></div><div>while PCBDDC eats up the memory (61 GB) when I tried </div><div><br></div><div>petsc-3.21.1/petsc/arch-linux-c-opt/bin/mpirun -n 4 ./app -pc_type bddc -pc_bddc_coarse_redundant_pc_type ilu -pc_bddc_use_vertices -ksp_error_if_not_converged -mat_type is -ksp_monitor -ksp_rtol 1e-8 -ksp_gmres_restart 30 -ksp_view -malloc_view -pc_bddc_monolithic -pc_bddc_neumann_pc_type ilu -pc_bddc_dirichlet_pc_type ilu <br></div><div><br></div><div>The following errors with BDDC came out. The memory usage for PCBDDC (different from PCASM) is also listed (I am assuming the unit is Bytes, right?). <b>Although the BDDC requires more memory, it still seems normal, right? </b></div><div><br></div><div><div>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div>[0]PETSC ERROR: Out of memory. This could be due to allocating</div><div>[0]PETSC ERROR: too large an object or bleeding by not properly</div><div>[0]PETSC ERROR: destroying unneeded objects.</div><div>[0] Maximum memory PetscMalloc()ed 30829727808 maximum size of entire process 16899194880</div><div>[0] Memory usage sorted by function</div></div><div>....</div><div><div><b>[0] 1 240 PCBDDCGraphCreate()</b></div><div><b>[0] 1 3551136 PCBDDCGraphInit()</b></div><div><b>[0] 2045 32720 PCBDDCGraphSetUp()</b></div><div><b>[0] 2 8345696 PCBDDCSetLocalAdjacencyGraph_BDDC()</b></div><div><b>[0] 1 784 PCCreate()</b></div><div><b>[0] 1 1216 PCCreate_BDDC()</b></div></div><div>....</div><div><br></div><div>Thanks for your help. </div><div><br></div><div>Xiaodong </div><div><div><br></div><div><br></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Stefano</div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Stefano</div>