<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi, <div class=""><br class=""></div><div class=""> I am trying to use an lu decomposition method for a relatively large matrix (~775,000 dofs) coming from a thermoelasticity problem. </div><div class=""><br class=""></div><div class=""> For the past few weeks, LU solver in 3.5.1 has been solving it just fine. I just upgraded to 3.5.2 from macports (running on Mac OS 10.10.2), and am getting the following “out of memory" error </div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">sab_old_mast_structural_analysis(378,0x7fff75f6e300) malloc: *** mach_vm_map(size=18446744066373115904) failed (error code=3)</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">*** error: can't allocate region</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">*** set a breakpoint in malloc_error_break to debug</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(195, 55, 32); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Out of memory. This could be due to allocating</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: too large an object or bleeding by not properly</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: destroying unneeded objects.</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Memory allocated 3649788624 Memory used by process 3943817216</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Memory requested 18446744066373113856</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" class="">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Petsc Release Version 3.5.2, unknown </div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: ./sab_old_mast_structural_analysis on a arch-macports named <a href="http://ws243-49.walker.dynamic.msstate.edu" class="">ws243-49.walker.dynamic.msstate.edu</a> by manav Thu Feb 5 17:30:18 2015</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: Configure options --prefix=/opt/local --prefix=/opt/local/lib/petsc --with-valgrind=0 --with-shared-libraries --with-c2html-dir=/opt/local --with-x=0 --with-blas-lapack-lib=/System/Library/Frameworks/Accelerate.framework/Versions/Current/Accelerate --with-hwloc-dir=/opt/local --with-suitesparse-dir=/opt/local --with-superlu-dir=/opt/local --with-metis-dir=/opt/local --with-parmetis-dir=/opt/local --with-scalapack-dir=/opt/local --with-mumps-dir=/opt/local CC=/opt/local/bin/mpicc-openmpi-mp CXX=/opt/local/bin/mpicxx-openmpi-mp FC=/opt/local/bin/mpif90-openmpi-mp F77=/opt/local/bin/mpif90-openmpi-mp F90=/opt/local/bin/mpif90-openmpi-mp COPTFLAGS=-Os CXXOPTFLAGS=-Os FOPTFLAGS=-Os LDFLAGS="-L/opt/local/lib -Wl,-headerpad_max_install_names" CPPFLAGS=-I/opt/local/include CFLAGS="-Os -arch x86_64" CXXFLAGS=-Os FFLAGS=-Os FCFLAGS=-Os F90FLAGS=-Os PETSC_ARCH=arch-macports --with-mpiexec=mpiexec-openmpi-mp</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #1 PetscMallocAlign() line 46 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/sys/memory/mal.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #2 PetscTrMallocDefault() line 184 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/sys/memory/mtr.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #3 PetscFreeSpaceGet() line 13 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/mat/utils/freespace.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #4 MatLUFactorSymbolic_SeqAIJ() line 362 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/mat/impls/aij/seq/aijfact.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #5 MatLUFactorSymbolic() line 2842 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/mat/interface/matrix.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #6 PCSetUp_LU() line 127 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/ksp/pc/impls/factor/lu/lu.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #7 PCSetUp() line 902 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/ksp/pc/interface/precon.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #8 KSPSetUp() line 305 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/ksp/ksp/interface/itfunc.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #9 KSPSolve() line 417 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/ksp/ksp/interface/itfunc.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #10 SNESSolve_NEWTONLS() line 232 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/snes/impls/ls/ls.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #11 SNESSolve() line 3743 in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_petsc/petsc/work/v3.5.2/src/snes/interface/snes.c</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">[0]PETSC ERROR: #12 solve() line 559 in src/solvers/petsc_nonlinear_solver.C</div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class="">--------------------------------------------------------------------------</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">A few questions: </div><div class=""><br class=""></div><div class="">— has something changed between 3.5.1 and 3.5.2 that might lead to this behavior? </div><div class=""><br class=""></div><div class="">— So far I have tried the following iterative solver option: -pc_type ilu -pc_factor_levels 1 (and 2) with very slow convergence. Is there a better preconditioner recommended for this problem? This is a solid mechanics problem with thermal load (not a coupled thermal-structural probelm). </div><div class=""><br class=""></div><div class="">— I tried using MUMPS through the option -pc_factor_mat_solver_package mumps -mat_mumps_icntl_ 22 1 -mat_mumps_icntl_ 23 8000 to try to get it to use the disk I/O and limit the memory to 8GB, but that too returned with an out of memory error. Is this the correct format to specify the options? If so, is the write to disk option expected to work with MUMPS called via petsc? </div><div class=""><br class=""></div><div class="">I would greatly appreciate your inputs. </div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Manav</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div></body></html>