<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1258">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p style="margin-top:0; margin-bottom:0"></p>
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p style="margin-top:0; margin-bottom:0"><span style="font-family:"Times New Roman",Times,serif">Hello,</span></p>
<span style="font-family:"Times New Roman",Times,serif"></span>
<p style="margin-top:0; margin-bottom:0"><a href="mailto:petsc-users@mcs.anl.gov" target="_blank" rel="noopener noreferrer" id="LPlnk3744" class="x_OWAAutoLink" style="color:rgb(17,85,204); font-family:arial,sans-serif; font-size:12.8px" previewremoved="true" tabindex="-1"></a><br>
<span style="font-family:"Times New Roman",Times,serif"></span></p>
<span style="font-family:"Times New Roman",Times,serif"></span>
<p style="margin-top:0; margin-bottom:0"><span style="font-family:"Times New Roman",Times,serif">I am constructing a large matrices in a generalized eigen value problem of order ~ 700,000*700,000 in MPIAIJ format across multiple nodes. The matrix construction
goes smoothly when I have smaller dimensional matrices or using less number of cores. However when I use more number of cores (~ 180 or more) I am getting these following error. </span></p>
<span style="font-family:"Times New Roman",Times,serif"></span>
<p style="margin-top:0; margin-bottom:0"><br>
<span style="font-family:"Times New Roman",Times,serif"></span></p>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #1 PetscCommBuildTwoSidedFReq_Ibarrier() line 418 in /petsc/petsc-3.9.3/src/sys/utils/mpits.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #2 PetscCommBuildTwoSidedFReq() line 570 in /petsc/petsc-3.9.3/src/sys/utils/mpits.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #3 MatStashScatterBegin_BTS() line 933 in /petsc/petsc-3.9.3/src/mat/utils/matstash.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #4 MatStashScatterBegin_Private() line 461 in /petsc/petsc-3.9.3/src/mat/utils/matstash.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #5 MatAssemblyBegin_MPIAIJ() line 683 in /petsc/petsc-3.9.3/src/mat/impls/aij/mpi/mpiaij.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div><span style="font-family:"Times New Roman",Times,serif">[169]PETSC ERROR: #6 MatAssemblyBegin() line 5158 in /petsc/petsc-3.9.3/src/mat/interface/matrix.c</span></div>
<span style="font-family:"Times New Roman",Times,serif"></span>
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<span style="font-family:"Times New Roman",Times,serif"></span><br>
<span style="font-family:"Times New Roman",Times,serif"></span></div>
<span style="font-family:"Times New Roman",Times,serif">I understand that the errors are in assembling the matrix in particular in subroutine "</span><span style="font-family:"Times New Roman",Times,serif">MPI_Testall" where it test the completion of test requests.
What might be the cause for such errors? Any thoughts on resolution is appreciated. I have attached the configure.log as well. </span></div>
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<br>
</div>
<span></span>Thanks,
<p></p>
<p style="margin-top:0; margin-bottom:0">D. N. Pham<br>
<br>
</p>
</div>
</body>
</html>