<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Hi Matt, Barry,</span>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Apologies for the extra dependency
on scipy. I can replicate the error by calling setValue (i,j,v) in a loop as well.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">In roughly half of 10 runs,
the following script fails because of an error in hashmapijv – the same as my original post.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">It successfully runs without
error the other times.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Barry is right that it's CUDA
specific. The script runs fine on the CPU.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Do you have any suggestions
or example scripts on assigning entries to a AIJCUSPARSE matrix?</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Here is a minimum snippet
that doesn't depend on scipy.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">```</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">from petsc4py import PETSc</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">import numpy as np</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">n = int(5e5); </span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">nnz = 3 * np.ones(n, dtype=np.int32)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">nnz[0] = nnz[-1] = 2</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A = PETSc.Mat(comm=PETSc.COMM_WORLD)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.createAIJ(size=[n,n],comm=PETSc.COMM_WORLD,nnz=nnz)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.setType('aijcusparse')</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.setValue(0, 0, 2)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.setValue(0, 1, -1)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.setValue(n-1, n-2, -1)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.setValue(n-1, n-1, 2)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">for index in range(1, n -
1):</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"> A.setValue(index,
index - 1, -1)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"> A.setValue(index,
index, 2)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"> A.setValue(index,
index + 1, -1)</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">A.assemble()</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">```</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">If it means anything to you,
when the hash error occurs, it is for index 67283 after filling 201851 nonzero values.</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;"><br>
</span></div>
<div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Thank you for your help and
suggestions!</span></div>
<div style="text-align: left; margin: 0px;" class="elementToProof"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); font-weight: 400;">Anna</span></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Barry Smith <bsmith@petsc.dev><br>
<b>Sent:</b> Thursday, January 18, 2024 2:35 PM<br>
<b>To:</b> Yesypenko, Anna <anna@oden.utexas.edu><br>
<b>Cc:</b> petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject:</b> Re: [petsc-users] HashMap Error when populating AIJCUSPARSE matrix</font>
<div> </div>
</div>
<div style="line-break:after-white-space">
<div><br>
</div>
Do you ever get a problem with <span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">'aij` ? Can you run in a loop with 'aij' to confirm it doesn't fail then?</span>
<div><font face="Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif" size="3"><br>
</font></div>
<div><font face="Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif" size="3"> </font></div>
<div><font face="Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif" size="3"><br>
</font></div>
<div><font face="Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif" size="3"> Barry</font></div>
<div><font face="Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif" size="3"><br>
</font>
<div><br>
<blockquote type="cite">
<div>On Jan 17, 2024, at 4:51 PM, Yesypenko, Anna <anna@oden.utexas.edu> wrote:</div>
<br class="x_Apple-interchange-newline">
<div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
Dear Petsc users/developers,</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
<br>
</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
I'm experiencing a bug when using petsc4py with GPU support. It may be my mistake in how I set up a AIJCUSPARSE matrix.</div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="letter-spacing:normal; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; background-color:rgb(255,255,255); font-weight:400">For larger matrices, I sometimes encounter a error in assigning matrix
values; the error is thrown in PetscHMapIJVQuerySet().</span></div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
Here is a minimum snippet that populates a sparse tridiagonal matrix. </div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
<br>
</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
```</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
from petsc4py import PETSc</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
from scipy.sparse import diags</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
import numpy as np</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
<br>
</div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">n = int(5e5); </span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt"><br>
</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">nnz = 3 * np.ones(n, dtype=np.int32); nnz[0] = nnz[-1] = 2</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">A = PETSc.Mat(comm=PETSc.COMM_WORLD)</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">A.createAIJ(size=[n,n],comm=PETSc.COMM_WORLD,nnz=nnz)</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">A.setType('aijcusparse')</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">tmp = diags([-1,2,-1],[-1,0,+1],shape=(n,n)).tocsr()</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">A.setValuesCSR(tmp.indptr,tmp.indices,tmp.data) ####### this is the line where the error is thrown.</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">A.assemble()</span></div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
```</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
<br>
</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
The error trace is below:</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
```</div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">File "petsc4py/PETSc/Mat.pyx", line 2603, in petsc4py.PETSc.Mat.setValuesCSR</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt"> File "petsc4py/PETSc/petscmat.pxi", line 1039, in petsc4py.PETSc.matsetvalues_csr</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt"> File "petsc4py/PETSc/petscmat.pxi", line 1032, in petsc4py.PETSc.matsetvalues_ijv</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">petsc4py.PETSc.Error: error code 76</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">[0] MatSetValues() at /work/06368/annayesy/ls6/petsc/src/mat/interface/matrix.c:1497</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">[0] MatSetValues_Seq_Hash() at /work/06368/annayesy/ls6/petsc/include/../src/mat/impls/aij/seq/seqhashmatsetvalues.h:52</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">[0] PetscHMapIJVQuerySet() at /work/06368/annayesy/ls6/petsc/include/petsc/private/hashmapijv.h:10</span></div>
<div style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">[0] Error in external library</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">[0] [khash] Assertion: `ret >= 0' failed.</span></div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
```</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
<br>
</div>
<div class="x_elementToProof" style="font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">
If I run the same script a handful of times, it will run without errors eventually.</div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">Does anyone have insight on why it is behaving this way? I'm running on a node with 3x NVIDIA A100 PCIE 40GB.</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt"><br>
</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">Thank you!</span></div>
<div class="x_elementToProof" style="font-family:Helvetica; font-size:18px; font-style:normal; font-variant-caps:normal; font-weight:400; letter-spacing:normal; text-align:start; text-indent:0px; text-transform:none; white-space:normal; word-spacing:0px; text-decoration:none">
<span style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt">Anna</span></div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>