<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On Aug 11, 2025, at 5:26 PM, Yongzhong Li via petsc-users <petsc-users@mcs.anl.gov> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Thank you Matt!<br><br>It appears that the HIP implementation is designed for sparse matvec on AMD GPUs, similar to CUDA for Nvidia GPUs, is that correct?<o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;"><o:p> </o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">We are primarily focusing on CPUs with AOCL. It would be great if PETSc offered an AOCL AIJ class. We could use AOCL in the same manner as we use MKL.<br><br>In the meantime, I wonder if I can simply use MATAIJ-type matrices and configure PETSc with AOCL during compilation. Will the MatMult() API then use AOCL BLAS as the backend?<br></span></div></div></div></blockquote><div><br></div> BLAS (and LAPACK) are for vectors and dense matrices so the quality (or lack of) of BLAS and LAPACK doesn't have any affect on the MATAIJ operations which are sparse matrix operations.</div><div><br></div><div> So yes use AOCL BLAS and MATAIJ for your sparse matrices.</div><div><br><blockquote type="cite"><div><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;"><br>Thanks,<o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Yongzhong<br><br> <br><br><o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div><div id="mail-editor-reference-message-container"><div><div><div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; border-image: none; padding: 3pt 0cm 0cm;"><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 10pt; font-family: Aptos, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;">Matthew Knepley <<a href="mailto:knepley@gmail.com" style="color: blue; text-decoration: underline;">knepley@gmail.com</a>><br><b>Date:<span class="Apple-converted-space"> </span></b>Monday, August 11, 2025 at 12:44</span><span style="font-size: 12pt; font-family: Arial, sans-serif;"> </span><span style="font-size: 12pt;">PM<br><b>To:<span class="Apple-converted-space"> </span></b>Yongzhong Li <<a href="mailto:yongzhong.li@mail.utoronto.ca" style="color: blue; text-decoration: underline;">yongzhong.li@mail.utoronto.ca</a>><br><b>Cc:<span class="Apple-converted-space"> </span></b><a href="mailto:petsc-users@mcs.anl.gov" style="color: blue; text-decoration: underline;">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" style="color: blue; text-decoration: underline;">petsc-users@mcs.anl.gov</a>>,<span class="Apple-converted-space"> </span><a href="mailto:petsc-maint@mcs.anl.gov" style="color: blue; text-decoration: underline;">petsc-maint@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-maint@mcs.anl.gov" style="color: blue; text-decoration: underline;">petsc-maint@mcs.anl.gov</a>>, Jasper Hatton <<a href="mailto:jasper.hatton@mail.utoronto.ca" style="color: blue; text-decoration: underline;">jasper.hatton@mail.utoronto.ca</a>>, Piero Triverio <<a href="mailto:piero.triverio@utoronto.ca" style="color: blue; text-decoration: underline;">piero.triverio@utoronto.ca</a>>, Atacan Tuhan <<a href="mailto:a.tuhan@mail.utoronto.ca" style="color: blue; text-decoration: underline;">a.tuhan@mail.utoronto.ca</a>><br><b>Subject:<span class="Apple-converted-space"> </span></b>Re: [petsc-users] Inquiry about configuring PETSc with AOCL BLAS/LAPACK<o:p></o:p></span></p></div><div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;">On Mon, Aug 11, 2025 at 12:32</span><span style="font-size: 12pt; font-family: Arial, sans-serif;"> </span><span style="font-size: 12pt;">PM Yongzhong Li <<a href="mailto:yongzhong.li@mail.utoronto.ca" style="color: blue; text-decoration: underline;">yongzhong.li@mail.utoronto.ca</a>> wrote:<o:p></o:p></span></div></div><div><blockquote style="border-width: medium medium medium 1pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204); border-image: none; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;"><div><div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Dear PETSc’s developer,<br><br>Hi, I am a user of PETSc. I have some questions about how we can configure PETSc with AOCL BLAS and LAPACK.</span><span style="font-size: 12pt;"><o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Previously, we linked PETSc with Intel MKL BLAS. This solution provides us with much better multithreading capability for sparse matrix-vector product compared with configuring PETSc with OpenBLAS. Now, our compute nodes have been upgraded with AMD CPUs, we are considering switching from Intel MKL to AMD AOCL.<br><br>My questions are:</span><span style="font-size: 12pt;"><o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div><ol start="1" type="1" style="margin-bottom: 0cm;"><li class="m-2228103146610721686msolistparagraph" style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">If we configure PETSc in compile time with<span class="Apple-converted-space"> </span><i>—with-blaslapack-dir = $AOCLROOT</i>, will we be able to use AOCL BLAS as the backend of PETSc MatMult() API?</span><o:p></o:p></li></ol></div></div></div></blockquote><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;">Do you mean use the AMD sparse matvec? We have a HIP implementation (<a href="https://urldefense.us/v3/__https://petsc.org/main/manualpages/Mat/MATAIJHIPSPARSE/__;!!G_uCfscf7eWS!erObo4bZtlWw2VuQgcNv1mF6FQ0629E3AUF29xJ1MC425B_G8NL2xhSbiULNL58ZGQcMnQWy85kVeTiFbTeeREz4Ubko4R7i9vQ$" style="color: blue; text-decoration: underline;">https://petsc.org/main/manualpages/Mat/MATAIJHIPSPARSE/</a>), but nothing for AOCL comparable to the AIJMKL class. If you think we need it, we would certainly help implement it.<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><blockquote style="border-width: medium medium medium 1pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204); border-image: none; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;"><div><div><div><ol start="1" type="1" style="margin-bottom: 0cm;"><li class="m-2228103146610721686msolistparagraph" style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></li><li class="m-2228103146610721686msolistparagraph" style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">What if AOCL BLAS and AOCL LAPACK are installed in two different directories, not under AOCLROOT?</span><o:p></o:p></li></ol></div></div></div></blockquote><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;">You would use -with-blaslapack-lib=[liblist]<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><blockquote style="border-width: medium medium medium 1pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204); border-image: none; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;"><div><div><div><ol start="1" type="1" style="margin-bottom: 0cm;"><li class="m-2228103146610721686msolistparagraph" style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></li><li class="m-2228103146610721686msolistparagraph" style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">PETSc has MatAIJMKL type for sparse matrix stored in Intel MKL format. Does PETSc also have another type for AMD AOCL?</span><o:p></o:p></li></ol></div></div></div></blockquote><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;">No, but it would be straightforward to add.<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"> Thanks,<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"> Matt<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><blockquote style="border-width: medium medium medium 1pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204); border-image: none; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;"><div><div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Thanks!</span><span style="font-size: 12pt;"><o:p></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="font-size: 11pt;">Yongzhong</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div></blockquote></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><br clear="all"></span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span class="gmailsignatureprefix"><span style="font-size: 12pt;">--</span></span><span style="font-size: 12pt;"><o:p></o:p></span></div><div><div><div><div><div><div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><o:p> </o:p></span></div></div><div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;"><span style="font-size: 12pt;"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!erObo4bZtlWw2VuQgcNv1mF6FQ0629E3AUF29xJ1MC425B_G8NL2xhSbiULNL58ZGQcMnQWy85kVeTiFbTeeREz4Ubko--hqJ1I$" target="_blank" style="color: blue; text-decoration: underline;">https://www.cse.buffalo.edu/~knepley/</a></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote></div><br></body></html>