<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.m-2228103146610721686msolistparagraph, li.m-2228103146610721686msolistparagraph, div.m-2228103146610721686msolistparagraph
{mso-style-name:m_-2228103146610721686msolistparagraph;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
span.gmailsignatureprefix
{mso-style-name:gmail_signature_prefix;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:9911333;
mso-list-template-ids:1082034494;}
@list l0:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1
{mso-list-id:88238492;
mso-list-template-ids:-1612801676;}
@list l1:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2
{mso-list-id:1808742356;
mso-list-template-ids:292721846;}
@list l2:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style>
</head>
<body lang="en-CN" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Thank you Matt!<br>
<br>
It appears that the HIP implementation is designed for sparse matvec on AMD GPUs, similar to CUDA for Nvidia GPUs, is that correct?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">We are primarily focusing on CPUs with AOCL. It would be great if PETSc offered an AOCL AIJ class. We could use AOCL in the same manner as we use MKL.
<br>
<br>
In the meantime, I wonder if I can simply use MATAIJ-type matrices and configure PETSc with AOCL during compilation. Will the MatMult() API then use AOCL BLAS as the backend?<br>
<br>
Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Yongzhong<br>
<br>
<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Matthew Knepley <knepley@gmail.com><br>
<b>Date: </b>Monday, August 11, 2025 at 12:44</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;color:black">PM<br>
<b>To: </b>Yongzhong Li <yongzhong.li@mail.utoronto.ca><br>
<b>Cc: </b>petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov>, petsc-maint@mcs.anl.gov <petsc-maint@mcs.anl.gov>, Jasper Hatton <jasper.hatton@mail.utoronto.ca>, Piero Triverio <piero.triverio@utoronto.ca>, Atacan Tuhan <a.tuhan@mail.utoronto.ca><br>
<b>Subject: </b>Re: [petsc-users] Inquiry about configuring PETSc with AOCL BLAS/LAPACK<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">On Mon, Aug 11, 2025 at 12:32</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:12.0pt">PM Yongzhong Li <<a href="mailto:yongzhong.li@mail.utoronto.ca">yongzhong.li@mail.utoronto.ca</a>>
wrote:<o:p></o:p></span></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Dear PETSc’s developer,<br>
<br>
Hi, I am a user of PETSc. I have some questions about how we can configure PETSc with AOCL BLAS and LAPACK.
</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Previously, we linked PETSc with Intel MKL BLAS. This solution provides us with much better multithreading capability for sparse matrix-vector
product compared with configuring PETSc with OpenBLAS. Now, our compute nodes have been upgraded with AMD CPUs, we are considering switching from Intel MKL to AMD AOCL.<br>
<br>
My questions are:</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<ol start="1" type="1">
<li class="m-2228103146610721686msolistparagraph" style="mso-list:l0 level1 lfo1">
<span lang="EN-US" style="font-size:11.0pt">If we configure PETSc in compile time with
<i>—with-blaslapack-dir = $AOCLROOT</i>, will we be able to use AOCL BLAS as the backend of PETSc MatMult() API?</span><o:p></o:p></li></ol>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Do you mean use the AMD sparse matvec? We have a HIP implementation (<a href="https://urldefense.us/v3/__https://petsc.org/main/manualpages/Mat/MATAIJHIPSPARSE/__;!!G_uCfscf7eWS!erObo4bZtlWw2VuQgcNv1mF6FQ0629E3AUF29xJ1MC425B_G8NL2xhSbiULNL58ZGQcMnQWy85kVeTiFbTeeREz4Ubko4R7i9vQ$">https://petsc.org/main/manualpages/Mat/MATAIJHIPSPARSE/</a>), but nothing
for AOCL comparable to the AIJMKL class. If you think we need it, we would certainly help implement it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<ol start="1" type="1">
<li class="m-2228103146610721686msolistparagraph" style="mso-list:l1 level1 lfo2">
<o:p> </o:p></li><li class="m-2228103146610721686msolistparagraph" style="mso-list:l1 level1 lfo2">
<span lang="EN-US" style="font-size:11.0pt">What if AOCL BLAS and AOCL LAPACK are installed in two different directories, not under AOCLROOT?</span><o:p></o:p></li></ol>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">You would use -with-blaslapack-lib=[liblist]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<ol start="1" type="1">
<li class="m-2228103146610721686msolistparagraph" style="mso-list:l2 level1 lfo3">
<o:p> </o:p></li><li class="m-2228103146610721686msolistparagraph" style="mso-list:l2 level1 lfo3">
<span lang="EN-US" style="font-size:11.0pt">PETSc has MatAIJMKL type for sparse matrix stored in Intel MKL format. Does PETSc also have another type for AMD AOCL?</span><o:p></o:p></li></ol>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">No, but it would be straightforward to add.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> Matt<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Thanks!</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Yongzhong</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><br clear="all">
</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix"><span style="font-size:12.0pt">--
</span></span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!erObo4bZtlWw2VuQgcNv1mF6FQ0629E3AUF29xJ1MC425B_G8NL2xhSbiULNL58ZGQcMnQWy85kVeTiFbTeeREz4Ubko--hqJ1I$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>