<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We have an example in src/ts/tutorials/autodiff on using AD for reaction-diffusion equations. It does exactly what Matt said - differentiating the stencil kernel to get the Jacobian kernel. More information is available in this report:
<a href="https://urldefense.us/v3/__https://arxiv.org/abs/1909.02836__;!!G_uCfscf7eWS!cMXnAlSzQJa8lo5JBEpmUizoHds-gACnH-ecvwbHQpvuta1pc1NbtArflhZa6Td7oV1qIFEndu5eX9P1yjvtqEGquw$" id="LPlnk842290">https://arxiv.org/abs/1909.02836</a></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hong (Mr.)</div>
<div id="appendonsend"></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display: inline-block; width: 98%;">
<div id="divRplyFwdMsg" dir="ltr"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> petsc-users <petsc-users-bounces@mcs.anl.gov> on behalf of Matthew Knepley <knepley@gmail.com><br>
<b>Sent:</b> Friday, January 17, 2025 6:22 AM<br>
<b>To:</b> Zou, Ling <lzou@anl.gov><br>
<b>Cc:</b> petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject:</b> Re: [petsc-users] Auto sparsity detection?</span>
<div> </div>
</div>
<div style="direction: ltr;">On Thu, Jan 16, 2025 at 10:43 PM Zou, Ling <<a href="mailto:lzou@anl.gov" id="OWA36742e38-9f7c-7a3d-9725-efe95b94d425" class="OWAAutoLink">lzou@anl.gov</a>> wrote:</div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<p style="direction: ltr;"><span style="font-size: 11pt;">Thank you, Matt.</span></p>
<p style="direction: ltr;"><span style="font-size: 11pt;">Seems that at least the matrix coloring part I am following the ‘best practice’.</span></p>
</blockquote>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">Yes, for FD approximations of the Jacobian.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">If you have a stencil operation (like FEM or FVM), then AD can be very useful because you</div>
<div style="direction: ltr;">only have to differentiate the kernel to get the Jacobian kernel.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"> Thanks,</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"> Matt</div>
<div style="direction: ltr;"> </div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<p style="direction: ltr;"><span style="font-size: 11pt;"> </span></p>
<p style="direction: ltr;"><span style="font-size: 11pt;">-Ling</span></p>
<p style="direction: ltr;"><span style="font-size: 11pt;"> </span></p>
<div id="x_m_6510721863087075056mail-editor-reference-message-container">
<div style="padding: 3pt 0in 0in; border-top: 1pt solid rgb(181, 196, 223);">
<p style="margin-bottom: 12pt;"><span style="font-size: 12pt; color: black;"><b>From:
</b>Matthew Knepley <<a href="mailto:knepley@gmail.com" id="OWAccc15827-f6e0-ec3a-18ed-62f079e243d3" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">knepley@gmail.com</a>><br>
<b>Date: </b>Thursday, January 16, 2025 at 9:01</span><span style="font-family: Arial, sans-serif; font-size: 12pt; color: black;"> </span><span style="font-size: 12pt; color: black;">PM<br>
<b>To: </b>Zou, Ling <<a href="mailto:lzou@anl.gov" id="OWAff1bcd10-c8f8-cdca-d511-502e7ef4128c" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">lzou@anl.gov</a>><br>
<b>Cc: </b><a href="mailto:petsc-users@mcs.anl.gov" id="OWAdfe4874a-a95b-09b9-c608-a970b833cec9" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" id="OWA11ce772a-0b29-085e-7d40-644ced977d6e" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Auto sparsity detection?</span></p>
</div>
<p><span style="font-size: 1pt; color: white;">On Thu, Jan 16, 2025 at 9:</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">50 PM Zou, Ling via petsc-users <petsc-users@</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">mcs.</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">anl.</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">gov>
wrote: Hi all, Does PETSc has some automatic matrix sparsity detection algorithm available? Something like: https:</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">//docs.</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">sciml.</span><span style="font-family: Arial, sans-serif; font-size: 1pt; color: white;"> </span><span style="font-size: 1pt; color: white;">ai/NonlinearSolve/stable/basics/sparsity_detection/</span></p>
<p><span style="font-size: 1pt; color: white;">ZjQcmQRYFpfptBannerStart</span></p>
<div id="x_m_6510721863087075056pfptBannert0a6dmn" style="border-width:initial; border-right-style:none; border-bottom-style:none; border-left-style:none; border-right-color:initial; border-bottom-color:initial; border-left-color:initial; display:block; text-align:left; margin:0px; padding:16px; border-radius:4px; min-width:200px; background-color:rgb(208,216,220); border-top-style:initial; border-top-color:rgb(144,164,174)">
<div id="x_m_6510721863087075056pfptBannert0a6dmn">
<div id="x_m_6510721863087075056pfptBannert0a6dmn">
<p style="line-height: 13.5pt; background-color: rgb(208, 216, 220);"><span style="font-family: Arial, sans-serif; font-size: 12pt; color: black;"><b>This Message Is From an External Sender</b></span></p>
</div>
<div id="x_m_6510721863087075056pfptBannert0a6dmn">
<p style="line-height: 13.5pt; background-color: rgb(208, 216, 220);"><span style="font-family: Arial, sans-serif; font-size: 12pt; color: black;">This message came from outside your organization.</span></p>
</div>
</div>
<p style="background-color: rgb(208, 216, 220);"><span style="font-size: 12pt; color: black;"> </span></p>
</div>
<p><span style="font-size: 1pt; color: white;">ZjQcmQRYFpfptBannerEnd</span></p>
<p><span style="font-size: 12pt;">On Thu, Jan 16, 2025 at 9:50</span><span style="font-family: Arial, sans-serif; font-size: 12pt;"> </span><span style="font-size: 12pt;">PM Zou, Ling via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" id="OWAc45322ad-14b3-b61b-d4db-73cc484e0c7a" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">petsc-users@mcs.anl.gov</a>>
wrote:</span></p>
<blockquote style="margin-right: 0in; margin-left: 4.8pt; padding: 0in 0in 0in 6pt; border-left: 1pt solid rgb(204, 204, 204);">
<p><span style="font-size: 11pt;">Hi all,</span></p>
<p><span style="font-size: 11pt;"> </span></p>
<p><span style="font-size: 11pt;">Does PETSc has some automatic matrix sparsity detection algorithm available?</span></p>
<p><span style="font-size: 11pt;">Something like: <a href="https://urldefense.us/v3/__https:/docs.sciml.ai/NonlinearSolve/stable/basics/sparsity_detection/__;!!G_uCfscf7eWS!ccEx6zmuNrVADqtN50hO2N0k4Qs-A70nztAjMLu-JElnjhK5w84BpYC8CAINd6KihSxaS2rx_LgpqUVM49U$" id="OWAfc3831dc-b2ec-edcb-fdac-a406452785eb" class="OWAAutoLink" data-auth="NotApplicable" style="margin-top: 0px; margin-bottom: 0px;">
https://docs.sciml.ai/NonlinearSolve/stable/basics/sparsity_detection/</a></span></p>
</blockquote>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;">Sparsity detection would rely on introspection of the user code for ComputeFunction(), which is not</span></p>
<p><span style="font-size: 12pt;">possible in C (unless you were to code up your evaluation in some symbolic framework).</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<blockquote style="margin-right: 0in; margin-left: 4.8pt; padding: 0in 0in 0in 6pt; border-left: 1pt solid rgb(204, 204, 204);">
<p><span style="font-size: 11pt;">The background is that I use finite differencing plus matrix coloring to (efficiently) get the Jacobian.</span></p>
<p><span style="font-size: 11pt;">For the matrix coloring part, I color the matrix based on mesh connectivity and variable dependencies, which is not bad, but just try to be lazy to even eliminating this part.</span></p>
</blockquote>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;">This is how the automatic frameworks also work. This is how we compute the sparsity pattern for PetscFE and PetscFV.</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<blockquote style="margin-right: 0in; margin-left: 4.8pt; padding: 0in 0in 0in 6pt; border-left: 1pt solid rgb(204, 204, 204);">
<p><span style="font-size: 11pt;">A related but different question, how much does PETSc support automatic differentiation?</span></p>
<p><span style="font-size: 11pt;">I see some old paper:</span></p>
<p><span style="font-size: 11pt;"><a href="https://ftp.mcs.anl.gov/pub/tech_reports/reports/P922.pdf" id="OWAe6b161e7-b713-44d5-eb83-29c8b4964c92" class="OWAAutoLink" data-auth="NotApplicable" style="margin-top: 0px; margin-bottom: 0px;">https://ftp.mcs.anl.gov/pub/tech_reports/reports/P922.pdf</a></span></p>
<p><span style="font-size: 11pt;">and discussion in the roadmap:</span></p>
<p><span style="font-size: 11pt;"><a href="https://urldefense.us/v3/__https:/petsc.org/release/community/roadmap/__;!!G_uCfscf7eWS!ccEx6zmuNrVADqtN50hO2N0k4Qs-A70nztAjMLu-JElnjhK5w84BpYC8CAINd6KihSxaS2rx_Lgpw6v6hKE$" id="OWAf203c0c3-a6db-30df-4e9a-958f6ccd857a" class="OWAAutoLink" data-auth="NotApplicable" style="margin-top: 0px; margin-bottom: 0px;">https://petsc.org/release/community/roadmap/</a></span></p>
<p><span style="font-size: 11pt;">I am thinking that if AD works so I don’t even need to do finite differencing Jacobian, or have it as another option.</span></p>
</blockquote>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;">Other people understand that better than I do.</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;"> Thanks,</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;"> Matt</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<blockquote style="margin-right: 0in; margin-left: 4.8pt; padding: 0in 0in 0in 6pt; border-left: 1pt solid rgb(204, 204, 204);">
<p><span style="font-size: 11pt;">Best,</span></p>
<p><span style="font-size: 11pt;"> </span></p>
<p><span style="font-size: 11pt;">-Ling</span></p>
</blockquote>
<p><span style="font-size: 12pt;"><br>
</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;">--</span></p>
<p><span style="font-size: 12pt;">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</span></p>
<p><span style="font-size: 12pt;"> </span></p>
<p><span style="font-size: 12pt;"><a href="https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!d-7O5V0pNvm_fDSKhNk_ilXP0jG-_MBnectBJ0BfVPOSzARXvYWAahGyRNf1cKCh9dJKEiFt2caV$" id="OWA330cfa90-f959-8d6d-7ebc-f704416adf35" class="OWAAutoLink" data-auth="NotApplicable" style="margin-top: 0px; margin-bottom: 0px;">https://www.cse.buffalo.edu/~knepley/</a></span></p>
</div>
</blockquote>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">--</div>
<div style="direction: ltr;">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!cWyHnKq-Gzasz3ooIUAgTl0RTGrzg0fW8jwVOi0AHE_Ydv4dnayXiG06EPQYvp6guWhXYTv8DMnOu7xNNzJR$" id="OWA313c3ad6-9a9e-01e2-2cb6-945b421aa551" class="OWAAutoLink" data-auth="NotApplicable">https://www.cse.buffalo.edu/~knepley/</a></div>
</body>
</html>