<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Matt: Thank you for the reply. The bulk of it makes a lot of sense. Yes! That need to keep track of the original mesh numbers (AKA "Natural") is what I find pressing for my research group. Awesome! I was separately keeping track of
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannerrt3zgl7" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannerrt3zgl7" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannerrt3zgl7" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannerrt3zgl7" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html>
<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannerrt3zgl7 { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonrt3zgl7:hover, .pfptPrimaryButtonrt3zgl7:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonrt3zgl7:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Matt:<br>
<br>
Thank you for the reply.</div>
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
The bulk of it makes a lot of sense.</div>
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Yes! That need to keep track of the original mesh numbers (AKA "Natural") is what I find pressing for my research group.</div>
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Awesome! I was separately keeping track of these numbers using a PetscSection that I was inputting into DMSetLocalSection() but of the coordinate DM, not the plex.</div>
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
It is good to know the "correct" way to do it.</div>
<div class="elementToProof" style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
"What is repetitive? It should be able to be automated."</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Absolutely as the intrinsic process is ubiquitous between mesh formats.</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
What I meant by "repetitive" is the information that is reused by different API calls (namely, global stratum sizes, and local point numbers corresponding to owned DAG points).</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
I need to define a struct to bookkeep this. It's not really an issue, rather a minor annoyance (for me).</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
I need the stratum sizes to offset DMPlex numbering cells in range [0,nCell) and vertices ranging in [nCell,nCell+nVert) to other mesh numberings where cells range from [1, nCell] and vertices range from [1, nVert]. In my experience, this information is needed
at least three (3) times, during coordinate writes, during element connectivity writes, and during DMLabel writes for BC's and other labelled data.</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
This information I determine using a code snippet like this:</div>
<div style="direction: ltr; margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
PetscCall(PetscObjectGetComm((PetscObject)plex,&mpiComm));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCallMPI(MPI_Comm_rank(mpiComm,&mpiRank));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCallMPI(MPI_Comm_size(mpiComm,&mpiCommSize));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(DMPlexCreatePointNumbering(plex,&GlobalNumberIS));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(ISGetIndices(GlobalNumberIS,&IdxPtr));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(DMPlexGetDepth(plex,&Depth));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(PetscMalloc3(//</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Depth,&LocalIdxPtrPtr,//Indices in the local stratum to owned points.</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Depth,&pOwnedPtr,//Number of points in the local stratum that are owned.</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Depth,&GlobalStratumSizePtr//Global stratum size.</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
for(PetscInt jj = 0;jj < Depth;jj++){</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(DMPlexGetDepthStratum(plex,jj,&pStart,&pEnd));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
pOwnedPtr[jj] = 0;</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
for(PetscInt ii = pStart;ii < pEnd;ii++){</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
if(IdxPtr[ii] >= 0) pOwnedPtr[jj]++;</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCallMPI(MPI_Allreduce(&pOwnedPtr[jj],&GlobalStratumSizePtr[jj],1,MPIU_INT,MPI_MAX,mpiComm));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(PetscMalloc1(pOwnedPtr[jj],&LocalIdxPtrPtr[jj]));</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
kk = 0;</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
for(PetscInt ii = pStart;ii < pEnd; ii++){</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
if(IdxPtr[ii] >= 0){</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
LocalIdxPtrPtr[jj][kk] = ii;</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
kk++;</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
}</div>
<div style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(ISRestoreIndices(GlobalNumberIS,&IdxPtr));</div>
<div class="elementToProof" style="margin-right: 40px; margin-left: 40px; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
PetscCall(ISDestroy(&GlobalNumberIS));</div>
<hr style="display: inline-block; width: 98%;">
<div id="divRplyFwdMsg" dir="ltr" style="color: inherit;"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> Matthew Knepley <knepley@gmail.com><br>
<b>Sent:</b> Thursday, July 11, 2024 8:32 PM<br>
<b>To:</b> Ferrand, Jesus A. <FERRANJ2@my.erau.edu><br>
<b>Cc:</b> petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject:</b> [EXTERNAL] Re: [petsc-users] What exactly is the GlobalToNatural PetscSF of DMPlex/DM?</span>
<div> </div>
</div>
<table style="text-align: left; line-height: 1.5; border-color: rgb(136, 136, 136); border-style: solid; border-width: 1px; background-color: rgb(255, 235, 156); width: 100%; min-width: 100%; color: black; box-sizing: border-box; border-collapse: collapse; border-spacing: 0px;">
<tbody>
<tr>
<td style="text-align: left; line-height: 1.5; padding: 5px 10px;">
<div style="text-align: left; line-height: 1.5; font-family: Arial, sans-serif; font-size: 14px;">
<span style="color: rgb(214, 0, 0);"><b>CAUTION:</b></span><span style="color: black;"> This email originated outside of Embry-Riddle Aeronautical University. Do not click links or open attachments unless you recognize the sender and know the content is safe.</span></div>
</td>
</tr>
</tbody>
</table>
<br>
<div style="direction: ltr;">On Mon, Jul 8, 2024 at 10:28 PM Ferrand, Jesus A. <<a href="mailto:FERRANJ2@my.erau.edu" id="OWA985dde05-f588-870e-0d8f-a23dd9439b34" class="OWAAutoLink">FERRANJ2@my.erau.edu</a>> wrote:</div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<div dir="ltr" id="x_m_6141427388992354471pfptBannercsz537f" style="display: block; text-align: left; margin: 16px 0px; padding: 8px 16px; border-radius: 4px; min-width: 200px; background-color: rgb(208, 216, 220); border-top: 4px solid rgb(144, 164, 174); color: inherit;">
<div id="x_m_6141427388992354471pfptBannercsz537f" style="float: left; display: block; margin: 0px 0px 1px; max-width: 600px; color: inherit;">
<div id="x_m_6141427388992354471pfptBannercsz537f" style="display: block; background-color: rgb(208, 216, 220); color: rgb(0, 0, 0); font-family: Arial, sans-serif; font-weight: bold; font-size: 14px; line-height: 18px;">
This Message Is From an External Sender</div>
<div id="x_m_6141427388992354471pfptBannercsz537f" style="font-weight: normal; display: block; background-color: rgb(208, 216, 220); color: rgb(0, 0, 0); font-family: Arial, sans-serif; font-size: 12px; line-height: 18px; margin-top: 2px;">
This message came from outside your organization.</div>
</div>
<div style="line-height: 0; display: block; font-size: 0.01px;"> </div>
</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Dear PETSc team:<br>
<br>
Greetings.</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
I keep working on mesh I/O utilities using DMPlex.</div>
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Specifically for the output stage, I need a solid grasp on the global numbers and ideally how to set them into the DMPlex during an input operation and carrying the global numbers through API calls to DMPlexDistribute() or DMPlexMigrate() and hopefully also
through some of the mesh adaption APIs. I was wondering if the GlobalToNatural PetscSF manages these global numbers. The next most useful object is the PointSF, but to me, it seems to only help establish DAG point ownership, not DAG point global indices.</div>
</blockquote>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">This is a good question, and gets at a design point of Plex. I don't believe global numbers are the "right" way to talk about mesh points, or</div>
<div style="direction: ltr;">even a very useful way to do it, for several reasons. Plex is designed to run just fine without any global numbers. It can, of course, produce</div>
<div style="direction: ltr;">them on command, as many people remain committed to their existence.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">Thus, the first idea is that global numbers should not be stored, since they can always be created on command very cheaply. It is much more</div>
<div style="direction: ltr;">costly to write global numbers to disk, or pull them through memory, than compute them.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">The second idea is that we use a combination of local numbers, namely (rank, point num) pairs, and PetscSF objects to establish sharing relations for parallel meshes. Global numbering is a particular traversal of a mesh, running
over the locally owned parts of each mesh in local order. Thus an SF + a local order = a global order, and the local order is provided by the point numbering.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">The third idea is that a "natural" order is just the global order in which a mesh is first fed to Plex. When I redistribute and reorder for good performance, I keep track of a PetscSF that can map the mesh back to the original order
in which it was provided. I see this as an unneeded expense, but many many people want output written in the original order (mostly because processing tools are so poor). This management is what we mean by GlobalToNatural.</div>
<div style="direction: ltr;"> </div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<div style="direction: ltr; font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);">
Otherwise, I have been working with the IS obtained from DMPlexGetPointNumbering() and manually determining global stratum sizes, offsets, and numbers by looking at the signs of the involuted index list that comes with that IS. It's working for now (I can monolithically
write meshes to CGNS in parallel), but it is resulting in repetitive code that I will need for another mesh format that I want to support.</div>
</blockquote>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">What is repetitive? It should be able to be automated.</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"> Thanks,</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"> Matt</div>
<div style="direction: ltr;"> </div>
<blockquote style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left: 1px solid rgb(204, 204, 204);">
<div id="x_m_6141427388992354471Signature" style="color: inherit;">
<div dir="ltr" id="x_m_6141427388992354471divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif;">
<p style="line-height: normal; background-color: white; margin: 0px 0px 11px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">Sincerely:</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Courier New", monospace; font-size: 14pt; color: red;"><b><u>J.A. Ferrand</u></b></span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">Embry-Riddle Aeronautical University - Daytona Beach - FL<br>
Ph.D. Candidate, Aerospace Engineering</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">M.Sc. Aerospace Engineering</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">B.Sc. Aerospace Engineering</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">B.Sc. Computational Mathematics</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;"><u>Phone:</u> (386)-843-1829</span></p>
<p style="line-height: normal; background-color: white; margin: 0px;"><span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;"><u>Email(s):</u> <a href="mailto:ferranj2@my.erau.edu" id="OWAa6111687-63d1-f757-e9c9-7f23e62671c9" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">ferranj2@my.erau.edu</a></span></p>
<p style="text-indent: 0.5in; line-height: normal; background-color: white; margin: 0px;">
<span style="font-family: "Times New Roman", Times, serif; font-size: 12pt; color: black;">
<a href="mailto:jesus.ferrand@gmail.com" id="OWAcd84424a-c088-0a90-7e52-0eb39630b6e9" class="OWAAutoLink" style="margin-top: 0px; margin-bottom: 0px;">
jesus.ferrand@gmail.com</a></span></p>
</div>
</div>
</blockquote>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;">--</div>
<div style="direction: ltr;">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr;"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!cOc1e73UmcgVF_64Q8DkGuEXm_3Lk1ZXoBu6H6FnWzrEQyMfXwYg-ZEB1SEFgw06kHuPQglPr-avz-FygftLGOQyrF8$" id="OWAf252ded5-bce8-1790-b901-97cab8af6306" class="OWAAutoLink" data-auth="NotApplicable">https://www.cse.buffalo.edu/~knepley/</a></div>
</body>
</html>