<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Times New Roman",serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:210654295;
mso-list-template-ids:-1484063740;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:1713577303;
mso-list-template-ids:-1514519776;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Pierre,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"> You are right. There are a few MatMultTransposeAdd that may need conforming layouts for the equality/inequality constraint vectors and equality/inequality constraint
Jacobian matrices. I need to check if that’s the case. We only have ex1 example currently, we need to add more examples. We are currently working on making PDIPM robust and while doing it will work on adding another example.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in">Very naive question, but given that I have a single constraint, how do I split a 1 x N matrix column-wise? I thought it was not possible.<o:p></o:p></p>
<p class="MsoNormal"><i><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">When setting the size of the constraint vector, you need to set the local size on one rank to 1 and all others to zero. For the Jacobian, the local row size on that rank
will be 1 and all others to zero. The column layout for the Jacobian should follow the layout for vector x. So each rank will set the local column size of the Jacobian to local size of x.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Shri<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">On 15 Sep 2020, at 2:21 AM, Abhyankar, Shrirang G <<a href="mailto:shrirang.abhyankar@pnnl.gov">shrirang.abhyankar@pnnl.gov</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Hello Pierre,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"> PDIPM works in parallel so you can have distributed Hessian, Jacobians, constraints, variables, gradients in any layout you want. If you are
using a DM then you can have it generate the Hessian.<span class="apple-converted-space"> </span></span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Could you please show an example where this is the case?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">pdipm->x, which I’m assuming is a working vector, is both used as input for Hessian and Jacobian functions, e.g., <a href="https://gitlab.com/petsc/petsc/-/blob/master/src/tao/constrained/impls/ipm/pdipm.c#L369">https://gitlab.com/petsc/petsc/-/blob/master/src/tao/constrained/impls/ipm/pdipm.c#L369</a> (Hessian)
+ <a href="https://gitlab.com/petsc/petsc/-/blob/master/src/tao/constrained/impls/ipm/pdipm.c#L473">https://gitlab.com/petsc/petsc/-/blob/master/src/tao/constrained/impls/ipm/pdipm.c#L473</a> (Jacobian)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I thus doubt that it is possible to have different layouts?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">In practice, I end up with the following error when I try this (2 processes, distributed Hessian with centralized Jacobian):<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: Nonconforming object sizes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: Vector wrong size 14172 for scatter 0 (scatter reverse and vector to != ctx from size)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: #1 VecScatterBegin() line 96 in /Users/jolivet/Documents/repositories/petsc/src/vec/vscat/interface/vscatfce.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: #2 MatMultTransposeAdd_MPIAIJ() line 1223 in /Users/jolivet/Documents/repositories/petsc/src/mat/impls/aij/mpi/mpiaij.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: #3 MatMultTransposeAdd() line 2648 in /Users/jolivet/Documents/repositories/petsc/src/mat/interface/matrix.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Nonconforming object sizes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Vector wrong size 13790 for scatter 27962 (scatter reverse and vector to != ctx from size)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: #4 TaoSNESFunction_PDIPM() line 510 in /Users/jolivet/Documents/repositories/petsc/src/tao/constrained/impls/ipm/pdipm.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #5 TaoSolve_PDIPM() line 712 in /Users/jolivet/Documents/repositories/petsc/src/tao/constrained/impls/ipm/pdipm.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1]PETSC ERROR: #6 TaoSolve() line 222 in /Users/jolivet/Documents/repositories/petsc/src/tao/interface/taosolver.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #1 VecScatterBegin() line 96 in /Users/jolivet/Documents/repositories/petsc/src/vec/vscat/interface/vscatfce.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #2 MatMultTransposeAdd_MPIAIJ() line 1223 in /Users/jolivet/Documents/repositories/petsc/src/mat/impls/aij/mpi/mpiaij.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #3 MatMultTransposeAdd() line 2648 in /Users/jolivet/Documents/repositories/petsc/src/mat/interface/matrix.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #4 TaoSNESFunction_PDIPM() line 510 in /Users/jolivet/Documents/repositories/petsc/src/tao/constrained/impls/ipm/pdipm.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #5 TaoSolve_PDIPM() line 712 in /Users/jolivet/Documents/repositories/petsc/src/tao/constrained/impls/ipm/pdipm.c<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #6 TaoSolve() line 222 in /Users/jolivet/Documents/repositories/petsc/src/tao/interface/taosolver.c<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I think this can be reproduced by ex1.c by just distributing the Hessian instead of having it centralized on rank 0.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Ideally, you want to have the layout below to minimize movement of matrix/vector elements across ranks.</span><o:p></o:p></p>
</div>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Times New Roman",serif">The layout of vectors x, bounds on x, and gradient is same.</span><o:p></o:p></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Times New Roman",serif">The row layout of the equality/inequality Jacobian is same as the equality/inequality constraint vector layout.</span><o:p></o:p></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Times New Roman",serif">The column layout of the equality/inequality Jacobian is same as that for x.</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Very naive question, but given that I have a single constraint, how do I split a 1 x N matrix column-wise? I thought it was not possible.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Pierre<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoListParagraph" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:1.0in;text-indent:-.25in;mso-list:l1 level1 lfo2">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Times New Roman",serif">The row and column layout for the Hessian is same as x.</span><o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">The tutorial example ex1 is extremely small (only 2 variables) so its implementation is very simplistic. I think, in parallel, it ships off constraints
etc. to rank 0. It’s not an ideal example w.r.t demonstrating a parallel implementation. We aim to add more examples as we develop PDIPM. If you have an example to contribute then we would most welcome it and provide help on adding it.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"> </span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Thanks,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Shri</span><o:p></o:p></p>
</div>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt">From:<span class="apple-converted-space"> </span></span></b><span style="font-size:12.0pt">petsc-dev <<a href="mailto:petsc-dev-bounces@mcs.anl.gov">petsc-dev-bounces@mcs.anl.gov</a>>
on behalf of Pierre Jolivet <<a href="mailto:pierre.jolivet@enseeiht.fr">pierre.jolivet@enseeiht.fr</a>><br>
<b>Date:<span class="apple-converted-space"> </span></b>Monday, September 14, 2020 at 1:52 PM<br>
<b>To:<span class="apple-converted-space"> </span></b>PETSc Development <<a href="mailto:petsc-dev@mcs.anl.gov">petsc-dev@mcs.anl.gov</a>><br>
<b>Subject:<span class="apple-converted-space"> </span></b>[petsc-dev] PDIPDM questions</span><o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">Hello,<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">In my quest to help users migrate from Ipopt to Tao, I’ve a new question.<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">When looking at src/tao/constrained/tutorials/ex1.c, it seems that almost everything is centralized on rank 0 (local sizes are 0 but on rank 0).<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">I’d like to have my Hessian distributed more naturally, as in (almost?) all other SNES/TS examples, but still keep the Jacobian of my equality constraint, which is of dimension 1 x N (N >> 1), centralized on rank
0.<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">Is this possible?<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">If not, is it possible to supply the transpose of the Jacobian, of dimension N x 1, which could then be distributed row-wise like the Hessian?<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">Or maybe use some trick to distribute a MatAIJ/MatDense of dimension 1 x N column-wise? Use a MatNest with as many blocks as processes?<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">So, just to sum up, how can I have a distributed Hessian with a Jacobian with a single row?<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">Thanks in advance for your help,<o:p></o:p></p>
</div>
</div>
<div>
<div style="margin-left:.5in">
<p class="MsoNormal" style="margin-left:.5in">Pierre<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>