<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Thu, 6 Dec 2018 at 13:40, Klaus Burkart <<a href="mailto:k_burkart@yahoo.com" target="_blank">k_burkart@yahoo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-7370751075108182716m_2339437057610483707ydp9c0591d3yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif;font-size:10px"><div></div>
<div><span>I was wrong with the decomposed case structure:<br><br>In case of a multicore simulation, I get a decomposed case with local data (matrix+rhs+x, neighbour processors (==rank) numbers) on each process/rank. There are no halo regions (instead the original application works with data streams to exchange processor border data). There's no global addressing available at this point. The underlying mesh is decomposed using scotch.<br></span></div></div></div></blockquote><div><br></div><div>It sounds like you are thus unable to explicitly assemble a sparse matrix of type MAT{SEQ,MPI}AIJ.</div><div>That severely limits you choice of "out-of-the-box" preconditioners available through PETSc.</div><div><br></div><div>If your code represented the halo parts and you stored the assembled sub-domain operator, I suspect you could use MATIS which would give you access to PCBDDC.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-7370751075108182716m_2339437057610483707ydp9c0591d3yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif;font-size:10px"><div><span><br>Is this data useable/sufficient to solve a linear system with petsc?<br></span></div></div></div></blockquote><div><br></div><div>You can wrap you action of y = A x in a MATSHELL and feed that to any Krylov method.</div><div>A Krylov method without a preconditioner is pretty useless. At the very least you should implement MatGetDiagonal. Then you could use GMRES/JACOBI</div><div> </div><div>Thanks,</div><div> Dave</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-7370751075108182716m_2339437057610483707ydp9c0591d3yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif;font-size:10px"><div><span><br>If so, what data structure would be required by petsc?<br><br>Klaus</span><br></div><div><br></div>
</div><div id="m_-7370751075108182716m_2339437057610483707yahoo_quoted_4340927883" class="m_-7370751075108182716m_2339437057610483707yahoo_quoted">
<div style="font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-size:13px;color:#26282a">
<div>
Am Montag, 3. Dezember 2018, 15:12:58 MEZ hat Dave May <<a href="mailto:dave.mayhem23@gmail.com" target="_blank">dave.mayhem23@gmail.com</a>> Folgendes geschrieben:
</div>
<div><br></div>
<div><br></div>
<div><div id="m_-7370751075108182716m_2339437057610483707yiv0349591767"><div><div><br clear="none"></div><div><br clear="none"><div class="m_-7370751075108182716m_2339437057610483707yiv0349591767gmail_quote"><div dir="ltr">On Mon, 3 Dec 2018 at 13:52, Klaus Burkart via petsc-users <<a rel="nofollow" shape="rect" href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br clear="none"></div><blockquote class="m_-7370751075108182716m_2339437057610483707yiv0349591767gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><span>Hello,<br clear="none"><br clear="none">I want to solve a cfd case, after decomposition, I get a sub matrix allocated to each process. The example below shows how the data is allocated to the processes (the sample data includes only the lower parts of the matrices). Row and column addresses are local.<br clear="none"><br clear="none">What petsc program setup/concept can be used to solve multiple linear systems in parallel, one on each process? </span></div></blockquote><div><br clear="none"></div><div>Create your KSP, Mat and Vec objects using PETSC_COMM_SELF. </div><div class="m_-7370751075108182716m_2339437057610483707yiv0349591767yqt1323112812" id="m_-7370751075108182716m_2339437057610483707yiv0349591767yqtfd61390"><div><br clear="none"></div><div><br clear="none"></div><blockquote class="m_-7370751075108182716m_2339437057610483707yiv0349591767gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><span><br clear="none"><br clear="none">Klaus<br clear="none"><br clear="none">Sample raw data:<br clear="none"><br clear="none">process matrix section value row column<br clear="none">0 lower 1,80E-05 1 0<br clear="none">0 lower 1,80E-05 5 0<br clear="none">0 lower 1,96E-05 2 1<br clear="none">0 lower 2,20E-05 6 1<br clear="none">0 lower 1,96E-05 3 2<br clear="none">0 lower 2,20E-05 7 2<br clear="none">0 lower 1,96E-05 4 3<br clear="none">0 lower 2,20E-05 8 3<br clear="none">0 lower 2,20E-05 9 4<br clear="none">0 lower 2,20E-05 6 5<br clear="none">0 lower 1,96E-05 10 5<br clear="none">0 lower 2,44E-05 7 6<br clear="none">0 lower 2,44E-05 11 6<br clear="none">0 lower 2,44E-05 8 7<br clear="none">0 lower 2,44E-05 12 7<br clear="none">0 lower 2,44E-05 9 8<br clear="none">0 lower 2,44E-05 13 8<br clear="none">0 lower 2,44E-05 14 9<br clear="none">0 lower 2,20E-05 11 10<br clear="none">0 lower 1,96E-05 15 10<br clear="none">0 lower 2,44E-05 12 11<br clear="none">0 lower 2,44E-05 16 11<br clear="none">0 lower 2,44E-05 13 12<br clear="none">0 lower 2,44E-05 17 12<br clear="none">0 lower 2,44E-05 14 13<br clear="none">0 lower 2,44E-05 18 13<br clear="none">0 lower 2,44E-05 19 14<br clear="none">0 lower 2,20E-05 16 15<br clear="none">0 lower 1,96E-05 20 15<br clear="none">0 lower 2,44E-05 17 16<br clear="none">0 lower 2,44E-05 21 16<br clear="none">0 lower 2,44E-05 18 17<br clear="none">0 lower 2,44E-05 22 17<br clear="none">0 lower 2,44E-05 19 18<br clear="none">0 lower 2,44E-05 23 18<br clear="none">0 lower 2,44E-05 24 19<br clear="none">0 lower 2,20E-05 21 20<br clear="none">0 lower 2,44E-05 22 21<br clear="none">0 lower 2,44E-05 23 22<br clear="none">0 lower 2,44E-05 24 23<br clear="none">1 lower 2,20E-05 1 0<br clear="none">1 lower 1,96E-05 5 0<br clear="none">1 lower 2,44E-05 2 1<br clear="none">1 lower 2,44E-05 6 1<br clear="none">1 lower 2,44E-05 3 2<br clear="none">1 lower 2,44E-05 7 2<br clear="none">1 lower 2,44E-05 4 3<br clear="none">1 lower 2,44E-05 8 3<br clear="none">1 lower 2,44E-05 9 4<br clear="none">1 lower 2,20E-05 6 5<br clear="none">1 lower 1,96E-05 10 5<br clear="none">1 lower 2,44E-05 7 6<br clear="none">1 lower 2,44E-05 11 6<br clear="none">1 lower 2,44E-05 8 7<br clear="none">1 lower 2,44E-05 12 7<br clear="none">1 lower 2,44E-05 9 8<br clear="none">1 lower 2,44E-05 13 8<br clear="none">1 lower 2,44E-05 14 9<br clear="none">1 lower 2,20E-05 11 10<br clear="none">1 lower 1,96E-05 15 10<br clear="none">1 lower 2,44E-05 12 11<br clear="none">1 lower 2,44E-05 16 11<br clear="none">1 lower 2,44E-05 13 12<br clear="none">1 lower 2,44E-05 17 12<br clear="none">1 lower 2,44E-05 14 13<br clear="none">1 lower 2,44E-05 18 13<br clear="none">1 lower 2,44E-05 19 14<br clear="none">1 lower 2,20E-05 16 15<br clear="none">1 lower 1,80E-05 20 15<br clear="none">1 lower 2,44E-05 17 16<br clear="none">1 lower 2,20E-05 21 16<br clear="none">1 lower 2,44E-05 18 17<br clear="none">1 lower 2,20E-05 22 17<br clear="none">1 lower 2,44E-05 19 18<br clear="none">1 lower 2,20E-05 23 18<br clear="none">1 lower 2,20E-05 24 19<br clear="none">1 lower 1,80E-05 21 20<br clear="none">1 lower 1,96E-05 22 21<br clear="none">1 lower 1,96E-05 23 22<br clear="none">1 lower 1,96E-05 24 23<br clear="none">2 lower 1,96E-05 1 0<br clear="none">2 lower 2,20E-05 5 0<br clear="none">2 lower 1,96E-05 2 1<br clear="none">2 lower 2,20E-05 6 1<br clear="none">2 lower 1,96E-05 3 2<br clear="none">2 lower 2,20E-05 7 2<br clear="none">2 lower 1,80E-05 4 3<br clear="none">2 lower 2,20E-05 8 3<br clear="none">2 lower 1,80E-05 9 4<br clear="none">2 lower 2,44E-05 6 5<br clear="none">2 lower 2,44E-05 10 5<br clear="none">2 lower 2,44E-05 7 6<br clear="none">2 lower 2,44E-05 11 6<br clear="none">2 lower 2,44E-05 8 7<br clear="none">2 lower 2,44E-05 12 7<br clear="none">2 lower 2,20E-05 9 8<br clear="none">2 lower 2,44E-05 13 8<br clear="none">2 lower 1,96E-05 14 9<br clear="none">2 lower 2,44E-05 11 10<br clear="none">2 lower 2,44E-05 15 10<br clear="none">2 lower 2,44E-05 12 11<br clear="none">2 lower 2,44E-05 16 11<br clear="none">2 lower 2,44E-05 13 12<br clear="none">2 lower 2,44E-05 17 12<br clear="none">2 lower 2,20E-05 14 13<br clear="none">2 lower 2,44E-05 18 13<br clear="none">2 lower 1,96E-05 19 14<br clear="none">2 lower 2,44E-05 16 15<br clear="none">2 lower 2,44E-05 20 15<br clear="none">2 lower 2,44E-05 17 16<br clear="none">2 lower 2,44E-05 21 16<br clear="none">2 lower 2,44E-05 18 17<br clear="none">2 lower 2,44E-05 22 17<br clear="none">2 lower 2,20E-05 19 18<br clear="none">2 lower 2,44E-05 23 18<br clear="none">2 lower 1,96E-05 24 19<br clear="none">2 lower 2,44E-05 21 20<br clear="none">2 lower 2,44E-05 22 21<br clear="none">2 lower 2,44E-05 23 22<br clear="none">2 lower 2,20E-05 24 23<br clear="none">3 lower 2,44E-05 1 0<br clear="none">3 lower 2,44E-05 5 0<br clear="none">3 lower 2,44E-05 2 1<br clear="none">3 lower 2,44E-05 6 1<br clear="none">3 lower 2,44E-05 3 2<br clear="none">3 lower 2,44E-05 7 2<br clear="none">3 lower 2,20E-05 4 3<br clear="none">3 lower 2,44E-05 8 3<br clear="none">3 lower 1,96E-05 9 4<br clear="none">3 lower 2,44E-05 6 5<br clear="none">3 lower 2,44E-05 10 5<br clear="none">3 lower 2,44E-05 7 6<br clear="none">3 lower 2,44E-05 11 6<br clear="none">3 lower 2,44E-05 8 7<br clear="none">3 lower 2,44E-05 12 7<br clear="none">3 lower 2,20E-05 9 8<br clear="none">3 lower 2,44E-05 13 8<br clear="none">3 lower 1,96E-05 14 9<br clear="none">3 lower 2,44E-05 11 10<br clear="none">3 lower 2,44E-05 15 10<br clear="none">3 lower 2,44E-05 12 11<br clear="none">3 lower 2,44E-05 16 11<br clear="none">3 lower 2,44E-05 13 12<br clear="none">3 lower 2,44E-05 17 12<br clear="none">3 lower 2,20E-05 14 13<br clear="none">3 lower 2,44E-05 18 13<br clear="none">3 lower 1,96E-05 19 14<br clear="none">3 lower 2,44E-05 16 15<br clear="none">3 lower 2,20E-05 20 15<br clear="none">3 lower 2,44E-05 17 16<br clear="none">3 lower 2,20E-05 21 16<br clear="none">3 lower 2,44E-05 18 17<br clear="none">3 lower 2,20E-05 22 17<br clear="none">3 lower 2,20E-05 19 18<br clear="none">3 lower 2,20E-05 23 18<br clear="none">3 lower 1,80E-05 24 19<br clear="none">3 lower 1,96E-05 21 20<br clear="none">3 lower 1,96E-05 22 21<br clear="none">3 lower 1,96E-05 23 22<br clear="none">3 lower 1,80E-05 24 23<br clear="none"></span><div><br clear="none"></div><div class="m_-7370751075108182716m_2339437057610483707yiv0349591767m_-1942437483139585720ydpd19d12bcyahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif;font-size:10px"></div></div></blockquote></div></div></div><div class="m_-7370751075108182716m_2339437057610483707yiv0349591767yqt1323112812" id="m_-7370751075108182716m_2339437057610483707yiv0349591767yqtfd92481">
</div></div></div></div>
</div>
</div></div></blockquote></div></div>