<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div><div> The following assumes you are not using the shell matrix context for some other purpose</div><div><br></div><blockquote type="cite"><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">subroutine formJacobian(snes,F_global,Jac,Jac_pre,dummy,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> SNES :: snes<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> Vec :: F_global<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! real(pREAL), dimension(3,3,cells(1),cells(2),cells3), intent(in) :: &<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! F<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> Mat :: Jac, Jac_pre<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscObject :: dummy<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscErrorCode :: err_PETSc<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, '@@ start build my jac'<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscCall(MatShellSetContext(Jac,F_global,ierr)) ! record the current base vector where the Jacobian is to be applied</span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, '@@ end build my jac'<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">end subroutine formJacobian</span></div></div></blockquote><div><br></div>subroutine Gk_op <div>...</div><div> Vec base</div><div> PetscCall(MatShellGetContext(Jac,base,ierr))</div><div><br></div><div> ! use base in the computation of your matrix-free Jacobian vector product</div><div>....</div><div><br><br><div><br><blockquote type="cite"><div>On Jan 11, 2024, at 5:55 AM, Yi Hu <y.hu@mpie.de> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Now I understand a bit more about the workflow of set jacobian. It seems that the SNES can be really fine-grained. As you point out, J is built via formJacobian() callback, and can be based on previous solution (or the base vector u, as you mentioned). And then KSP can use a customized MATOP_MULT to solve the linear equations J(u)*x=rhs.<span class="Apple-converted-space"> </span><o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">So I followed your idea about removing DMSNESSetJacobianLocal() and did the following.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">……<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,&<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> 9*product(cells(1:2))*cells3,9*product(cells(1:2))*cells3,&<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> 0,Jac_PETSc,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatShellSetOperation(Jac_PETSc,MATOP_MULT,GK_op,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call SNESSetJacobian(SNES_mech,Jac_PETSc,Jac_PETSc,formJacobian,0,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">……<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">And my formJacobian() is<span class="Apple-converted-space"> </span><o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">subroutine formJacobian(snes,F_global,Jac,Jac_pre,dummy,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <span class="Apple-converted-space"> </span><o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> SNES :: snes<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> Vec :: F_global<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! real(pREAL), dimension(3,3,cells(1),cells(2),cells3), intent(in) :: &<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! F<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> Mat :: Jac, Jac_pre<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscObject :: dummy<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscErrorCode :: err_PETSc<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, '@@ start build my jac'<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <span class="Apple-converted-space"> </span><o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCopy(Jac_PETSc,Jac,SAME_NONZERO_PATTERN,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCopy(Jac_PETSc,Jac_pre,SAME_NONZERO_PATTERN,err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! Jac = Jac_PETSc<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! Jac_pre = Jac_PETSc<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, '@@ end build my jac'<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">end subroutine formJacobian<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">it turns out that no matter by a simple assignment or MatCopy(), the compiled program gives me the same error as before. So I guess the real jacobian is still not set. I wonder how to get around this and let this built jac in formJacobian() to be the same as my shell matrix.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Yi<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(225, 225, 225) currentcolor currentcolor; border-image: none; padding: 3pt 0in 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span lang="EN-US"><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" style="color: purple; text-decoration: underline;">bsmith@petsc.dev</a>><span class="Apple-converted-space"> </span><br><b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, January 10, 2024 4:27 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;">y.hu@mpie.de</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" style="color: purple; text-decoration: underline;">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] SNES seems not use my matrix-free operation<o:p></o:p></span></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> By default if SNESSetJacobian() is not called with a function pointer PETSc attempts to compute the Jacobian matrix explicitly with finite differences and coloring. This doesn't makes sense with a shell matrix. Hence the error message below regarding MatFDColoringCreate().<o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> DMSNESSetJacobianLocal() calls SNESSetJacobian() with a function pointer of SNESComputeJacobian_DMLocal() so preventing the error from triggering in your code.<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> You can provide your own function to SNESSetJacobian() and thus not need to call DMSNESSetJacobianLocal(). What you do depends on how you want to record the "base" vector that tells your matrix-free multiply routine where the Jacobian matrix vector product is being applied, that is J(u)*x. u is the "base" vector which is passed to the function provided with SNESSetJacobian().<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> Barry<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><br><br><o:p></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">On Jan 10, 2024, at 6:20 AM, Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;">y.hu@mpie.de</a>> wrote:<o:p></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Thanks for the clarification. It is more clear to me now about the global to local processes after checking the examples, e.g. ksp/ksp/tutorials/ex14f.F90.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">And for using Vec locally, I followed your advice of VecGet.. and VecRestore… In fact I used DMDAVecGetArrayReadF90() and some other relevant subroutines.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">For your comment on DMSNESSetJacobianLocal(). It seems that I need to use both SNESSetJacobian() and then DMSNESSetJacobianLocal() to get things working. When I do only SNESSetJacobian(), it does not work, meaning the following does not work</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">……<span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="font-size: 10pt; font-family: "Courier New";"> </span><span style="font-size: 10pt; font-family: "Courier New";">call DMDASNESsetFunctionLocal(DM_mech,INSERT_VALUES,formResidual,PETSC_NULL_SNES,err_PETSc) <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,&</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> 9*product(cells(1:2))*cells3,9*product(cells(1:2))*cells3,&</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> 0,Jac_PETSc,err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatShellSetOperation(Jac_PETSc,MATOP_MULT,GK_op,err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call SNESSetJacobian(SNES_mech,Jac_PETSc,Jac_PETSc,PETSC_NULL_FUNCTION,0,err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> !call DMSNESsetJacobianLocal(DM_mech,formJacobian,PETSC_NULL_SNES,err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> !CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call SNESsetConvergenceTest(SNES_mech,converged,PETSC_NULL_SNES,PETSC_NULL_FUNCTION,err_PETSc) <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call SNESSetDM(SNES_mech,DM_mech,err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">……</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">It gives me the message<span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: No support for this operation for this object type </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: Code not yet written for matrix type shell <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: See<span class="apple-converted-space"> </span><a href="https://petsc.org/release/faq/" style="color: purple; text-decoration: underline;"><span style="color: purple;">https://petsc.org/release/faq/</span></a><span class="apple-converted-space"> </span>for trouble shooting. <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: Petsc Release Version 3.16.4, Feb 02, 2022</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu --with-fortran-bindings --with-mpi-f90module-visibility=0 --download-fftw --download-hdf5 --download-hdf5-fortran-bindings --download-fblaslapack --download-ml --download-zlib </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #1 MatFDColoringCreate() at /home/yi/app/petsc-3.16.4/src/mat/matfd/fdmatrix.c:471 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #2 SNESComputeJacobian_DMDA() at /home/yi/app/petsc-3.16.4/src/snes/utils/dmdasnes.c:173[0]PETSC ERROR: #3 SNESComputeJacobian() at /home/yi/app/petsc-3.16.4/src/snes/interface/snes.c:2864 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #4 SNESSolve_NEWTONLS() at /home/yi/app/petsc-3.16.4/src/snes/impls/ls/ls.c:222 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #5 SNESSolve() at /home/yi/app/petsc-3.16.4/src/snes/interface/snes.c:4809 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #6 User provided function() at User file:0 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #7 VecSetErrorIfLocked() at /home/yi/app/petsc-3.16.4/include/petscvec.h:623 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #8 VecGetArray() at /home/yi/app/petsc-3.16.4/src/vec/vec/interface/rvector.c:1769 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #9 User provided function() at User file:0 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #10 MatFDColoringCreate() at /home/yi/app/petsc-3.16.4/src/mat/matfd/fdmatrix.c:471 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #11 SNESComputeJacobian_DMDA() at /home/yi/app/petsc-3.16.4/src/snes/utils/dmdasnes.c:173 </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #12 SNESComputeJacobian() at /home/yi/app/petsc-3.16.4/src/snes/interface/snes.c:2864 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #13 SNESSolve_NEWTONLS() at /home/yi/app/petsc-3.16.4/src/snes/impls/ls/ls.c:222 <span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">[0]PETSC ERROR: #14 SNESSolve() at /home/yi/app/petsc-3.16.4/src/snes/interface/snes.c:4809</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">It seems that I have to use a DMSNESSetJacobianLocal() to “activate” the use of my shell matrix, although the formJacobian() in the DMSNESSetJacobianLocal() is doing nothing.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Best wishes,</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Yi</span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div><div><div style="border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-color: currentcolor; border-image: none;"><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">Barry Smith <<a href="mailto:bsmith@petsc.dev" style="color: purple; text-decoration: underline;"><span style="color: purple;">bsmith@petsc.dev</span></a>><span class="apple-converted-space"> </span><br><b>Sent:</b><span class="apple-converted-space"> </span>Tuesday, January 9, 2024 4:49 PM<br><b>To:</b><span class="apple-converted-space"> </span>Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;"><span style="color: purple;">y.hu@mpie.de</span></a>><br><b>Cc:</b><span class="apple-converted-space"> </span>petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" style="color: purple; text-decoration: underline;"><span style="color: purple;">petsc-users@mcs.anl.gov</span></a>><br><b>Subject:</b><span class="apple-converted-space"> </span>Re: [petsc-users] SNES seems not use my matrix-free operation</span><o:p></o:p></div></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">However, my GK_op (which is the reloaded MATOP_MULT) gives me some problem. It is entered but then crashed with Segmentation Violation error. So I guess my indices may be wrong. I wonder do I need to use the local Vec (of dF), and should my output Vec also be in the correct shape (i.e. after calculation I need to transform back into a Vec)? As you can see here, my dF is a tensor defined on every grid point. </span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></blockquote><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> The input for the matrix-vector product is a global vector, as is the result. (Not like the arguments to DMSNESSetJacobianLocal).<o:p></o:p></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> This means that your MATOP_MULT function needs to do the DMGlobalToLocal() vector operation first then the "unwrapping" from the vector to the array format at the beginning of the routine. Similarly it needs to "unwrap" the result vector as an array. See src/snes/tutorials/ex14f.F90 and in particular the code block<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> PetscCall(DMGlobalToLocalBegin(da,X,INSERT_VALUES,localX,ierr))<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> PetscCall(DMGlobalToLocalEnd(da,X,INSERT_VALUES,localX,ierr))<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">! Get pointers to vector data<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> PetscCall(VecGetArrayReadF90(localX,xx,ierr))<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> PetscCall(VecGetArrayF90(F,ff,ierr))<o:p></o:p></div></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> Barry<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">You really shouldn't be using DMSNESSetJacobianLocal() for your code. Basically all the DMSNESSetJacobianLocal() gives you is that it automatically handles the global to local mapping and unwrapping of the vector to an array, but it doesn't work for shell matrices.<o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><br><br><br><o:p></o:p></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">On Jan 9, 2024, at 6:30 AM, Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;"><span style="color: purple;">y.hu@mpie.de</span></a>> wrote:<o:p></o:p></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Dear Barry,</span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Thanks for your help.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">It works when doing first SNESSetJacobian() with my created shell matrix Jac in the main (or module) and then DMSNESSetJacobianLocal() to associate with my DM and an dummy formJacobian callback (which is doing nothing). My SNES can now recognize my shell matrix and do my customized operation.</span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">However, my GK_op (which is the reloaded MATOP_MULT) gives me some problem. It is entered but then crashed with Segmentation Violation error. So I guess my indices may be wrong. I wonder do I need to use the local Vec (of dF), and should my output Vec also be in the correct shape (i.e. after calculation I need to transform back into a Vec)? As you can see here, my dF is a tensor defined on every grid point.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Best wishes,</span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Yi</span><o:p></o:p></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div style="border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-color: currentcolor; border-image: none;"><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">Barry Smith <<a href="mailto:bsmith@petsc.dev" style="color: purple; text-decoration: underline;"><span style="color: purple;">bsmith@petsc.dev</span></a>><span class="apple-converted-space"> </span><br><b>Sent:</b><span class="apple-converted-space"> </span>Monday, January 8, 2024 6:41 PM<br><b>To:</b><span class="apple-converted-space"> </span>Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;"><span style="color: purple;">y.hu@mpie.de</span></a>><br><b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" style="color: purple; text-decoration: underline;"><span style="color: purple;">petsc-users@mcs.anl.gov</span></a><br><b>Subject:</b><span class="apple-converted-space"> </span>Re: [petsc-users] SNES seems not use my matrix-free operation</span><o:p></o:p></div></div></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> "formJacobian" should not be __creating__ the matrices. Here "form" means computing the numerical values in the matrix (or when using a shell matrix it means keeping a copy of X so that your custom matrix-free multiply knows the base location where the matrix free Jacobian-vector products are computed.)<o:p></o:p></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> You create the shell matrices up in your main program and pass them in with SNESSetJacobian(). <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> Try first calling SNESSetJacobian() to provide the matrices (provide a dummy function argument) and then call DMSNESSetJacobianLocal() to provide your "formjacobian" function (that does not create the matrices).<o:p></o:p></div></div></div></div><div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> Barry<o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> Yes, "form" is a bad word that should not have been used in our code.<o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><br><br><br><br><o:p></o:p></div></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">On Jan 8, 2024, at 12:24 PM, Yi Hu <<a href="mailto:y.hu@mpie.de" style="color: purple; text-decoration: underline;"><span style="color: purple;">y.hu@mpie.de</span></a>> wrote:<o:p></o:p></div></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Dear PETSc Experts,</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I am implementing a matrix-free jacobian for my SNES solver in Fortran. (command line option -snes_type newtonls -ksp_type gmres)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">In the main program, I define my residual and jacobian and matrix-free jacobian like the following,</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">…</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">call DMDASNESSetFunctionLocal(DM_mech, INSERT_VALUES, formResidual, PETSC_NULL_SNES, err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">call DMSNESSetJacobianLocal(DM_mech, formJacobian, PETSC_NULL_SNES, err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">…</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">subroutine formJacobian(residual_subdomain,F,Jac_pre,Jac,dummy,err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">#include <petsc/finclude/petscmat.h></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> use petscmat</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> implicit None</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> DMDALocalInfo, dimension(DMDA_LOCAL_INFO_SIZE) :: &</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> residual_subdomain !< DMDA info (needs to be named "in" for macros like XRANGE to work)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> real(pREAL), dimension(3,3,cells(1),cells(2),cells3), intent(in) :: &</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> F !< deformation gradient field</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> Mat :: Jac, Jac_pre</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscObject :: dummy</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscErrorCode :: err_PETSc</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> PetscInt :: N_dof ! global number of DoF, maybe only a placeholder</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> N_dof = 9*product(cells(1:2))*cells3<span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, 'in my jac'</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,N_dof,N_dof,0,Jac,err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatShellSetOperation(Jac,MATOP_MULT,GK_op,err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> <span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, 'in my jac'</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> ! for jac preconditioner</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,N_dof,N_dof,0,Jac_pre,err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> call MatShellSetOperation(Jac_pre,MATOP_MULT,GK_op,err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> CHKERRQ(err_PETSc)</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> print*, 'in my jac'</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10pt; font-family: "Courier New";">end subroutine formJacobian</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="hljs-keyword">subroutine</span><span class="line"> GK_op(Jac,dF,output,err_PETSc)</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="hljs-keyword">real</span><span class="line">(pREAL), </span><span class="hljs-keyword">dimension</span><span class="line">(</span><span class="hljs-number">3</span><span class="line">,</span><span class="hljs-number">3</span><span class="line">,cells(</span><span class="hljs-number">1</span><span class="line">),cells(</span><span class="hljs-number">2</span><span class="line">),cells3), </span><span class="hljs-keyword">intent</span><span class="line">(</span><span class="hljs-keyword">in</span><span class="line">) :: &</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> dF </span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="hljs-keyword">real</span><span class="line">(pREAL), </span><span class="hljs-keyword">dimension</span><span class="line">(</span><span class="hljs-number">3</span><span class="line">,</span><span class="hljs-number">3</span><span class="line">,cells(</span><span class="hljs-number">1</span><span class="line">),cells(</span><span class="hljs-number">2</span><span class="line">),cells3), </span><span class="hljs-keyword">intent</span><span class="line">(</span><span class="hljs-keyword">out</span><span class="line">) :: &</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> output </span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="hljs-keyword">real</span><span class="line">(pREAL), </span><span class="hljs-keyword">dimension</span><span class="line">(</span><span class="hljs-number">3</span><span class="line">,</span><span class="hljs-number">3</span><span class="line">) :: &</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> deltaF_aim = </span><span class="hljs-number">0.0_pREAL</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> Mat :: Jac</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> PetscErrorCode :: err_PETSc</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="hljs-keyword">integer</span><span class="line"> :: i, j, k, e</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="line"><span lang="EN-US">… a lot of calculations …</span></span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"><span lang="EN-US"> </span></span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><span class="hljs-builtin">print</span><span class="line">*, </span><span class="hljs-string">'in GK op'</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="line"> </span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"><span class="hljs-keyword">end</span><span class="line"> </span><span class="hljs-keyword">subroutine</span><span class="line"> GK_op</span><o:p></o:p></pre><pre style="margin: 0in 0in 0.0001pt; font-size: 10pt; font-family: "Courier New";"> <o:p></o:p></pre><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">The first question is that: it seems I still need to explicitly define the interface of MatCreateShell() and MatShellSetOperation() to properly use them, even though I include them via “use petscmat”. It is a little bit strange to me, since some examples do not perform this step.<span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Then the main issue is that I can build my own Jacobian from my call back function formJacobian, and confirm my Jacobian is a shell matrix (by MatView). However, my customized operator GK_op is not called when solving the nonlinear system (not print my “in GK op”). When I try to monitor my SNES, it gives me some conventional output not mentioning my matrix-free operations. So I guess my customized MATOP_MULT may be not associated with Jacobian. Or my configuration is somehow wrong. Could you help me solve this issue?<span class="apple-converted-space"> </span></span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Thanks,</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Yi</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><o:p></o:p></div></div></div></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br><br><br><br></span><o:p></o:p></div></div></div><div class="MsoNormal" align="center" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-align: center;"><hr size="2" width="100%" align="center"></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;">-------------------------------------------------<br>Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br><br>Max-Planck-Institut für Eisenforschung GmbH<br>Max-Planck-Straße 1<br>D-40237 Düsseldorf<br> <br>Handelsregister B 2533 <br>Amtsgericht Düsseldorf<br> <br>Geschäftsführung<br>Prof. Dr. Gerhard Dehm<br>Prof. Dr. Jörg Neugebauer<br>Prof. Dr. Dierk Raabe<br>Dr. Kai de Weldige<br> <br>Ust.-Id.-Nr.: DE 11 93 58 514 <br>Steuernummer: 105 5891 1000<br><br><br>Please consider that invitations and e-mails of our institute are <br>only valid if they end with …@mpie.de. <br>If you are not sure of the validity please contact </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br><br>Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <br>In Zweifelsfällen wenden Sie sich bitte an </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br>-------------------------------------------------</span><o:p></o:p></div></div></div></div></blockquote></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br><br><br></span><o:p></o:p></div></div><div class="MsoNormal" align="center" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-align: center;"><hr size="2" width="100%" align="center"></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;">-------------------------------------------------<br>Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br><br>Max-Planck-Institut für Eisenforschung GmbH<br>Max-Planck-Straße 1<br>D-40237 Düsseldorf<br> <br>Handelsregister B 2533 <br>Amtsgericht Düsseldorf<br> <br>Geschäftsführung<br>Prof. Dr. Gerhard Dehm<br>Prof. Dr. Jörg Neugebauer<br>Prof. Dr. Dierk Raabe<br>Dr. Kai de Weldige<br> <br>Ust.-Id.-Nr.: DE 11 93 58 514 <br>Steuernummer: 105 5891 1000<br><br><br>Please consider that invitations and e-mails of our institute are <br>only valid if they end with …@mpie.de. <br>If you are not sure of the validity please contact </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: purple;">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br><br>Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <br>In Zweifelsfällen wenden Sie sich bitte an </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: purple;">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br>-------------------------------------------------</span><o:p></o:p></div></div></div></blockquote></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; text-align: start; -webkit-text-stroke-width: 0px; word-spacing: 0px;"><br></span><o:p></o:p></div><div class="MsoNormal" align="center" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-align: center;"><hr size="2" width="100%" align="center"></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;">-------------------------------------------------<br>Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br><br>Max-Planck-Institut für Eisenforschung GmbH<br>Max-Planck-Straße 1<br>D-40237 Düsseldorf<br> <br>Handelsregister B 2533 <br>Amtsgericht Düsseldorf<br> <br>Geschäftsführung<br>Prof. Dr. Gerhard Dehm<br>Prof. Dr. Jörg Neugebauer<br>Prof. Dr. Dierk Raabe<br>Dr. Kai de Weldige<br> <br>Ust.-Id.-Nr.: DE 11 93 58 514 <br>Steuernummer: 105 5891 1000<br><br><br>Please consider that invitations and e-mails of our institute are <br>only valid if they end with …@mpie.de. <br>If you are not sure of the validity please contact </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: purple;">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br><br>Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <br>In Zweifelsfällen wenden Sie sich bitte an </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline;"><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: purple;">rco@mpie.de</span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;"><br>-------------------------------------------------</span><o:p></o:p></div></div></blockquote></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div></div><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><hr style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">-------------------------------------------------</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Stay up to date and follow us on LinkedIn, Twitter and YouTube.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Max-Planck-Institut für Eisenforschung GmbH</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Max-Planck-Straße 1</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">D-40237 Düsseldorf</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"> </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Handelsregister B 2533 </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Amtsgericht Düsseldorf</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"> </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Geschäftsführung</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Prof. Dr. Gerhard Dehm</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Prof. Dr. Jörg Neugebauer</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Prof. Dr. Dierk Raabe</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Dr. Kai de Weldige</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"> </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Ust.-Id.-Nr.: DE 11 93 58 514 </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Steuernummer: 105 5891 1000</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Please consider that invitations and e-mails of our institute are </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">only valid if they end with …@mpie.de. </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">If you are not sure of the validity please contact </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">rco@mpie.de</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">aus unserem Haus nur mit der Endung …@mpie.de gültig sind. </span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">In Zweifelsfällen wenden Sie sich bitte an </span><a href="mailto:rco@mpie.de" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">rco@mpie.de</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">-------------------------------------------------</span></div></blockquote></div><br></div></body></html>