<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> MPI_Comm_size(<span style="font-family: monospace;" class="">PETSC_COMM_WORLD,&size);</span><div class=""><font face="monospace" class=""> MPI_Comm_rank(</font><span style="font-family: monospace;" class="">PETSC_COMM_WORLD,&rank);</span></div><div class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><div class=""><div class=""><font face="monospace" class="">SNESNASMGetSNES(snes,0,&subsnes);<br class=""> char prefix[10];<br class=""> sprintf(prefix,"sub_%d_",rank);<br class=""> SNESSetOptionsPrefix(subsnes,prefix);</font></div></div></div></blockquote></div></div></blockquote><div class=""><br class=""></div><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 17, 2022, at 9:35 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">On Fri, Jun 17, 2022 at 9:22 AM Takahashi, Tadanaga <<a href="mailto:tt73@njit.edu" class="">tt73@njit.edu</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="">I'm having some trouble pulling out the subsolver. I tried to use SNESNASMGetSNES in a loop over each subdomain. However I get an error when I run the code with more than one MPI processors. Here is a snippet from my code: </div><div class=""><br class=""><font face="monospace" class=""> SNES snes, subsnes;<br class=""> PetscMPIInt rank, size;<br class=""></font></div><div class=""><font face="monospace" class=""> ... </font></div><div class=""><div class=""><font face="monospace" class=""> ierr = SNESCreate(PETSC_COMM_WORLD,&snes); CHKERRQ(ierr);<br class=""> ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);<br class=""> ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);<br class=""> ... <br class=""> ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);<br class=""> ierr = SNESSetUp(snes); CHKERRQ(ierr);<br class=""> PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);<br class=""> for (i=0; i<size; i++) {<br class=""> PetscPrintf(PETSC_COMM_WORLD, "rank = %d\n",i);<br class=""> SNESNASMGetSNES(snes,i,&subsnes);<br class=""> // char prefix[10];<br class=""> // sprintf(prefix,"sub_%d_",i);<br class=""> // SNESSetOptionsPrefix(subsnes,prefix);<br class=""> }</font><br class=""></div></div><div class=""><font face="monospace" class=""> ... </font></div><div class=""><font face="monospace" class=""> ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);<br class=""></font></div><div class=""><font face="monospace" class=""><br class=""></font></div><div class=""><br class=""></div><div class="">And, here is the output of the code when I run with 2 MPI procs: </div></div></blockquote><div class=""><br class=""></div><div class=""><span style="font-family:monospace" class="">SNESNASMGetSNES() gets the local subsolvers. It seems you only have one per process.</span><br class=""></div><div class=""><span style="font-family:monospace" class="">You can check </span><a href="https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/" class="">https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/</a></div><div class=""><br class=""></div><div class="">Notice that your current code will not work because, according to your explanation, you only want to change</div><div class="">the prefix on a single rank, so you need to check the rank when you do it.</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class=""><font face="monospace" class="">takahashi@ubuntu:~/Desktop/MA-DDM/C/Rectangle$ mpiexec -n 2 ./test1<br class="">Size = 2<br class="">rank = 0<br class="">rank = 1<br class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br class="">[0]PETSC ERROR: Argument out of range<br class="">[0]PETSC ERROR: No such subsolver<br class="">[0]PETSC ERROR: See <a href="https://petsc.org/release/faq/" target="_blank" class="">https://petsc.org/release/faq/</a> for trouble shooting.<br class="">[0]PETSC ERROR: Petsc Release Version 3.17.1, unknown <br class="">[0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi Fri Jun 17 06:06:38 2022<br class="">[0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0<br class="">[0]PETSC ERROR: #1 SNESNASMGetSNES() at /home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923<br class=""><br class="">===================================================================================<br class="">= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<br class="">= RANK 0 PID 976566 RUNNING AT ubuntu<br class="">= KILLED BY SIGNAL: 9 (Killed)<br class="">===================================================================================</font><br class=""></div><div class=""><br class=""></div><div class="">This error doesn't occur when I run this without MPI. However, I tried to change the prefix of the subdomain to `sub_0_` but I am not able to change the snes_type using this prefix. Running<font face="monospace" class=""> ./test1 -snes_view -help | grep sub_0_snes_type </font><font face="arial, sans-serif" class="">prints nothing. </font></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class="">On Thu, Jun 16, 2022 at 5:57 PM tt73 <<a href="mailto:tt73@njit.edu" target="_blank" class="">tt73@njit.edu</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class=""><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Hi, </div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">I am using NASM as the outer solver for a nonlinear problem. For one of the subdomains, I want to run the local solve with a different set of options form the others. Is there any way to set options for each subdomain? </div></div></blockquote></div><br clear="all" class=""><div class="">I can see two ways:</div><div class=""><br class=""></div><div class=""> 1) Pull out the subsolver and set it using the API</div><div class=""><br class=""></div><div class=""> 2) Pull out the subsolver and give it a different prefix</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all" class=""><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></div>
</div></blockquote></div><br class=""></div></body></html>