<div dir="ltr"><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)">Thank you so much for your heart warming in time reply, and I just need to ask you a few more questions: </div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)"><br></div><div class="gmail_default" style><font color="#38761d" style="font-family:'comic sans ms',sans-serif">1, Correct me if I'm wrong, I am kinda new in this field. So, I can only print the matrix or vectors entirely, to know the structure stored in each processor I could only cal MatGetLocalSize or MatGetOwnershipRange to get general ideas (Are there more of them? what is </font><font color="#38761d" face="comic sans ms, sans-serif">PetscViewerASCIISynchronizedPrintf used for?</font><font color="#38761d" style="font-family:'comic sans ms',sans-serif">). Communicator </font><span style="font-family:arial,sans-serif;color:rgb(80,0,80);font-size:12.8000001907349px">PETSC_COMM_SELE</span><span style="font-family:arial,sans-serif;font-size:12.8000001907349px"><font color="#38761d"> is only useful when there is only one process going on. </font></span></div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif"><span style="font-size:12.8000001907349px;font-family:arial,sans-serif"><font color="#38761d"> </font></span></div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)">2, Our project is based on another group's project, and they are physists.... So I am trying to get hold of when every processor communicates what with each other. The question is not that complicated, they first initialize the matrix and the vectors, set some operators and values, assembly them and now solve Ax=b using FGMRES. From this point I just want to know how the processors divide the matrix A, because I looked into KSPsolve(), there doesn't seem to be any communication right? (Maybe I wasn't paying enough attention). So, could you give me some hints how to achieve this? to know how they communicate? I didn't find much documentation about this.</div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)"><br></div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)">3, related to question 2....how the matrix was generated generally, for example, all the processors genereate one matrix together? or each of them generate a whole matrix deparately by itself? Every one processor holds one copy or there's only one copy in Rank0?</div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)"><br></div><div class="gmail_default" style="font-family:'comic sans ms',sans-serif;color:rgb(56,118,29)">Thank you, you are the best!</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><font face="comic sans ms, sans-serif" color="#0b5394">Longyin Cui (or you know me as Eric);</font><div><font face="comic sans ms, sans-serif" color="#0b5394">Student from C.S. division;</font></div><div><font face="comic sans ms, sans-serif" color="#0b5394">Cell: 7407047169;</font></div><div><font face="comic sans ms, sans-serif" color="#0b5394">return 0;</font></div><div><font color="#00ffff"><br></font></div></div></div></div></div></div>
<br><div class="gmail_quote">On Sat, Jun 20, 2015 at 1:38 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Eric,<br>
<span class=""><br>
<br>
> On Jun 20, 2015, at 1:42 AM, Longyin Cui <<a href="mailto:cuilongyin@gmail.com">cuilongyin@gmail.com</a>> wrote:<br>
><br>
> OMG you are real!!!<br>
> OK, All my error message looks like this:<br>
> PETSc Error ... exiting<br>
> --------------------------------------------------------------------------<br>
> mpirun has exited due to process rank 13 with PID 1816 on<br>
> node cnode174.local exiting improperly. There are two reasons this could occur:<br>
><br>
> 1. this process did not call "init" before exiting, but others in<br>
> the job did. This can cause a job to hang indefinitely while it waits<br>
> for all processes to call "init". By rule, if one process calls "init",<br>
> then ALL processes must call "init" prior to termination.<br>
><br>
> 2. this process called "init", but exited without calling "finalize".<br>
> By rule, all processes that call "init" MUST call "finalize" prior to<br>
> exiting or it will be considered an "abnormal termination"<br>
><br>
> This may have caused other processes in the application to be<br>
> terminated by signals sent by mpirun (as reported here).<br>
<br>
</span> This crash doesn't seem to have anything to do in particular with code below. Do the PETSc examples run in parallel? Does your code that you ran have a PetscInitialize() in it? What about running on two processors, does that work?<br>
<span class=""><br>
><br>
> You are right, I did use PETSC_COMM_SELE, and when I used PETSC_COMM_WORLD alone I could get the entire matrix printed. But, this is one whole matrix in one file. The reason I used: PetscViewerASCIIOpen ( PETSC_COMM_SELF, "mat.output", &viewer); and MatView (matrix,viewer); was because it says "Each processor can instead write its own independent output by specifying the communicator PETSC_COMM_SELF".<br>
<br>
</span> Yikes, this is completely untrue and has been for decades. We have no way of saving the matrix in its parts; you cannot use use a PETSC_COMM_SELF viewer with a parallel matrix. Sorry about the wrong information in the documentation; I have fixed it.<br>
<br>
Why can't you just save the matrix in one file and then compare it? We don't provide a way to save objects one part per process because we think it is a bad model for parallel computing since the result depends on the number of processors you are using.<br>
<span class="HOEnZb"><font color="#888888"><br>
Barry<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> Also, I tried this as well, which failed, same error message :<br>
> PetscMPIInt my_rank;<br>
> MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);<br>
> string str = KILLME(my_rank); // KILLME is a int to string function...<br>
> const char * c = str.c_str();<br>
> PetscViewer viewer;<br>
> PetscViewerASCIIOpen(PETSC_COMM_WORLD, c , &viewer);<br>
> MatView(impOP,viewer); //impOP is the huge matrix.<br>
> PetscViewerDestroy(&viewer);<br>
><br>
> I was trying to generate 16 files recording each matrix hold by each processor so I can conpare them with the big matrix...so, what do you think<br>
><br>
> Thank you very muuuch.<br>
><br>
> Longyin Cui (or you know me as Eric);<br>
> Student from C.S. division;<br>
> Cell: <a href="tel:7407047169" value="+17407047169">7407047169</a>;<br>
> return 0;<br>
><br>
><br>
> On Sat, Jun 20, 2015 at 1:34 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> You need to cut and paste and send the entire error message: "not working" makes it very difficult for us to know what has gone wrong.<br>
> Based on the code fragment you sent I guess one of your problems is that the viewer communicator is not the same as the matrix communicator. Since the matrix is on 16 processors (I am guessing PETSC_COMM_WORLD) the viewer communicator must also be the same (also PETSC_COMM_WORLD).<br>
> The simplest code you can do is<br>
><br>
> > PetscViewerASCIIOpen(PETSC_COMM_WORLD,"stdout",&viewer);<br>
> > MatView(impOP,viewer);<br>
><br>
> but you can get a similar effect with the command line option -mat_view and not write any code at all (the less code you have to write the better).<br>
><br>
> Barry<br>
><br>
><br>
> > On Jun 19, 2015, at 10:42 PM, Longyin Cui <<a href="mailto:cuilongyin@gmail.com">cuilongyin@gmail.com</a>> wrote:<br>
> ><br>
> > Hi dear whoever reads this:<br>
> ><br>
> > I have a quick question:<br>
> > After matrix assembly, suppouse I have matrix A. Assuming I used 16 processors, if I want each processor to print out their local contents of the A, how do I proceed? (I simply want to know how the matrix is stored from generating to communicating to solving, so I get to display all the time to get better undersdtanding)<br>
> ><br>
> > I read the examples, and I tried things like below and sooooo many different codes from examples, it still is not working.<br>
> > PetscViewer viewer;<br>
> > PetscMPIInt my_rank;<br>
> > MPI_Comm_rank(PETSC_COMM_WORLD,&my_rank);<br>
> > PetscPrintf(MPI_COMM_SELF,"[%d] rank\n",my_rank);<br>
> > PetscViewerASCIIOpen(MPI_COMM_SELF,NULL,&viewer);<br>
> > PetscViewerPushFormat(viewer,PETSC_VIEWER_ASCII_INFO);<br>
> > MatView(impOP,viewer);<br>
> ><br>
> > Plea......se give me some hints<br>
> ><br>
> > Thank you so very much!<br>
> ><br>
> ><br>
> > Longyin Cui (or you know me as Eric);<br>
> > Student from C.S. division;<br>
> > Cell: <a href="tel:7407047169" value="+17407047169">7407047169</a>;<br>
> > return 0;<br>
> ><br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>