From gcheng at steele.mgh.harvard.edu Fri Feb 3 16:29:26 2006 From: gcheng at steele.mgh.harvard.edu (Gang Cheng) Date: Fri, 3 Feb 2006 17:29:26 -0500 (EST) Subject: Questions about petsc matrix and KSP Message-ID: <3676.132.183.110.197.1139005766.squirrel@steele.mgh.harvard.edu> Hi, guys, I have some questions about the usage of petsc matrix and KSP. The parallel Fortran code I'm working on simulates a biological process in which a diffusion-reaction PDE needs to be solved at each time step in a loop for many steps. Moreover, the coefficient matrix (H) for the discretized (using finite difference) PDE changes as the system evolves. So it is necessary to update H at each time step. My current algorithm is: Before the loop --------------- 1A. Create the parallel matrix H using MatCreateMPIAIJ and save it in a common block. 2A. According to the initial condition of the domain, fill non-zero values in H row by row using MatSetValues. H is then assembed by calling MatAssemblyBegin and MatAssemblyEnd. At each time step within the loop --------------------------------- 1B. Each processor scans local domain and ONLY updates (using MatSetValues) the rows in H that need to be updated (e.g., a certain grid has changed and a new boundary condition needs to be applied to its interface with other grids). MatAssemblyBegin and MatAssemblyEnd are called even if no update is done in the local processor to avoid deadlock. 2B. Create a KSP from H, calculate right-hand-side vector and solve the linear system using KSPSolve. 3B. Destroy the KSP. Here is the problem I'm having with the above algorithm. In step 1B, I didn't update the entire H with the intention to save some CPU time (I found that the MatSetValues operiaiton is very slow for parallel matrices). However, I found that the updated values were not inserted in the supposed postion in H, causing errors in the solution of the PDE (I'm certain that the input parameters for MatSetValues were right). If I instead update the entire H while keeping everything else the same, the PDE solution is correct. So my specific question is, if a matrix needs to be updated, do we need to update ALL its rows before assembling it? Can we update just some of them? Also, I'm sure some of you guys are using petsc for problems similar to mine. If you have any suggestions on how I may improve the efficiency of my algorithm, I'd greatly appreciate them. Sorry for this long email. -GC From bsmith at mcs.anl.gov Fri Feb 3 17:02:45 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 3 Feb 2006 17:02:45 -0600 (CST) Subject: Questions about petsc matrix and KSP In-Reply-To: <3676.132.183.110.197.1139005766.squirrel@steele.mgh.harvard.edu> References: <3676.132.183.110.197.1139005766.squirrel@steele.mgh.harvard.edu> Message-ID: On Fri, 3 Feb 2006, Gang Cheng wrote: > Hi, guys, > > I have some questions about the usage of petsc matrix and KSP. The > parallel Fortran code I'm working on simulates a biological process in > which a diffusion-reaction PDE needs to be solved at each time step in a > loop for many steps. Moreover, the coefficient matrix (H) for the > discretized (using finite difference) PDE changes as the system evolves. > So it is necessary to update H at each time step. My current algorithm is: > > Before the loop > --------------- > 1A. Create the parallel matrix H using MatCreateMPIAIJ and save it in a > common block. > 2A. According to the initial condition of the domain, fill non-zero values > in H row by row using MatSetValues. H is then assembed by calling > MatAssemblyBegin and MatAssemblyEnd. > > At each time step within the loop > --------------------------------- > 1B. Each processor scans local domain and ONLY updates (using > MatSetValues) the rows in H that need to be updated (e.g., a certain grid > has changed and a new boundary condition needs to be applied to its > interface with other grids). MatAssemblyBegin and MatAssemblyEnd are > called even if no update is done in the local processor to avoid deadlock. > 2B. Create a KSP from H, calculate right-hand-side vector and solve the > linear system using KSPSolve. > 3B. Destroy the KSP. You should not create a new KSP each time, reuse the old one and just call KSPSetOperators() again with the matrix. > > Here is the problem I'm having with the above algorithm. In step 1B, I > didn't update the entire H with the intention to save some CPU time (I > found that the MatSetValues operiaiton is very slow for parallel > matrices). However, I found that the updated values were not inserted in > the supposed postion in H, causing errors in the solution of the PDE (I'm > certain that the input parameters for MatSetValues were right). If I > instead update the entire H while keeping everything else the same, the > PDE solution is correct. So my specific question is, if a matrix needs to > be updated, do we need to update ALL its rows before assembling it? Can we > update just some of them? You are absolutely suppose to be able to just change some of the values (those that are not changed should stay where they are), but if you call MatZeroEntries() then obviously you need to reset everything. I would do a MatView() on the matrix (the second time through the loop) for both approaches (1) change only some values and (2) set all the values and try to see what is different about the two matrices. In theory they should be the same. > Also, I'm sure some of you guys are using petsc > for problems similar to mine. If you have any suggestions on how I may > improve the efficiency of my algorithm, I'd greatly appreciate them. After you have it working well you can try to lag the preconditioner. You can call KSPSetOperators() with the flag SAME_PRECONDITIONER for several iterations, this may speed things up since it will not have to spend the time rebuilding the preconditioner each step. As the matrix changes likely the preconditioner will work less and less well and so you have to call KSPSetOperators() without that flag to get it to build a new preconditioner. Of course, this approach need not always work; if the matrix spectrum is changing rapidly you will need to rebuild the preconditioner each time. But it is worth trying. Barry > > Sorry for this long email. > > -GC > > > > > > > From tomjan at jay.au.poznan.pl Sat Feb 4 08:28:25 2006 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Sat, 4 Feb 2006 15:28:25 +0100 (CET) Subject: how to implement parallel integer optimization with petsc? Message-ID: hello group because i'm not sure what kind (MILP or IP) of optimization it is I attached short example at the end of this email my real problem is to find integer solution for matrix which is about 100X3500 large. I think it is good idea to do this task in parallel mode. I found yesterday PETSC pakage. It is incredible (for me), but compilation and testing proces has passed smooooth on my cluster (as much as 2x1PC :-)). Because of such painless implementation it would be great if icould use PETSC for solving my problem. my question is: is it posible with PETSC? if yes could you count(and shortly describe) consecutive steps? maybe someone already have done it? thank you for any replyes and sugesstions. tom ps. short example suposse we have two kinds of elements type A nad B which has such features like price. A1 200$ A2 300$ A3 400$ B1 500$ B2 600$ B3 700$ B4 900$ we want to find set of pairs AxB which acomplish constrain for mean price. (when we make pair we sum prices of their elements) additionaly constrain is that we may use elements A only once and elements B only twice. so, our solution is binary matrix (1=make pair)like bellow B1 B2 B3 B4 B5 A1 1 0 0 0 0 A2 1 0 0 0 0 A3 0 1 0 0 0 ######################################################## # tomjan at jay.au.poznan.pl # # jay.au.poznan.pl/~tomjan/ # ######################################################## From knepley at mcs.anl.gov Sat Feb 4 11:23:07 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Sat, 04 Feb 2006 11:23:07 -0600 Subject: how to implement parallel integer optimization with petsc? In-Reply-To: (Tomasz Jankowski's message of "Sat, 4 Feb 2006 15:28:25 +0100 (CET)") References: Message-ID: Tomasz Jankowski writes: PETSc basically offers parallel linear algebra and nonlinear solvers. If you can phrase the algorithm in linear algebraic terms, then we can probably help. In addition, it appears that you have a logically Cartesian space, so the DA object could handle allocation, etc. Matt > hello group > > because i'm not sure what kind (MILP or IP) of optimization it is I attached short example at the end of this email > > my real problem is to find integer solution for matrix which is about 100X3500 large. I think it is good idea to do this > task in parallel mode. > I found yesterday PETSC pakage. It is incredible (for me), but compilation > and testing proces has passed smooooth on my cluster (as much as 2x1PC :-)). Because of such painless implementation it > would be great if icould use > PETSC for solving my problem. > > my question is: > > is it posible with PETSC? > if yes could you count(and shortly describe) consecutive steps? > maybe someone already have done it? > > thank you for any replyes and sugesstions. > > tom > > > ps. short example > suposse we have two kinds of elements type A nad B which has such features like price. > A1 200$ > A2 300$ > A3 400$ > B1 500$ > B2 600$ > B3 700$ > B4 900$ > > > we want to find set of pairs AxB which acomplish constrain for mean price. (when we make pair we sum prices of their > elements) > additionaly constrain is that we may use elements A only once and elements B only twice. > so, our solution is binary matrix (1=make pair)like bellow > B1 B2 B3 B4 B5 > A1 1 0 0 0 0 > A2 1 0 0 0 0 > A3 0 1 0 0 0 > > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From tomjan at jay.au.poznan.pl Sat Feb 4 13:06:54 2006 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Sat, 4 Feb 2006 20:06:54 +0100 (CET) Subject: how to implement parallel integer optimization with petsc? In-Reply-To: References: Message-ID: > PETSc basically offers parallel linear algebra and nonlinear solvers. > If you can phrase the algorithm in linear algebraic terms, then we can probably help. unfortunetly I can't (sorry I'm not mathematicst) but I can phrase my problem in AMPL language. please look at the two files which I have pasted at the end of this email. mate-selection.mod contain optimization model in AMPL. the second file mate-selection.dat contain the example data. Please forgive me my previous email with unclear description of my problem (its because of my english).once again I need to assert that I don't expect the full recipe for implementation of my problem with PETSc. I will realy appreciate if you give some sugesstion. Thank you for your reply tom (ps. if AMPL model is not good I provide my algorithm in algebraic terms) ---------mate-selection.mod---------------- set BULLS; set COWS; param ebv {COWS,BULLS} >= 0; param cost {BULLS} > 0; param f {COWS,BULLS} >= 0; param l_cows > 0; var Pairs {COWS,BULLS} binary; maximize Total_Ebv: sum {i in COWS, j in BULLS} ebv[i,j] * Pairs[i,j]; subject to Max_cows {i in COWS}: sum{j in BULLS} Pairs[i,j] = 1; subject to Max_bulls {j in BULLS}: sum{i in COWS} Pairs[i,j] <= 2; subject to F: (sum{i in COWS, j in BULLS} f[i,j] * Pairs[i,j])/l_cows <= 0.03; subject to Cost: sum{ i in COWS, j in BULLS} Pairs[i,j] * cost[j] <=500; --------------------------------------------- -----------------mate-selection.dat--------------------- data; set BULLS := male0 male1 male2 male3 male4; set COWS := female0 female1 female2; param ebv: male0 male1 male2 male3 male4:= female0 290 410 298 400 471 female1 901 807 956 653 173 female2 185 453 733 316 367 ; param f: male0 male1 male2 male3 male4:= female0 0.076 0.048 0.068 0.064 0.064 female1 0.078 0.057 0.035 0.018 0.014 female2 0.068 0.042 0.062 0.076 0.059 ; param cost:= male0 72 male1 544 male2 822 male3 825 male4 127 ; param l_cows := 3; end; -------------------------------------------- From knepley at mcs.anl.gov Sat Feb 4 13:27:02 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Sat, 04 Feb 2006 13:27:02 -0600 Subject: how to implement parallel integer optimization with petsc? In-Reply-To: (Tomasz Jankowski's message of "Sat, 4 Feb 2006 20:06:54 +0100 (CET)") References: Message-ID: Tomasz Jankowski writes: Unfortunately, AMPL does a lot of processing. It would be a big job to translate this to PETSc. Matt >> PETSc basically offers parallel linear algebra and nonlinear solvers. >> If you can phrase the algorithm in linear algebraic terms, then we can probably help. > > unfortunetly I can't (sorry I'm not mathematicst) but I can phrase my problem in AMPL language. please look at the two > files which I have pasted at the end of this email. mate-selection.mod contain optimization model in AMPL. the second > file mate-selection.dat contain the example data. > > Please forgive me my previous email with unclear description of my problem (its because of my english).once again I need > to assert that I don't expect the full recipe for implementation of my problem with PETSc. > > I will realy appreciate if you give some sugesstion. > > Thank you for your reply > > tom > > (ps. if AMPL model is not good I provide my algorithm in algebraic terms) > > ---------mate-selection.mod---------------- > set BULLS; > set COWS; > > param ebv {COWS,BULLS} >= 0; > param cost {BULLS} > 0; > param f {COWS,BULLS} >= 0; > param l_cows > 0; > var Pairs {COWS,BULLS} binary; > > maximize Total_Ebv: > sum {i in COWS, j in BULLS} ebv[i,j] * Pairs[i,j]; > > subject to Max_cows {i in COWS}: sum{j in BULLS} Pairs[i,j] = 1; > subject to Max_bulls {j in BULLS}: sum{i in COWS} Pairs[i,j] <= 2; > subject to F: (sum{i in COWS, j in BULLS} f[i,j] * Pairs[i,j])/l_cows <= 0.03; > subject to Cost: sum{ i in COWS, j in BULLS} Pairs[i,j] * cost[j] <=500; > --------------------------------------------- > > -----------------mate-selection.dat--------------------- > > data; > > set BULLS := male0 male1 male2 male3 male4; > set COWS := female0 female1 female2; > param ebv: male0 male1 male2 male3 male4:= > female0 290 410 298 400 471 > female1 901 807 956 653 173 > female2 185 453 733 316 367 > ; > param f: male0 male1 male2 male3 male4:= > female0 0.076 0.048 0.068 0.064 0.064 > female1 0.078 0.057 0.035 0.018 0.014 > female2 0.068 0.042 0.062 0.076 0.059 > ; > param cost:= > male0 72 > male1 544 > male2 822 > male3 825 > male4 127 > ; > param l_cows := 3; > end; > > -------------------------------------------- > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From r.gori at cineca.it Mon Feb 6 05:41:34 2006 From: r.gori at cineca.it (Roberto Gori) Date: Mon, 6 Feb 2006 12:41:34 +0100 (MET) Subject: mat creation Message-ID: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> Hi, guys, I'm trying to solve a linear system with M equations and N unknowns (M >> N) using the LSQR method. I have a C parallel sparse matrix MxN with NZ nonzero elements stored in this way: double values[NZ]; // the nonzero values int idxn[NZ]; // the column indices int nz[M]; // the number of nonzero values for each row what is the best way to create a PETSc matrix? Do exists a statement like VecCreateMPIWithArray for already distributed vectors that could do all the work in one shot without copy? Just another detail. Initially M is unknown. All my structures are allocated dynamically because the number of rows M (and the number of nonzero NZ also) is determined just before to solve the system, so I think it's no feasible for me to call MatCreateMPIAIJ at the begin of my code and to perform the MatSetValues loop. Roberto ================================================================ | Roberto Gori CINECA | High Performance System Group Via Magnanelli 6/3 40033 | e-mail: r.gori at cineca.it Casalecchio di Reno | Tel: 051/6171522 Bologna - ITALY | Fax: 051/6132198 http://www.cineca.it ================================================================ From bsmith at mcs.anl.gov Mon Feb 6 09:15:15 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 6 Feb 2006 09:15:15 -0600 (CST) Subject: mat creation In-Reply-To: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> References: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> Message-ID: On Mon, 6 Feb 2006, Roberto Gori wrote: > Hi, guys, > > I'm trying to solve a linear system with M equations and N unknowns (M >> N) using the LSQR method. > > I have a C parallel sparse matrix MxN with NZ nonzero elements stored in this way: > > double values[NZ]; // the nonzero values > int idxn[NZ]; // the column indices > int nz[M]; // the number of nonzero values for each row This needs to be instead indices to start of each row in idxn and values instead of the length of each row. > > what is the best way to create a PETSc matrix? MatMPIAIJSetPreallocationCSR() > > Do exists a statement like VecCreateMPIWithArray for already distributed vectors that could do all the work in one shot without copy? Yes > Just another detail. Initially M is unknown. All my structures are > allocated dynamically because the number of rows M (and the number > of nonzero NZ also) is determined just before to solve the system, > so I think it's no feasible for me to call MatCreateMPIAIJ at the > begin of my code and to perform the MatSetValues loop. There is nothing saying that you must "call MatCreateMPIAIJ at the beginning of your code", you can call it anytime that you have the needed information. Barry > > Roberto > > ================================================================ > | Roberto Gori CINECA > | High Performance System Group Via Magnanelli 6/3 40033 > | e-mail: r.gori at cineca.it Casalecchio di Reno > | Tel: 051/6171522 Bologna - ITALY > | Fax: 051/6132198 http://www.cineca.it > ================================================================ > > From r.gori at cineca.it Mon Feb 6 12:44:56 2006 From: r.gori at cineca.it (Roberto Gori) Date: Mon, 6 Feb 2006 19:44:56 +0100 (MET) Subject: mat creation References: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> Message-ID: <022b01c62b4d$6cc42d70$3907ba82@int.cineca.it> Many thanks Barry, MatMPIAIJSetPreallocationCSR could be a good solution for me. It seems that to satisfy the MatMPIAIJSetPreallocationCSR requirements I should convert the nz array from a relative to an absolute representation of the row offsets but ... I think this implies a not negligible drawback: my matrix is too large and offsets must be promoted to a long long int (64 bit) type. Moreover the idxn array type also must be promoted to be compliant with a 64 bit PetscInt type. This requires the double amount of memory for nz and idxn. What do you think about the opportunity to provide another version of MatMPIAIJSetPreallocationCSR based on relative offsets to save memory? Absolute offsets could be managed internally ... Finally I've look into the body of MatMPIAIJSetPreallocationCSR_MPIAIJ function ad I found this snippet of code for (i=0; i Hi, guys, > > I'm trying to solve a linear system with M equations and N unknowns (M >> N) using the LSQR method. > > I have a C parallel sparse matrix MxN with NZ nonzero elements stored in this way: > > double values[NZ]; // the nonzero values > int idxn[NZ]; // the column indices > int nz[M]; // the number of nonzero values for each row This needs to be instead indices to start of each row in idxn and values instead of the length of each row. > > what is the best way to create a PETSc matrix? MatMPIAIJSetPreallocationCSR() > > Do exists a statement like VecCreateMPIWithArray for already distributed vectors that could do all the work in one shot without copy? Yes > Just another detail. Initially M is unknown. All my structures are > allocated dynamically because the number of rows M (and the number > of nonzero NZ also) is determined just before to solve the system, > so I think it's no feasible for me to call MatCreateMPIAIJ at the > begin of my code and to perform the MatSetValues loop. There is nothing saying that you must "call MatCreateMPIAIJ at the beginning of your code", you can call it anytime that you have the needed information. Barry > > Roberto > > ================================================================ > | Roberto Gori CINECA > | High Performance System Group Via Magnanelli 6/3 40033 > | e-mail: r.gori at cineca.it Casalecchio di Reno > | Tel: 051/6171522 Bologna - ITALY > | Fax: 051/6132198 http://www.cineca.it > ================================================================ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Feb 6 19:45:58 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 6 Feb 2006 19:45:58 -0600 (CST) Subject: mat creation In-Reply-To: <022b01c62b4d$6cc42d70$3907ba82@int.cineca.it> References: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> <022b01c62b4d$6cc42d70$3907ba82@int.cineca.it> Message-ID: On Mon, 6 Feb 2006, Roberto Gori wrote: > Many thanks Barry, > > MatMPIAIJSetPreallocationCSR could be a good solution for me. > It seems that to satisfy the MatMPIAIJSetPreallocationCSR requirements I should convert the nz array from a relative to an absolute representation of the row offsets but ... > I think this implies a not negligible drawback: my matrix is too large and offsets must be promoted to a long long int (64 bit) type. > Moreover the idxn array type also must be promoted to be compliant with a 64 bit PetscInt type. This requires the double amount of memory for nz and idxn. > > What do you think about the opportunity to provide another version of MatMPIAIJSetPreallocationCSR based on relative offsets to save memory? > Absolute offsets could be managed internally ... Your point is valid and noted but this would require writing another whole set of the AIJ routines that use the relative offsets. Maybe not so hard but a lot of code duplication. > > Finally I've look into the body of MatMPIAIJSetPreallocationCSR_MPIAIJ function ad I found this snippet of code > > for (i=0; i ii = i + rstart; > nnz = I[i+1]- I[i]; > ierr = MatSetValues_MPIAIJ(B,1,&ii,nnz,J+I[i],values+(v ? I[i] : 0),INSERT_VALUES);CHKERRQ(ierr); > } > > There's no way to avoid the copy of the values? This doubles one more time the memory for the values array and spend time into the loop. > Morevover nnz corresponds exactly to my nz[i] ... The thing is PETSc does not just store the MPIAIJ matrices in a simple CSR format (the matrix is actually stored in two parts, see the MATMPIAIJ manual page). So the copy is needed. What we do. Is first count the number of nonzeros per row then preallocate the memory then compute the values in the matrix, see MatMPIAIJSetPreallocatio(). This may seem expensive but really is not, the counting is fast. You build the CSR matrix for your sparse matrix; using our approach is no harder and prevents the need for a copy. (In fact the reason PETSc has the Preallocation() and MatSetValues() stuff is so that users do not have to construct the CSR format directly. Barry > > ----- Original Message ----- > From: Barry Smith > To: Roberto Gori > Cc: petsc-users at mcs.anl.gov > Sent: Monday, February 06, 2006 4:15 PM > Subject: Re: mat creation > > > > > On Mon, 6 Feb 2006, Roberto Gori wrote: > > > Hi, guys, > > > > I'm trying to solve a linear system with M equations and N unknowns (M >> N) using the LSQR method. > > > > I have a C parallel sparse matrix MxN with NZ nonzero elements stored in this way: > > > > double values[NZ]; // the nonzero values > > int idxn[NZ]; // the column indices > > int nz[M]; // the number of nonzero values for each row > > This needs to be instead indices to start of each row in idxn and values > instead of the length of each row. > > > > what is the best way to create a PETSc matrix? > > MatMPIAIJSetPreallocationCSR() > > > > > > Do exists a statement like VecCreateMPIWithArray for already distributed vectors that could do all the work in one shot without copy? > > Yes > > > Just another detail. Initially M is unknown. All my structures are > > allocated dynamically because the number of rows M (and the number > > of nonzero NZ also) is determined just before to solve the system, > > so I think it's no feasible for me to call MatCreateMPIAIJ at the > > begin of my code and to perform the MatSetValues loop. > > There is nothing saying that you must "call MatCreateMPIAIJ at the > beginning of your code", you can call it anytime that you have the > needed information. > > Barry > > > > > Roberto > > > > ================================================================ > > | Roberto Gori CINECA > > | High Performance System Group Via Magnanelli 6/3 40033 > > | e-mail: r.gori at cineca.it Casalecchio di Reno > > | Tel: 051/6171522 Bologna - ITALY > > | Fax: 051/6132198 http://www.cineca.it > > ================================================================ > > > > > > From bsmith at mcs.anl.gov Mon Feb 6 20:50:58 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 6 Feb 2006 20:50:58 -0600 (CST) Subject: mat creation In-Reply-To: References: <00d901c62b12$47c5aa20$3907ba82@int.cineca.it> <022b01c62b4d$6cc42d70$3907ba82@int.cineca.it> Message-ID: On Mon, 6 Feb 2006, Barry Smith wrote: > > > On Mon, 6 Feb 2006, Roberto Gori wrote: > > > Many thanks Barry, > > > > MatMPIAIJSetPreallocationCSR could be a good solution for me. > > It seems that to satisfy the MatMPIAIJSetPreallocationCSR requirements I should convert the nz array from a relative to an absolute representation of the row offsets but ... > > I think this implies a not negligible drawback: my matrix is too large and offsets must be promoted to a long long int (64 bit) type. > > Moreover the idxn array type also must be promoted to be compliant with a 64 bit PetscInt type. This requires the double amount of memory for nz and idxn. > > > > What do you think about the opportunity to provide another version of MatMPIAIJSetPreallocationCSR based on relative offsets to save memory? > > Absolute offsets could be managed internally ... > These may be some confusing here; the idxn values are only relative to that process (i.e. idxn[0] == 0) this means there won't be a problem so long as you have less then about 2 billion nonzeros on a process (which would take > 28 gigabytes of memory). Barry > Your point is valid and noted but this would require writing another > whole set of the AIJ routines that use the relative offsets. Maybe not so > hard but a lot of code duplication. > > > > > Finally I've look into the body of MatMPIAIJSetPreallocationCSR_MPIAIJ function ad I found this snippet of code > > > > for (i=0; i > ii = i + rstart; > > nnz = I[i+1]- I[i]; > > ierr = MatSetValues_MPIAIJ(B,1,&ii,nnz,J+I[i],values+(v ? I[i] : 0),INSERT_VALUES);CHKERRQ(ierr); > > } > > > > There's no way to avoid the copy of the values? This doubles one more time the memory for the values array and spend time into the loop. > > Morevover nnz corresponds exactly to my nz[i] ... > > The thing is PETSc does not just store the MPIAIJ matrices in a simple > CSR format (the matrix is actually stored in two parts, see the MATMPIAIJ > manual page). So the copy is needed. > > What we do. Is first count the number of nonzeros per row then > preallocate the memory then compute the values in the matrix, see > MatMPIAIJSetPreallocatio(). This may > seem expensive but really is not, the counting is fast. You build the CSR > matrix for your sparse matrix; using our approach is no harder and > prevents the need for a copy. (In fact the reason PETSc has the > Preallocation() and MatSetValues() stuff is so that users do not have > to construct the CSR format directly. > > Barry > > > > > > > ----- Original Message ----- > > From: Barry Smith > > To: Roberto Gori > > Cc: petsc-users at mcs.anl.gov > > Sent: Monday, February 06, 2006 4:15 PM > > Subject: Re: mat creation > > > > > > > > > > On Mon, 6 Feb 2006, Roberto Gori wrote: > > > > > Hi, guys, > > > > > > I'm trying to solve a linear system with M equations and N unknowns (M >> N) using the LSQR method. > > > > > > I have a C parallel sparse matrix MxN with NZ nonzero elements stored in this way: > > > > > > double values[NZ]; // the nonzero values > > > int idxn[NZ]; // the column indices > > > int nz[M]; // the number of nonzero values for each row > > > > This needs to be instead indices to start of each row in idxn and values > > instead of the length of each row. > > > > > > what is the best way to create a PETSc matrix? > > > > MatMPIAIJSetPreallocationCSR() > > > > > > > > > > Do exists a statement like VecCreateMPIWithArray for already distributed vectors that could do all the work in one shot without copy? > > > > Yes > > > > > Just another detail. Initially M is unknown. All my structures are > > > allocated dynamically because the number of rows M (and the number > > > of nonzero NZ also) is determined just before to solve the system, > > > so I think it's no feasible for me to call MatCreateMPIAIJ at the > > > begin of my code and to perform the MatSetValues loop. > > > > There is nothing saying that you must "call MatCreateMPIAIJ at the > > beginning of your code", you can call it anytime that you have the > > needed information. > > > > Barry > > > > > > > > Roberto > > > > > > ================================================================ > > > | Roberto Gori CINECA > > > | High Performance System Group Via Magnanelli 6/3 40033 > > > | e-mail: r.gori at cineca.it Casalecchio di Reno > > > | Tel: 051/6171522 Bologna - ITALY > > > | Fax: 051/6132198 http://www.cineca.it > > > ================================================================ > > > > > > > > > > > From tomjan at jay.au.poznan.pl Tue Feb 7 05:57:16 2006 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Tue, 7 Feb 2006 12:57:16 +0100 (CET) Subject: how to use PETSC with C++ Message-ID: hello group I know that no-one like newbie questions, but... unfortunetly I have one Sorry but I cant find in google or manual the clear example how to use PETSC from C++. If there are such, please just point me to them. Initaially I have started with ./config/configure.py --download-f-blas-lapack=1 I have sucessfully compiled a few examples and run in parallel mode. Now I want to implement PETSC for solwing my linear models. I need to switch to C++ because whole code for data manipulation (reading,renumbering) and model manipulation (representation, building) I have written in C++. Maybe it is strange that I'm writing c/c++ code and have such problems but (i'm geneticst not informaticst) compiling and linking ,static and shared libs building it is just night mare for me. So, I have configured PETSC with such options Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 --with-shared=0 -configModules=PETSc.Configure and can't compile even such code ------------------------------------------- #include #include "petscksp.h" #undef __FUNCT__ #define __FUNCT__ "main" int main(int argc,char **args) { PetscErrorCode ierr; PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); ierr = PetscFinalize(); CHKERRQ(ierr); return 0; } ---------------------------------------- When I try to compile it I obtain this: mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C mpiCC -Wall -g3 -o ex4 ex4.o -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu -L/home/work/petsc-2.3.0/lib/linux-gnu -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lnsl -lrt -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi -llam -lutil -ldl -lg2c -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s -lpthread -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio -llammpi++ -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread ex4.o(.text+0x2e): In function `main': /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to `PetscInitialize(int*, char***, char const*, char const*)' ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: undefined reference to `PetscFinalize()' collect2: ld returned 1 exit status mpiCC: No such file or directory make: [ex4] B??d 1 (zignorowany) ######################################################## # tomjan at jay.au.poznan.pl # # jay.au.poznan.pl/~tomjan/ # ######################################################## From knepley at mcs.anl.gov Tue Feb 7 07:57:35 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Tue, 07 Feb 2006 07:57:35 -0600 Subject: how to use PETSC with C++ In-Reply-To: (Tomasz Jankowski's message of "Tue, 7 Feb 2006 12:57:16 +0100 (CET)") References: Message-ID: Tomasz Jankowski writes: The easiest way to use PETSc from C++ is to reconfigure with -with-clanguage=C++ and rebuild. You could even put this in a different arch directory using -PETSC_ARCH=. Thanks, Matt > hello group > > I know that no-one like newbie questions, but... unfortunetly I have one > > Sorry but I cant find in google or manual the clear example how to use PETSC from C++. > If there are such, please just point me to them. > > > Initaially I have started with > ./config/configure.py --download-f-blas-lapack=1 > > I have sucessfully compiled a few examples and run in parallel mode. > Now I want to implement PETSC for solwing my linear models. > I need to switch to C++ because whole code for data manipulation > (reading,renumbering) and model manipulation (representation, building) I have written in C++. > > Maybe it is strange that I'm writing c/c++ code and have such problems but (i'm geneticst not informaticst) compiling > and linking ,static and shared libs building it is just night mare for me. > > So, > > I have configured PETSC with such options > Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 --with-shared=0 -configModules=PETSc.Configure > > and can't compile even such code > ------------------------------------------- > #include > #include "petscksp.h" > > #undef __FUNCT__ > #define __FUNCT__ "main" > int main(int argc,char **args) > { > PetscErrorCode ierr; > PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); > ierr = PetscFinalize(); > CHKERRQ(ierr); > return 0; > } > ---------------------------------------- > > > When I try to compile it I obtain this: > > mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include > -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C > mpiCC -Wall -g3 -o ex4 ex4.o -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu -L/home/work/petsc-2.3.0/lib/linux-gnu > -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lnsl -lrt > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi > -llam -lutil -ldl -lg2c -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s > -lpthread -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio > -llammpi++ -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread > ex4.o(.text+0x2e): In function `main': > /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to `PetscInitialize(int*, char***, char const*, char > const*)' > ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: undefined reference to `PetscFinalize()' > collect2: ld returned 1 exit status > mpiCC: No such file or directory > make: [ex4] B??d 1 (zignorowany) > > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From bsmith at mcs.anl.gov Tue Feb 7 08:26:38 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 7 Feb 2006 08:26:38 -0600 (CST) Subject: how to use PETSC with C++ In-Reply-To: References: Message-ID: Please download and install PETSc 2.3.1 (just released). Regardless of whether you run with --with-clanguage=c++ or not, what you want to do SHOULD work! If the problem persists with 2.3.1 please send configure.log make_log*** and the output from linking your example to petsc-maint at mcs.anl.gov so we can determine what has gone wrong. Barry On Tue, 7 Feb 2006, Tomasz Jankowski wrote: > hello group > > I know that no-one like newbie questions, but... unfortunetly I have one > > Sorry but I cant find in google or manual the clear example how to use > PETSC from C++. > If there are such, please just point me to them. > > > Initaially I have started with > ./config/configure.py --download-f-blas-lapack=1 > > I have sucessfully compiled a few examples and run in parallel mode. > Now I want to implement PETSC for solwing my linear models. > I need to switch to C++ because whole code for data manipulation > (reading,renumbering) and model manipulation (representation, building) I > have written in C++. > > Maybe it is strange that I'm writing c/c++ code and have such problems but > (i'm geneticst not informaticst) compiling and linking ,static and shared > libs building it is just night mare for me. > > So, > > I have configured PETSC with such options > Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 > --with-shared=0 -configModules=PETSc.Configure > > and can't compile even such code > ------------------------------------------- > #include > #include "petscksp.h" > > #undef __FUNCT__ > #define __FUNCT__ "main" > int main(int argc,char **args) > { > PetscErrorCode ierr; > PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); > ierr = PetscFinalize(); > CHKERRQ(ierr); > return 0; > } > ---------------------------------------- > > > When I try to compile it I obtain this: > > mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 > -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include > -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C > mpiCC -Wall -g3 -o ex4 ex4.o > -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu > -L/home/work/petsc-2.3.0/lib/linux-gnu -lpetscksp -lpetscdm -lpetscmat > -lpetscvec -lpetsc -lnsl -lrt > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas > -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi -llam -lutil -ldl -lg2c > -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 > -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s -lpthread > -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 > -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio -llammpi++ > -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread > ex4.o(.text+0x2e): In function `main': > /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to > `PetscInitialize(int*, char***, char const*, char const*)' > ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: > undefined reference to `PetscFinalize()' > collect2: ld returned 1 exit status > mpiCC: No such file or directory > make: [ex4] B??d 1 (zignorowany) > > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## From tomjan at jay.au.poznan.pl Tue Feb 7 05:57:16 2006 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Tue, 7 Feb 2006 12:57:16 +0100 (CET) Subject: how to use PETSC with C++ Message-ID: hello group I know that no-one like newbie questions, but... unfortunetly I have one Sorry but I cant find in google or manual the clear example how to use PETSC from C++. If there are such, please just point me to them. Initaially I have started with ./config/configure.py --download-f-blas-lapack=1 I have sucessfully compiled a few examples and run in parallel mode. Now I want to implement PETSC for solwing my linear models. I need to switch to C++ because whole code for data manipulation (reading,renumbering) and model manipulation (representation, building) I have written in C++. Maybe it is strange that I'm writing c/c++ code and have such problems but (i'm geneticst not informaticst) compiling and linking ,static and shared libs building it is just night mare for me. So, I have configured PETSC with such options Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 --with-shared=0 -configModules=PETSc.Configure and can't compile even such code ------------------------------------------- #include #include "petscksp.h" #undef __FUNCT__ #define __FUNCT__ "main" int main(int argc,char **args) { PetscErrorCode ierr; PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); ierr = PetscFinalize(); CHKERRQ(ierr); return 0; } ---------------------------------------- When I try to compile it I obtain this: mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C mpiCC -Wall -g3 -o ex4 ex4.o -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu -L/home/work/petsc-2.3.0/lib/linux-gnu -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lnsl -lrt -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi -llam -lutil -ldl -lg2c -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s -lpthread -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio -llammpi++ -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread ex4.o(.text+0x2e): In function `main': /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to `PetscInitialize(int*, char***, char const*, char const*)' ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: undefined reference to `PetscFinalize()' collect2: ld returned 1 exit status mpiCC: No such file or directory make: [ex4] B??d 1 (zignorowany) ######################################################## # tomjan at jay.au.poznan.pl # # jay.au.poznan.pl/~tomjan/ # ######################################################## From bsmith at mcs.anl.gov Tue Feb 7 08:26:38 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 7 Feb 2006 08:26:38 -0600 (CST) Subject: how to use PETSC with C++ In-Reply-To: References: Message-ID: Please download and install PETSc 2.3.1 (just released). Regardless of whether you run with --with-clanguage=c++ or not, what you want to do SHOULD work! If the problem persists with 2.3.1 please send configure.log make_log*** and the output from linking your example to petsc-maint at mcs.anl.gov so we can determine what has gone wrong. Barry On Tue, 7 Feb 2006, Tomasz Jankowski wrote: > hello group > > I know that no-one like newbie questions, but... unfortunetly I have one > > Sorry but I cant find in google or manual the clear example how to use > PETSC from C++. > If there are such, please just point me to them. > > > Initaially I have started with > ./config/configure.py --download-f-blas-lapack=1 > > I have sucessfully compiled a few examples and run in parallel mode. > Now I want to implement PETSC for solwing my linear models. > I need to switch to C++ because whole code for data manipulation > (reading,renumbering) and model manipulation (representation, building) I > have written in C++. > > Maybe it is strange that I'm writing c/c++ code and have such problems but > (i'm geneticst not informaticst) compiling and linking ,static and shared > libs building it is just night mare for me. > > So, > > I have configured PETSC with such options > Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 > --with-shared=0 -configModules=PETSc.Configure > > and can't compile even such code > ------------------------------------------- > #include > #include "petscksp.h" > > #undef __FUNCT__ > #define __FUNCT__ "main" > int main(int argc,char **args) > { > PetscErrorCode ierr; > PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); > ierr = PetscFinalize(); > CHKERRQ(ierr); > return 0; > } > ---------------------------------------- > > > When I try to compile it I obtain this: > > mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 > -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include > -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C > mpiCC -Wall -g3 -o ex4 ex4.o > -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu > -L/home/work/petsc-2.3.0/lib/linux-gnu -lpetscksp -lpetscdm -lpetscmat > -lpetscvec -lpetsc -lnsl -lrt > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas > -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi -llam -lutil -ldl -lg2c > -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 > -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s -lpthread > -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 > -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio -llammpi++ > -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread > ex4.o(.text+0x2e): In function `main': > /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to > `PetscInitialize(int*, char***, char const*, char const*)' > ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: > undefined reference to `PetscFinalize()' > collect2: ld returned 1 exit status > mpiCC: No such file or directory > make: [ex4] B??d 1 (zignorowany) > > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## From knepley at mcs.anl.gov Tue Feb 7 07:57:35 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Tue, 07 Feb 2006 07:57:35 -0600 Subject: how to use PETSC with C++ In-Reply-To: (Tomasz Jankowski's message of "Tue, 7 Feb 2006 12:57:16 +0100 (CET)") References: Message-ID: Tomasz Jankowski writes: The easiest way to use PETSc from C++ is to reconfigure with -with-clanguage=C++ and rebuild. You could even put this in a different arch directory using -PETSC_ARCH=. Thanks, Matt > hello group > > I know that no-one like newbie questions, but... unfortunetly I have one > > Sorry but I cant find in google or manual the clear example how to use PETSC from C++. > If there are such, please just point me to them. > > > Initaially I have started with > ./config/configure.py --download-f-blas-lapack=1 > > I have sucessfully compiled a few examples and run in parallel mode. > Now I want to implement PETSC for solwing my linear models. > I need to switch to C++ because whole code for data manipulation > (reading,renumbering) and model manipulation (representation, building) I have written in C++. > > Maybe it is strange that I'm writing c/c++ code and have such problems but (i'm geneticst not informaticst) compiling > and linking ,static and shared libs building it is just night mare for me. > > So, > > I have configured PETSC with such options > Configure Options: --with-clanguage=C++ --download-f-blas-lapack=1 --with-shared=0 -configModules=PETSc.Configure > > and can't compile even such code > ------------------------------------------- > #include > #include "petscksp.h" > > #undef __FUNCT__ > #define __FUNCT__ "main" > int main(int argc,char **args) > { > PetscErrorCode ierr; > PetscInitialize(argc,args,PETSC_NULL,PETSC_NULL); > ierr = PetscFinalize(); > CHKERRQ(ierr); > return 0; > } > ---------------------------------------- > > > When I try to compile it I obtain this: > > mpiCC -c -Wall -g3 -I/home/work/petsc-2.3.0 -I/home/work/petsc-2.3.0/bmake/linux-gnu -I/home/work/petsc-2.3.0/include > -I/usr/local/include -D__SDIR__="/home/work/blup_pcg" ex4.C > mpiCC -Wall -g3 -o ex4 ex4.o -Wl,-rpath,/home/work/petsc-2.3.0/lib/linux-gnu -L/home/work/petsc-2.3.0/lib/linux-gnu > -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lnsl -lrt > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu-lflapack > -Wl,-rpath,/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu > -L/home/work/petsc-2.3.0/externalpackages/fblaslapack/linux-gnu -lfblas -lm -L/usr/local/lib -llammpio -llamf77mpi -lmpi > -llam -lutil -ldl -lg2c -lm -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s > -lpthread -L/usr/local/lib -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -llammpio > -llammpi++ -llamf77mpi -lmpi -llam -lutil -ldl -lstdc++ -lgcc_s -lpthread > ex4.o(.text+0x2e): In function `main': > /usr/local/include/mpi2cxx/datatype_inln.h:174: undefined reference to `PetscInitialize(int*, char***, char const*, char > const*)' > ex4.o(.text+0x33):/usr/local/include/mpi2cxx/datatype_inln.h:213: undefined reference to `PetscFinalize()' > collect2: ld returned 1 exit status > mpiCC: No such file or directory > make: [ex4] B??d 1 (zignorowany) > > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From tomjan at jay.au.poznan.pl Wed Feb 8 00:40:26 2006 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Wed, 8 Feb 2006 07:40:26 +0100 (CET) Subject: how to use PETSC with C++ In-Reply-To: References: Message-ID: > Please download and install PETSc 2.3.1 (just released). > Regardless of whether you run with --with-clanguage=c++ or > not, what you want to do SHOULD work! > > If the problem persists with 2.3.1 please send configure.log > make_log*** and the output from linking your example to > petsc-maint at mcs.anl.gov so we > can determine what has gone wrong. Ewerything has gone smooth. Thank you, mister. tom ######################################################## # tomjan at jay.au.poznan.pl # # jay.au.poznan.pl/~tomjan/ # ######################################################## From billy at dem.uminho.pt Fri Feb 17 12:02:42 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Fri, 17 Feb 2006 18:02:42 +0000 Subject: makefile. Message-ID: <1140199362.43f60fc28f2fc@serv-g1.ccom.uminho.pt> I am new to PETSc, so I don't really know how it works. I added this line to my makefile: include $(PETSC_DIR)/bmake/common/base but I had a target named 'clean' defined. Now, when I run my makefile it says: ... ignoring old commands for target 'clean' Does PETSc also define a target named 'clean'? If yes, what does it do? Billy. From balay at mcs.anl.gov Fri Feb 17 13:29:42 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 17 Feb 2006 13:29:42 -0600 (CST) Subject: makefile. In-Reply-To: <1140199362.43f60fc28f2fc@serv-g1.ccom.uminho.pt> References: <1140199362.43f60fc28f2fc@serv-g1.ccom.uminho.pt> Message-ID: On Fri, 17 Feb 2006, billy at dem.uminho.pt wrote: > > I am new to PETSc, so I don't really know how it works. > > I added this line to my makefile: > > include $(PETSC_DIR)/bmake/common/base > > but I had a target named 'clean' defined. Now, when I run my makefile it says: > > ... ignoring old commands for target 'clean' > > Does PETSc also define a target named 'clean'? If yes, what does it do? Yes - it also defines a target clean. Its in bmake/common/rules. You can check it to see what all default files it deletes. To get additional files to be deleted by this default target - specify the following in the user makefile CLEANFILES = filenames Satish From knepley at mcs.anl.gov Fri Feb 17 14:06:29 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Fri, 17 Feb 2006 14:06:29 -0600 Subject: makefile. In-Reply-To: <1140199362.43f60fc28f2fc@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Fri, 17 Feb 2006 18:02:42 +0000") References: <1140199362.43f60fc28f2fc@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: > I am new to PETSc, so I don't really know how it works. > > I added this line to my makefile: > > include $(PETSC_DIR)/bmake/common/base > > but I had a target named 'clean' defined. Now, when I run my makefile it says: > > ... ignoring old commands for target 'clean' > > Does PETSc also define a target named 'clean'? If yes, what does it do? It does define 'clean' to cleanup up after compilation. You could change the name of your target if you need it. I will look at changing it in an up coming release. Matt -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From billy at dem.uminho.pt Fri Feb 17 19:19:34 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sat, 18 Feb 2006 01:19:34 +0000 Subject: PETSC headers Message-ID: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> I added this line to my makefile: include $(PETSC_DIR)/bmake/common/base In my main.c file I added the header: #include "petsc.h" and it worked fine, but when I use instead: #include "petscao.h" #include "petscsnes.h" I get the following errors during compilation: In file included from /home/billy/PETSC/petsc-2.3.0/include/petscao.h:8, from source/main.c:50: /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: error: parse error before numeric constant /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: warning: no semicolon at end of struct or union /home/billy/PETSC/petsc-2.3.0/include/petscis.h:172: error: parse error before '}' token Billy. From knepley at mcs.anl.gov Fri Feb 17 19:22:21 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Fri, 17 Feb 2006 19:22:21 -0600 Subject: PETSC headers In-Reply-To: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Sat, 18 Feb 2006 01:19:34 +0000") References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: Do you have the latest patchlevel for 2.3.0? Also, we just released 2.3.1 and stringly recommend you upgrade since there are no interface changes. I think this problem was fixed in a patch. Matt > I added this line to my makefile: > > include $(PETSC_DIR)/bmake/common/base > > In my main.c file I added the header: > > #include "petsc.h" > > and it worked fine, but when I use instead: > > #include "petscao.h" > #include "petscsnes.h" > > I get the following errors during compilation: > > In file included from /home/billy/PETSC/petsc-2.3.0/include/petscao.h:8, > from source/main.c:50: > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: error: parse error before > numeric constant > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: warning: no semicolon at > end of struct or union > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:172: error: parse error before > '}' token > > > Billy. > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From balay at mcs.anl.gov Fri Feb 17 22:15:23 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 17 Feb 2006 22:15:23 -0600 (CST) Subject: PETSC headers In-Reply-To: References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> Message-ID: Not sure what was fixed in patches.. Bill - can you reproduce this error with a PETSc example? [say src/snes/examples/tutorials/ex1.c] Satish On Fri, 17 Feb 2006, Matthew Knepley wrote: > billy at dem.uminho.pt writes: > > Do you have the latest patchlevel for 2.3.0? Also, we just released > 2.3.1 and stringly recommend you upgrade since there are no interface changes. > I think this problem was fixed in a patch. > > Matt > > > I added this line to my makefile: > > > > include $(PETSC_DIR)/bmake/common/base > > > > In my main.c file I added the header: > > > > #include "petsc.h" > > > > and it worked fine, but when I use instead: > > > > #include "petscao.h" > > #include "petscsnes.h" > > > > I get the following errors during compilation: > > > > In file included from /home/billy/PETSC/petsc-2.3.0/include/petscao.h:8, > > from source/main.c:50: > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: error: parse error before > > numeric constant > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: warning: no semicolon at > > end of struct or union > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:172: error: parse error before > > '}' token > > > > > > Billy. > > > > > > > > From billy at dem.uminho.pt Sat Feb 18 10:32:49 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sat, 18 Feb 2006 16:32:49 +0000 Subject: PETSC headers In-Reply-To: References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> Message-ID: <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> No, the examples are working OK. I compiled petsc/src/snes/examples/tutorials/ex10d/ex10.c and there is no problem. I must be doing something wrong in my makefile. # METIS METIS_DIR = $(SOURCE_DIR)/metis METIS_LIB = $(METIS_DIR)/libmetis.a # Multiple processor - MPICH ifeq ($(MPI_VERSION), MPICH) MPI_LIB = -L$(HOME)/MPICH/mpich-1.2.7/lib MPI_INC = -I$(HOME)/MPICH/mpich-1.2.7/include CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) $(PETSC_INCLUDE) $(PETSC_LIB) -lm CC = $(CLINKER) endif # Multiple processor - LAM ifeq ($(MPI_VERSION), LAM) MPI_LIB = -L$(HOME)/LAMMPI/lam-7.1.1/linux/lib MPI_INC = -I$(HOME)/LAMMPI/lam-7.1.1/linux/include CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) $(PETSC_INCLUDE) $(PETSC_LIB) -lm -lmpi CC = $(CLINKER) endif # PETSC include ${PETSC_DIR}/bmake/common/base EXECUTABLE = examples/ex all: $(SOURCE_OBJ) $(LASPACK_OBJ) ifeq ($(MPI_VERSION),LAM) @echo "" @echo "***** Building with LAM option *****" @echo "" endif ifeq ($(MPI_VERSION),MPICH) @echo "" @echo "***** Building with MPICH option *****" @echo "" endif $(CC) $(SOURCE_OBJ) $(METIS_LIB) $(PETSC_LIB) -o $(EXECUTABLE) Billy. Quoting Satish Balay : > Not sure what was fixed in patches.. > > Bill - can you reproduce this error with a PETSc example? [say > src/snes/examples/tutorials/ex1.c] > > Satish > > On Fri, 17 Feb 2006, Matthew Knepley wrote: > > > billy at dem.uminho.pt writes: > > > > Do you have the latest patchlevel for 2.3.0? Also, we just released > > 2.3.1 and stringly recommend you upgrade since there are no interface > changes. > > I think this problem was fixed in a patch. > > > > Matt > > > > > I added this line to my makefile: > > > > > > include $(PETSC_DIR)/bmake/common/base > > > > > > In my main.c file I added the header: > > > > > > #include "petsc.h" > > > > > > and it worked fine, but when I use instead: > > > > > > #include "petscao.h" > > > #include "petscsnes.h" > > > > > > I get the following errors during compilation: > > > > > > In file included from > /home/billy/PETSC/petsc-2.3.0/include/petscao.h:8, > > > from source/main.c:50: > > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: error: parse error > before > > > numeric constant > > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:167: warning: no > semicolon at > > > end of struct or union > > > /home/billy/PETSC/petsc-2.3.0/include/petscis.h:172: error: parse error > before > > > '}' token > > > > > > > > > Billy. > > > > > > > > > > > > > > > From balay at mcs.anl.gov Sat Feb 18 11:19:46 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 18 Feb 2006 11:19:46 -0600 (CST) Subject: PETSC headers In-Reply-To: <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> Message-ID: PETSc is installed with MPI, LAPACK - and can be installed with PARMETIS [which has metis.a]. So there is no reason to have a non-petsc makefile. And you can't use a different MPI than what PETSc is installed with. The simplest makefile would be: -------------------------------------- CFLAGS = FFLAGS = CPPFLAGS = FPPFLAGS = CLEANFILES = include ${PETSC_DIR}/bmake/common/base ex1: ex1.o chkopts -${CLINKER} -o ex1 ex1.o ${PETSC_LIB} ${RM} ex1.o -------------------------------------- Where CLEANFILES - is additonal files you want to get deleted with 'make clean' CPPFLAGS, FPPFLAGS are additional user required preprocessor flags [for eg -DFOOBAR] CFLAGS,FFLAGS are additional user required compiler flags [perhaps like -r8] If you wish to just use PETSc as installed [without parmetis] - and 'metis' required by your code, use: -------------------------------------- SOURCE_DIR = ??? METIS_INC = -I$(SOURCE_DIR)/metis/include METIS_LIB = $(METIS_DIR)/libmetis.a EXECUTABLE = examples/ex SOURCE_OBJ = ex1.o CFLAGS = FFLAGS = CPPFLAGS = $(METIS_INC) FPPFLAGS = CLEANFILES = $(EXECUTABLE) include ${PETSC_DIR}/bmake/common/base all: $(SOURCE_OBJ) chkopts -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} -------------------------------------- Satish On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > > > No, the examples are working OK. I compiled > petsc/src/snes/examples/tutorials/ex10d/ex10.c and there is no problem. > > I must be doing something wrong in my makefile. > > # METIS > METIS_DIR = $(SOURCE_DIR)/metis > METIS_LIB = $(METIS_DIR)/libmetis.a > > # Multiple processor - MPICH > ifeq ($(MPI_VERSION), MPICH) > MPI_LIB = -L$(HOME)/MPICH/mpich-1.2.7/lib > MPI_INC = -I$(HOME)/MPICH/mpich-1.2.7/include > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > $(PETSC_INCLUDE) $(PETSC_LIB) -lm > CC = $(CLINKER) > endif > > # Multiple processor - LAM > ifeq ($(MPI_VERSION), LAM) > MPI_LIB = -L$(HOME)/LAMMPI/lam-7.1.1/linux/lib > MPI_INC = -I$(HOME)/LAMMPI/lam-7.1.1/linux/include > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > $(PETSC_INCLUDE) $(PETSC_LIB) -lm -lmpi > CC = $(CLINKER) > endif > > # PETSC > include ${PETSC_DIR}/bmake/common/base > > EXECUTABLE = examples/ex > > all: $(SOURCE_OBJ) $(LASPACK_OBJ) > > ifeq ($(MPI_VERSION),LAM) > @echo "" > @echo "***** Building with LAM option *****" > @echo "" > endif > > ifeq ($(MPI_VERSION),MPICH) > @echo "" > @echo "***** Building with MPICH option *****" > @echo "" > endif > > $(CC) $(SOURCE_OBJ) $(METIS_LIB) $(PETSC_LIB) -o $(EXECUTABLE) > From billy at dem.uminho.pt Sat Feb 18 15:30:59 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sat, 18 Feb 2006 21:30:59 +0000 Subject: PETSC headers In-Reply-To: References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> Message-ID: <1140298259.43f7921338f83@serv-g1.ccom.uminho.pt> -------------------------------------- SOURCE_DIR = ??? METIS_INC = -I$(SOURCE_DIR)/metis/include METIS_LIB = $(METIS_DIR)/libmetis.a EXECUTABLE = examples/ex SOURCE_OBJ = ex1.o CFLAGS = FFLAGS = CPPFLAGS = $(METIS_INC) FPPFLAGS = CLEANFILES = $(EXECUTABLE) include ${PETSC_DIR}/bmake/common/base all: $(SOURCE_OBJ) chkopts -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} -------------------------------------- If I use that is says: make: *** No rule to make target `debug/main.o', needed by `all'. Stop. Billy. Quoting Satish Balay : > PETSc is installed with MPI, LAPACK - and can be installed with PARMETIS > [which has metis.a]. So there is no reason to have a non-petsc > makefile. > > And you can't use a different MPI than what PETSc is installed with. > > The simplest makefile would be: > > -------------------------------------- > CFLAGS = > FFLAGS = > CPPFLAGS = > FPPFLAGS = > CLEANFILES = > > include ${PETSC_DIR}/bmake/common/base > > ex1: ex1.o chkopts > -${CLINKER} -o ex1 ex1.o ${PETSC_LIB} > ${RM} ex1.o > -------------------------------------- > > Where > > CLEANFILES - is additonal files you want to get deleted with 'make clean' > CPPFLAGS, FPPFLAGS are additional user required preprocessor flags [for eg > -DFOOBAR] > CFLAGS,FFLAGS are additional user required compiler flags [perhaps like > -r8] > > If you wish to just use PETSc as installed [without parmetis] - and > 'metis' required by your code, use: > > -------------------------------------- > SOURCE_DIR = ??? > METIS_INC = -I$(SOURCE_DIR)/metis/include > METIS_LIB = $(METIS_DIR)/libmetis.a > EXECUTABLE = examples/ex > SOURCE_OBJ = ex1.o > > CFLAGS = > FFLAGS = > CPPFLAGS = $(METIS_INC) > FPPFLAGS = > CLEANFILES = $(EXECUTABLE) > > include ${PETSC_DIR}/bmake/common/base > > all: $(SOURCE_OBJ) chkopts > -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} > > -------------------------------------- > > > Satish > > On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > > > > > > > No, the examples are working OK. I compiled > > petsc/src/snes/examples/tutorials/ex10d/ex10.c and there is no problem. > > > > I must be doing something wrong in my makefile. > > > > # METIS > > METIS_DIR = $(SOURCE_DIR)/metis > > METIS_LIB = $(METIS_DIR)/libmetis.a > > > > # Multiple processor - MPICH > > ifeq ($(MPI_VERSION), MPICH) > > MPI_LIB = -L$(HOME)/MPICH/mpich-1.2.7/lib > > MPI_INC = -I$(HOME)/MPICH/mpich-1.2.7/include > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm > > CC = $(CLINKER) > > endif > > > > # Multiple processor - LAM > > ifeq ($(MPI_VERSION), LAM) > > MPI_LIB = -L$(HOME)/LAMMPI/lam-7.1.1/linux/lib > > MPI_INC = -I$(HOME)/LAMMPI/lam-7.1.1/linux/include > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm -lmpi > > CC = $(CLINKER) > > endif > > > > # PETSC > > include ${PETSC_DIR}/bmake/common/base > > > > EXECUTABLE = examples/ex > > > > all: $(SOURCE_OBJ) $(LASPACK_OBJ) > > > > ifeq ($(MPI_VERSION),LAM) > > @echo "" > > @echo "***** Building with LAM option *****" > > @echo "" > > endif > > > > ifeq ($(MPI_VERSION),MPICH) > > @echo "" > > @echo "***** Building with MPICH option *****" > > @echo "" > > endif > > > > $(CC) $(SOURCE_OBJ) $(METIS_LIB) $(PETSC_LIB) -o $(EXECUTABLE) > > > > From balay at mcs.anl.gov Sat Feb 18 15:41:37 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 18 Feb 2006 15:41:37 -0600 (CST) Subject: PETSC headers In-Reply-To: <1140298259.43f7921338f83@serv-g1.ccom.uminho.pt> References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> <1140298259.43f7921338f83@serv-g1.ccom.uminho.pt> Message-ID: For one you have correctly modify SOURCE_DIR [I didn't know what was correct value for you]. And then make sure other things are correctly specified - like SOURCE_OBJ etc.. I don't see any 'debug/main.o' specified in the makefile - not sure where its coming from. This discusion is better done at petsc-maint at mcs.anl.gov. If you have more isues - send us all the relavent files - so that we can reproduce problems. Satish On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > -------------------------------------- > > SOURCE_DIR = ??? > METIS_INC = -I$(SOURCE_DIR)/metis/include > METIS_LIB = $(METIS_DIR)/libmetis.a > EXECUTABLE = examples/ex > SOURCE_OBJ = ex1.o > > CFLAGS = > FFLAGS = > CPPFLAGS = $(METIS_INC) > FPPFLAGS = > CLEANFILES = $(EXECUTABLE) > > include ${PETSC_DIR}/bmake/common/base > > all: $(SOURCE_OBJ) chkopts > -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} > > -------------------------------------- > > > If I use that is says: > > make: *** No rule to make target `debug/main.o', needed by `all'. Stop. > > Billy. > > > > Quoting Satish Balay : > > > PETSc is installed with MPI, LAPACK - and can be installed with PARMETIS > > [which has metis.a]. So there is no reason to have a non-petsc > > makefile. > > > > And you can't use a different MPI than what PETSc is installed with. > > > > The simplest makefile would be: > > > > -------------------------------------- > > CFLAGS = > > FFLAGS = > > CPPFLAGS = > > FPPFLAGS = > > CLEANFILES = > > > > include ${PETSC_DIR}/bmake/common/base > > > > ex1: ex1.o chkopts > > -${CLINKER} -o ex1 ex1.o ${PETSC_LIB} > > ${RM} ex1.o > > -------------------------------------- > > > > Where > > > > CLEANFILES - is additonal files you want to get deleted with 'make clean' > > CPPFLAGS, FPPFLAGS are additional user required preprocessor flags [for eg > > -DFOOBAR] > > CFLAGS,FFLAGS are additional user required compiler flags [perhaps like > > -r8] > > > > If you wish to just use PETSc as installed [without parmetis] - and > > 'metis' required by your code, use: > > > > -------------------------------------- > > SOURCE_DIR = ??? > > METIS_INC = -I$(SOURCE_DIR)/metis/include > > METIS_LIB = $(METIS_DIR)/libmetis.a > > EXECUTABLE = examples/ex > > SOURCE_OBJ = ex1.o > > > > CFLAGS = > > FFLAGS = > > CPPFLAGS = $(METIS_INC) > > FPPFLAGS = > > CLEANFILES = $(EXECUTABLE) > > > > include ${PETSC_DIR}/bmake/common/base > > > > all: $(SOURCE_OBJ) chkopts > > -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} > > > > -------------------------------------- > > > > > > Satish > > > > On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > > > > > > > > > > > No, the examples are working OK. I compiled > > > petsc/src/snes/examples/tutorials/ex10d/ex10.c and there is no problem. > > > > > > I must be doing something wrong in my makefile. > > > > > > # METIS > > > METIS_DIR = $(SOURCE_DIR)/metis > > > METIS_LIB = $(METIS_DIR)/libmetis.a > > > > > > # Multiple processor - MPICH > > > ifeq ($(MPI_VERSION), MPICH) > > > MPI_LIB = -L$(HOME)/MPICH/mpich-1.2.7/lib > > > MPI_INC = -I$(HOME)/MPICH/mpich-1.2.7/include > > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm > > > CC = $(CLINKER) > > > endif > > > > > > # Multiple processor - LAM > > > ifeq ($(MPI_VERSION), LAM) > > > MPI_LIB = -L$(HOME)/LAMMPI/lam-7.1.1/linux/lib > > > MPI_INC = -I$(HOME)/LAMMPI/lam-7.1.1/linux/include > > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm -lmpi > > > CC = $(CLINKER) > > > endif > > > > > > # PETSC > > > include ${PETSC_DIR}/bmake/common/base > > > > > > EXECUTABLE = examples/ex > > > > > > all: $(SOURCE_OBJ) $(LASPACK_OBJ) > > > > > > ifeq ($(MPI_VERSION),LAM) > > > @echo "" > > > @echo "***** Building with LAM option *****" > > > @echo "" > > > endif > > > > > > ifeq ($(MPI_VERSION),MPICH) > > > @echo "" > > > @echo "***** Building with MPICH option *****" > > > @echo "" > > > endif > > > > > > $(CC) $(SOURCE_OBJ) $(METIS_LIB) $(PETSC_LIB) -o $(EXECUTABLE) > > > > > > > > > > From billy at dem.uminho.pt Sat Feb 18 18:11:24 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sun, 19 Feb 2006 00:11:24 +0000 Subject: PETSC headers In-Reply-To: References: <1140225574.43f6762646cc3@serv-g1.ccom.uminho.pt> <1140280369.43f74c31f36c2@serv-g1.ccom.uminho.pt> <1140298259.43f7921338f83@serv-g1.ccom.uminho.pt> Message-ID: <1140307884.43f7b7ac8cd67@serv-g1.ccom.uminho.pt> Thank you very much for your help. I think it is going work now. I am going to try and adapt my code to use PETSc. It is an unstructured finite volume solver. Billy. Quoting Satish Balay : > For one you have correctly modify SOURCE_DIR [I didn't know what was > correct value for you]. And then make sure other things are correctly > specified - like SOURCE_OBJ etc.. > > I don't see any 'debug/main.o' specified in the makefile - not sure > where its coming from. > > This discusion is better done at petsc-maint at mcs.anl.gov. If you have > more isues - send us all the relavent files - so that we can reproduce > problems. > > Satish > > On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > > > -------------------------------------- > > > > SOURCE_DIR = ??? > > METIS_INC = -I$(SOURCE_DIR)/metis/include > > METIS_LIB = $(METIS_DIR)/libmetis.a > > EXECUTABLE = examples/ex > > SOURCE_OBJ = ex1.o > > > > CFLAGS = > > FFLAGS = > > CPPFLAGS = $(METIS_INC) > > FPPFLAGS = > > CLEANFILES = $(EXECUTABLE) > > > > include ${PETSC_DIR}/bmake/common/base > > > > all: $(SOURCE_OBJ) chkopts > > -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} > > > > -------------------------------------- > > > > > > If I use that is says: > > > > make: *** No rule to make target `debug/main.o', needed by `all'. Stop. > > > > Billy. > > > > > > > > Quoting Satish Balay : > > > > > PETSc is installed with MPI, LAPACK - and can be installed with > PARMETIS > > > [which has metis.a]. So there is no reason to have a non-petsc > > > makefile. > > > > > > And you can't use a different MPI than what PETSc is installed with. > > > > > > The simplest makefile would be: > > > > > > -------------------------------------- > > > CFLAGS = > > > FFLAGS = > > > CPPFLAGS = > > > FPPFLAGS = > > > CLEANFILES = > > > > > > include ${PETSC_DIR}/bmake/common/base > > > > > > ex1: ex1.o chkopts > > > -${CLINKER} -o ex1 ex1.o ${PETSC_LIB} > > > ${RM} ex1.o > > > -------------------------------------- > > > > > > Where > > > > > > CLEANFILES - is additonal files you want to get deleted with 'make > clean' > > > CPPFLAGS, FPPFLAGS are additional user required preprocessor flags [for > eg > > > -DFOOBAR] > > > CFLAGS,FFLAGS are additional user required compiler flags [perhaps like > > > -r8] > > > > > > If you wish to just use PETSc as installed [without parmetis] - and > > > 'metis' required by your code, use: > > > > > > -------------------------------------- > > > SOURCE_DIR = ??? > > > METIS_INC = -I$(SOURCE_DIR)/metis/include > > > METIS_LIB = $(METIS_DIR)/libmetis.a > > > EXECUTABLE = examples/ex > > > SOURCE_OBJ = ex1.o > > > > > > CFLAGS = > > > FFLAGS = > > > CPPFLAGS = $(METIS_INC) > > > FPPFLAGS = > > > CLEANFILES = $(EXECUTABLE) > > > > > > include ${PETSC_DIR}/bmake/common/base > > > > > > all: $(SOURCE_OBJ) chkopts > > > -${CLINKER} -o $(EXECUTABLE) $(SOURCE_OBJ) $(METIS_LIB) ${PETSC_LIB} > > > > > > -------------------------------------- > > > > > > > > > Satish > > > > > > On Sat, 18 Feb 2006, billy at dem.uminho.pt wrote: > > > > > > > > > > > > > > > No, the examples are working OK. I compiled > > > > petsc/src/snes/examples/tutorials/ex10d/ex10.c and there is no > problem. > > > > > > > > I must be doing something wrong in my makefile. > > > > > > > > # METIS > > > > METIS_DIR = $(SOURCE_DIR)/metis > > > > METIS_LIB = $(METIS_DIR)/libmetis.a > > > > > > > > # Multiple processor - MPICH > > > > ifeq ($(MPI_VERSION), MPICH) > > > > MPI_LIB = -L$(HOME)/MPICH/mpich-1.2.7/lib > > > > MPI_INC = -I$(HOME)/MPICH/mpich-1.2.7/include > > > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm > > > > CC = $(CLINKER) > > > > endif > > > > > > > > # Multiple processor - LAM > > > > ifeq ($(MPI_VERSION), LAM) > > > > MPI_LIB = -L$(HOME)/LAMMPI/lam-7.1.1/linux/lib > > > > MPI_INC = -I$(HOME)/LAMMPI/lam-7.1.1/linux/include > > > > CFLAGS = -I$(SOURCE_DIR) -I$(LASPACK_DIR) $(MPI_INC) $(MPI_LIB) > > > > $(PETSC_INCLUDE) $(PETSC_LIB) -lm -lmpi > > > > CC = $(CLINKER) > > > > endif > > > > > > > > # PETSC > > > > include ${PETSC_DIR}/bmake/common/base > > > > > > > > EXECUTABLE = examples/ex > > > > > > > > all: $(SOURCE_OBJ) $(LASPACK_OBJ) > > > > > > > > ifeq ($(MPI_VERSION),LAM) > > > > @echo "" > > > > @echo "***** Building with LAM option *****" > > > > @echo "" > > > > endif > > > > > > > > ifeq ($(MPI_VERSION),MPICH) > > > > @echo "" > > > > @echo "***** Building with MPICH option *****" > > > > @echo "" > > > > endif > > > > > > > > $(CC) $(SOURCE_OBJ) $(METIS_LIB) $(PETSC_LIB) -o $(EXECUTABLE) > > > > > > > > > > > > > > > > > > From buket at be.itu.edu.tr Sun Feb 19 11:34:34 2006 From: buket at be.itu.edu.tr (buket at be.itu.edu.tr) Date: Sun, 19 Feb 2006 19:34:34 +0200 (EET) Subject: divergence? Message-ID: <4267.160.75.90.192.1140370474.squirrel@www.be.itu.edu.tr> Hello, I am trying to solve a linear system of vibroacoustic Problem with 12328 dimension. When I use bcgs and ilu(1) with n=1 proc, I get: ---Norm of residual=4.58868e+17 ---Norm of error(u-x)=75.0051 ---Number of iteration=7 ---Solution Time=3.896039 second When I use bcgs and ilu(2) with n=1 proc, I get a different error message: ---[0]PETSC ERROR: MatLUFactorNumeric_SeqAIJ() line 537 in src/mat/impls/aij/seq/aijfact.c [0]PETSC ERROR: Detected zero pivot in LU factorization! [0]PETSC ERROR: Zero pivot row 10816 value 5.0874e+06 tolerance 1e-12 * rs 7.29416e+20! [0]PETSC ERROR: MatLUFactorNumeric() line 1831 in src/mat/interface/matrix.c .................... That system seems to converge immediately, at first, with 7 iters. But the norm of residual is increadibly high. Did you face with that situation? Can that be the result of divergence? Best Regards, Buket Benek From bsmith at mcs.anl.gov Sun Feb 19 16:22:30 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 19 Feb 2006 16:22:30 -0600 (CST) Subject: divergence? In-Reply-To: <4267.160.75.90.192.1140370474.squirrel@www.be.itu.edu.tr> References: <4267.160.75.90.192.1140370474.squirrel@www.be.itu.edu.tr> Message-ID: Buket, ---Norm of residual=4.58868e+17 This is usually indicative of a matrix with poor scaling or very small matrices along the diagonal. I a very small pivot is encountered causing a "crazy" preconditioner with tiny entries producing a really absurd preconditioned residual norm (most likely the true residual norm is much more reasonable, use -ksp_truemonitor) It has not converged. You can run (the latest PETSc 2.3.1) with -pc_factor_shift_positive_definite to "damp the matrix used to construct the preconditioner to prevent zero or very small pivots". For this size problem you also may as well just use the direct solver -pc_type lu Good luck, Barry On Sun, 19 Feb 2006, buket at be.itu.edu.tr wrote: > Hello, > I am trying to solve a linear system of vibroacoustic Problem with 12328 > dimension. > > When I use bcgs and ilu(1) with n=1 proc, I get: > > ---Norm of residual=4.58868e+17 > ---Norm of error(u-x)=75.0051 > ---Number of iteration=7 > ---Solution Time=3.896039 second > > When I use bcgs and ilu(2) with n=1 proc, I get a different error message: > > ---[0]PETSC ERROR: MatLUFactorNumeric_SeqAIJ() line 537 in > src/mat/impls/aij/seq/aijfact.c > [0]PETSC ERROR: Detected zero pivot in LU factorization! > [0]PETSC ERROR: Zero pivot row 10816 value 5.0874e+06 tolerance 1e-12 > * rs 7.29416e+20! > [0]PETSC ERROR: MatLUFactorNumeric() line 1831 in > src/mat/interface/matrix.c > .................... > > That system seems to converge immediately, at first, with 7 iters. But the > norm of residual is increadibly high. Did you face with that situation? > Can that be the result of divergence? > > Best Regards, > Buket Benek > > > From billy at dem.uminho.pt Mon Feb 20 15:23:38 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Mon, 20 Feb 2006 21:23:38 +0000 Subject: Matrix operation Message-ID: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> Hello, I would like to do the following matrix-vector operation in PETSc: h += (Diag(A) - A) * u where A is a matrix, Diag(A) is a matrix and h, u are vectors. It is the multiplication of a matrix with diagonals set to zero with a vector. How can I do this in PETSc? because MatGetDiagonal gives a vector not a matrix. Billy. From hzhang at mcs.anl.gov Mon Feb 20 16:52:01 2006 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Mon, 20 Feb 2006 16:52:01 -0600 (CST) Subject: Matrix operation In-Reply-To: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> References: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> Message-ID: Billy, You can do MatGetDiagonal(A,diag); MatMult(A,u,&vtmp); VecGetArray(diag,diag_array); VecGetArray(vtmp,vtmp_array); VecGetArray(u,u_arrar); VecGetArray(h,h_array); VecGetLocalSize(h,&n); for (i=0; i > Hello, > > I would like to do the following matrix-vector operation in PETSc: > > h += (Diag(A) - A) * u > > where A is a matrix, Diag(A) is a matrix and h, u are vectors. It is the > multiplication of a matrix with diagonals set to zero with a vector. > > > How can I do this in PETSc? because MatGetDiagonal gives a vector not a matrix. > > > Billy. > > > From knepley at mcs.anl.gov Mon Feb 20 17:15:34 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Mon, 20 Feb 2006 17:15:34 -0600 Subject: Matrix operation In-Reply-To: (Hong Zhang's message of "Mon, 20 Feb 2006 16:52:01 -0600 (CST)") References: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> Message-ID: Hong Zhang writes: I would recommend the later MatGetDiagonal(A, diag); VecPointwiseMult(w, diag, u); MatMult(A, u, v); VecAXPY(w, -1.0, v); Matt > Billy, > > You can do > MatGetDiagonal(A,diag); > MatMult(A,u,&vtmp); > VecGetArray(diag,diag_array); > VecGetArray(vtmp,vtmp_array); > VecGetArray(u,u_arrar); > VecGetArray(h,h_array); > VecGetLocalSize(h,&n); > for (i=0; i h_array[i] += diag_array[i]*u_array[i] - vtmp_array[i]; > } > VecRestoreArray(h,h_array); > VecRestoreArray() > ... > > or you can use VecPointwiseMult() got get > Diag(A)*u. > > Hong > > On Mon, 20 Feb 2006 billy at dem.uminho.pt wrote: > >> >> Hello, >> >> I would like to do the following matrix-vector operation in PETSc: >> >> h += (Diag(A) - A) * u >> >> where A is a matrix, Diag(A) is a matrix and h, u are vectors. It is the >> multiplication of a matrix with diagonals set to zero with a vector. >> >> >> How can I do this in PETSc? because MatGetDiagonal gives a vector not a matrix. >> >> >> Billy. >> >> >> > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From abdul-rahman at tu-harburg.de Tue Feb 21 09:34:38 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Tue, 21 Feb 2006 16:34:38 +0100 (MET) Subject: New changes for CG set type in 2.3.1-p7 ? Message-ID: Hi all, There seems to be some changes for the CG type in PETSc 2.3.1-p7. I can't have -ksp_cg_symmetric anymore - it simply says: Option left: name:-ksp_cg_symmetric no value Hard coding into my routine does not help either: call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) does not show any effect. I still get CG of Hermitian type. Are the changes documented anywhere? I cannot make much sense from the bitkeeper history. It works with 2.2.1 and *I think* 2.3.0 too. By the way there's a *huge* difference in iterations between Hermitian and Symmetric types in my test problem. Thanks! Razi From letian.wang at ghiocel-tech.com Tue Feb 21 09:57:52 2006 From: letian.wang at ghiocel-tech.com (Letian Wang) Date: Tue, 21 Feb 2006 10:57:52 -0500 Subject: Help on installation of PETSC+prometheus on Cygwin Message-ID: <001601c636ff$92bf9fb0$0b00a8c0@lele> Hello, I had problems to install PETSc-2.3.1-p5 and Prometheus on my notebook under Cygwin environment. Here are the options I used: --with-blas-lapack-dir=$PETSC_DIR/externalpackages/fblaslapack/$PETSC_ARCH --with-debugging=0 --with-mpi-dir=$HOME/mpich2-install --download-parmetis=1 --download-prometheus=1 --download-spooles=1 --download-hypre=1 I could download Parmetis and it seemed the compiling was ok because I saw files libparmetis.a and libmetis.a. However, PETSc complains the downloaded Parmetis cannot be used. Attached is the configure.log file. Any helps are greatly appreciated. Thanks. Letian Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: application/octet-stream Size: 2537362 bytes Desc: not available URL: From balay at mcs.anl.gov Tue Feb 21 10:21:54 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 21 Feb 2006 10:21:54 -0600 (CST) Subject: Help on installation of PETSC+prometheus on Cygwin In-Reply-To: <001601c636ff$92bf9fb0$0b00a8c0@lele> References: <001601c636ff$92bf9fb0$0b00a8c0@lele> Message-ID: These external packages are not tested on windows. >> /home/feap/Library/petsc-2.3.1-p5/externalpackages/ParMetis/cygwin-c-real-opt/include/parmetis.h:20:1: warning: "__cdecl" redefined >> You could try removing the following code from /home/feap/Library/petsc-2.3.1-p5/externalpackages/ParMetis/parmetis.h and see if it helps #ifndef _MSC_VER #define __cdecl #endif However I'll recommend using linux instead [due to all the external package requirements] Satish On Tue, 21 Feb 2006, Letian Wang wrote: > Hello, > > > > I had problems to install PETSc-2.3.1-p5 and Prometheus on my notebook under > Cygwin environment. Here are the options I used: > > > > --with-blas-lapack-dir=$PETSC_DIR/externalpackages/fblaslapack/$PETSC_ARCH > > --with-debugging=0 > > --with-mpi-dir=$HOME/mpich2-install > > --download-parmetis=1 > > --download-prometheus=1 > > --download-spooles=1 > > --download-hypre=1 > > > > I could download Parmetis and it seemed the compiling was ok because I saw > files libparmetis.a and libmetis.a. However, PETSc complains the downloaded > Parmetis cannot be used. Attached is the configure.log file. > > > > Any helps are greatly appreciated. Thanks. > > > > Letian Wang > > From letian.wang at ghiocel-tech.com Tue Feb 21 10:45:46 2006 From: letian.wang at ghiocel-tech.com (Letian Wang) Date: Tue, 21 Feb 2006 11:45:46 -0500 Subject: Help on installation of PETSC+prometheus on Cygwin In-Reply-To: Message-ID: <000001c63706$4336e3c0$0b00a8c0@lele> Satish: Thank you very much for the help. Right now it works on cygwin shell after removing the three lines. Letian -----Original Message----- From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay Sent: Tuesday, February 21, 2006 11:22 AM To: petsc-users at mcs.anl.gov Subject: Re: Help on installation of PETSC+prometheus on Cygwin These external packages are not tested on windows. >> /home/feap/Library/petsc-2.3.1-p5/externalpackages/ParMetis/cygwin-c-real-op t/include/parmetis.h:20:1: warning: "__cdecl" redefined >> You could try removing the following code from /home/feap/Library/petsc-2.3.1-p5/externalpackages/ParMetis/parmetis.h and see if it helps #ifndef _MSC_VER #define __cdecl #endif However I'll recommend using linux instead [due to all the external package requirements] Satish On Tue, 21 Feb 2006, Letian Wang wrote: > Hello, > > > > I had problems to install PETSc-2.3.1-p5 and Prometheus on my notebook under > Cygwin environment. Here are the options I used: > > > > --with-blas-lapack-dir=$PETSC_DIR/externalpackages/fblaslapack/$PETSC_ARCH > > --with-debugging=0 > > --with-mpi-dir=$HOME/mpich2-install > > --download-parmetis=1 > > --download-prometheus=1 > > --download-spooles=1 > > --download-hypre=1 > > > > I could download Parmetis and it seemed the compiling was ok because I saw > files libparmetis.a and libmetis.a. However, PETSc complains the downloaded > Parmetis cannot be used. Attached is the configure.log file. > > > > Any helps are greatly appreciated. Thanks. > > > > Letian Wang > > From balay at mcs.anl.gov Tue Feb 21 10:57:57 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 21 Feb 2006 10:57:57 -0600 (CST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: On Tue, 21 Feb 2006, abdul-rahman at tu-harburg.de wrote: > Hi all, > > There seems to be some changes for the CG type in PETSc 2.3.1-p7. > I can't have -ksp_cg_symmetric anymore - it simply says: > > Option left: name:-ksp_cg_symmetric no value > petsc-2.3.0 changelog has the following entry: >>> -ksp_cg_Hermitian and -ksp_cg_symmetric have been changed to -ksp_cg_type Hermitian or symmetric >>> > > Hard coding into my routine does not help either: > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) > > does not show any effect. I still get CG of Hermitian type. Looks like there is a bug here. KSP_CG_SYMMETRIC value was changed on C side - but not on fortran side. Try editing include/finclude/petscksp.h and change to: parameter (KSP_CG_SYMMETRIC=0,KSP_CG_HERMITIAN=1) I'll add this to the next patch update to petsc-2.3.1 > Are the changes documented anywhere? I cannot make much sense from the > bitkeeper history. > > It works with 2.2.1 and *I think* 2.3.0 too. By the way there's a *huge* > difference in iterations between Hermitian and Symmetric types in my test > problem. If you still have problems after the above fix - let us know. Satish From billy at dem.uminho.pt Tue Feb 21 15:22:33 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Tue, 21 Feb 2006 21:22:33 +0000 Subject: VecGetValue? Message-ID: <1140556953.43fb84992f0fc@serv-g1.ccom.uminho.pt> Hi, Why does PETSc have a VecSetValue and not a VecGetValue? Billy. From billy at dem.uminho.pt Tue Feb 21 18:55:36 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Wed, 22 Feb 2006 00:55:36 +0000 Subject: Matrix operation In-Reply-To: References: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> Message-ID: <1140569736.43fbb688a55d3@serv-g1.ccom.uminho.pt> Thank you very much for your answers. I was thinking why doesn't PETSc allow functions inside functions (like LASPack)? h += (Diag(A) - A) * u <=> AddAsgn_VV (h, Mul_QV (Sub_QQ (Diag_Q (A), A), u)); Billy Quoting Matthew Knepley : > Hong Zhang writes: > > I would recommend the later > > MatGetDiagonal(A, diag); > VecPointwiseMult(w, diag, u); > MatMult(A, u, v); > VecAXPY(w, -1.0, v); > > Matt > > > Billy, > > > > You can do > > MatGetDiagonal(A,diag); > > MatMult(A,u,&vtmp); > > VecGetArray(diag,diag_array); > > VecGetArray(vtmp,vtmp_array); > > VecGetArray(u,u_arrar); > > VecGetArray(h,h_array); > > VecGetLocalSize(h,&n); > > for (i=0; i > h_array[i] += diag_array[i]*u_array[i] - vtmp_array[i]; > > } > > VecRestoreArray(h,h_array); > > VecRestoreArray() > > ... > > > > or you can use VecPointwiseMult() got get > > Diag(A)*u. > > > > Hong > > > > On Mon, 20 Feb 2006 billy at dem.uminho.pt wrote: > > > >> > >> Hello, > >> > >> I would like to do the following matrix-vector operation in PETSc: > >> > >> h += (Diag(A) - A) * u > >> > >> where A is a matrix, Diag(A) is a matrix and h, u are vectors. It is the > >> multiplication of a matrix with diagonals set to zero with a vector. > >> > >> > >> How can I do this in PETSc? because MatGetDiagonal gives a vector not a > matrix. > >> > >> > >> Billy. > >> > >> > >> > > > > > > > > -- > "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec > Guiness > > From abdul-rahman at tu-harburg.de Wed Feb 22 05:51:06 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Wed, 22 Feb 2006 12:51:06 +0100 (MET) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: Satish, On Tue, 21 Feb 2006, Satish Balay wrote: > petsc-2.3.0 changelog has the following entry: > > >>> > -ksp_cg_Hermitian and -ksp_cg_symmetric have been changed > to -ksp_cg_type Hermitian or symmetric oops, thanks for pointing it out. It works as above. However, I still have problem with the below: i've changed the said file to: > parameter (KSP_CG_SYMMETRIC=0,KSP_CG_HERMITIAN=1) But still does not seem to be affected by my routine call: > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) Maybe I goofed up somewhere - will let you know more about it. Thanks much, Razi From balay at mcs.anl.gov Wed Feb 22 14:56:03 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 22 Feb 2006 14:56:03 -0600 (CST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: On Wed, 22 Feb 2006, abdul-rahman at tu-harburg.de wrote: > Satish, > > On Tue, 21 Feb 2006, Satish Balay wrote: > > > petsc-2.3.0 changelog has the following entry: > > > > >>> > > -ksp_cg_Hermitian and -ksp_cg_symmetric have been changed > > to -ksp_cg_type Hermitian or symmetric > > oops, thanks for pointing it out. It works as above. However, I still have > problem with the below: > > i've changed the said file to: > > parameter (KSP_CG_SYMMETRIC=0,KSP_CG_HERMITIAN=1) > > But still does not seem to be affected by my routine call: > > > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) > > Maybe I goofed up somewhere - will let you know more about it. You should be able to verify if the correct type is used with '-ksp_view' option. Satish From knepley at mcs.anl.gov Wed Feb 22 15:19:31 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Wed, 22 Feb 2006 15:19:31 -0600 Subject: Matrix operation In-Reply-To: <1140569736.43fbb688a55d3@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Wed, 22 Feb 2006 00:55:36 +0000") References: <1140470618.43fa335ad91bb@serv-g1.ccom.uminho.pt> <1140569736.43fbb688a55d3@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: > Thank you very much for your answers. > > I was thinking why doesn't PETSc allow functions inside functions (like LASPack)? > > h += (Diag(A) - A) * u <=> AddAsgn_VV (h, Mul_QV (Sub_QQ (Diag_Q (A), A), u)); Its an error reporting discipline since C does not allow exceptions. Matt > Billy > > > > Quoting Matthew Knepley : > >> Hong Zhang writes: >> >> I would recommend the later >> >> MatGetDiagonal(A, diag); >> VecPointwiseMult(w, diag, u); >> MatMult(A, u, v); >> VecAXPY(w, -1.0, v); >> >> Matt >> >> > Billy, >> > >> > You can do >> > MatGetDiagonal(A,diag); >> > MatMult(A,u,&vtmp); >> > VecGetArray(diag,diag_array); >> > VecGetArray(vtmp,vtmp_array); >> > VecGetArray(u,u_arrar); >> > VecGetArray(h,h_array); >> > VecGetLocalSize(h,&n); >> > for (i=0; i> > h_array[i] += diag_array[i]*u_array[i] - vtmp_array[i]; >> > } >> > VecRestoreArray(h,h_array); >> > VecRestoreArray() >> > ... >> > >> > or you can use VecPointwiseMult() got get >> > Diag(A)*u. >> > >> > Hong >> > >> > On Mon, 20 Feb 2006 billy at dem.uminho.pt wrote: >> > >> >> >> >> Hello, >> >> >> >> I would like to do the following matrix-vector operation in PETSc: >> >> >> >> h += (Diag(A) - A) * u >> >> >> >> where A is a matrix, Diag(A) is a matrix and h, u are vectors. It is the >> >> multiplication of a matrix with diagonals set to zero with a vector. >> >> >> >> >> >> How can I do this in PETSc? because MatGetDiagonal gives a vector not a >> matrix. >> >> >> >> >> >> Billy. >> >> >> >> >> >> >> > >> > >> > >> >> -- >> "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec >> Guiness >> >> > > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From knepley at mcs.anl.gov Wed Feb 22 15:20:46 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Wed, 22 Feb 2006 15:20:46 -0600 Subject: VecGetValue? In-Reply-To: <1140556953.43fb84992f0fc@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Tue, 21 Feb 2006 21:22:33 +0000") References: <1140556953.43fb84992f0fc@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: > Hi, > > Why does PETSc have a VecSetValue and not a VecGetValue? We could introduce one, but its not usually what you want since it would require a collective Assemble() call before you could use the value. You can accomplish the same thing with a scatter. Matt -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From abdul-rahman at tu-harburg.de Thu Feb 23 09:29:42 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Thu, 23 Feb 2006 16:29:42 +0100 (MET) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: On Wed, 22 Feb 2006, Satish Balay wrote: I know where I did wrong now: the call below: > > > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) should have come _after_: call KSPSetFromOptions(ksp,ierr) Otherwise it just goes with the default Hermitian even if I don't set the switch to Hermitian. > You should be able to verify if the correct type is used with > '-ksp_view' option. > > Satish > Thank you again Razi From bsmith at mcs.anl.gov Thu Feb 23 10:15:31 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 23 Feb 2006 08:15:31 -0800 (PST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: Satish, Can you please check this? We should be able to call KSPCGSetType() either before or afterwards. It could be that KSPSetFromOptions_CG() wrongly resets to the default value. I looked at the code and didn't see any wrong logic. Probably it may be easiest to just run in the debugger and check if it wrongly reset. Thanks Barry On Thu, 23 Feb 2006, abdul-rahman at tu-harburg.de wrote: > > > On Wed, 22 Feb 2006, Satish Balay wrote: > > I know where I did wrong now: the call below: > >>>>> call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) > > should have come _after_: > > call KSPSetFromOptions(ksp,ierr) > > Otherwise it just goes with the default Hermitian even if I don't set the > switch to Hermitian. > > >> You should be able to verify if the correct type is used with >> '-ksp_view' option. >> >> Satish >> > > Thank you again > > > Razi > > From balay at mcs.anl.gov Thu Feb 23 14:50:44 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 23 Feb 2006 14:50:44 -0600 (CST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: I just tried the following [with both c & fortran examples ksp/examples/tutorials/ex2.c, ex2f.F] And I get the expected behavior. Maybe KSPSetType() was not called before KSPCGSetType()? Satish -------- >>> call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) call KSPSetType(ksp,KSPCG,ierr) call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr) call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) >>> asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials>./ex2f -ksp_view KSP Object: type: cg CG or CGNE: variant SYMMETRIC maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: ilu ILU: 0 levels of fill ILU: max fill ratio allocated 1 ILU: tolerance for zero pivot 1e-12 out-of-place factorization matrix ordering: natural Factored matrix follows Matrix Object: type=aij, rows=9, cols=9 total: nonzeros=33, allocated nonzeros=45 not using I-node routines linear system matrix = precond matrix: Matrix Object: type=aij, rows=9, cols=9 total: nonzeros=33, allocated nonzeros=45 not using I-node routines Norm of error 0.1215E-05 iterations 4 asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials> On Thu, 23 Feb 2006, Barry Smith wrote: > > Satish, > > Can you please check this? We should be able to > call KSPCGSetType() either before or afterwards. It could > be that KSPSetFromOptions_CG() wrongly resets to the default > value. I looked at the code and didn't see any wrong logic. > Probably it may be easiest to just run in the debugger and > check if it wrongly reset. > > Thanks > > Barry > > > On Thu, 23 Feb 2006, abdul-rahman at tu-harburg.de wrote: > > > > > > > On Wed, 22 Feb 2006, Satish Balay wrote: > > > > I know where I did wrong now: the call below: > > > > > > > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) > > > > should have come _after_: > > > > call KSPSetFromOptions(ksp,ierr) > > > > Otherwise it just goes with the default Hermitian even if I don't set the > > switch to Hermitian. > > > > > > > You should be able to verify if the correct type is used with > > > '-ksp_view' option. > > > > > > Satish > > > > > > > Thank you again > > > > > > Razi > > > > > > From bsmith at mcs.anl.gov Thu Feb 23 16:03:19 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 23 Feb 2006 14:03:19 -0800 (PST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: What about calling SetFromOptions after the call to KSPCGSetType()? Thanks Barry On Thu, 23 Feb 2006, Satish Balay wrote: > > I just tried the following [with both c & fortran examples > ksp/examples/tutorials/ex2.c, ex2f.F] And I get the expected behavior. > > Maybe KSPSetType() was not called before KSPCGSetType()? > > Satish > > -------- > >>>> > call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) > call KSPSetType(ksp,KSPCG,ierr) > call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr) > call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) >>>> > > asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials>./ex2f -ksp_view > KSP Object: > type: cg > CG or CGNE: variant SYMMETRIC > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: ilu > ILU: 0 levels of fill > ILU: max fill ratio allocated 1 > ILU: tolerance for zero pivot 1e-12 > out-of-place factorization > matrix ordering: natural > Factored matrix follows > Matrix Object: > type=aij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > linear system matrix = precond matrix: > Matrix Object: > type=aij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > Norm of error 0.1215E-05 iterations 4 > asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials> > > > On Thu, 23 Feb 2006, Barry Smith wrote: > >> >> Satish, >> >> Can you please check this? We should be able to >> call KSPCGSetType() either before or afterwards. It could >> be that KSPSetFromOptions_CG() wrongly resets to the default >> value. I looked at the code and didn't see any wrong logic. >> Probably it may be easiest to just run in the debugger and >> check if it wrongly reset. >> >> Thanks >> >> Barry >> >> >> On Thu, 23 Feb 2006, abdul-rahman at tu-harburg.de wrote: >> >>> >>> >>> On Wed, 22 Feb 2006, Satish Balay wrote: >>> >>> I know where I did wrong now: the call below: >>> >>>>>>> call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) >>> >>> should have come _after_: >>> >>> call KSPSetFromOptions(ksp,ierr) >>> >>> Otherwise it just goes with the default Hermitian even if I don't set the >>> switch to Hermitian. >>> >>> >>>> You should be able to verify if the correct type is used with >>>> '-ksp_view' option. >>>> >>>> Satish >>>> >>> >>> Thank you again >>> >>> >>> Razi >>> >>> >> >> > > From balay at mcs.anl.gov Thu Feb 23 16:10:31 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 23 Feb 2006 16:10:31 -0600 (CST) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: Sorry - the test code had SetFromOptions after KSPCGSetType() Satish --- call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) call KSPSetType(ksp,KSPCG,ierr) call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr) call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) call KSPSetFromOptions(ksp,ierr) call KSPSolve(ksp,b,x,ierr) On Thu, 23 Feb 2006, Barry Smith wrote: > > What about calling SetFromOptions after the call to KSPCGSetType()? > > Thanks > Barry > > > On Thu, 23 Feb 2006, Satish Balay wrote: > > > > > I just tried the following [with both c & fortran examples > > ksp/examples/tutorials/ex2.c, ex2f.F] And I get the expected behavior. > > > > Maybe KSPSetType() was not called before KSPCGSetType()? > > > > Satish > > > > -------- > > > > > > > > > call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) > > call KSPSetType(ksp,KSPCG,ierr) > > call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr) > > call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) > > > > > > > > > asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials>./ex2f -ksp_view > > KSP Object: > > type: cg > > CG or CGNE: variant SYMMETRIC > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > > left preconditioning > > PC Object: > > type: ilu > > ILU: 0 levels of fill > > ILU: max fill ratio allocated 1 > > ILU: tolerance for zero pivot 1e-12 > > out-of-place factorization > > matrix ordering: natural > > Factored matrix follows > > Matrix Object: > > type=aij, rows=9, cols=9 > > total: nonzeros=33, allocated nonzeros=45 > > not using I-node routines > > linear system matrix = precond matrix: > > Matrix Object: > > type=aij, rows=9, cols=9 > > total: nonzeros=33, allocated nonzeros=45 > > not using I-node routines > > Norm of error 0.1215E-05 iterations 4 > > asterix:/home/balay/spetsc/src/ksp/ksp/examples/tutorials> > > > > > > On Thu, 23 Feb 2006, Barry Smith wrote: > > > > > > > > Satish, > > > > > > Can you please check this? We should be able to > > > call KSPCGSetType() either before or afterwards. It could > > > be that KSPSetFromOptions_CG() wrongly resets to the default > > > value. I looked at the code and didn't see any wrong logic. > > > Probably it may be easiest to just run in the debugger and > > > check if it wrongly reset. > > > > > > Thanks > > > > > > Barry > > > > > > > > > On Thu, 23 Feb 2006, abdul-rahman at tu-harburg.de wrote: > > > > > > > > > > > > > > > On Wed, 22 Feb 2006, Satish Balay wrote: > > > > > > > > I know where I did wrong now: the call below: > > > > > > > > > > > > call KSPCGSetType( ksp, KSP_CG_SYMMETRIC, ierr ) > > > > > > > > should have come _after_: > > > > > > > > call KSPSetFromOptions(ksp,ierr) > > > > > > > > Otherwise it just goes with the default Hermitian even if I don't set > > > > the > > > > switch to Hermitian. > > > > > > > > > > > > > You should be able to verify if the correct type is used with > > > > > '-ksp_view' option. > > > > > > > > > > Satish > > > > > > > > > > > > > Thank you again > > > > > > > > > > > > Razi > > > > > > > > > > > > > > > > > > > > From billy at dem.uminho.pt Thu Feb 23 18:07:38 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Fri, 24 Feb 2006 00:07:38 +0000 Subject: final residual of iterative solver? Message-ID: <1140739658.43fe4e4ac5180@serv-g1.ccom.uminho.pt> How do you retrieve the final residual of the iterative solver? Billy. From balay at mcs.anl.gov Thu Feb 23 18:12:17 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 23 Feb 2006 18:12:17 -0600 (CST) Subject: final residual of iterative solver? In-Reply-To: <1140739658.43fe4e4ac5180@serv-g1.ccom.uminho.pt> References: <1140739658.43fe4e4ac5180@serv-g1.ccom.uminho.pt> Message-ID: Call KSPGetResidualNorm() after KSPSolve() Satish On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > > > How do you retrieve the final residual of the iterative solver? > > > Billy. > > From abdul-rahman at tu-harburg.de Fri Feb 24 05:24:32 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Fri, 24 Feb 2006 12:24:32 +0100 (MET) Subject: New changes for CG set type in 2.3.1-p7 ? In-Reply-To: References: Message-ID: Satish, On Thu, 23 Feb 2006, Satish Balay wrote: > > I just tried the following [with both c & fortran examples > ksp/examples/tutorials/ex2.c, ex2f.F] And I get the expected behavior. > > Maybe KSPSetType() was not called before KSPCGSetType()? You're right. I didn't have KSPSetType beforehand. > Satish > > -------- > > >>> > call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) > call KSPSetType(ksp,KSPCG,ierr) > call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr) > call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) > >>> Thanks! Razi From abdul-rahman at tu-harburg.de Fri Feb 24 07:14:31 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Fri, 24 Feb 2006 14:14:31 +0100 (MET) Subject: PETSc lib performance tuning Message-ID: Hi all, I have a couple questions about the PETSc performance: Does having ATLAS ( math-atlas.sourceforge.net ) help ? If so how do I hook it to PETSc ? How do I best customize the compiler flags? Should I edit the variables directly in petscconf, or can I do it with configure ? Thanks, Razi From berend at chalmers.se Fri Feb 24 07:28:20 2006 From: berend at chalmers.se (Berend van Wachem) Date: Fri, 24 Feb 2006 14:28:20 +0100 Subject: PETSc lib performance tuning In-Reply-To: References: Message-ID: <43FF09F4.5000302@chalmers.se> Hi Razi, I have tested both atlas and other blas/lapack and found small effects on linux. For using ATLAS with petsc, you configure ATLAS, build it, and then specify when you configure petsc --with-blas-lapack-dir= where your ATLAS libs are. In my case, I use --with-blas-lapack-dir=/usr/local/ATLAS/Linux_X86_64/ Good luck, Berend. > > I have a couple questions about the PETSc performance: > > Does having ATLAS ( math-atlas.sourceforge.net ) help ? If so how do I > hook it to PETSc ? > > How do I best customize the compiler flags? Should I edit the > variables directly in petscconf, or can I do it with configure ? > > > Thanks, > > Razi > From abdul-rahman at tu-harburg.de Fri Feb 24 09:09:11 2006 From: abdul-rahman at tu-harburg.de (abdul-rahman at tu-harburg.de) Date: Fri, 24 Feb 2006 16:09:11 +0100 (MET) Subject: PETSc lib performance tuning In-Reply-To: <43FF09F4.5000302@chalmers.se> References: <43FF09F4.5000302@chalmers.se> Message-ID: I asked: > > How do I best customize the compiler flags? Should I edit the > > variables directly in petscconf, or can I do it with configure ? OK. I found this out. It's quite obvious :) ./config/configure.py --help gives _lots_ of helpful info. On Fri, 24 Feb 2006, Berend van Wachem wrote: > Hi Razi, > > I have tested both atlas and other blas/lapack and found small effects > on linux. Thanks a lot Berend. If that's so I'll leave it out for now. Razi From balay at mcs.anl.gov Fri Feb 24 10:16:40 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 24 Feb 2006 10:16:40 -0600 (CST) Subject: PETSc lib performance tuning In-Reply-To: References: Message-ID: On Fri, 24 Feb 2006, abdul-rahman at tu-harburg.de wrote: > How do I best customize the compiler flags? Should I edit the > variables directly in petscconf, or can I do it with configure ? To compile with optimizationflags - you can use the following configure options: --with-debugging=0 COPTFLAGS=-O3 FOPTFLAGS=-O3 CXXOPTFLAGS=-O3 etc.. Satish From billy at dem.uminho.pt Fri Feb 24 11:24:43 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Fri, 24 Feb 2006 17:24:43 +0000 Subject: VecGetValues and tutorail? Message-ID: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> Hello, "VecGetValues - Gets values from certain locations of a vector. Currently can only get values on the same processor" How can I get a value of the vector belonging to a different processor? I am trying to learn how to adapt my code to work in parallel. Have looked at src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on how to use petsc with unstructured meshes. Is there any simple step-by-step tutorial? Billy. From balay at mcs.anl.gov Fri Feb 24 11:56:45 2006 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 24 Feb 2006 11:56:45 -0600 (CST) Subject: VecGetValues and tutorail? In-Reply-To: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> Message-ID: On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > > Hello, > > > "VecGetValues - Gets values from certain locations of a vector. Currently can > only get values on the same processor" > > How can I get a value of the vector belonging to a different processor? > > > I am trying to learn how to adapt my code to work in parallel. Have looked at > src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on how to use > petsc with unstructured meshes. Is there any simple step-by-step tutorial? Basically - you'll have to use VecScatter routines to specify the communication required. Then use VecScatterBegin/End() to invoke this communication. Some of this info is in the tutorial slides [check arround slide 150] in: http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf Not usre if there is an unstructured grid example code. [but there are examples using VecScatters in src/dm/da/examples/tutorials] Satish From billy at dem.uminho.pt Sun Feb 26 17:00:25 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sun, 26 Feb 2006 23:00:25 +0000 Subject: VecGetValues and tutorail? In-Reply-To: References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> Message-ID: <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> Hi, I want to parallelize an unstructured finite volume CFD code. The code is second-order in space, so I only need one layer of overlap between domains. The time advancement is implicit. I renumbered the cells in my application to respect PETSc ordering. Do I need to use AO: Application Orderings? Or is this optional? P0 P1 ---------- ---------- | 0| 1| 2| <-> | 9|10|11| | 3| 4| 5| <-> |12|13|14| | 6| 7| 8| <-> |15|16|17| ---------- ---------- Should I use VecCreateGhost to include the ghost cells or another type of vector like VecCreateSeq? P0 P1 ------------- ------------- | 0| 1| 2| 9| <-> |18| 9|10|11| | 3| 4| 5|10| <-> |19|12|13|14| | 6| 7| 8|11| <-> |21|15|16|17| ------------- ------------- Do I need to create IS: Index Sets? P0 P1 ------------- ------------- | 0| 1| 2| 9| <-> | 9| 0| 1| 2| | 3| 4| 5|10| <-> |10| 3| 4| 5| | 6| 7| 8|11| <-> |11| 6| 7| 8| ------------- ------------- Then I should apply VecScatter routines to update ghost cells, right? Sorry for asking so many questions, but this is all very confusing to me. Billy. Quoting Satish Balay : > On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > > > > > Hello, > > > > > > "VecGetValues - Gets values from certain locations of a vector. Currently > can > > only get values on the same processor" > > > > How can I get a value of the vector belonging to a different processor? > > > > > > I am trying to learn how to adapt my code to work in parallel. Have looked > at > > src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on how to > use > > petsc with unstructured meshes. Is there any simple step-by-step > tutorial? > > Basically - you'll have to use VecScatter routines to specify the > communication required. Then use VecScatterBegin/End() to invoke this > communication. Some of this info is in the tutorial slides [check > arround slide 150] in: > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > > Not usre if there is an unstructured grid example code. [but there are > examples using VecScatters in src/dm/da/examples/tutorials] > > Satish > > From bsmith at mcs.anl.gov Sun Feb 26 17:07:38 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 26 Feb 2006 15:07:38 -0800 (PST) Subject: VecGetValues and tutorail? In-Reply-To: <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> Message-ID: If you are using a structured grid you should use DA's to manage all this. You won't need to monkey with IS or VectorScatter or VecCreateGhost. Otherwise, ... see below.... On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > Hi, > > I want to parallelize an unstructured finite volume CFD code. The code is > second-order in space, so I only need one layer of overlap between domains. The > time advancement is implicit. > > I renumbered the cells in my application to respect PETSc ordering. > Do I need to use AO: Application Orderings? Or is this optional? You do not have to use AO to manage getting the "PETSc style" ordering, you can do it anyway you like, so long as you have a "PETSc style" ordering. > > P0 P1 > ---------- ---------- > | 0| 1| 2| <-> | 9|10|11| > | 3| 4| 5| <-> |12|13|14| > | 6| 7| 8| <-> |15|16|17| > ---------- ---------- > > Should I use VecCreateGhost to include the ghost cells or another type of vector > like VecCreateSeq? Likely you will wish to use VecCreateGhost() > > P0 P1 > ------------- ------------- > | 0| 1| 2| 9| <-> |18| 9|10|11| > | 3| 4| 5|10| <-> |19|12|13|14| > | 6| 7| 8|11| <-> |21|15|16|17| > ------------- ------------- > > Do I need to create IS: Index Sets? If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), VecGhostGetLocalForm() etc) you will not need to deal with index sets. > > P0 P1 > ------------- ------------- > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > | 3| 4| 5|10| <-> |10| 3| 4| 5| > | 6| 7| 8|11| <-> |11| 6| 7| 8| > ------------- ------------- > > Then I should apply VecScatter routines to update ghost cells, right? If you use VecCreateGhost() then it manages the scatters during the VecGhostUpdateBegin/End() for you; you do not need to use IS or VecScatter. If you do not use VecCreateGhost() then you need to use IS followed by VecScatterCreate() to build the scatter data structures and VecScatterBegin/End() to update the ghost values. Barry > > Sorry for asking so many questions, but this is all very confusing to me. > > > Billy. > > > Quoting Satish Balay : > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: >> >>> >>> Hello, >>> >>> >>> "VecGetValues - Gets values from certain locations of a vector. Currently >> can >>> only get values on the same processor" >>> >>> How can I get a value of the vector belonging to a different processor? >>> >>> >>> I am trying to learn how to adapt my code to work in parallel. Have looked >> at >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on how to >> use >>> petsc with unstructured meshes. Is there any simple step-by-step >> tutorial? >> >> Basically - you'll have to use VecScatter routines to specify the >> communication required. Then use VecScatterBegin/End() to invoke this >> communication. Some of this info is in the tutorial slides [check >> arround slide 150] in: >> http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf >> >> Not usre if there is an unstructured grid example code. [but there are >> examples using VecScatters in src/dm/da/examples/tutorials] >> >> Satish >> >> > > > From billy at dem.uminho.pt Sun Feb 26 17:15:12 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Sun, 26 Feb 2006 23:15:12 +0000 Subject: VecGetValues and tutorail? In-Reply-To: References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> Message-ID: <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> Thank you very much for your help. :) I think I will use VecCreateGhost() then. Billy. Quoting Barry Smith : > > If you are using a structured grid you should use DA's to > manage all this. You won't need to monkey with IS or VectorScatter > or VecCreateGhost. Otherwise, ... see below.... > > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > > > Hi, > > > > I want to parallelize an unstructured finite volume CFD code. The code is > > second-order in space, so I only need one layer of overlap between domains. > The > > time advancement is implicit. > > > > I renumbered the cells in my application to respect PETSc ordering. > > Do I need to use AO: Application Orderings? Or is this optional? > > You do not have to use AO to manage getting the "PETSc style" ordering, > you can do it anyway you like, so long as you have a "PETSc style" > ordering. > > > > > P0 P1 > > ---------- ---------- > > | 0| 1| 2| <-> | 9|10|11| > > | 3| 4| 5| <-> |12|13|14| > > | 6| 7| 8| <-> |15|16|17| > > ---------- ---------- > > > > Should I use VecCreateGhost to include the ghost cells or another type of > vector > > like VecCreateSeq? > > Likely you will wish to use VecCreateGhost() > > > > > P0 P1 > > ------------- ------------- > > | 0| 1| 2| 9| <-> |18| 9|10|11| > > | 3| 4| 5|10| <-> |19|12|13|14| > > | 6| 7| 8|11| <-> |21|15|16|17| > > ------------- ------------- > > > > Do I need to create IS: Index Sets? > > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), > VecGhostGetLocalForm() etc) you will not need to deal with index sets. > > > > P0 P1 > > ------------- ------------- > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > > | 3| 4| 5|10| <-> |10| 3| 4| 5| > > | 6| 7| 8|11| <-> |11| 6| 7| 8| > > ------------- ------------- > > > > Then I should apply VecScatter routines to update ghost cells, right? > > If you use VecCreateGhost() then it manages the scatters during the > VecGhostUpdateBegin/End() for you; you do not need to use IS or VecScatter. > > If you do not use VecCreateGhost() then you need to use IS followed by > VecScatterCreate() to build the scatter data structures and > VecScatterBegin/End() to update the ghost values. > > Barry > > > > > Sorry for asking so many questions, but this is all very confusing to me. > > > > > > Billy. > > > > > > Quoting Satish Balay : > > > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > >> > >>> > >>> Hello, > >>> > >>> > >>> "VecGetValues - Gets values from certain locations of a vector. > Currently > >> can > >>> only get values on the same processor" > >>> > >>> How can I get a value of the vector belonging to a different processor? > >>> > >>> > >>> I am trying to learn how to adapt my code to work in parallel. Have > looked > >> at > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on how > to > >> use > >>> petsc with unstructured meshes. Is there any simple step-by-step > >> tutorial? > >> > >> Basically - you'll have to use VecScatter routines to specify the > >> communication required. Then use VecScatterBegin/End() to invoke this > >> communication. Some of this info is in the tutorial slides [check > >> arround slide 150] in: > >> > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > >> > >> Not usre if there is an unstructured grid example code. [but there are > >> examples using VecScatters in src/dm/da/examples/tutorials] > >> > >> Satish > >> > >> > > > > > > > > From nilserik at gmail.com Mon Feb 27 08:07:51 2006 From: nilserik at gmail.com (=?ISO-8859-1?Q?Nils_Erik_Svang=E5rd?=) Date: Mon, 27 Feb 2006 15:07:51 +0100 Subject: SNES Problem Message-ID: <82f936c20602270607t775d7e03o6a37afa4c0b3d2a8@mail.gmail.com> Hi all, I have problems solving some CFD problems using SNES and my custom back-euler. I have 7 equations that I want to solve. First my FormFunction copies the values from the Vec that SNES uses to the variables that my code use: RO(L)=xx(1,L) RU(L)=xx(2,L) RV(L)=xx(3,L) RW(L)=xx(4,L) ET(L)=xx(5,L) RQ(L)= xx(6,L) REPS(L)=xx(7,L) Here I also print the values of ET(2) for debugging: write(6,*)"ET(2) = xx(5,2) : ",ET(2)," = ",xx(5,2) Then I get the fluxes by calling custom functions call AUXVR call VGRAD call FLUX call KESRC Then I perform back-euler save the new value to so that I can use it in the next iterationi (TSF(L) is Time step function, which is set by hand and is the same for all L): ff(1,L) = RO(L)-OLD(1,L)-TSF(L)*DRO(L) ff(2,L) = RU(L)-OLD(2,L)-TSF(L)*DRU(L) ff(3,L) = RV(L)-OLD(3,L)-TSF(L)*DRV(L) ff(4,L) = RW(L)-OLD(4,L)-TSF(L)*DRW(L) ff(5,L) = ET(L)-OLD(5,L)-TSF(L)*DET(L) ff(6,L) = RQ(L)-OLD(6,L)-TSF(L)*DRQ(L) ff(7,L) = REPS(L)-OLD(7,L)-TSF(L)*DREPS(L) And save the new value of RO-REPS to use in the next iteration of back-euler: OLD(1,L)=RO(L) OLD(2,L)=RU(L) OLD(3,L)=RV(L) OLD(4,L)=RW(L) OLD(5,L)=ET(L) OLD(6,L)=RQ(L) OLD(7,L)=REPS(L) Here I print the values of L=2 as above for debugging: write(6,*)ff(5,2)," = ",ET(2),"-",OLD(5,2),"-",TSF(2),"*",DET(2) The program exit abnormaly after 33 runs of the FormFunction. The cause of this is that AUXVR tries to perform sqrt(ET(2)) when ET(2) is negative, ET is the total energy and should never be negative. It seems that all of a sudden the PETSc SNES solver decides to supply the FormFunction with a negative ET value. I need help understanding why and how to fix it (if it is fixable). Here is the output of: write(6,*)"ET(2) = xx(5,2) : ",ET(2)," = ",xx(5,2) Which is in the beginning of my FormFunction. And: write(6,*)ff(5,2)," = ",ET(2),"-",OLD(5,2),"-",TSF(2),"*",DET(2) Which is in the end of my Formfunction. 1. ET(2) = xx(5,2) : 253250. = 253250.000000000 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 2. ET(2) = xx(5,2) : 253250. = 253250.000010259 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 3. ET(2) = xx(5,2) : 253250. = 253249.999999988 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4132419437260 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941324E+08 4. ET(2) = xx(5,2) : 253250. = 253250.000048432 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 5. ET(2) = xx(5,2) : 253250. = 253249.999999872 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4177283437784 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941773E+08 6. ET(2) = xx(5,2) : 253250. = 253250.000071113 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 7. ET(2) = xx(5,2) : 253250. = 253249.999980115 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4453347441010 = 253250. - 253250.000000000 - 1.000000E-07 * -2.944533E+08 8. ET(2) = xx(5,2) : 253250. = 253250.000037298 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4082723436679 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940827E+08 9. ET(2) = xx(5,2) : 253250. = 253249.999942706 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4082723436679 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940827E+08 10. ET(2) = xx(5,2) : 253250. = 253249.999948703 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 11. ET(2) = xx(5,2) : 253250. = 253250.000007808 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.7283811474088 = 253250. - 253250.000000000 - 1.000000E-07 * -2.972838E+08 12. ET(2) = xx(5,2) : 253250. = 253250.000018146 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 13. ET(2) = xx(5,2) : 253250. = 253250.000001865 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 28.3742819315846 = 253250. - 253250.000000000 - 1.000000E-07 * -2.837428E+08 14. ET(2) = xx(5,2) : 253250. = 253249.999997296 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 15. ET(2) = xx(5,2) : 253250. = 253250.000000049 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 30.6548963582361 = 253250. - 253250.000000000 - 1.000000E-07 * -3.065490E+08 16. ET(2) = xx(5,2) : 253250. = 253250.000022870 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 17. ET(2) = xx(5,2) : 253250. = 253249.999999716 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.5326051451209 = 253250. - 253250.000000000 - 1.000000E-07 * -2.953260E+08 18. ET(2) = xx(5,2) : 253250. = 253250.000074328 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 19. ET(2) = xx(5,2) : 253250. = 253250.000049809 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 20. ET(2) = xx(5,2) : 253250. = 253250.000019253 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 21. ET(2) = xx(5,2) : 253250. = 253250.000000189 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.8851075492403 = 253250. - 253250.000000000 - 1.000000E-07 * -2.988511E+08 22. ET(2) = xx(5,2) : 253250. = 253249.999997897 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 23. ET(2) = xx(5,2) : 253250. = 253249.999975142 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.3961539435263 = 253250. - 253250.000000000 - 1.000000E-07 * -2.939615E+08 24. ET(2) = xx(5,2) : 253250. = 253250.000018324 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 25. ET(2) = xx(5,2) : 253250. = 253249.999987073 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 26. ET(2) = xx(5,2) : 253250. = 253249.999980982 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 27. ET(2) = xx(5,2) : 253250. = 253249.999994645 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 28. ET(2) = xx(5,2) : 253250. = 253249.999962364 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 29. ET(2) = xx(5,2) : 253250. = 253249.999999921 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 30. ET(2) = xx(5,2) : 253250. = 253250.000000058 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 28.4019971319085 = 253250. - 253250.000000000 - 1.000000E-07 * -2.840200E+08 31. ET(2) = xx(5,2) : 253250. = 253250.000040348 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 32. ET(2) = xx(5,2) : 253250. = 253250.000005245 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 33. ET(2) = xx(5,2) : 253250. = 253250.000005245 ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 34. ET(2) = xx(5,2) : -848141. = -848141.388090847 -------------------------------------------------------------------------- Petsc Release Version 2.3.0, Patch 32, April, 26, 2005 See docs/changes/index.html for recent updates. See docs/faq.html for hints about trouble shooting. See docs/index.html for manual pages. ----------------------------------------------------------------------- ./SNESG3D2 on a linux-gnu named sethnx004.vac.com by yy26539 Mon Feb 27 14:21:13 2006 Libraries linked from /home/yy26539/work/NISSE/petsc-2.3.0/lib/linux-gnu Configure run at Thu Oct 13 08:23:56 2005 Configure options --with-cc=gcc --with-fc="f77 -N109" --download-mpich=1 --download-mpich-pm=gforker --download-f-blas-lapack=1 --with-shared=0 ----------------------------------------------------------------------- [0]PETSC ERROR: Caught signal number 8 FPE: Floating Point Exception,probably divide by zero [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: --------------- Stack Frames --------------- [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] SNES user function line 0 unknownunknown [0]PETSC ERROR: [0] SNESComputeFunction line 788 src/snes/interface/snes.c [0]PETSC ERROR: [0] SNESLineSearchCubic line 514 src/snes/impls/ls/ls.c [0]PETSC ERROR: -------------------------------------------- [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ! Suggestions and tips are very welcome! /nisse -- Nils-Erik Svang?rd MSN: schweingaard at hotmail.com Skype: schweingaard From bsmith at mcs.anl.gov Mon Feb 27 09:07:18 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 27 Feb 2006 09:07:18 -0600 (CST) Subject: SNES Problem In-Reply-To: <82f936c20602270607t775d7e03o6a37afa4c0b3d2a8@mail.gmail.com> References: <82f936c20602270607t775d7e03o6a37afa4c0b3d2a8@mail.gmail.com> Message-ID: SNES works by computing p = -approxinv(J)*F(uold) and then does a line search on unew = uold + lambda*p to get the new u. First it uses a test value of 1 for lambda so it tries to compute F(uold + p). It is possible that uold + p has some "non-physical" values in it. There are two ways you can try handling it: 1) Before doing the linesearch SNES calls a "precheck" function, that can change the step if it decides there is a problem with the step (like it is too long). You can provide your own precheck function with SNESLineSearchSetPreCheck() it could, for example, shrink the Newton direction to make it remain physical. or 2) when your form function detects an illegal value it calls SETERRQ(PETSC_ERR_ARG_DOMAIN,"Nonphysical function input"); Note: this is before it takes the square root of the negative number, so check the number and call the error before calling SETERRQ(). (If using Fortran then simply set ierr = PETSC_ERR_ARG_DOMAIN and then immediately return). Then replace your call to SNESSolve with ierr = PetscExceptionTry1(SNESSolve(snes,b,x),PETSC_ERR_ARG_DOMAIN); if (PetscExceptionCaught(ierr,PETSC_ERR_ARG_DOMAIN)) { /* this means your function found a non-physical value so cut your time-step and continue through the loop again. */ Put code to do this here. } If using fortran then do call PetscExceptionPush(PETSC_ERR_ARG_DOMAIN,ierr) call SNESSolve(snes,b,x,ierr) call PetscExceptionPop(PETSC_ERR_ARG_DOMAIN,anotherierr) if (ierr == PETSC_ERR_ARG_DOMAIN) then non-physical value so cut the time-step and try again else ! everything is normal so take the next time-step Barry The Fortran interface may be missing PetscExceptionPush() and Pop() if so let us know and we'll provide the patch. On Mon, 27 Feb 2006, Nils Erik Svang?rd wrote: > Hi all, > > I have problems solving some CFD problems using SNES and my custom > back-euler. I have 7 equations that I want to solve. > > First my FormFunction copies the values from the Vec that SNES uses to > the variables that my code use: > > RO(L)=xx(1,L) > RU(L)=xx(2,L) > RV(L)=xx(3,L) > RW(L)=xx(4,L) > ET(L)=xx(5,L) > RQ(L)= xx(6,L) > REPS(L)=xx(7,L) > > Here I also print the values of ET(2) for debugging: > > write(6,*)"ET(2) = xx(5,2) : ",ET(2)," = ",xx(5,2) > > Then I get the fluxes by calling custom functions > > call AUXVR > call VGRAD > call FLUX > call KESRC > > Then I perform back-euler save the new value to so that I can use it > in the next iterationi (TSF(L) is Time step function, which is set by > hand and is the same for all L): > > ff(1,L) = RO(L)-OLD(1,L)-TSF(L)*DRO(L) > ff(2,L) = RU(L)-OLD(2,L)-TSF(L)*DRU(L) > ff(3,L) = RV(L)-OLD(3,L)-TSF(L)*DRV(L) > ff(4,L) = RW(L)-OLD(4,L)-TSF(L)*DRW(L) > ff(5,L) = ET(L)-OLD(5,L)-TSF(L)*DET(L) > ff(6,L) = RQ(L)-OLD(6,L)-TSF(L)*DRQ(L) > ff(7,L) = REPS(L)-OLD(7,L)-TSF(L)*DREPS(L) > > And save the new value of RO-REPS to use in the next iteration of back-euler: > > OLD(1,L)=RO(L) > OLD(2,L)=RU(L) > OLD(3,L)=RV(L) > OLD(4,L)=RW(L) > OLD(5,L)=ET(L) > OLD(6,L)=RQ(L) > OLD(7,L)=REPS(L) > > Here I print the values of L=2 as above for debugging: > > write(6,*)ff(5,2)," = ",ET(2),"-",OLD(5,2),"-",TSF(2),"*",DET(2) > > > > The program exit abnormaly after 33 runs of the FormFunction. The > cause of this is that AUXVR tries to perform sqrt(ET(2)) when ET(2) is > negative, ET is the total energy and should never be negative. It > seems that all of a sudden the PETSc SNES solver decides to supply the > FormFunction with a negative ET value. I need help understanding why > and how to fix it (if it is fixable). > > Here is the output of: > write(6,*)"ET(2) = xx(5,2) : ",ET(2)," = ",xx(5,2) > Which is in the beginning of my FormFunction. > And: > write(6,*)ff(5,2)," = ",ET(2),"-",OLD(5,2),"-",TSF(2),"*",DET(2) > Which is in the end of my Formfunction. > > 1. ET(2) = xx(5,2) : 253250. = 253250.000000000 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 2. ET(2) = xx(5,2) : 253250. = 253250.000010259 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 3. ET(2) = xx(5,2) : 253250. = 253249.999999988 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4132419437260 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941324E+08 > 4. ET(2) = xx(5,2) : 253250. = 253250.000048432 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 5. ET(2) = xx(5,2) : 253250. = 253249.999999872 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4177283437784 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941773E+08 > 6. ET(2) = xx(5,2) : 253250. = 253250.000071113 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 7. ET(2) = xx(5,2) : 253250. = 253249.999980115 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4453347441010 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.944533E+08 > 8. ET(2) = xx(5,2) : 253250. = 253250.000037298 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4082723436679 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940827E+08 > 9. ET(2) = xx(5,2) : 253250. = 253249.999942706 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4082723436679 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940827E+08 > 10. ET(2) = xx(5,2) : 253250. = 253249.999948703 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 11. ET(2) = xx(5,2) : 253250. = 253250.000007808 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.7283811474088 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.972838E+08 > 12. ET(2) = xx(5,2) : 253250. = 253250.000018146 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 13. ET(2) = xx(5,2) : 253250. = 253250.000001865 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 28.3742819315846 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.837428E+08 > 14. ET(2) = xx(5,2) : 253250. = 253249.999997296 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 15. ET(2) = xx(5,2) : 253250. = 253250.000000049 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 30.6548963582361 = > 253250. - 253250.000000000 - 1.000000E-07 * -3.065490E+08 > 16. ET(2) = xx(5,2) : 253250. = 253250.000022870 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 17. ET(2) = xx(5,2) : 253250. = 253249.999999716 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.5326051451209 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.953260E+08 > 18. ET(2) = xx(5,2) : 253250. = 253250.000074328 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 > 19. ET(2) = xx(5,2) : 253250. = 253250.000049809 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 > 20. ET(2) = xx(5,2) : 253250. = 253250.000019253 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 21. ET(2) = xx(5,2) : 253250. = 253250.000000189 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.8851075492403 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.988511E+08 > 22. ET(2) = xx(5,2) : 253250. = 253249.999997897 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 > 23. ET(2) = xx(5,2) : 253250. = 253249.999975142 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.3961539435263 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.939615E+08 > 24. ET(2) = xx(5,2) : 253250. = 253250.000018324 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 > 25. ET(2) = xx(5,2) : 253250. = 253249.999987073 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 > 26. ET(2) = xx(5,2) : 253250. = 253249.999980982 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 > 27. ET(2) = xx(5,2) : 253250. = 253249.999994645 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 28. ET(2) = xx(5,2) : 253250. = 253249.999962364 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4165187437643 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941652E+08 > 29. ET(2) = xx(5,2) : 253250. = 253249.999999921 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 30. ET(2) = xx(5,2) : 253250. = 253250.000000058 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 28.4019971319085 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.840200E+08 > 31. ET(2) = xx(5,2) : 253250. = 253250.000040348 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4085059436707 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.940851E+08 > 32. ET(2) = xx(5,2) : 253250. = 253250.000005245 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 33. ET(2) = xx(5,2) : 253250. = 253250.000005245 > ff(5,2) = ET(2)-OLD(5,2)-TSF(2)*DET(2) : 29.4126563437192 = > 253250. - 253250.000000000 - 1.000000E-07 * -2.941266E+08 > 34. ET(2) = xx(5,2) : -848141. = -848141.388090847 > -------------------------------------------------------------------------- > Petsc Release Version 2.3.0, Patch 32, April, 26, 2005 > See docs/changes/index.html for recent updates. > See docs/faq.html for hints about trouble shooting. > See docs/index.html for manual pages. > ----------------------------------------------------------------------- > ./SNESG3D2 on a linux-gnu named sethnx004.vac.com by yy26539 Mon Feb > 27 14:21:13 2006 > Libraries linked from /home/yy26539/work/NISSE/petsc-2.3.0/lib/linux-gnu > Configure run at Thu Oct 13 08:23:56 2005 > Configure options --with-cc=gcc --with-fc="f77 -N109" > --download-mpich=1 --download-mpich-pm=gforker > --download-f-blas-lapack=1 --with-shared=0 > ----------------------------------------------------------------------- > [0]PETSC ERROR: Caught signal number 8 FPE: Floating Point > Exception,probably divide by zero > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [0]PETSC ERROR: likely location of problem given in stack below > [0]PETSC ERROR: --------------- Stack Frames --------------- > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, > [0]PETSC ERROR: INSTEAD the line number of the start of the function > [0]PETSC ERROR: is given. > [0]PETSC ERROR: [0] SNES user function line 0 unknownunknown > [0]PETSC ERROR: [0] SNESComputeFunction line 788 src/snes/interface/snes.c > [0]PETSC ERROR: [0] SNESLineSearchCubic line 514 src/snes/impls/ls/ls.c > [0]PETSC ERROR: -------------------------------------------- > [0]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > [0]PETSC ERROR: Signal received! > [0]PETSC ERROR: ! > > > Suggestions and tips are very welcome! > /nisse > > > -- > Nils-Erik Svang???rd > MSN: schweingaard at hotmail.com > Skype: schweingaard > > From billy at dem.uminho.pt Mon Feb 27 12:37:51 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Mon, 27 Feb 2006 18:37:51 +0000 Subject: VecGetValues? In-Reply-To: <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> Message-ID: <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> Once I update the ghosts cells values, how do I access them using the global index? I have used VecGetValues but it says: "Can only get local values, trying 100!" Billy. Quoting billy at dem.uminho.pt: > > Thank you very much for your help. :) > > I think I will use VecCreateGhost() then. > > Billy. > > > Quoting Barry Smith : > > > > > If you are using a structured grid you should use DA's to > > manage all this. You won't need to monkey with IS or VectorScatter > > or VecCreateGhost. Otherwise, ... see below.... > > > > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > > > > > Hi, > > > > > > I want to parallelize an unstructured finite volume CFD code. The code > is > > > second-order in space, so I only need one layer of overlap between > domains. > > The > > > time advancement is implicit. > > > > > > I renumbered the cells in my application to respect PETSc ordering. > > > Do I need to use AO: Application Orderings? Or is this optional? > > > > You do not have to use AO to manage getting the "PETSc style" ordering, > > you can do it anyway you like, so long as you have a "PETSc style" > > ordering. > > > > > > > > P0 P1 > > > ---------- ---------- > > > | 0| 1| 2| <-> | 9|10|11| > > > | 3| 4| 5| <-> |12|13|14| > > > | 6| 7| 8| <-> |15|16|17| > > > ---------- ---------- > > > > > > Should I use VecCreateGhost to include the ghost cells or another type > of > > vector > > > like VecCreateSeq? > > > > Likely you will wish to use VecCreateGhost() > > > > > > > > P0 P1 > > > ------------- ------------- > > > | 0| 1| 2| 9| <-> |18| 9|10|11| > > > | 3| 4| 5|10| <-> |19|12|13|14| > > > | 6| 7| 8|11| <-> |21|15|16|17| > > > ------------- ------------- > > > > > > Do I need to create IS: Index Sets? > > > > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), > > VecGhostGetLocalForm() etc) you will not need to deal with index sets. > > > > > > P0 P1 > > > ------------- ------------- > > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > > > | 3| 4| 5|10| <-> |10| 3| 4| 5| > > > | 6| 7| 8|11| <-> |11| 6| 7| 8| > > > ------------- ------------- > > > > > > Then I should apply VecScatter routines to update ghost cells, right? > > > > If you use VecCreateGhost() then it manages the scatters during the > > VecGhostUpdateBegin/End() for you; you do not need to use IS or > VecScatter. > > > > If you do not use VecCreateGhost() then you need to use IS followed by > > VecScatterCreate() to build the scatter data structures and > > VecScatterBegin/End() to update the ghost values. > > > > Barry > > > > > > > > Sorry for asking so many questions, but this is all very confusing to > me. > > > > > > > > > Billy. > > > > > > > > > Quoting Satish Balay : > > > > > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > > >> > > >>> > > >>> Hello, > > >>> > > >>> > > >>> "VecGetValues - Gets values from certain locations of a vector. > > Currently > > >> can > > >>> only get values on the same processor" > > >>> > > >>> How can I get a value of the vector belonging to a different > processor? > > >>> > > >>> > > >>> I am trying to learn how to adapt my code to work in parallel. Have > > looked > > >> at > > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on > how > > to > > >> use > > >>> petsc with unstructured meshes. Is there any simple step-by-step > > >> tutorial? > > >> > > >> Basically - you'll have to use VecScatter routines to specify the > > >> communication required. Then use VecScatterBegin/End() to invoke this > > >> communication. Some of this info is in the tutorial slides [check > > >> arround slide 150] in: > > >> > > > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > > >> > > >> Not usre if there is an unstructured grid example code. [but there are > > >> examples using VecScatters in src/dm/da/examples/tutorials] > > >> > > >> Satish > > >> > > >> > > > > > > > > > > > > > > > > From knepley at mcs.anl.gov Mon Feb 27 13:05:23 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Mon, 27 Feb 2006 13:05:23 -0600 Subject: VecGetValues? In-Reply-To: <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Mon, 27 Feb 2006 18:37:51 +0000") References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: > Once I update the ghosts cells values, how do I access them using the global index? > > I have used VecGetValues but it says: "Can only get local values, trying 100!" We do not have access to local values through global indices expect through the DA. This is something we might provide with the unstructured update in the next release, but it is really more common to refer to them through local indices, as in FEM methods. Matt > Billy. > > Quoting billy at dem.uminho.pt: > >> >> Thank you very much for your help. :) >> >> I think I will use VecCreateGhost() then. >> >> Billy. >> >> >> Quoting Barry Smith : >> >> > >> > If you are using a structured grid you should use DA's to >> > manage all this. You won't need to monkey with IS or VectorScatter >> > or VecCreateGhost. Otherwise, ... see below.... >> > >> > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: >> > >> > > Hi, >> > > >> > > I want to parallelize an unstructured finite volume CFD code. The code >> is >> > > second-order in space, so I only need one layer of overlap between >> domains. >> > The >> > > time advancement is implicit. >> > > >> > > I renumbered the cells in my application to respect PETSc ordering. >> > > Do I need to use AO: Application Orderings? Or is this optional? >> > >> > You do not have to use AO to manage getting the "PETSc style" ordering, >> > you can do it anyway you like, so long as you have a "PETSc style" >> > ordering. >> > >> > > >> > > P0 P1 >> > > ---------- ---------- >> > > | 0| 1| 2| <-> | 9|10|11| >> > > | 3| 4| 5| <-> |12|13|14| >> > > | 6| 7| 8| <-> |15|16|17| >> > > ---------- ---------- >> > > >> > > Should I use VecCreateGhost to include the ghost cells or another type >> of >> > vector >> > > like VecCreateSeq? >> > >> > Likely you will wish to use VecCreateGhost() >> > >> > > >> > > P0 P1 >> > > ------------- ------------- >> > > | 0| 1| 2| 9| <-> |18| 9|10|11| >> > > | 3| 4| 5|10| <-> |19|12|13|14| >> > > | 6| 7| 8|11| <-> |21|15|16|17| >> > > ------------- ------------- >> > > >> > > Do I need to create IS: Index Sets? >> > >> > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), >> > VecGhostGetLocalForm() etc) you will not need to deal with index sets. >> > > >> > > P0 P1 >> > > ------------- ------------- >> > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| >> > > | 3| 4| 5|10| <-> |10| 3| 4| 5| >> > > | 6| 7| 8|11| <-> |11| 6| 7| 8| >> > > ------------- ------------- >> > > >> > > Then I should apply VecScatter routines to update ghost cells, right? >> > >> > If you use VecCreateGhost() then it manages the scatters during the >> > VecGhostUpdateBegin/End() for you; you do not need to use IS or >> VecScatter. >> > >> > If you do not use VecCreateGhost() then you need to use IS followed by >> > VecScatterCreate() to build the scatter data structures and >> > VecScatterBegin/End() to update the ghost values. >> > >> > Barry >> > >> > > >> > > Sorry for asking so many questions, but this is all very confusing to >> me. >> > > >> > > >> > > Billy. >> > > >> > > >> > > Quoting Satish Balay : >> > > >> > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: >> > >> >> > >>> >> > >>> Hello, >> > >>> >> > >>> >> > >>> "VecGetValues - Gets values from certain locations of a vector. >> > Currently >> > >> can >> > >>> only get values on the same processor" >> > >>> >> > >>> How can I get a value of the vector belonging to a different >> processor? >> > >>> >> > >>> >> > >>> I am trying to learn how to adapt my code to work in parallel. Have >> > looked >> > >> at >> > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused on >> how >> > to >> > >> use >> > >>> petsc with unstructured meshes. Is there any simple step-by-step >> > >> tutorial? >> > >> >> > >> Basically - you'll have to use VecScatter routines to specify the >> > >> communication required. Then use VecScatterBegin/End() to invoke this >> > >> communication. Some of this info is in the tutorial slides [check >> > >> arround slide 150] in: >> > >> >> > >> http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf >> > >> >> > >> Not usre if there is an unstructured grid example code. [but there are >> > >> examples using VecScatters in src/dm/da/examples/tutorials] >> > >> >> > >> Satish >> > >> >> > >> >> > > >> > > >> > > >> > >> > >> >> >> > > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From billy at dem.uminho.pt Mon Feb 27 13:40:58 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Mon, 27 Feb 2006 19:40:58 +0000 Subject: VecGetValues? In-Reply-To: References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> Message-ID: <1141069258.440355ca46bac@serv-g1.ccom.uminho.pt> I establish the ghost cells and I have a vector gx created with VecCreateGhost() which looks like this: ---gx---: Process [0] 0 0.875 1 0.625 2 0.875 3 0.625 Process [1] 4 0.375 5 0.125 6 0.375 7 0.125 Process [2] 8 0.375 9 0.125 10 0.375 11 0.125 Process [3] 12 0.875 13 0.625 14 0.875 15 0.625 Then I can use VecGhostGetLocalForm to get the local vector with ghost cell values: ---lx---: 0 0.875 1 0.625 2 0.875 3 0.625 4 0.375 5 0.875 6 0.375 7 0.625 Then if I can access the ghost cell using local index: xl[4]: 0.375000 I have several global vectors like gx and if I have local representation for each one of them, isn't this a duplication of memory? Can I access the ghost cell value with creating the local vector? (VecGhostGetLocalForm) Billy. Quoting Matthew Knepley : > billy at dem.uminho.pt writes: > > > Once I update the ghosts cells values, how do I access them using the > global index? > > > > I have used VecGetValues but it says: "Can only get local values, trying > 100!" > > We do not have access to local values through global indices expect through > the DA. > This is something we might provide with the unstructured update in the next > release, but it is really > more common to refer to them through local indices, as in FEM methods. > > Matt > > > Billy. > > > > Quoting billy at dem.uminho.pt: > > > >> > >> Thank you very much for your help. :) > >> > >> I think I will use VecCreateGhost() then. > >> > >> Billy. > >> > >> > >> Quoting Barry Smith : > >> > >> > > >> > If you are using a structured grid you should use DA's to > >> > manage all this. You won't need to monkey with IS or VectorScatter > >> > or VecCreateGhost. Otherwise, ... see below.... > >> > > >> > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > >> > > >> > > Hi, > >> > > > >> > > I want to parallelize an unstructured finite volume CFD code. The > code > >> is > >> > > second-order in space, so I only need one layer of overlap between > >> domains. > >> > The > >> > > time advancement is implicit. > >> > > > >> > > I renumbered the cells in my application to respect PETSc ordering. > >> > > Do I need to use AO: Application Orderings? Or is this optional? > >> > > >> > You do not have to use AO to manage getting the "PETSc style" > ordering, > >> > you can do it anyway you like, so long as you have a "PETSc style" > >> > ordering. > >> > > >> > > > >> > > P0 P1 > >> > > ---------- ---------- > >> > > | 0| 1| 2| <-> | 9|10|11| > >> > > | 3| 4| 5| <-> |12|13|14| > >> > > | 6| 7| 8| <-> |15|16|17| > >> > > ---------- ---------- > >> > > > >> > > Should I use VecCreateGhost to include the ghost cells or another > type > >> of > >> > vector > >> > > like VecCreateSeq? > >> > > >> > Likely you will wish to use VecCreateGhost() > >> > > >> > > > >> > > P0 P1 > >> > > ------------- ------------- > >> > > | 0| 1| 2| 9| <-> |18| 9|10|11| > >> > > | 3| 4| 5|10| <-> |19|12|13|14| > >> > > | 6| 7| 8|11| <-> |21|15|16|17| > >> > > ------------- ------------- > >> > > > >> > > Do I need to create IS: Index Sets? > >> > > >> > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), > >> > VecGhostGetLocalForm() etc) you will not need to deal with index sets. > >> > > > >> > > P0 P1 > >> > > ------------- ------------- > >> > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > >> > > | 3| 4| 5|10| <-> |10| 3| 4| 5| > >> > > | 6| 7| 8|11| <-> |11| 6| 7| 8| > >> > > ------------- ------------- > >> > > > >> > > Then I should apply VecScatter routines to update ghost cells, > right? > >> > > >> > If you use VecCreateGhost() then it manages the scatters during the > >> > VecGhostUpdateBegin/End() for you; you do not need to use IS or > >> VecScatter. > >> > > >> > If you do not use VecCreateGhost() then you need to use IS followed by > >> > VecScatterCreate() to build the scatter data structures and > >> > VecScatterBegin/End() to update the ghost values. > >> > > >> > Barry > >> > > >> > > > >> > > Sorry for asking so many questions, but this is all very confusing > to > >> me. > >> > > > >> > > > >> > > Billy. > >> > > > >> > > > >> > > Quoting Satish Balay : > >> > > > >> > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > >> > >> > >> > >>> > >> > >>> Hello, > >> > >>> > >> > >>> > >> > >>> "VecGetValues - Gets values from certain locations of a vector. > >> > Currently > >> > >> can > >> > >>> only get values on the same processor" > >> > >>> > >> > >>> How can I get a value of the vector belonging to a different > >> processor? > >> > >>> > >> > >>> > >> > >>> I am trying to learn how to adapt my code to work in parallel. > Have > >> > looked > >> > >> at > >> > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused > on > >> how > >> > to > >> > >> use > >> > >>> petsc with unstructured meshes. Is there any simple step-by-step > >> > >> tutorial? > >> > >> > >> > >> Basically - you'll have to use VecScatter routines to specify the > >> > >> communication required. Then use VecScatterBegin/End() to invoke > this > >> > >> communication. Some of this info is in the tutorial slides [check > >> > >> arround slide 150] in: > >> > >> > >> > > >> > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > >> > >> > >> > >> Not usre if there is an unstructured grid example code. [but there > are > >> > >> examples using VecScatters in src/dm/da/examples/tutorials] > >> > >> > >> > >> Satish > >> > >> > >> > >> > >> > > > >> > > > >> > > > >> > > >> > > >> > >> > >> > > > > > > > > > > -- > "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec > Guiness > > From billy at dem.uminho.pt Mon Feb 27 13:47:38 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Mon, 27 Feb 2006 19:47:38 +0000 Subject: VecGetValues? (correction) In-Reply-To: References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> Message-ID: <1141069658.4403575a20099@serv-g1.ccom.uminho.pt> I establish the ghost cells and I have a vector gx created with VecCreateGhost() that looks like this: ---gx---: Process [0] 0 0.875 1 0.625 2 0.875 3 0.625 Process [1] 4 0.375 5 0.125 6 0.375 7 0.125 Process [2] 8 0.375 9 0.125 10 0.375 11 0.125 Process [3] 12 0.875 13 0.625 14 0.875 15 0.625 Then I can use VecGhostGetLocalForm to get the local vector with ghost cell values. For process [0] looks like this: ---lx---: 0 0.875 1 0.625 2 0.875 3 0.625 4 0.375 5 0.875 6 0.375 7 0.625 Then I can access the ghost cell using local index: xl[4]: 0.375000 I have several global vectors like gx and if I have a local representation for each one of them, isn't this a duplication of memory? My question is: Can I access the ghost cell value without creating the local vector? (VecGhostGetLocalForm) Billy. Quoting Matthew Knepley : > billy at dem.uminho.pt writes: > > > Once I update the ghosts cells values, how do I access them using the > global index? > > > > I have used VecGetValues but it says: "Can only get local values, trying > 100!" > > We do not have access to local values through global indices expect through > the DA. > This is something we might provide with the unstructured update in the next > release, but it is really > more common to refer to them through local indices, as in FEM methods. > > Matt > > > Billy. > > > > Quoting billy at dem.uminho.pt: > > > >> > >> Thank you very much for your help. :) > >> > >> I think I will use VecCreateGhost() then. > >> > >> Billy. > >> > >> > >> Quoting Barry Smith : > >> > >> > > >> > If you are using a structured grid you should use DA's to > >> > manage all this. You won't need to monkey with IS or VectorScatter > >> > or VecCreateGhost. Otherwise, ... see below.... > >> > > >> > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > >> > > >> > > Hi, > >> > > > >> > > I want to parallelize an unstructured finite volume CFD code. The > code > >> is > >> > > second-order in space, so I only need one layer of overlap between > >> domains. > >> > The > >> > > time advancement is implicit. > >> > > > >> > > I renumbered the cells in my application to respect PETSc ordering. > >> > > Do I need to use AO: Application Orderings? Or is this optional? > >> > > >> > You do not have to use AO to manage getting the "PETSc style" > ordering, > >> > you can do it anyway you like, so long as you have a "PETSc style" > >> > ordering. > >> > > >> > > > >> > > P0 P1 > >> > > ---------- ---------- > >> > > | 0| 1| 2| <-> | 9|10|11| > >> > > | 3| 4| 5| <-> |12|13|14| > >> > > | 6| 7| 8| <-> |15|16|17| > >> > > ---------- ---------- > >> > > > >> > > Should I use VecCreateGhost to include the ghost cells or another > type > >> of > >> > vector > >> > > like VecCreateSeq? > >> > > >> > Likely you will wish to use VecCreateGhost() > >> > > >> > > > >> > > P0 P1 > >> > > ------------- ------------- > >> > > | 0| 1| 2| 9| <-> |18| 9|10|11| > >> > > | 3| 4| 5|10| <-> |19|12|13|14| > >> > > | 6| 7| 8|11| <-> |21|15|16|17| > >> > > ------------- ------------- > >> > > > >> > > Do I need to create IS: Index Sets? > >> > > >> > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), > >> > VecGhostGetLocalForm() etc) you will not need to deal with index sets. > >> > > > >> > > P0 P1 > >> > > ------------- ------------- > >> > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > >> > > | 3| 4| 5|10| <-> |10| 3| 4| 5| > >> > > | 6| 7| 8|11| <-> |11| 6| 7| 8| > >> > > ------------- ------------- > >> > > > >> > > Then I should apply VecScatter routines to update ghost cells, > right? > >> > > >> > If you use VecCreateGhost() then it manages the scatters during the > >> > VecGhostUpdateBegin/End() for you; you do not need to use IS or > >> VecScatter. > >> > > >> > If you do not use VecCreateGhost() then you need to use IS followed by > >> > VecScatterCreate() to build the scatter data structures and > >> > VecScatterBegin/End() to update the ghost values. > >> > > >> > Barry > >> > > >> > > > >> > > Sorry for asking so many questions, but this is all very confusing > to > >> me. > >> > > > >> > > > >> > > Billy. > >> > > > >> > > > >> > > Quoting Satish Balay : > >> > > > >> > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > >> > >> > >> > >>> > >> > >>> Hello, > >> > >>> > >> > >>> > >> > >>> "VecGetValues - Gets values from certain locations of a vector. > >> > Currently > >> > >> can > >> > >>> only get values on the same processor" > >> > >>> > >> > >>> How can I get a value of the vector belonging to a different > >> processor? > >> > >>> > >> > >>> > >> > >>> I am trying to learn how to adapt my code to work in parallel. > Have > >> > looked > >> > >> at > >> > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused > on > >> how > >> > to > >> > >> use > >> > >>> petsc with unstructured meshes. Is there any simple step-by-step > >> > >> tutorial? > >> > >> > >> > >> Basically - you'll have to use VecScatter routines to specify the > >> > >> communication required. Then use VecScatterBegin/End() to invoke > this > >> > >> communication. Some of this info is in the tutorial slides [check > >> > >> arround slide 150] in: > >> > >> > >> > > >> > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > >> > >> > >> > >> Not usre if there is an unstructured grid example code. [but there > are > >> > >> examples using VecScatters in src/dm/da/examples/tutorials] > >> > >> > >> > >> Satish > >> > >> > >> > >> > >> > > > >> > > > >> > > > >> > > >> > > >> > >> > >> > > > > > > > > > > -- > "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec > Guiness > > From knepley at mcs.anl.gov Mon Feb 27 14:00:33 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Mon, 27 Feb 2006 14:00:33 -0600 Subject: VecGetValues? In-Reply-To: <1141069258.440355ca46bac@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Mon, 27 Feb 2006 19:40:58 +0000") References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> <1141069258.440355ca46bac@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: If you look inside the code, we do not duplicate memory for the local rep, jsut wrap it with a seq vector. Matt > I establish the ghost cells and I have a vector gx created with VecCreateGhost() > which looks like this: > > ---gx---: > Process [0] > 0 0.875 > 1 0.625 > 2 0.875 > 3 0.625 > Process [1] > 4 0.375 > 5 0.125 > 6 0.375 > 7 0.125 > Process [2] > 8 0.375 > 9 0.125 > 10 0.375 > 11 0.125 > Process [3] > 12 0.875 > 13 0.625 > 14 0.875 > 15 0.625 > > Then I can use VecGhostGetLocalForm to get the local vector with ghost cell values: > > ---lx---: > 0 0.875 > 1 0.625 > 2 0.875 > 3 0.625 > 4 0.375 > 5 0.875 > 6 0.375 > 7 0.625 > > Then if I can access the ghost cell using local index: > > xl[4]: 0.375000 > > I have several global vectors like gx and if I have local representation for > each one of them, isn't this a duplication of memory? > > Can I access the ghost cell value with creating the local vector? > (VecGhostGetLocalForm) > > > Billy. > > > Quoting Matthew Knepley : > >> billy at dem.uminho.pt writes: >> >> > Once I update the ghosts cells values, how do I access them using the >> global index? >> > >> > I have used VecGetValues but it says: "Can only get local values, trying >> 100!" >> >> We do not have access to local values through global indices expect through >> the DA. >> This is something we might provide with the unstructured update in the next >> release, but it is really >> more common to refer to them through local indices, as in FEM methods. >> >> Matt >> >> > Billy. >> > >> > Quoting billy at dem.uminho.pt: >> > >> >> >> >> Thank you very much for your help. :) >> >> >> >> I think I will use VecCreateGhost() then. >> >> >> >> Billy. >> >> >> >> >> >> Quoting Barry Smith : >> >> >> >> > >> >> > If you are using a structured grid you should use DA's to >> >> > manage all this. You won't need to monkey with IS or VectorScatter >> >> > or VecCreateGhost. Otherwise, ... see below.... >> >> > >> >> > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: >> >> > >> >> > > Hi, >> >> > > >> >> > > I want to parallelize an unstructured finite volume CFD code. The >> code >> >> is >> >> > > second-order in space, so I only need one layer of overlap between >> >> domains. >> >> > The >> >> > > time advancement is implicit. >> >> > > >> >> > > I renumbered the cells in my application to respect PETSc ordering. >> >> > > Do I need to use AO: Application Orderings? Or is this optional? >> >> > >> >> > You do not have to use AO to manage getting the "PETSc style" >> ordering, >> >> > you can do it anyway you like, so long as you have a "PETSc style" >> >> > ordering. >> >> > >> >> > > >> >> > > P0 P1 >> >> > > ---------- ---------- >> >> > > | 0| 1| 2| <-> | 9|10|11| >> >> > > | 3| 4| 5| <-> |12|13|14| >> >> > > | 6| 7| 8| <-> |15|16|17| >> >> > > ---------- ---------- >> >> > > >> >> > > Should I use VecCreateGhost to include the ghost cells or another >> type >> >> of >> >> > vector >> >> > > like VecCreateSeq? >> >> > >> >> > Likely you will wish to use VecCreateGhost() >> >> > >> >> > > >> >> > > P0 P1 >> >> > > ------------- ------------- >> >> > > | 0| 1| 2| 9| <-> |18| 9|10|11| >> >> > > | 3| 4| 5|10| <-> |19|12|13|14| >> >> > > | 6| 7| 8|11| <-> |21|15|16|17| >> >> > > ------------- ------------- >> >> > > >> >> > > Do I need to create IS: Index Sets? >> >> > >> >> > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), >> >> > VecGhostGetLocalForm() etc) you will not need to deal with index sets. >> >> > > >> >> > > P0 P1 >> >> > > ------------- ------------- >> >> > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| >> >> > > | 3| 4| 5|10| <-> |10| 3| 4| 5| >> >> > > | 6| 7| 8|11| <-> |11| 6| 7| 8| >> >> > > ------------- ------------- >> >> > > >> >> > > Then I should apply VecScatter routines to update ghost cells, >> right? >> >> > >> >> > If you use VecCreateGhost() then it manages the scatters during the >> >> > VecGhostUpdateBegin/End() for you; you do not need to use IS or >> >> VecScatter. >> >> > >> >> > If you do not use VecCreateGhost() then you need to use IS followed by >> >> > VecScatterCreate() to build the scatter data structures and >> >> > VecScatterBegin/End() to update the ghost values. >> >> > >> >> > Barry >> >> > >> >> > > >> >> > > Sorry for asking so many questions, but this is all very confusing >> to >> >> me. >> >> > > >> >> > > >> >> > > Billy. >> >> > > >> >> > > >> >> > > Quoting Satish Balay : >> >> > > >> >> > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: >> >> > >> >> >> > >>> >> >> > >>> Hello, >> >> > >>> >> >> > >>> >> >> > >>> "VecGetValues - Gets values from certain locations of a vector. >> >> > Currently >> >> > >> can >> >> > >>> only get values on the same processor" >> >> > >>> >> >> > >>> How can I get a value of the vector belonging to a different >> >> processor? >> >> > >>> >> >> > >>> >> >> > >>> I am trying to learn how to adapt my code to work in parallel. >> Have >> >> > looked >> >> > >> at >> >> > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still confused >> on >> >> how >> >> > to >> >> > >> use >> >> > >>> petsc with unstructured meshes. Is there any simple step-by-step >> >> > >> tutorial? >> >> > >> >> >> > >> Basically - you'll have to use VecScatter routines to specify the >> >> > >> communication required. Then use VecScatterBegin/End() to invoke >> this >> >> > >> communication. Some of this info is in the tutorial slides [check >> >> > >> arround slide 150] in: >> >> > >> >> >> > >> >> >> http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf >> >> > >> >> >> > >> Not usre if there is an unstructured grid example code. [but there >> are >> >> > >> examples using VecScatters in src/dm/da/examples/tutorials] >> >> > >> >> >> > >> Satish >> >> > >> >> >> > >> >> >> > > >> >> > > >> >> > > >> >> > >> >> > >> >> >> >> >> >> >> > >> > >> > >> > >> >> -- >> "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec >> Guiness >> >> > > > > -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness From billy at dem.uminho.pt Mon Feb 27 14:25:41 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Mon, 27 Feb 2006 20:25:41 +0000 Subject: VecGetValues? In-Reply-To: References: <1140801883.43ff415babb48@serv-g1.ccom.uminho.pt> <1140994825.44023309b03f1@serv-g1.ccom.uminho.pt> <1140995712.44023680cd8b8@serv-g1.ccom.uminho.pt> <1141065471.440346ffb4871@serv-g1.ccom.uminho.pt> <1141069258.440355ca46bac@serv-g1.ccom.uminho.pt> Message-ID: <1141071941.44036045778ea@serv-g1.ccom.uminho.pt> Thanks for your prompt answer! Regards, Billy. Quoting Matthew Knepley : > billy at dem.uminho.pt writes: > > If you look inside the code, we do not duplicate memory for the local > rep, > jsut wrap it with a seq vector. > > Matt > > > I establish the ghost cells and I have a vector gx created with > VecCreateGhost() > > which looks like this: > > > > ---gx---: > > Process [0] > > 0 0.875 > > 1 0.625 > > 2 0.875 > > 3 0.625 > > Process [1] > > 4 0.375 > > 5 0.125 > > 6 0.375 > > 7 0.125 > > Process [2] > > 8 0.375 > > 9 0.125 > > 10 0.375 > > 11 0.125 > > Process [3] > > 12 0.875 > > 13 0.625 > > 14 0.875 > > 15 0.625 > > > > Then I can use VecGhostGetLocalForm to get the local vector with ghost cell > values: > > > > ---lx---: > > 0 0.875 > > 1 0.625 > > 2 0.875 > > 3 0.625 > > 4 0.375 > > 5 0.875 > > 6 0.375 > > 7 0.625 > > > > Then if I can access the ghost cell using local index: > > > > xl[4]: 0.375000 > > > > I have several global vectors like gx and if I have local representation > for > > each one of them, isn't this a duplication of memory? > > > > Can I access the ghost cell value with creating the local vector? > > (VecGhostGetLocalForm) > > > > > > Billy. > > > > > > Quoting Matthew Knepley : > > > >> billy at dem.uminho.pt writes: > >> > >> > Once I update the ghosts cells values, how do I access them using the > >> global index? > >> > > >> > I have used VecGetValues but it says: "Can only get local values, > trying > >> 100!" > >> > >> We do not have access to local values through global indices expect > through > >> the DA. > >> This is something we might provide with the unstructured update in the > next > >> release, but it is really > >> more common to refer to them through local indices, as in FEM methods. > >> > >> Matt > >> > >> > Billy. > >> > > >> > Quoting billy at dem.uminho.pt: > >> > > >> >> > >> >> Thank you very much for your help. :) > >> >> > >> >> I think I will use VecCreateGhost() then. > >> >> > >> >> Billy. > >> >> > >> >> > >> >> Quoting Barry Smith : > >> >> > >> >> > > >> >> > If you are using a structured grid you should use DA's to > >> >> > manage all this. You won't need to monkey with IS or VectorScatter > >> >> > or VecCreateGhost. Otherwise, ... see below.... > >> >> > > >> >> > On Sun, 26 Feb 2006, billy at dem.uminho.pt wrote: > >> >> > > >> >> > > Hi, > >> >> > > > >> >> > > I want to parallelize an unstructured finite volume CFD code. The > >> code > >> >> is > >> >> > > second-order in space, so I only need one layer of overlap > between > >> >> domains. > >> >> > The > >> >> > > time advancement is implicit. > >> >> > > > >> >> > > I renumbered the cells in my application to respect PETSc > ordering. > >> >> > > Do I need to use AO: Application Orderings? Or is this optional? > >> >> > > >> >> > You do not have to use AO to manage getting the "PETSc style" > >> ordering, > >> >> > you can do it anyway you like, so long as you have a "PETSc style" > >> >> > ordering. > >> >> > > >> >> > > > >> >> > > P0 P1 > >> >> > > ---------- ---------- > >> >> > > | 0| 1| 2| <-> | 9|10|11| > >> >> > > | 3| 4| 5| <-> |12|13|14| > >> >> > > | 6| 7| 8| <-> |15|16|17| > >> >> > > ---------- ---------- > >> >> > > > >> >> > > Should I use VecCreateGhost to include the ghost cells or another > >> type > >> >> of > >> >> > vector > >> >> > > like VecCreateSeq? > >> >> > > >> >> > Likely you will wish to use VecCreateGhost() > >> >> > > >> >> > > > >> >> > > P0 P1 > >> >> > > ------------- ------------- > >> >> > > | 0| 1| 2| 9| <-> |18| 9|10|11| > >> >> > > | 3| 4| 5|10| <-> |19|12|13|14| > >> >> > > | 6| 7| 8|11| <-> |21|15|16|17| > >> >> > > ------------- ------------- > >> >> > > > >> >> > > Do I need to create IS: Index Sets? > >> >> > > >> >> > If you use VecCreateGhost() and friends (VecGhostUpdateBegin/End(), > > >> >> > VecGhostGetLocalForm() etc) you will not need to deal with index > sets. > >> >> > > > >> >> > > P0 P1 > >> >> > > ------------- ------------- > >> >> > > | 0| 1| 2| 9| <-> | 9| 0| 1| 2| > >> >> > > | 3| 4| 5|10| <-> |10| 3| 4| 5| > >> >> > > | 6| 7| 8|11| <-> |11| 6| 7| 8| > >> >> > > ------------- ------------- > >> >> > > > >> >> > > Then I should apply VecScatter routines to update ghost cells, > >> right? > >> >> > > >> >> > If you use VecCreateGhost() then it manages the scatters during the > > >> >> > VecGhostUpdateBegin/End() for you; you do not need to use IS or > >> >> VecScatter. > >> >> > > >> >> > If you do not use VecCreateGhost() then you need to use IS followed > by > >> >> > VecScatterCreate() to build the scatter data structures and > >> >> > VecScatterBegin/End() to update the ghost values. > >> >> > > >> >> > Barry > >> >> > > >> >> > > > >> >> > > Sorry for asking so many questions, but this is all very > confusing > >> to > >> >> me. > >> >> > > > >> >> > > > >> >> > > Billy. > >> >> > > > >> >> > > > >> >> > > Quoting Satish Balay : > >> >> > > > >> >> > >> On Fri, 24 Feb 2006, billy at dem.uminho.pt wrote: > >> >> > >> > >> >> > >>> > >> >> > >>> Hello, > >> >> > >>> > >> >> > >>> > >> >> > >>> "VecGetValues - Gets values from certain locations of a vector. > >> >> > Currently > >> >> > >> can > >> >> > >>> only get values on the same processor" > >> >> > >>> > >> >> > >>> How can I get a value of the vector belonging to a different > >> >> processor? > >> >> > >>> > >> >> > >>> > >> >> > >>> I am trying to learn how to adapt my code to work in parallel. > >> Have > >> >> > looked > >> >> > >> at > >> >> > >>> src/snes/examples/tutorials/ex10d/ex10.c, but I am still > confused > >> on > >> >> how > >> >> > to > >> >> > >> use > >> >> > >>> petsc with unstructured meshes. Is there any simple > step-by-step > >> >> > >> tutorial? > >> >> > >> > >> >> > >> Basically - you'll have to use VecScatter routines to specify > the > >> >> > >> communication required. Then use VecScatterBegin/End() to invoke > >> this > >> >> > >> communication. Some of this info is in the tutorial slides > [check > >> >> > >> arround slide 150] in: > >> >> > >> > >> >> > > >> >> > >> > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/PetscTu06.pdf > >> >> > >> > >> >> > >> Not usre if there is an unstructured grid example code. [but > there > >> are > >> >> > >> examples using VecScatters in src/dm/da/examples/tutorials] > >> >> > >> > >> >> > >> Satish > >> >> > >> > >> >> > >> > >> >> > > > >> >> > > > >> >> > > > >> >> > > >> >> > > >> >> > >> >> > >> >> > >> > > >> > > >> > > >> > > >> > >> -- > >> "Failure has a thousand explanations. Success doesn't need one" -- Sir > Alec > >> Guiness > >> > >> > > > > > > > > > > -- > "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec > Guiness > > From harald at tapir.caltech.edu Tue Feb 28 13:40:10 2006 From: harald at tapir.caltech.edu (Harald Pfeiffer) Date: Tue, 28 Feb 2006 11:40:10 -0800 Subject: attaching pathdb Message-ID: <4404A71A.8070405@tapir.caltech.edu> Hello, I'd like to use the pathscale debugger, pathdb, when debugging petsc-codes. However, petsc seems to have hardcoded the list of debuggers: -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] start the debugger in new xterm -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] start all processes in the debugger How do I go about doing this? Thanks, Harald -- Harald P. Pfeiffer harald at tapir.caltech.edu Theoretical Astrophysics Phone (626) 395-8413 Caltech 130-33 Fax (626) 796-5675 Pasadena, CA 91125, USA From bsmith at mcs.anl.gov Tue Feb 28 13:51:48 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Feb 2006 13:51:48 -0600 (CST) Subject: attaching pathdb In-Reply-To: <4404A71A.8070405@tapir.caltech.edu> References: <4404A71A.8070405@tapir.caltech.edu> Message-ID: Harald, We need to know the command line arguments for pathdb to attach properly (each debugger is a little different, hence the hardwiring). Please send us the pathdb man page and all the output when you run it with -help and we'll make a tenative patch for you to try. Barry On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > Hello, > > I'd like to use the pathscale debugger, pathdb, when debugging petsc-codes. > However, petsc seems to have hardcoded the list of debuggers: > > -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] > start the debugger in new xterm > -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] > start all processes in the debugger > > > How do I go about doing this? > > Thanks, > Harald > > From harald at tapir.caltech.edu Tue Feb 28 14:05:47 2006 From: harald at tapir.caltech.edu (Harald Pfeiffer) Date: Tue, 28 Feb 2006 12:05:47 -0800 Subject: attaching pathdb In-Reply-To: References: <4404A71A.8070405@tapir.caltech.edu> Message-ID: <4404AD1B.7060003@tapir.caltech.edu> Hi Barry, the usage seems to be similar or even identical to gdb: Output of pathdb -help: > usage: pathdb [option...] [program [core|pid]] The man page is online under http://www.pathscale.com/docs/pathdb.html, and there is also the full user guidem http://www.pathscale.com/docs/PathDB_UserGuide.pdf (look at page 29). Harald Barry Smith wrote: > > Harald, > > We need to know the command line arguments for pathdb to attach > properly (each debugger is a little different, hence the hardwiring). > Please send us the pathdb man page and all the output when you > run it with -help and we'll make a tenative patch for you to try. > > Barry > > > On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > >> Hello, >> >> I'd like to use the pathscale debugger, pathdb, when debugging >> petsc-codes. However, petsc seems to have hardcoded the list of >> debuggers: >> >> -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] >> start the debugger in new xterm >> -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] >> start all processes in the debugger >> >> >> How do I go about doing this? >> >> Thanks, >> Harald >> >> -- Harald P. Pfeiffer harald at tapir.caltech.edu Theoretical Astrophysics Phone (626) 395-8413 Caltech 130-33 Fax (626) 796-5675 Pasadena, CA 91125, USA From bsmith at mcs.anl.gov Tue Feb 28 14:13:36 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Feb 2006 14:13:36 -0600 (CST) Subject: attaching pathdb In-Reply-To: <4404AD1B.7060003@tapir.caltech.edu> References: <4404A71A.8070405@tapir.caltech.edu> <4404AD1B.7060003@tapir.caltech.edu> Message-ID: Then it should "just work", what happens when you run with -start_in_debugger noxterm,pathdb or -start_in_debugger pathdb? Please send all output from the error. Thanks Barry On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > Hi Barry, > > the usage seems to be similar or even identical to gdb: > > Output of pathdb -help: >> usage: pathdb [option...] [program [core|pid]] > > The man page is online under http://www.pathscale.com/docs/pathdb.html, > and there is also the full user guidem > http://www.pathscale.com/docs/PathDB_UserGuide.pdf (look at page 29). > > Harald > > > > Barry Smith wrote: >> >> Harald, >> >> We need to know the command line arguments for pathdb to attach >> properly (each debugger is a little different, hence the hardwiring). >> Please send us the pathdb man page and all the output when you >> run it with -help and we'll make a tenative patch for you to try. >> >> Barry >> >> >> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >> >>> Hello, >>> >>> I'd like to use the pathscale debugger, pathdb, when debugging >>> petsc-codes. However, petsc seems to have hardcoded the list of debuggers: >>> >>> -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] >>> start the debugger in new xterm >>> -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] >>> start all processes in the debugger >>> >>> >>> How do I go about doing this? >>> >>> Thanks, >>> Harald >>> >>> > > From harald at tapir.caltech.edu Tue Feb 28 14:22:52 2006 From: harald at tapir.caltech.edu (Harald Pfeiffer) Date: Tue, 28 Feb 2006 12:22:52 -0800 Subject: attaching pathdb In-Reply-To: References: <4404A71A.8070405@tapir.caltech.edu> <4404AD1B.7060003@tapir.caltech.edu> Message-ID: <4404B11C.5010402@tapir.caltech.edu> -start_in_debugger pathdb brings up gdb?! Ditto with noxterm. Harald Barry Smith wrote: > > Then it should "just work", what happens when you run > with -start_in_debugger noxterm,pathdb or > -start_in_debugger pathdb? Please send all output from > the error. > > Thanks > > Barry > > > On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > >> Hi Barry, >> >> the usage seems to be similar or even identical to gdb: >> >> Output of pathdb -help: >>> usage: pathdb [option...] [program [core|pid]] >> >> The man page is online under http://www.pathscale.com/docs/pathdb.html, >> and there is also the full user guidem >> http://www.pathscale.com/docs/PathDB_UserGuide.pdf (look at page 29). >> >> Harald >> >> >> >> Barry Smith wrote: >>> >>> Harald, >>> >>> We need to know the command line arguments for pathdb to attach >>> properly (each debugger is a little different, hence the hardwiring). >>> Please send us the pathdb man page and all the output when you >>> run it with -help and we'll make a tenative patch for you to try. >>> >>> Barry >>> >>> >>> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >>> >>>> Hello, >>>> >>>> I'd like to use the pathscale debugger, pathdb, when debugging >>>> petsc-codes. However, petsc seems to have hardcoded the list of >>>> debuggers: >>>> >>>> -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>> start the debugger in new xterm >>>> -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>> start all processes in the debugger >>>> >>>> >>>> How do I go about doing this? >>>> >>>> Thanks, >>>> Harald >>>> >>>> >> >> -- Harald P. Pfeiffer harald at tapir.caltech.edu Theoretical Astrophysics Phone (626) 395-8413 Caltech 130-33 Fax (626) 796-5675 Pasadena, CA 91125, USA From bsmith at mcs.anl.gov Tue Feb 28 15:06:05 2006 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Feb 2006 15:06:05 -0600 (CST) Subject: attaching pathdb In-Reply-To: <4404B11C.5010402@tapir.caltech.edu> References: <4404A71A.8070405@tapir.caltech.edu> <4404AD1B.7060003@tapir.caltech.edu> <4404B11C.5010402@tapir.caltech.edu> Message-ID: Ahh yes, found the offensive code and it is offensive. In src/sys/error/adebug.c in PetscSetDebuggerFromString() is a horrible list of choices. We should fix this code to be more general. Anyway, you can just add another entry for pathdb and do make lib shared in that directory. Please let us know if this does not resolve the problem, Barry This will be fixed in our next 2.3.1 patch. On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > -start_in_debugger pathdb brings up gdb?! Ditto with noxterm. > > Harald > > > Barry Smith wrote: >> >> Then it should "just work", what happens when you run >> with -start_in_debugger noxterm,pathdb or >> -start_in_debugger pathdb? Please send all output from >> the error. >> >> Thanks >> >> Barry >> >> >> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >> >>> Hi Barry, >>> >>> the usage seems to be similar or even identical to gdb: >>> >>> Output of pathdb -help: >>>> usage: pathdb [option...] [program [core|pid]] >>> >>> The man page is online under http://www.pathscale.com/docs/pathdb.html, >>> and there is also the full user guidem >>> http://www.pathscale.com/docs/PathDB_UserGuide.pdf (look at page 29). >>> >>> Harald >>> >>> >>> >>> Barry Smith wrote: >>>> >>>> Harald, >>>> >>>> We need to know the command line arguments for pathdb to attach >>>> properly (each debugger is a little different, hence the hardwiring). >>>> Please send us the pathdb man page and all the output when you >>>> run it with -help and we'll make a tenative patch for you to try. >>>> >>>> Barry >>>> >>>> >>>> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >>>> >>>>> Hello, >>>>> >>>>> I'd like to use the pathscale debugger, pathdb, when debugging >>>>> petsc-codes. However, petsc seems to have hardcoded the list of >>>>> debuggers: >>>>> >>>>> -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>>> start the debugger in new xterm >>>>> -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>>> start all processes in the debugger >>>>> >>>>> >>>>> How do I go about doing this? >>>>> >>>>> Thanks, >>>>> Harald >>>>> >>>>> >>> >>> > > From harald at tapir.caltech.edu Tue Feb 28 15:38:33 2006 From: harald at tapir.caltech.edu (Harald Pfeiffer) Date: Tue, 28 Feb 2006 13:38:33 -0800 Subject: attaching pathdb In-Reply-To: References: <4404A71A.8070405@tapir.caltech.edu> <4404AD1B.7060003@tapir.caltech.edu> <4404B11C.5010402@tapir.caltech.edu> Message-ID: <4404C2D9.1040409@tapir.caltech.edu> Works now, thanks! Barry Smith wrote: > > Ahh yes, found the offensive code and it is offensive. In > src/sys/error/adebug.c in PetscSetDebuggerFromString() is > a horrible list of choices. We should fix this code to be more > general. Anyway, you can just add another entry for pathdb > and do make lib shared in that directory. > > Please let us know if this does not resolve the problem, > > Barry > > This will be fixed in our next 2.3.1 patch. > > > On Tue, 28 Feb 2006, Harald Pfeiffer wrote: > >> -start_in_debugger pathdb brings up gdb?! Ditto with noxterm. >> >> Harald >> >> >> Barry Smith wrote: >>> >>> Then it should "just work", what happens when you run >>> with -start_in_debugger noxterm,pathdb or >>> -start_in_debugger pathdb? Please send all output from >>> the error. >>> >>> Thanks >>> >>> Barry >>> >>> >>> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >>> >>>> Hi Barry, >>>> >>>> the usage seems to be similar or even identical to gdb: >>>> >>>> Output of pathdb -help: >>>>> usage: pathdb [option...] [program [core|pid]] >>>> >>>> The man page is online under >>>> http://www.pathscale.com/docs/pathdb.html, >>>> and there is also the full user guidem >>>> http://www.pathscale.com/docs/PathDB_UserGuide.pdf (look at page 29). >>>> >>>> Harald >>>> >>>> >>>> >>>> Barry Smith wrote: >>>>> >>>>> Harald, >>>>> >>>>> We need to know the command line arguments for pathdb to attach >>>>> properly (each debugger is a little different, hence the hardwiring). >>>>> Please send us the pathdb man page and all the output when you >>>>> run it with -help and we'll make a tenative patch for you to try. >>>>> >>>>> Barry >>>>> >>>>> >>>>> On Tue, 28 Feb 2006, Harald Pfeiffer wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I'd like to use the pathscale debugger, pathdb, when debugging >>>>>> petsc-codes. However, petsc seems to have hardcoded the list of >>>>>> debuggers: >>>>>> >>>>>> -on_error_attach_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>>>> start the debugger in new xterm >>>>>> -start_in_debugger [gdb,dbx,xxgdb,ups,noxterm] >>>>>> start all processes in the debugger >>>>>> >>>>>> >>>>>> How do I go about doing this? >>>>>> >>>>>> Thanks, >>>>>> Harald >>>>>> >>>>>> >>>> >>>> >> >> -- Harald P. Pfeiffer harald at tapir.caltech.edu Theoretical Astrophysics Phone (626) 395-8413 Caltech 130-33 Fax (626) 796-5675 Pasadena, CA 91125, USA From billy at dem.uminho.pt Tue Feb 28 18:27:34 2006 From: billy at dem.uminho.pt (billy at dem.uminho.pt) Date: Wed, 1 Mar 2006 00:27:34 +0000 Subject: MatSetValues() Message-ID: <1141172854.4404ea76a0d90@serv-g1.ccom.uminho.pt> Using MatSetValues() any process can insert values into any location of the global matrix? Billy. From knepley at mcs.anl.gov Tue Feb 28 18:29:07 2006 From: knepley at mcs.anl.gov (Matthew Knepley) Date: Tue, 28 Feb 2006 18:29:07 -0600 Subject: MatSetValues() In-Reply-To: <1141172854.4404ea76a0d90@serv-g1.ccom.uminho.pt> (billy@dem.uminho.pt's message of "Wed, 1 Mar 2006 00:27:34 +0000") References: <1141172854.4404ea76a0d90@serv-g1.ccom.uminho.pt> Message-ID: billy at dem.uminho.pt writes: > Using MatSetValues() any process can insert values into any location of the > global matrix? Yes. Matt -- "Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness