From zonexo at gmail.com Mon Dec 1 05:50:54 2008 From: zonexo at gmail.com (Ben Tay) Date: Mon, 01 Dec 2008 19:50:54 +0800 Subject: Error when using MKL 10.0.5.025 Message-ID: <4933CF9E.1040204@gmail.com> Hi, I met with some problems when using the MKL 10.0.5.025. My PETSc was compiled using --with-shared=0. However, after compilation, when I run some of the tests such as ex1f and ex2f, I got the error: ./ex2f: error while loading shared libraries: libmkl_lapack.so: cannot open shared object file: No such file or directory I thought I had specified to use static. My command to compile is : ./config/configure.py --with-vendor-compilers=intel --with-x=0 --with-hypre-dir=/home/svu/g0306332/lib/hypre --with-debugging=0 --with-batch=1 --with-mpi-dir=/app1/mvapich/current/ --with-mpi-shared=0 --with-shared=0 --with-blas-lapack-dir=/opt/intel/cmkl/10.0.5.025/lib/em64t However, if I use back MKL 8.1.1, with similar command, then there's no such problem. Hope you can help. Thanks! -- Yours sincerely, Ben Tay From knepley at gmail.com Mon Dec 1 08:33:50 2008 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 1 Dec 2008 08:33:50 -0600 Subject: Error when using MKL 10.0.5.025 In-Reply-To: <4933CF9E.1040204@gmail.com> References: <4933CF9E.1040204@gmail.com> Message-ID: Perhaps you forgot to update your LD_LIBRARY_PATH when running the first one. Matt On Mon, Dec 1, 2008 at 5:50 AM, Ben Tay wrote: > Hi, > > I met with some problems when using the MKL 10.0.5.025. My PETSc was > compiled using --with-shared=0. However, after compilation, when I run some > of the tests such as ex1f and ex2f, I got the error: > > ./ex2f: error while loading shared libraries: libmkl_lapack.so: cannot open > shared object file: No such file or directory > > I thought I had specified to use static. My command to compile is : > > ./config/configure.py --with-vendor-compilers=intel --with-x=0 > --with-hypre-dir=/home/svu/g0306332/lib/hypre --with-debugging=0 > --with-batch=1 --with-mpi-dir=/app1/mvapich/current/ --with-mpi-shared=0 > --with-shared=0 --with-blas-lapack-dir=/opt/intel/cmkl/ > 10.0.5.025/lib/em64t > > However, if I use back MKL 8.1.1, with similar command, then there's no > such problem. > > Hope you can help. Thanks! > > -- > Yours sincerely, > > Ben Tay > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From e.tadeu at gmail.com Mon Dec 1 12:28:48 2008 From: e.tadeu at gmail.com (Edson Tadeu) Date: Mon, 1 Dec 2008 16:28:48 -0200 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian Message-ID: Hi, I'm solving non-linear problems with Line Search, and sometimes I need to change an entire row of the Jacobian together with the corresponding element in the Function, but I only know which rows/elements need to be changed when I'm computing the Jacobian. So, what I'd like to know first is: 1) Am I allowed to change the Function vector from inside FormJacobian? (I've looked inside SNESSolve_LS, and it seems that this is not allowed, because "fnorm" is evaluated before SNESComputeJacobian, and if I change F inside FormJacobian, fnorm would be wrong). or 2) Am I allowed to change the Function vector from PreCheck routine? If none of that is allowed, another solution that I thought was to compute the Jacobian while computing the Function. The FAQ says that "You are free to have your 'FormFunction' compute as much of the Jacobian at that point as you like, keep the information in the user context (the final argument to FormFunction and FormJacobian) and then retreive the information in your FormJacobian() function". The problem is that FormFunction is called many times more than FormJacobian, and Jacobian calculation is slow, so: 3) Is there any way to know when I should really recompute the Jacobian from inside FormFunction? (I don't think so, but...) Well, is there any way to solve this problem? ;) Thanks in advance, Edson -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Dec 1 12:56:57 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 1 Dec 2008 12:56:57 -0600 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian In-Reply-To: References: Message-ID: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> On Dec 1, 2008, at 12:28 PM, Edson Tadeu wrote: > Hi, > > I'm solving non-linear problems with Line Search, and sometimes I > need to change an entire row of the Jacobian together with the > corresponding element in the Function, but I only know which rows/ > elements need to be changed when I'm computing the Jacobian. So, > what I'd like to know first is: Maybe if you explain why you need to "change an entire row of the Jacobian together with the corresponding element in the Function" we could come up with some suggestions on how to proceed. > > > 1) Am I allowed to change the Function vector from inside > FormJacobian? (I've looked inside SNESSolve_LS, and it seems that > this is not allowed, because "fnorm" is evaluated before > SNESComputeJacobian, and if I change F inside FormJacobian, fnorm > would be wrong). > > or > > 2) Am I allowed to change the Function vector from PreCheck routine? > > If none of that is allowed, another solution that I thought was to > compute the Jacobian while computing the Function. The FAQ says that > "You are free to have your 'FormFunction' compute as much of the > Jacobian at that point as you like, keep the information in the user > context (the final argument to FormFunction and FormJacobian) and > then retreive the information in your FormJacobian() function". The > problem is that FormFunction is called many times more than > FormJacobian, and Jacobian calculation is slow, so: > > 3) Is there any way to know when I should really recompute the > Jacobian from inside FormFunction? (I don't think so, but...) No. But if you do not use any line search at all then each function evaluation will have an associated Jacobian so you can compute them together. (Of course you lose the globalization of the line search so SNES may not converge). You can turn off all line searches with - snes_ls basic from the command line or from the code SNESLineSearchSet(snes,SNESLineSeachNo,PETSC_NULL); Barry > > > Well, is there any way to solve this problem? ;) > > Thanks in advance, > Edson > From e.tadeu at gmail.com Mon Dec 1 14:04:56 2008 From: e.tadeu at gmail.com (Edson Tadeu) Date: Mon, 1 Dec 2008 18:04:56 -0200 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian In-Reply-To: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> References: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> Message-ID: Barry, Thanks for the reply; On Mon, Dec 1, 2008 at 4:56 PM, Barry Smith wrote: > > On Dec 1, 2008, at 12:28 PM, Edson Tadeu wrote: > > Hi, >> >> I'm solving non-linear problems with Line Search, and sometimes I need >> to change an entire row of the Jacobian together with the corresponding >> element in the Function, but I only know which rows/elements need to be >> changed when I'm computing the Jacobian. So, what I'd like to know first is: >> > > Maybe if you explain why you need to "change an entire row of the > Jacobian together with the corresponding element in the Function" we could > come up > with some suggestions on how to proceed. > It's because, depending on some conditions, I need to make some change of variables in the original equations, otherwise the Jacobian will have linear dependent rows. For example, I need to tell that x(4) = 1.0-x(3) (instead of the original equation for x(4)). I would do it by changing the Jacobian like this: J(4,3) = 1; J(4,4) = 1; all other columns of row 4 of J are zero. F(4) = 0. This way, I would have delta_x(4) = -delta_x(3). I've solved this now by directly changing the delta solution vector (delta_x) in the pre check phase... it seems to be working, but it doesn't seem to be the right way, I don't know... > > >> >> 1) Am I allowed to change the Function vector from inside FormJacobian? >> (I've looked inside SNESSolve_LS, and it seems that this is not allowed, >> because "fnorm" is evaluated before SNESComputeJacobian, and if I change F >> inside FormJacobian, fnorm would be wrong). >> >> or >> >> 2) Am I allowed to change the Function vector from PreCheck routine? >> >> If none of that is allowed, another solution that I thought was to >> compute the Jacobian while computing the Function. The FAQ says that "You >> are free to have your 'FormFunction' compute as much of the Jacobian at that >> point as you like, keep the information in the user context (the final >> argument to FormFunction and FormJacobian) and then retreive the information >> in your FormJacobian() function". The problem is that FormFunction is called >> many times more than FormJacobian, and Jacobian calculation is slow, so: >> >> 3) Is there any way to know when I should really recompute the Jacobian >> from inside FormFunction? (I don't think so, but...) >> > > No. But if you do not use any line search at all then each function > evaluation will have an associated Jacobian so you can compute them > together. (Of course you lose the globalization of the line search so SNES > may not converge). You can turn off all line searches with -snes_ls basic > from the command line or from the code > SNESLineSearchSet(snes,SNESLineSeachNo,PETSC_NULL); > > Barry > Ok, but the whole reason that I'm using SNES is to speed up my computations by using the Line Search algorithms (or maybe Trust Region). I had coded an algorithm before that manually computed using full steps, and it were working, but were slow! Thanks, Edson -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Dec 1 14:29:07 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 1 Dec 2008 14:29:07 -0600 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian In-Reply-To: References: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> Message-ID: <09C5748D-70A7-46DC-9BB1-4747513CC214@mcs.anl.gov> The mathematician side of me cringes (actually every side of me cringes) when I read this. You are changing the definition of the function you are finding the zero of WHILE you are finding the zero of it. Yuck. How do you know what you get at the end is anything but garbage? It seems to me that either 1) there is a bug in your Jacobian or 2) near (or at) the solution (when F(x) = 0) the Jacobian is singular For 1) you can run with -snes_ls_type test to do some simple tests on the Jacobian. For 2) I submit that what you really want to do (instead of changing the "Jacobian" to something that is not the Jacobian) is to properly solve the singular linear system. How to do this depends on the nature of the null space of the Jacobian and if you have a handle on what the null space is). In PETSc the simplest thing to try (sequentially) is -pc_type lu -pc_factor_shift_nonzero and see what happens. Barry On Dec 1, 2008, at 2:04 PM, Edson Tadeu wrote: > Barry, > > Thanks for the reply; > > On Mon, Dec 1, 2008 at 4:56 PM, Barry Smith > wrote: > > On Dec 1, 2008, at 12:28 PM, Edson Tadeu wrote: > > Hi, > > I'm solving non-linear problems with Line Search, and sometimes I > need to change an entire row of the Jacobian together with the > corresponding element in the Function, but I only know which rows/ > elements need to be changed when I'm computing the Jacobian. So, > what I'd like to know first is: > > Maybe if you explain why you need to "change an entire row of the > Jacobian together with the corresponding element in the Function" we > could come up > with some suggestions on how to proceed. > > It's because, depending on some conditions, I need to make some > change of variables in the original equations, otherwise the > Jacobian will have linear dependent rows. For example, I need to > tell that x(4) = 1.0-x(3) (instead of the original equation for > x(4)). I would do it by changing the Jacobian like this: > > J(4,3) = 1; J(4,4) = 1; all other columns of row 4 of J are zero. > F(4) = 0. > > This way, I would have delta_x(4) = -delta_x(3). I've solved this > now by directly changing the delta solution vector (delta_x) in the > pre check phase... it seems to be working, but it doesn't seem to be > the right way, I don't know... > > > > > > 1) Am I allowed to change the Function vector from inside > FormJacobian? (I've looked inside SNESSolve_LS, and it seems that > this is not allowed, because "fnorm" is evaluated before > SNESComputeJacobian, and if I change F inside FormJacobian, fnorm > would be wrong). > > or > > 2) Am I allowed to change the Function vector from PreCheck routine? > > If none of that is allowed, another solution that I thought was to > compute the Jacobian while computing the Function. The FAQ says that > "You are free to have your 'FormFunction' compute as much of the > Jacobian at that point as you like, keep the information in the user > context (the final argument to FormFunction and FormJacobian) and > then retreive the information in your FormJacobian() function". The > problem is that FormFunction is called many times more than > FormJacobian, and Jacobian calculation is slow, so: > > 3) Is there any way to know when I should really recompute the > Jacobian from inside FormFunction? (I don't think so, but...) > > No. But if you do not use any line search at all then each function > evaluation will have an associated Jacobian so you can compute them > together. (Of course you lose the globalization of the line search > so SNES may not converge). You can turn off all line searches with - > snes_ls basic > from the command line or from the code > SNESLineSearchSet(snes,SNESLineSeachNo,PETSC_NULL); > > Barry > > Ok, but the whole reason that I'm using SNES is to speed up my > computations by using the Line Search algorithms (or maybe Trust > Region). I had coded an algorithm before that manually computed > using full steps, and it were working, but were slow! > > Thanks, > Edson > From e.tadeu at gmail.com Mon Dec 1 15:21:03 2008 From: e.tadeu at gmail.com (Edson Tadeu) Date: Mon, 1 Dec 2008 19:21:03 -0200 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian In-Reply-To: <09C5748D-70A7-46DC-9BB1-4747513CC214@mcs.anl.gov> References: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> <09C5748D-70A7-46DC-9BB1-4747513CC214@mcs.anl.gov> Message-ID: Barry, It's not that I'm changing the definition of the function... the problem is that the function have some "ifs" on it. This comes ultimately from the physics of the problem... there is some change of dependency between the variables, like this: A is an unknown, B is an unknown, C is dependent (not on the system): C = 1 - A - B + delta_C (where delta_C is also dependent from some other variables). But if C == 0 and delta_C == 0, then the dependency changes: A is an unknown, but B is now dependent: B = 1 - A This is why, when this happens, the Jacobian row of B becomes linearly dependent with the row of A. So, actually, B should be taken out of the system! [See that there is a real change here, because now the derivative dB/dA == -1, but before, A and B were unrelated.] Well... this scheme was working before using full Newton steps... ;) But thanks for the tip, I'll also try to treat accordingly the null space that appears with the linear dependence. Hmm... but by removing the null space of J, I'll be effectively removing the row corresponding to the B unknown... so, I would need to define delta_B = -delta_A later, anyway, wouldn't I (or maybe PETSc would define it automatically, given the basis vector [1 -1 0 0 0...] for the null space)? Thanks, Edson On Mon, Dec 1, 2008 at 6:29 PM, Barry Smith wrote: > > The mathematician side of me cringes (actually every side of me cringes) > when I read this. > > You are changing the definition of the function you are finding the zero > of WHILE you are > finding the zero of it. Yuck. How do you know what you get at the end is > anything but garbage? > > It seems to me that either > 1) there is a bug in your Jacobian or > 2) near (or at) the solution (when F(x) = 0) the Jacobian is singular > > For 1) you can run with -snes_ls_type test to do some simple tests on the > Jacobian. > > For 2) I submit that what you really want to do (instead of changing the > "Jacobian" to something > that is not the Jacobian) is to properly solve the singular linear system. > How to do this > depends on the nature of the null space of the Jacobian and if you have a > handle on what > the null space is). In PETSc the simplest thing to try (sequentially) is > -pc_type lu -pc_factor_shift_nonzero > and see what happens. > > Barry > > > > > On Dec 1, 2008, at 2:04 PM, Edson Tadeu wrote: > > Barry, >> >> Thanks for the reply; >> >> On Mon, Dec 1, 2008 at 4:56 PM, Barry Smith wrote: >> >> On Dec 1, 2008, at 12:28 PM, Edson Tadeu wrote: >> >> Hi, >> >> I'm solving non-linear problems with Line Search, and sometimes I need to >> change an entire row of the Jacobian together with the corresponding element >> in the Function, but I only know which rows/elements need to be changed when >> I'm computing the Jacobian. So, what I'd like to know first is: >> >> Maybe if you explain why you need to "change an entire row of the >> Jacobian together with the corresponding element in the Function" we could >> come up >> with some suggestions on how to proceed. >> >> It's because, depending on some conditions, I need to make some change of >> variables in the original equations, otherwise the Jacobian will have linear >> dependent rows. For example, I need to tell that x(4) = 1.0-x(3) (instead of >> the original equation for x(4)). I would do it by changing the Jacobian like >> this: >> >> J(4,3) = 1; J(4,4) = 1; all other columns of row 4 of J are zero. F(4) = >> 0. >> >> This way, I would have delta_x(4) = -delta_x(3). I've solved this now by >> directly changing the delta solution vector (delta_x) in the pre check >> phase... it seems to be working, but it doesn't seem to be the right way, I >> don't know... >> >> >> >> >> >> 1) Am I allowed to change the Function vector from inside FormJacobian? >> (I've looked inside SNESSolve_LS, and it seems that this is not allowed, >> because "fnorm" is evaluated before SNESComputeJacobian, and if I change F >> inside FormJacobian, fnorm would be wrong). >> >> or >> >> 2) Am I allowed to change the Function vector from PreCheck routine? >> >> If none of that is allowed, another solution that I thought was to >> compute the Jacobian while computing the Function. The FAQ says that "You >> are free to have your 'FormFunction' compute as much of the Jacobian at that >> point as you like, keep the information in the user context (the final >> argument to FormFunction and FormJacobian) and then retreive the information >> in your FormJacobian() function". The problem is that FormFunction is called >> many times more than FormJacobian, and Jacobian calculation is slow, so: >> >> 3) Is there any way to know when I should really recompute the Jacobian >> from inside FormFunction? (I don't think so, but...) >> >> No. But if you do not use any line search at all then each function >> evaluation will have an associated Jacobian so you can compute them >> together. (Of course you lose the globalization of the line search so SNES >> may not converge). You can turn off all line searches with -snes_ls basic >> from the command line or from the code >> SNESLineSearchSet(snes,SNESLineSeachNo,PETSC_NULL); >> >> Barry >> >> Ok, but the whole reason that I'm using SNES is to speed up my >> computations by using the Line Search algorithms (or maybe Trust Region). I >> had coded an algorithm before that manually computed using full steps, and >> it were working, but were slow! >> >> Thanks, >> Edson >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Dec 1 15:36:11 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 1 Dec 2008 15:36:11 -0600 Subject: (Petsc SNES) Reevaluating Function right after evaluating the Jacobian In-Reply-To: References: <536D5197-9B3C-4640-B401-3BE369D729C6@mcs.anl.gov> <09C5748D-70A7-46DC-9BB1-4747513CC214@mcs.anl.gov> Message-ID: On Dec 1, 2008, at 3:21 PM, Edson Tadeu wrote: > Barry, > > It's not that I'm changing the definition of the function... the > problem is that the function have some "ifs" on it. This comes > ultimately from the physics of the problem... there is some change > of dependency between the variables, like this: > > A is an unknown, B is an unknown, C is dependent (not on the system): > > C = 1 - A - B + delta_C > > (where delta_C is also dependent from some other variables). > > But if C == 0 and delta_C == 0, then the dependency changes: > > A is an unknown, but B is now dependent: > > B = 1 - A > > This is why, when this happens, the Jacobian row of B becomes > linearly dependent with the row of A. So, actually, B should be > taken out of the system! [See that there is a real change here, > because now the derivative dB/dA == -1, but before, A and B were > unrelated.] > > Well... this scheme was working before using full Newton steps... ;) Yes, but it may explain why Newton's method was converging slowly for you. > > > But thanks for the tip, I'll also try to treat accordingly the null > space that appears with the linear dependence. Hmm... but by > removing the null space of J, I'll be effectively removing the row > corresponding to the B unknown... so, I would need to define delta_B > = -delta_A later, anyway, wouldn't I (or maybe PETSc would define it > automatically, given the basis vector [1 -1 0 0 0...] for the null > space)? If you know the null space then you can use KSPSetNullSpace() to set the null space just before the linear solve. It is unfortunately a bit of a hack to do that. In the end of your FormJacobian() do a SNESGetKSP() and then create your null space object with MatNullSpaceCreate() and then attach it to the KSP with KSPSetNullSpace(). If you do this then YOU don't need to monkey with the changing the search direction manually. Barry BTW: I don't totally know what I am talking about here. > > > Thanks, > Edson > > > On Mon, Dec 1, 2008 at 6:29 PM, Barry Smith > wrote: > > The mathematician side of me cringes (actually every side of me > cringes) when I read this. > > You are changing the definition of the function you are finding > the zero of WHILE you are > finding the zero of it. Yuck. How do you know what you get at the > end is anything but garbage? > > It seems to me that either > 1) there is a bug in your Jacobian or > 2) near (or at) the solution (when F(x) = 0) the Jacobian is singular > > For 1) you can run with -snes_ls_type test to do some simple tests > on the Jacobian. > > For 2) I submit that what you really want to do (instead of changing > the "Jacobian" to something > that is not the Jacobian) is to properly solve the singular linear > system. How to do this > depends on the nature of the null space of the Jacobian and if you > have a handle on what > the null space is). In PETSc the simplest thing to try > (sequentially) is -pc_type lu -pc_factor_shift_nonzero > and see what happens. > > Barry > > > > > On Dec 1, 2008, at 2:04 PM, Edson Tadeu wrote: > > Barry, > > Thanks for the reply; > > On Mon, Dec 1, 2008 at 4:56 PM, Barry Smith > wrote: > > On Dec 1, 2008, at 12:28 PM, Edson Tadeu wrote: > > Hi, > > I'm solving non-linear problems with Line Search, and sometimes I > need to change an entire row of the Jacobian together with the > corresponding element in the Function, but I only know which rows/ > elements need to be changed when I'm computing the Jacobian. So, > what I'd like to know first is: > > Maybe if you explain why you need to "change an entire row of the > Jacobian together with the corresponding element in the Function" we > could come up > with some suggestions on how to proceed. > > It's because, depending on some conditions, I need to make some > change of variables in the original equations, otherwise the > Jacobian will have linear dependent rows. For example, I need to > tell that x(4) = 1.0-x(3) (instead of the original equation for > x(4)). I would do it by changing the Jacobian like this: > > J(4,3) = 1; J(4,4) = 1; all other columns of row 4 of J are zero. > F(4) = 0. > > This way, I would have delta_x(4) = -delta_x(3). I've solved this > now by directly changing the delta solution vector (delta_x) in the > pre check phase... it seems to be working, but it doesn't seem to be > the right way, I don't know... > > > > > > 1) Am I allowed to change the Function vector from inside > FormJacobian? (I've looked inside SNESSolve_LS, and it seems that > this is not allowed, because "fnorm" is evaluated before > SNESComputeJacobian, and if I change F inside FormJacobian, fnorm > would be wrong). > > or > > 2) Am I allowed to change the Function vector from PreCheck routine? > > If none of that is allowed, another solution that I thought was to > compute the Jacobian while computing the Function. The FAQ says that > "You are free to have your 'FormFunction' compute as much of the > Jacobian at that point as you like, keep the information in the user > context (the final argument to FormFunction and FormJacobian) and > then retreive the information in your FormJacobian() function". The > problem is that FormFunction is called many times more than > FormJacobian, and Jacobian calculation is slow, so: > > 3) Is there any way to know when I should really recompute the > Jacobian from inside FormFunction? (I don't think so, but...) > > No. But if you do not use any line search at all then each function > evaluation will have an associated Jacobian so you can compute them > together. (Of course you lose the globalization of the line search > so SNES may not converge). You can turn off all line searches with - > snes_ls basic > from the command line or from the code > SNESLineSearchSet(snes,SNESLineSeachNo,PETSC_NULL); > > Barry > > Ok, but the whole reason that I'm using SNES is to speed up my > computations by using the Line Search algorithms (or maybe Trust > Region). I had coded an algorithm before that manually computed > using full steps, and it were working, but were slow! > > Thanks, > Edson > > > From Lars.Rindorf at teknologisk.dk Tue Dec 2 05:55:20 2008 From: Lars.Rindorf at teknologisk.dk (Lars Rindorf) Date: Tue, 2 Dec 2008 12:55:20 +0100 Subject: Timesolver: Newmark method? Message-ID: Dear all Is it planned to do the Newmark time scheme for second order time differential equations in petsc? KR _____________________ Lars Rindorf Technology consultant, Ph.D. Mobile +45 7220 3367 Surfaces and microtechnology Danish Technological Institute Gregersensvej 2630 Taastrup Denmark http://www.dti.dk Phone +45 7220 2000 -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Dec 2 07:28:52 2008 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 2 Dec 2008 07:28:52 -0600 Subject: Timesolver: Newmark method? In-Reply-To: References: Message-ID: On Tue, Dec 2, 2008 at 5:55 AM, Lars Rindorf wrote: > Dear all > > Is it planned to do the Newmark time scheme for second order time > differential equations in petsc? > We have not planned to do it, however if you would like to contribute it, we would help you do it. The TS strcutures are fairly straightforward and looking at Crank-Nicholson should help. Thanks, Matt > KR > > *_____________________* > > *Lars Rindorf* > > Technology consultant, Ph.D. > > > > Mobile +45 7220 3367 > > > > *Surfaces and microtechnology* > > Danish Technological Institute > > Gregersensvej > > 2630 Taastrup > > Denmark > > http://www.dti.dk > > Phone +45 7220 2000 > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at al.com.au Tue Dec 2 19:52:32 2008 From: chrisc at al.com.au (Chris Cooper) Date: Wed, 03 Dec 2008 12:52:32 +1100 Subject: DMMG solver convergence with single precision? Message-ID: <4935E660.6010401@al.com.au> Hi, Just wondering if there are any techniques for improving DMMG convergence on a linear problem when operating in single precision? eg using ksp example 34 in Petsc 2.3.3 with multigrid levels changed to 6... double precision: 0 KSP Residual norm 4.451733404502e-02 1 KSP Residual norm 2.801676364873e-04 2 KSP Residual norm 3.215426166316e-06 3 KSP Residual norm 5.446277176355e-08 Residual norm 5.44628e-08 Error norm 0.000356798 Error norm 9.21724e-05 Error norm 1.34208e-07 single precision: 0 KSP Residual norm 4.450287297368e-02 1 KSP Residual norm 2.799208450597e-04 2 KSP Residual norm 2.232483529951e-05 3 KSP Residual norm 1.575733222126e-05 4 KSP Residual norm 1.286925271415e-05 ... 27 KSP Residual norm 4.369261660031e-06 28 KSP Residual norm 4.287577212381e-06 29 KSP Residual norm 4.210308816255e-06 30 KSP Residual norm 2.235170904896e-05 31 KSP Residual norm 1.709007477757e-07 Residual norm 8.45584e-07 Error norm 0.000370784 Error norm 9.54664e-05 Error norm 1.38935e-07 thanks, Chris Cooper Animal Logic Animal Logic http://www.animallogic.com Please think of the environment before printing this email. This email and any attachments may be confidential and/or privileged. If you are not the intended recipient of this email, you must not disclose or use the information contained in it. Please notify the sender immediately and delete this document if you have received it in error. We do not guarantee this email is error or virus free. From knepley at gmail.com Tue Dec 2 20:02:03 2008 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 2 Dec 2008 20:02:03 -0600 Subject: DMMG solver convergence with single precision? In-Reply-To: <4935E660.6010401@al.com.au> References: <4935E660.6010401@al.com.au> Message-ID: On Tue, Dec 2, 2008 at 7:52 PM, Chris Cooper wrote: > Hi, > > Just wondering if there are any techniques for improving DMMG convergence > on a linear problem when operating in single precision? > > eg using ksp example 34 in Petsc 2.3.3 with multigrid levels changed to > 6... > > double precision: > 0 KSP Residual norm 4.451733404502e-02 > 1 KSP Residual norm 2.801676364873e-04 > 2 KSP Residual norm 3.215426166316e-06 > 3 KSP Residual norm 5.446277176355e-08 > Residual norm 5.44628e-08 > Error norm 0.000356798 > Error norm 9.21724e-05 > Error norm 1.34208e-07 > > single precision: > 0 KSP Residual norm 4.450287297368e-02 > 1 KSP Residual norm 2.799208450597e-04 > 2 KSP Residual norm 2.232483529951e-05 > 3 KSP Residual norm 1.575733222126e-05 > 4 KSP Residual norm 1.286925271415e-05 > ... > 27 KSP Residual norm 4.369261660031e-06 > 28 KSP Residual norm 4.287577212381e-06 > 29 KSP Residual norm 4.210308816255e-06 > 30 KSP Residual norm 2.235170904896e-05 > 31 KSP Residual norm 1.709007477757e-07 This bothers me. Shouldn't the Residual norm be the same as the last residual, as it is above? Matt Residual norm 8.45584e-07 > Error norm 0.000370784 > Error norm 9.54664e-05 > Error norm 1.38935e-07 > > thanks, > Chris Cooper > Animal Logic > > Animal Logic > http://www.animallogic.com > > Please think of the environment before printing this email. > > This email and any attachments may be confidential and/or privileged. If > you are not the intended recipient of this email, you must not disclose or > use the information contained in it. Please notify the sender immediately > and delete this document if you have received it in error. We do not > guarantee this email is error or virus free. > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at al.com.au Tue Dec 2 21:47:51 2008 From: chrisc at al.com.au (Chris Cooper) Date: Wed, 03 Dec 2008 14:47:51 +1100 Subject: DMMG solver convergence with single precision? In-Reply-To: References: <4935E660.6010401@al.com.au> Message-ID: <49360167.1010304@al.com.au> Matthew Knepley wrote: > On Tue, Dec 2, 2008 at 7:52 PM, Chris Cooper > wrote: > > Hi, > > Just wondering if there are any techniques for improving DMMG > convergence on a linear problem when operating in single precision? > > eg using ksp example 34 in Petsc 2.3.3 with multigrid levels > changed to 6... > > double precision: > 0 KSP Residual norm 4.451733404502e-02 > 1 KSP Residual norm 2.801676364873e-04 > 2 KSP Residual norm 3.215426166316e-06 > 3 KSP Residual norm 5.446277176355e-08 > Residual norm 5.44628e-08 > Error norm 0.000356798 > Error norm 9.21724e-05 > Error norm 1.34208e-07 > > single precision: > 0 KSP Residual norm 4.450287297368e-02 > 1 KSP Residual norm 2.799208450597e-04 > 2 KSP Residual norm 2.232483529951e-05 > 3 KSP Residual norm 1.575733222126e-05 > 4 KSP Residual norm 1.286925271415e-05 > ... > 27 KSP Residual norm 4.369261660031e-06 > 28 KSP Residual norm 4.287577212381e-06 > 29 KSP Residual norm 4.210308816255e-06 > 30 KSP Residual norm 2.235170904896e-05 > 31 KSP Residual norm 1.709007477757e-07 > > > This bothers me. Shouldn't the Residual norm be the same as the last > residual, as it is above? Hi Matt, If I use ksp_monitor_true_residual the numbers match... 30 KSP preconditioned resid norm 2.235170904896e-05 true resid norm 2.235170904896e-05 ||Ae||/||Ax|| 5.022531840950e-04 31 KSP preconditioned resid norm 1.709007477757e-07 true resid norm 8.455837132715e-07 ||Ae||/||Ax|| 1.900065399241e-05 Residual norm 8.45584e-07 Error norm 0.000370784 Error norm 9.54664e-05 Error norm 1.38935e-07 I do get better convergence on that example with -ksp_type stcg but would appreciate any other recommendations for single precision linear problems. thanks, Chris > > Matt > > Residual norm 8.45584e-07 > Error norm 0.000370784 > Error norm 9.54664e-05 > Error norm 1.38935e-07 > > thanks, > Chris Cooper > Animal Logic > > Animal Logic > http://www.animallogic.com > > Please think of the environment before printing this email. > > This email and any attachments may be confidential and/or > privileged. If you are not the intended recipient of this email, > you must not disclose or use the information contained in it. > Please notify the sender immediately and delete this document if > you have received it in error. We do not guarantee this email is > error or virus free. > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener Animal Logic http://www.animallogic.com Please think of the environment before printing this email. This email and any attachments may be confidential and/or privileged. If you are not the intended recipient of this email, you must not disclose or use the information contained in it. Please notify the sender immediately and delete this document if you have received it in error. We do not guarantee this email is error or virus free. From bsmith at mcs.anl.gov Tue Dec 2 22:43:46 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 2 Dec 2008 22:43:46 -0600 Subject: DMMG solver convergence with single precision? In-Reply-To: <4935E660.6010401@al.com.au> References: <4935E660.6010401@al.com.au> Message-ID: <61A79110-6187-48AB-AF20-6F638CC565F9@mcs.anl.gov> I have reproduced this. I expect that you would not be able to decrease the residual by more than a factor of 10^6 but the results below are worse than I expected. After first running it I thought I would have to say "I have no idea". But then I ran barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 - ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres - ksp_gmres_restart 4 0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.803983807098e-04 true resid norm 2.803510287777e-04 ||Ae||/||Ax|| 6.297569256276e-03 2 KSP preconditioned resid norm 7.431745416397e-06 true resid norm 7.503178949264e-06 ||Ae||/||Ax|| 1.685450988589e-04 3 KSP preconditioned resid norm 4.979479399481e-06 true resid norm 6.953502179385e-06 ||Ae||/||Ax|| 1.561976241646e-04 4 KSP preconditioned resid norm 6.971503353270e-06 true resid norm 6.971503353270e-06 ||Ae||/||Ax|| 1.566019782331e-04 5 KSP preconditioned resid norm 4.069146442021e-08 true resid norm 8.315442414641e-07 ||Ae||/||Ax|| 1.867910941655e-05 Residual norm 8.31544e-07 Error norm 0.000372035 Error norm 9.54604e-05 Error norm 1.38928e-07 Part of the problem is coming from the gmres not generating a good orthonormal basis of directions, so I ran [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 - ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres - ksp_gmres_modifiedgramschmidt 0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.803631359711e-04 true resid norm 2.803224197123e-04 ||Ae||/||Ax|| 6.296926643699e-03 2 KSP preconditioned resid norm 6.318659416138e-06 true resid norm 6.395913715096e-06 ||Ae||/||Ax|| 1.436724269297e-04 3 KSP preconditioned resid norm 5.441804660222e-06 true resid norm 5.559954843193e-06 ||Ae||/||Ax|| 1.248941407539e-04 4 KSP preconditioned resid norm 5.441734174383e-06 true resid norm 5.580118795478e-06 ||Ae||/||Ax|| 1.253470836673e-04 5 KSP preconditioned resid norm 4.763217020809e-06 true resid norm 4.965124844603e-06 ||Ae||/||Ax|| 1.115323830163e-04 6 KSP preconditioned resid norm 1.149661784439e-06 true resid norm 1.742098334034e-06 ||Ae||/||Ax|| 3.913302862202e-05 7 KSP preconditioned resid norm 1.129046836468e-06 true resid norm 1.607488229638e-06 ||Ae||/||Ax|| 3.610926069086e-05 8 KSP preconditioned resid norm 1.129005681832e-06 true resid norm 1.682757897470e-06 ||Ae||/||Ax|| 3.780005499721e-05 9 KSP preconditioned resid norm 9.863339300864e-07 true resid norm 1.657332745708e-06 ||Ae||/||Ax|| 3.722892870428e-05 10 KSP preconditioned resid norm 3.080494650476e-07 true resid norm 1.298029587815e-06 ||Ae||/||Ax|| 2.915784352808e-05 Residual norm 1.29803e-06 Error norm 0.00100342 Error norm 0.000631685 Error norm 6.85996e-07 It looks like using the modified gram-schmidt orthogonalization helps (that is it takes less iterations than with the classical (default) BUT looking more closely we see that it actually terminated too early and the answer is crappy. So lets turn off ALL the GMRES stuff [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 - ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson - mg_levels_ksp_type richardson 0 KSP preconditioned resid norm 3.698072814941e+02 true resid norm 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 1 KSP preconditioned resid norm 4.145919799805e+01 true resid norm 9.632258675992e-03 ||Ae||/||Ax|| 2.163709551096e-01 2 KSP preconditioned resid norm 4.848244190216e+00 true resid norm 1.894293935038e-03 ||Ae||/||Ax|| 4.255182296038e-02 3 KSP preconditioned resid norm 5.902504324913e-01 true resid norm 3.106203221250e-04 ||Ae||/||Ax|| 6.977513432503e-03 4 KSP preconditioned resid norm 7.403665781021e-02 true resid norm 4.669937698054e-05 ||Ae||/||Ax|| 1.049015438184e-03 5 KSP preconditioned resid norm 9.483236819506e-03 true resid norm 6.756537914043e-06 ||Ae||/||Ax|| 1.517731725471e-04 6 KSP preconditioned resid norm 1.249413820915e-03 true resid norm 1.256137807104e-06 ||Ae||/||Ax|| 2.821682210197e-05 Residual norm 1.25614e-06 Error norm 0.000369181 Error norm 9.46438e-05 Error norm 1.37756e-07 The final residual doesn't look that great but if you look at the error norms you see that the solution has fully converged. This final run is actually classical multigrid. So it looks like that is the way to go; BUT you have to be very careful with how you decide when it has converged. Obviously you want to stop as soon as the algebraic error is smaller than the truncation error. But, sadly we don't know what the algebraic error is (we only know the residual) and we do not know what the truncation error is. My hack convergence criteria would be to stop as soon as the residual norm DOES NOT decrease by at least (say) 20% To do a little better we can use SOR as the smoother instead of ILU [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 - ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson - mg_levels_ksp_type richardson -mg_levels_pc_type sor 0 KSP preconditioned resid norm 3.202437744141e+02 true resid norm 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.194643592834e+01 true resid norm 2.088646404445e-03 ||Ae||/||Ax|| 4.691759496927e-02 2 KSP preconditioned resid norm 4.755175709724e-01 true resid norm 1.131777826231e-04 ||Ae||/||Ax|| 2.542330417782e-03 3 KSP preconditioned resid norm 1.988304778934e-02 true resid norm 6.019943612046e-06 ||Ae||/||Ax|| 1.352269464405e-04 4 KSP preconditioned resid norm 8.689521928318e-04 true resid norm 8.886481737136e-07 ||Ae||/||Ax|| 1.996184437303e-05 Residual norm 8.88648e-07 Error norm 0.000368883 Error norm 9.4803e-05 Error norm 1.37996e-07 Hope this helps, Barry On Dec 2, 2008, at 7:52 PM, Chris Cooper wrote: > Hi, > > Just wondering if there are any techniques for improving DMMG > convergence on a linear problem when operating in single precision? > > eg using ksp example 34 in Petsc 2.3.3 with multigrid levels changed > to 6... > > double precision: > 0 KSP Residual norm 4.451733404502e-02 > 1 KSP Residual norm 2.801676364873e-04 > 2 KSP Residual norm 3.215426166316e-06 > 3 KSP Residual norm 5.446277176355e-08 > Residual norm 5.44628e-08 > Error norm 0.000356798 > Error norm 9.21724e-05 > Error norm 1.34208e-07 > > single precision: > 0 KSP Residual norm 4.450287297368e-02 > 1 KSP Residual norm 2.799208450597e-04 > 2 KSP Residual norm 2.232483529951e-05 > 3 KSP Residual norm 1.575733222126e-05 > 4 KSP Residual norm 1.286925271415e-05 > ... > 27 KSP Residual norm 4.369261660031e-06 > 28 KSP Residual norm 4.287577212381e-06 > 29 KSP Residual norm 4.210308816255e-06 > 30 KSP Residual norm 2.235170904896e-05 > 31 KSP Residual norm 1.709007477757e-07 > Residual norm 8.45584e-07 > Error norm 0.000370784 > Error norm 9.54664e-05 > Error norm 1.38935e-07 > > thanks, > Chris Cooper > Animal Logic > > Animal Logic > http://www.animallogic.com > > Please think of the environment before printing this email. > > This email and any attachments may be confidential and/or > privileged. If you are not the intended recipient of this email, you > must not disclose or use the information contained in it. Please > notify the sender immediately and delete this document if you have > received it in error. We do not guarantee this email is error or > virus free. > > > From Lars.Rindorf at teknologisk.dk Wed Dec 3 07:39:28 2008 From: Lars.Rindorf at teknologisk.dk (Lars Rindorf) Date: Wed, 3 Dec 2008 14:39:28 +0100 Subject: SV: Timesolver: Newmark method? In-Reply-To: References: Message-ID: Hi Matt I could implement the Newmark method with your assistance. No problem. By it has to make sense for my work. How fast and memory efficient is the petsc TS? Is it optimized in some way or implemented in some clever way numerically? KR, Lars Fra: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] P? vegne af Matthew Knepley Sendt: 2. december 2008 14:29 Til: petsc-users at mcs.anl.gov Emne: Re: Timesolver: Newmark method? On Tue, Dec 2, 2008 at 5:55 AM, Lars Rindorf wrote: Dear all Is it planned to do the Newmark time scheme for second order time differential equations in petsc? We have not planned to do it, however if you would like to contribute it, we would help you do it. The TS strcutures are fairly straightforward and looking at Crank-Nicholson should help. Thanks, Matt KR _____________________ Lars Rindorf Technology consultant, Ph.D. Mobile +45 7220 3367 Surfaces and microtechnology Danish Technological Institute Gregersensvej 2630 Taastrup Denmark http://www.dti.dk Phone +45 7220 2000 -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahuramazda10 at gmail.com Wed Dec 3 08:00:49 2008 From: ahuramazda10 at gmail.com (Santolo Felaco) Date: Wed, 3 Dec 2008 15:00:49 +0100 Subject: Mpiexec e machinelist Message-ID: <5f76eef60812030600w1dec97b9ocba05c4bb35deef4@mail.gmail.com> Hi. How can I set the machine list with comand mpiexec of release of mpiech integrate with Petsc 2.3.1? With mpirun I set -machine-file. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Dec 3 09:01:09 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 3 Dec 2008 09:01:09 -0600 Subject: SV: Timesolver: Newmark method? In-Reply-To: References: Message-ID: The TS uses KSP for linear problems and SNES for nonlinear problems and SNES uses KSP (when using implicit integrators), these take all the run time so the TS is as efficient as the particular linear solver used (for implicit integrators). For explicit integrators, you just write the integrator and TS is just a little wrapper around it that doesn't take any time so the efficiency is the efficiency of the explicit integrator you write. In other words, TS is just a thin layer of code that does not cause a performance hit, so if you code a Newmark integrator than the efficiency of the TS is the efficiency of the code you write. Barry On Dec 3, 2008, at 7:39 AM, Lars Rindorf wrote: > Hi Matt > > I could implement the Newmark method with your assistance. No > problem. By it has to make sense for my work. > > How fast and memory efficient is the petsc TS? Is it optimized in > some way or implemented in some clever way numerically? > > KR, Lars > > Fra: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov > ] P? vegne af Matthew Knepley > Sendt: 2. december 2008 14:29 > Til: petsc-users at mcs.anl.gov > Emne: Re: Timesolver: Newmark method? > > On Tue, Dec 2, 2008 at 5:55 AM, Lars Rindorf > wrote: > Dear all > > Is it planned to do the Newmark time scheme for second order time > differential equations in petsc? > > > We have not planned to do it, however if you would like to > contribute it, we would help you do it. The > TS strcutures are fairly straightforward and looking at Crank- > Nicholson should help. > > Thanks, > > Matt > > KR > > _____________________ > > Lars Rindorf > > Technology consultant, Ph.D. > > > > Mobile +45 7220 3367 > > > > Surfaces and microtechnology > > Danish Technological Institute > > Gregersensvej > > 2630 Taastrup > > Denmark > > http://www.dti.dk > > Phone +45 7220 2000 > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener From balay at mcs.anl.gov Wed Dec 3 09:09:06 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 3 Dec 2008 09:09:06 -0600 (CST) Subject: Mpiexec e machinelist In-Reply-To: <5f76eef60812030600w1dec97b9ocba05c4bb35deef4@mail.gmail.com> References: <5f76eef60812030600w1dec97b9ocba05c4bb35deef4@mail.gmail.com> Message-ID: On Wed, 3 Dec 2008, Santolo Felaco wrote: > Hi. How can I set the machine list with comand mpiexec of release of > mpiech integrate with Petsc 2.3.1? With mpirun I set -machine-file. If you use the option --download-mpich=1, we just install MPICH2. We however default to using --with-pm=gforker - for convinence. If you wish to run across multiple machines - you need MPICH2 installed with pm=mpd, and then configure mpd across multiple machines - with the machine file. [mpich install with mpd can be done with petsc configure option --download-mpich-device=mpd. But additionally - you would have to manually configure mpd with the required machinefile] Alternatively - you can install your own MPI - as you desire - and then install PETSc with the option --with-mpi-dir [instad of --with-cc --with-fc --download-mpich=1] option. Each MPI impl [and version] will have a slightly different way of installing/configuring machinefiles. BTW: You are uisng a very old version of PETSc. Current release is petsc-2.3.3 Satish From saswata at umd.edu Wed Dec 3 11:11:23 2008 From: saswata at umd.edu (Saswata Hier-Majumder) Date: Wed, 03 Dec 2008 12:11:23 -0500 Subject: Local Jacobian subroutine in fortan Message-ID: <4936BDBB.5050007@umd.edu> Dear All, I am looking for a fortran-based example of prescribing the local Jacobian for a multicomponent PDE problem, using DA structure. If anyone can point me to a fortran version of /snes/examples/tutorials/ex7.c or a similar problem, I'll greatly appreciate it. Thanks Sash -- www.geol.umd.edu/~saswata From bsmith at mcs.anl.gov Wed Dec 3 15:58:24 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 3 Dec 2008 15:58:24 -0600 Subject: Mpiexec e machinelist In-Reply-To: References: <5f76eef60812030600w1dec97b9ocba05c4bb35deef4@mail.gmail.com> Message-ID: <9234B07A-271D-4DAA-8FFB-3F29AD620400@mcs.anl.gov> I have modified the MPICH install process (from PETSc) to print the how to start up the demon instead of trying to start up the demon automatically. Barry On Dec 3, 2008, at 9:09 AM, Satish Balay wrote: > On Wed, 3 Dec 2008, Santolo Felaco wrote: > >> Hi. How can I set the machine list with comand mpiexec of release of >> mpiech integrate with Petsc 2.3.1? With mpirun I set -machine-file. > > If you use the option --download-mpich=1, we just install MPICH2. > > We however default to using --with-pm=gforker - for convinence. If you > wish to run across multiple machines - you need MPICH2 installed with > pm=mpd, and then configure mpd across multiple machines - with the > machine file. [mpich install with mpd can be done with petsc configure > option --download-mpich-device=mpd. But additionally - you would have > to manually configure mpd with the required machinefile] > > Alternatively - you can install your own MPI - as you desire - and > then install PETSc with the option --with-mpi-dir [instad of --with-cc > --with-fc --download-mpich=1] option. Each MPI impl [and version] will > have a slightly different way of installing/configuring machinefiles. > > BTW: You are uisng a very old version of PETSc. Current release is > petsc-2.3.3 > > > Satish > From chrisc at al.com.au Wed Dec 3 19:59:54 2008 From: chrisc at al.com.au (Chris Cooper) Date: Thu, 04 Dec 2008 12:59:54 +1100 Subject: DMMG solver convergence with single precision? In-Reply-To: <61A79110-6187-48AB-AF20-6F638CC565F9@mcs.anl.gov> References: <4935E660.6010401@al.com.au> <61A79110-6187-48AB-AF20-6F638CC565F9@mcs.anl.gov> Message-ID: <4937399A.9040401@al.com.au> Thanks Barry. Initial indications are that fgmres is converging better than richardson on a custom test setup, with cg somewhere in between. The solver is a little different to ex34 as well though. Thanks for the pointers, the convergence is much better now. My main concern was why the multigrid seemed to be stalling with single precision, but that doesn't appear to be an issue now. cheers, Chris Barry Smith wrote: > > I have reproduced this. I expect that you would not be able to > decrease the residual by more than a factor > of 10^6 but the results below are worse than I expected. > > After first running it I thought I would have to say "I have no > idea". But then I ran > > barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 > -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres > -ksp_gmres_restart 4 > 0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm > 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 2.803983807098e-04 true resid norm > 2.803510287777e-04 ||Ae||/||Ax|| 6.297569256276e-03 > 2 KSP preconditioned resid norm 7.431745416397e-06 true resid norm > 7.503178949264e-06 ||Ae||/||Ax|| 1.685450988589e-04 > 3 KSP preconditioned resid norm 4.979479399481e-06 true resid norm > 6.953502179385e-06 ||Ae||/||Ax|| 1.561976241646e-04 > 4 KSP preconditioned resid norm 6.971503353270e-06 true resid norm > 6.971503353270e-06 ||Ae||/||Ax|| 1.566019782331e-04 > 5 KSP preconditioned resid norm 4.069146442021e-08 true resid norm > 8.315442414641e-07 ||Ae||/||Ax|| 1.867910941655e-05 > Residual norm 8.31544e-07 > Error norm 0.000372035 > Error norm 9.54604e-05 > Error norm 1.38928e-07 > > Part of the problem is coming from the gmres not generating a good > orthonormal basis of directions, so I ran > [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 > -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres > -ksp_gmres_modifiedgramschmidt > 0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm > 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 2.803631359711e-04 true resid norm > 2.803224197123e-04 ||Ae||/||Ax|| 6.296926643699e-03 > 2 KSP preconditioned resid norm 6.318659416138e-06 true resid norm > 6.395913715096e-06 ||Ae||/||Ax|| 1.436724269297e-04 > 3 KSP preconditioned resid norm 5.441804660222e-06 true resid norm > 5.559954843193e-06 ||Ae||/||Ax|| 1.248941407539e-04 > 4 KSP preconditioned resid norm 5.441734174383e-06 true resid norm > 5.580118795478e-06 ||Ae||/||Ax|| 1.253470836673e-04 > 5 KSP preconditioned resid norm 4.763217020809e-06 true resid norm > 4.965124844603e-06 ||Ae||/||Ax|| 1.115323830163e-04 > 6 KSP preconditioned resid norm 1.149661784439e-06 true resid norm > 1.742098334034e-06 ||Ae||/||Ax|| 3.913302862202e-05 > 7 KSP preconditioned resid norm 1.129046836468e-06 true resid norm > 1.607488229638e-06 ||Ae||/||Ax|| 3.610926069086e-05 > 8 KSP preconditioned resid norm 1.129005681832e-06 true resid norm > 1.682757897470e-06 ||Ae||/||Ax|| 3.780005499721e-05 > 9 KSP preconditioned resid norm 9.863339300864e-07 true resid norm > 1.657332745708e-06 ||Ae||/||Ax|| 3.722892870428e-05 > 10 KSP preconditioned resid norm 3.080494650476e-07 true resid norm > 1.298029587815e-06 ||Ae||/||Ax|| 2.915784352808e-05 > Residual norm 1.29803e-06 > Error norm 0.00100342 > Error norm 0.000631685 > Error norm 6.85996e-07 > > It looks like using the modified gram-schmidt orthogonalization helps > (that is it takes less iterations than with the classical (default) BUT > looking more closely we see that it actually terminated too early and > the answer is crappy. > > So lets turn off ALL the GMRES stuff > [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 > -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson > -mg_levels_ksp_type richardson > 0 KSP preconditioned resid norm 3.698072814941e+02 true resid norm > 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 4.145919799805e+01 true resid norm > 9.632258675992e-03 ||Ae||/||Ax|| 2.163709551096e-01 > 2 KSP preconditioned resid norm 4.848244190216e+00 true resid norm > 1.894293935038e-03 ||Ae||/||Ax|| 4.255182296038e-02 > 3 KSP preconditioned resid norm 5.902504324913e-01 true resid norm > 3.106203221250e-04 ||Ae||/||Ax|| 6.977513432503e-03 > 4 KSP preconditioned resid norm 7.403665781021e-02 true resid norm > 4.669937698054e-05 ||Ae||/||Ax|| 1.049015438184e-03 > 5 KSP preconditioned resid norm 9.483236819506e-03 true resid norm > 6.756537914043e-06 ||Ae||/||Ax|| 1.517731725471e-04 > 6 KSP preconditioned resid norm 1.249413820915e-03 true resid norm > 1.256137807104e-06 ||Ae||/||Ax|| 2.821682210197e-05 > Residual norm 1.25614e-06 > Error norm 0.000369181 > Error norm 9.46438e-05 > Error norm 1.37756e-07 > > The final residual doesn't look that great but if you look at the > error norms you see that the solution has fully converged. > > This final run is actually classical multigrid. So it looks like that > is the way to go; BUT you have to be very careful with how > you decide when it has converged. Obviously you want to stop as soon > as the algebraic error is smaller than the truncation error. > But, sadly we don't know what the algebraic error is (we only know the > residual) and we do not know what the truncation error is. > > My hack convergence criteria would be to stop as soon as the residual > norm DOES NOT decrease by at least (say) 20% > > To do a little better we can use SOR as the smoother instead of ILU > [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 > -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson > -mg_levels_ksp_type richardson -mg_levels_pc_type sor > 0 KSP preconditioned resid norm 3.202437744141e+02 true resid norm > 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 1.194643592834e+01 true resid norm > 2.088646404445e-03 ||Ae||/||Ax|| 4.691759496927e-02 > 2 KSP preconditioned resid norm 4.755175709724e-01 true resid norm > 1.131777826231e-04 ||Ae||/||Ax|| 2.542330417782e-03 > 3 KSP preconditioned resid norm 1.988304778934e-02 true resid norm > 6.019943612046e-06 ||Ae||/||Ax|| 1.352269464405e-04 > 4 KSP preconditioned resid norm 8.689521928318e-04 true resid norm > 8.886481737136e-07 ||Ae||/||Ax|| 1.996184437303e-05 > Residual norm 8.88648e-07 > Error norm 0.000368883 > Error norm 9.4803e-05 > Error norm 1.37996e-07 > > Hope this helps, > > Barry > > > > > > On Dec 2, 2008, at 7:52 PM, Chris Cooper wrote: > >> Hi, >> >> Just wondering if there are any techniques for improving DMMG >> convergence on a linear problem when operating in single precision? >> >> eg using ksp example 34 in Petsc 2.3.3 with multigrid levels changed >> to 6... >> >> double precision: >> 0 KSP Residual norm 4.451733404502e-02 >> 1 KSP Residual norm 2.801676364873e-04 >> 2 KSP Residual norm 3.215426166316e-06 >> 3 KSP Residual norm 5.446277176355e-08 >> Residual norm 5.44628e-08 >> Error norm 0.000356798 >> Error norm 9.21724e-05 >> Error norm 1.34208e-07 >> >> single precision: >> 0 KSP Residual norm 4.450287297368e-02 >> 1 KSP Residual norm 2.799208450597e-04 >> 2 KSP Residual norm 2.232483529951e-05 >> 3 KSP Residual norm 1.575733222126e-05 >> 4 KSP Residual norm 1.286925271415e-05 >> ... >> 27 KSP Residual norm 4.369261660031e-06 >> 28 KSP Residual norm 4.287577212381e-06 >> 29 KSP Residual norm 4.210308816255e-06 >> 30 KSP Residual norm 2.235170904896e-05 >> 31 KSP Residual norm 1.709007477757e-07 >> Residual norm 8.45584e-07 >> Error norm 0.000370784 >> Error norm 9.54664e-05 >> Error norm 1.38935e-07 >> >> thanks, >> Chris Cooper >> Animal Logic >> >> Animal Logic >> http://www.animallogic.com >> >> Please think of the environment before printing this email. >> >> This email and any attachments may be confidential and/or privileged. >> If you are not the intended recipient of this email, you must not >> disclose or use the information contained in it. Please notify the >> sender immediately and delete this document if you have received it >> in error. We do not guarantee this email is error or virus free. >> >> >> > Animal Logic http://www.animallogic.com Please think of the environment before printing this email. This email and any attachments may be confidential and/or privileged. If you are not the intended recipient of this email, you must not disclose or use the information contained in it. Please notify the sender immediately and delete this document if you have received it in error. We do not guarantee this email is error or virus free. From achatter at cse.psu.edu Thu Dec 4 12:26:50 2008 From: achatter at cse.psu.edu (Anirban Chatterjee) Date: Thu, 04 Dec 2008 13:26:50 -0500 Subject: Is there a way to provide graph coordinates? Message-ID: <493820EA.9040505@cse.psu.edu> Hi, Is there a way to provide coordinates to the nodes of a graph/matrix when declaring it? I have the adjacency matrix and want to provide (x y) coordinates to the nodes of the adjacency graph corresponding to the matrix. How do I do this in PETSc? I want to use a geometric partitioner on this adjacency matrix. Thanks, Anirban From achatter at cse.psu.edu Thu Dec 4 12:26:50 2008 From: achatter at cse.psu.edu (Anirban Chatterjee) Date: Thu, 04 Dec 2008 13:26:50 -0500 Subject: Is there a way to provide graph coordinates? Message-ID: <493820EA.9040505@cse.psu.edu> Hi, Is there a way to provide coordinates to the nodes of a graph/matrix when declaring it? I have the adjacency matrix and want to provide (x y) coordinates to the nodes of the adjacency graph corresponding to the matrix. How do I do this in PETSc? I want to use a geometric partitioner on this adjacency matrix. Thanks, Anirban From knepley at gmail.com Thu Dec 4 12:37:24 2008 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 4 Dec 2008 12:37:24 -0600 Subject: Is there a way to provide graph coordinates? In-Reply-To: <493820EA.9040505@cse.psu.edu> References: <493820EA.9040505@cse.psu.edu> Message-ID: We have no specific structure for this. I would just use a Vec. Matt On Thu, Dec 4, 2008 at 12:26 PM, Anirban Chatterjee wrote: > Hi, > > Is there a way to provide coordinates to the nodes of a graph/matrix when > declaring it? > I have the adjacency matrix and want to provide (x y) coordinates to the > nodes of the adjacency graph corresponding to the matrix. > How do I do this in PETSc? I want to use a geometric partitioner on this > adjacency matrix. > > Thanks, > Anirban > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Dec 4 14:45:11 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 4 Dec 2008 14:45:11 -0600 Subject: Timesolver: Newmark method? In-Reply-To: References: Message-ID: Lars, We do not have the resources or time to implement this method. We very happily accept contributions of new algorithms to the PETSc source from anyone. Barry On Nov 28, 2008, at 2:40 PM, Lars Rindorf wrote: > Dear all > > Is it planned to do the Newmark time scheme for second order time > differential equations in petsc? > > KR > _____________________ > Lars Rindorf > Technology consultant, Ph.D. > > Mobile +45 7220 3367 > > Surfaces and microtechnology > Danish Technological Institute > Gregersensvej > 2630 Taastrup > Denmark > http://www.dti.dk > Phone +45 7220 2000 > From Lars.Rindorf at teknologisk.dk Thu Dec 4 17:38:19 2008 From: Lars.Rindorf at teknologisk.dk (Lars Rindorf) Date: Fri, 5 Dec 2008 00:38:19 +0100 Subject: SV: Timesolver: Newmark method? In-Reply-To: References: Message-ID: Hi Barry I have sent the message below twice. For some reason, the first one stayed out in cyberspace for some days. It is dated Nov. 28. KR, Lars -----Oprindelig meddelelse----- Fra: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] P? vegne af Barry Smith Sendt: 4. december 2008 21:45 Til: petsc-users at mcs.anl.gov Emne: Re: Timesolver: Newmark method? Lars, We do not have the resources or time to implement this method. We very happily accept contributions of new algorithms to the PETSc source from anyone. Barry On Nov 28, 2008, at 2:40 PM, Lars Rindorf wrote: > Dear all > > Is it planned to do the Newmark time scheme for second order time > differential equations in petsc? > > KR > _____________________ > Lars Rindorf > Technology consultant, Ph.D. > > Mobile +45 7220 3367 > > Surfaces and microtechnology > Danish Technological Institute > Gregersensvej > 2630 Taastrup > Denmark > http://www.dti.dk > Phone +45 7220 2000 > From bsmith at mcs.anl.gov Fri Dec 5 15:39:12 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 5 Dec 2008 15:39:12 -0600 Subject: Local Jacobian subroutine in fortan In-Reply-To: <4936BDBB.5050007@umd.edu> References: <4936BDBB.5050007@umd.edu> Message-ID: <05E7C12F-A4C0-4876-A3B6-03077278DBFA@mcs.anl.gov> Sorry for the delay in responding. We had not implemented support for DMMGSetSNESLocal() for use from Fortran, hence the delay. I have added the needed code to PETSc and an example src/snes/ examples/tutorials/ex40f90.F You will need to use the development copy of PETSc petsc-dev http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html to access the new example. Good luck, and please report problems to petsc-maint at mcs.anl.gov Barry On Dec 3, 2008, at 11:11 AM, Saswata Hier-Majumder wrote: > Dear All, > I am looking for a fortran-based example of prescribing the local > Jacobian for a multicomponent PDE problem, using DA structure. If > anyone can point me to a fortran version of /snes/examples/tutorials/ > ex7.c or a similar problem, I'll greatly appreciate it. > Thanks > Sash > > -- > www.geol.umd.edu/~saswata > From saswata at umd.edu Sat Dec 6 09:30:07 2008 From: saswata at umd.edu (Saswata Hier-Majumder) Date: Sat, 06 Dec 2008 10:30:07 -0500 Subject: Local Jacobian subroutine in fortan In-Reply-To: <05E7C12F-A4C0-4876-A3B6-03077278DBFA@mcs.anl.gov> References: <4936BDBB.5050007@umd.edu> <05E7C12F-A4C0-4876-A3B6-03077278DBFA@mcs.anl.gov> Message-ID: <493A9A7F.4020406@umd.edu> Thanks a lot for taking the time, Barry. I really appreciate it. Sash Barry Smith wrote: > > Sorry for the delay in responding. We had not implemented support for > DMMGSetSNESLocal() for use from > Fortran, hence the delay. > > I have added the needed code to PETSc and an example > src/snes/examples/tutorials/ex40f90.F > You will need to use the development copy of PETSc petsc-dev > http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html > to access the new example. > > Good luck, and please report problems to petsc-maint at mcs.anl.gov > > > Barry > > On Dec 3, 2008, at 11:11 AM, Saswata Hier-Majumder wrote: > >> Dear All, >> I am looking for a fortran-based example of prescribing the local >> Jacobian for a multicomponent PDE problem, using DA structure. If >> anyone can point me to a fortran version of >> /snes/examples/tutorials/ex7.c or a similar problem, I'll greatly >> appreciate it. >> Thanks >> Sash >> >> --www.geol.umd.edu/~saswata >> > -- www.geol.umd.edu/~saswata From ntardieu at giref.ulaval.ca Mon Dec 8 12:59:45 2008 From: ntardieu at giref.ulaval.ca (ntardieu at giref.ulaval.ca) Date: Mon, 8 Dec 2008 13:59:45 -0500 (EST) Subject: Double solve on the coarse level? Message-ID: <18149.132.203.7.32.1228762785.squirrel@interne.giref.ulaval.ca> Dear PETSc users, We designed a multigrid solver based on the PCMG framework. But when one looks at the output, it seems that the system on the coarsest level is solved twice : indeed the same residuals appear twice. Is it a consequence of using a W cycle with an iteratif solver on the coarsest level? The PC object and a typical output are given below. Thanks, Nicolas ======================================================================= PC Object: type: mg MG: type is MULTIPLICATIVE, levels=2 cycles=w, pre-smooths=1, post-smooths=1 Coarse gride solver -- level 0 ------------------------------- KSP Object:(mg_coarse_) type: cr maximum iterations=30, initial guess is zero tolerances: relative=1e-06, absolute=1e-50, divergence=10000 left preconditioning PC Object:(mg_coarse_) type: sor SOR: type = local_symmetric, iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=24497, cols=24497 total: nonzeros=936401, allocated nonzeros=0 not using I-node routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object:(mg_levels_1_) type: chebychev Chebychev: eigenvalue estimates: min = 0.01, max = 100 maximum iterations=3, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning PC Object:(mg_levels_1_) type: sor SOR: type = local_symmetric, iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=59833, cols=59833 total: nonzeros=670149, allocated nonzeros=670149 not using I-node routines Up solver (post-smoother) on level 1 ------------------------------- KSP Object:(mg_levels_1_) type: chebychev Chebychev: eigenvalue estimates: min = 0.01, max = 100 maximum iterations=3 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning PC Object:(mg_levels_1_) type: sor SOR: type = local_symmetric, iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=59833, cols=59833 total: nonzeros=670149, allocated nonzeros=670149 not using I-node routines linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=59833, cols=59833 total: nonzeros=670149, allocated nonzeros=670149 not using I-node routines ======================================================================= Typical output : 0 KSP Residual norm 8.529532792227e-01 1 KSP Residual norm 8.312168822014e-01 2 KSP Residual norm 7.962289819435e-01 3 KSP Residual norm 7.498665547818e-01 0 KSP Residual norm 8.623070846983e-01 <<<<<<<<<< 1 KSP Residual norm 4.994728639387e-01 2 KSP Residual norm 4.441045285060e-01 3 KSP Residual norm 4.036899174922e-01 4 KSP Residual norm 3.741251351308e-01 5 KSP Residual norm 3.449669992123e-01 6 KSP Residual norm 3.220670285389e-01 7 KSP Residual norm 2.974753633724e-01 8 KSP Residual norm 2.786493949211e-01 9 KSP Residual norm 2.666365822398e-01 10 KSP Residual norm 2.615313007214e-01 11 KSP Residual norm 2.580757152907e-01 12 KSP Residual norm 2.549505151556e-01 13 KSP Residual norm 2.509670840632e-01 14 KSP Residual norm 2.464315316006e-01 15 KSP Residual norm 2.416198261586e-01 16 KSP Residual norm 2.359854861997e-01 17 KSP Residual norm 2.298769116813e-01 18 KSP Residual norm 2.232203276735e-01 19 KSP Residual norm 2.153461917946e-01 20 KSP Residual norm 2.072492142809e-01 21 KSP Residual norm 1.983342705665e-01 22 KSP Residual norm 1.894778760982e-01 23 KSP Residual norm 1.803772952130e-01 24 KSP Residual norm 1.718394016178e-01 25 KSP Residual norm 1.639952865611e-01 26 KSP Residual norm 1.571843225056e-01 27 KSP Residual norm 1.512651261054e-01 28 KSP Residual norm 1.459688114323e-01 29 KSP Residual norm 1.410245577886e-01 30 KSP Residual norm 1.364936912778e-01 0 KSP Residual norm 8.623070846983e-01 <<<<<<<<<< 1 KSP Residual norm 4.994728639387e-01 2 KSP Residual norm 4.441045285060e-01 3 KSP Residual norm 4.036899174922e-01 4 KSP Residual norm 3.741251351308e-01 5 KSP Residual norm 3.449669992123e-01 6 KSP Residual norm 3.220670285389e-01 7 KSP Residual norm 2.974753633724e-01 8 KSP Residual norm 2.786493949211e-01 9 KSP Residual norm 2.666365822398e-01 10 KSP Residual norm 2.615313007214e-01 11 KSP Residual norm 2.580757152907e-01 12 KSP Residual norm 2.549505151556e-01 13 KSP Residual norm 2.509670840632e-01 14 KSP Residual norm 2.464315316006e-01 15 KSP Residual norm 2.416198261586e-01 16 KSP Residual norm 2.359854861997e-01 17 KSP Residual norm 2.298769116813e-01 18 KSP Residual norm 2.232203276735e-01 19 KSP Residual norm 2.153461917946e-01 20 KSP Residual norm 2.072492142809e-01 21 KSP Residual norm 1.983342705665e-01 22 KSP Residual norm 1.894778760982e-01 23 KSP Residual norm 1.803772952130e-01 24 KSP Residual norm 1.718394016178e-01 25 KSP Residual norm 1.639952865611e-01 26 KSP Residual norm 1.571843225056e-01 27 KSP Residual norm 1.512651261054e-01 28 KSP Residual norm 1.459688114323e-01 29 KSP Residual norm 1.410245577886e-01 30 KSP Residual norm 1.364936912778e-01 0 KSP Residual norm 2.864424649239e+00 1 KSP Residual norm 2.719941822422e+00 2 KSP Residual norm 2.486679927440e+00 3 KSP Residual norm 2.176182306332e+00 0 KSP preconditioned resid norm 3.133018965496e+02 true resid norm 5.229629619561e+01 ||Ae||/||Ax|| 1.000000000000e+00 ======================================================================= From keita at cray.com Mon Dec 8 13:59:02 2008 From: keita at cray.com (Keita Teranishi) Date: Mon, 8 Dec 2008 13:59:02 -0600 Subject: Scotch5.1 support Message-ID: <925346A443D4E340BEB20248BAFCDBDF089F5741@CFEVS1-IP.americas.cray.com> Hi, Do you have any plan to support scotch-5.1? Thanks, ================================ Keita Teranishi Math Software Group Cray, Inc. keita at cray.com ================================ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Dec 8 14:59:29 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 8 Dec 2008 14:59:29 -0600 Subject: Scotch5.1 support In-Reply-To: <925346A443D4E340BEB20248BAFCDBDF089F5741@CFEVS1-IP.americas.cray.com> References: <925346A443D4E340BEB20248BAFCDBDF089F5741@CFEVS1-IP.americas.cray.com> Message-ID: This is what is being used in petsc-dev but it is not clear how much it is working. Barry BTW: we will be releasing PETSc 3.0 very soon On Dec 8, 2008, at 1:59 PM, Keita Teranishi wrote: > Hi, > > Do you have any plan to support scotch-5.1? > > Thanks, > ================================ > Keita Teranishi > Math Software Group > Cray, Inc. > keita at cray.com > ================================ From bsmith at mcs.anl.gov Mon Dec 8 15:51:05 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 8 Dec 2008 15:51:05 -0600 Subject: Double solve on the coarse level? In-Reply-To: <18149.132.203.7.32.1228762785.squirrel@interne.giref.ulaval.ca> References: <18149.132.203.7.32.1228762785.squirrel@interne.giref.ulaval.ca> Message-ID: <7E566FBC-91F2-4823-9127-472B333A6BE2@mcs.anl.gov> On Dec 8, 2008, at 12:59 PM, ntardieu at giref.ulaval.ca wrote: > Dear PETSc users, > > We designed a multigrid solver based on the PCMG framework. > But when one looks at the output, it seems that the system on the > coarsest > level is solved twice : indeed the same residuals appear twice. Is > it a > consequence of using a W cycle with an iteratif solver on the coarsest > level? No. It happens also with a direct coarse grid solver. This is an error that happens with the W cycle. In PCMGMCycle_Private() is the line while (cycles--) { ierr = PCMGMCycle_Private(pc,mglevels-1,reason);CHKERRQ(ierr); } when cycles is 2 (which is w cycle). I have put a correction into petsc-dev to define cycles to be 1 on the 1st level even for w cycle. PetscInt cycles = (mg->level == 1) ? 1 : (PetscInt) mg->cycles; Thanks for reporting the problem, Barry > > The PC object and a typical output are given below. > > Thanks, > > Nicolas > > = > ====================================================================== > PC Object: > type: mg > MG: type is MULTIPLICATIVE, levels=2 cycles=w, pre-smooths=1, > post-smooths=1 > Coarse gride solver -- level 0 ------------------------------- > KSP Object:(mg_coarse_) > type: cr > maximum iterations=30, initial guess is zero > tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > left preconditioning > PC Object:(mg_coarse_) > type: sor > SOR: type = local_symmetric, iterations = 1, omega = 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=24497, cols=24497 > total: nonzeros=936401, allocated nonzeros=0 > not using I-node routines > Down solver (pre-smoother) on level 1 ------------------------------- > KSP Object:(mg_levels_1_) > type: chebychev > Chebychev: eigenvalue estimates: min = 0.01, max = 100 > maximum iterations=3, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object:(mg_levels_1_) > type: sor > SOR: type = local_symmetric, iterations = 1, omega = 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=59833, cols=59833 > total: nonzeros=670149, allocated nonzeros=670149 > not using I-node routines > Up solver (post-smoother) on level 1 ------------------------------- > KSP Object:(mg_levels_1_) > type: chebychev > Chebychev: eigenvalue estimates: min = 0.01, max = 100 > maximum iterations=3 > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object:(mg_levels_1_) > type: sor > SOR: type = local_symmetric, iterations = 1, omega = 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=59833, cols=59833 > total: nonzeros=670149, allocated nonzeros=670149 > not using I-node routines > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=59833, cols=59833 > total: nonzeros=670149, allocated nonzeros=670149 > not using I-node routines > > = > ====================================================================== > Typical output : > > 0 KSP Residual norm 8.529532792227e-01 > 1 KSP Residual norm 8.312168822014e-01 > 2 KSP Residual norm 7.962289819435e-01 > 3 KSP Residual norm 7.498665547818e-01 > 0 KSP Residual norm 8.623070846983e-01 <<<<<<<<<< > 1 KSP Residual norm 4.994728639387e-01 > 2 KSP Residual norm 4.441045285060e-01 > 3 KSP Residual norm 4.036899174922e-01 > 4 KSP Residual norm 3.741251351308e-01 > 5 KSP Residual norm 3.449669992123e-01 > 6 KSP Residual norm 3.220670285389e-01 > 7 KSP Residual norm 2.974753633724e-01 > 8 KSP Residual norm 2.786493949211e-01 > 9 KSP Residual norm 2.666365822398e-01 > 10 KSP Residual norm 2.615313007214e-01 > 11 KSP Residual norm 2.580757152907e-01 > 12 KSP Residual norm 2.549505151556e-01 > 13 KSP Residual norm 2.509670840632e-01 > 14 KSP Residual norm 2.464315316006e-01 > 15 KSP Residual norm 2.416198261586e-01 > 16 KSP Residual norm 2.359854861997e-01 > 17 KSP Residual norm 2.298769116813e-01 > 18 KSP Residual norm 2.232203276735e-01 > 19 KSP Residual norm 2.153461917946e-01 > 20 KSP Residual norm 2.072492142809e-01 > 21 KSP Residual norm 1.983342705665e-01 > 22 KSP Residual norm 1.894778760982e-01 > 23 KSP Residual norm 1.803772952130e-01 > 24 KSP Residual norm 1.718394016178e-01 > 25 KSP Residual norm 1.639952865611e-01 > 26 KSP Residual norm 1.571843225056e-01 > 27 KSP Residual norm 1.512651261054e-01 > 28 KSP Residual norm 1.459688114323e-01 > 29 KSP Residual norm 1.410245577886e-01 > 30 KSP Residual norm 1.364936912778e-01 > 0 KSP Residual norm 8.623070846983e-01 <<<<<<<<<< > 1 KSP Residual norm 4.994728639387e-01 > 2 KSP Residual norm 4.441045285060e-01 > 3 KSP Residual norm 4.036899174922e-01 > 4 KSP Residual norm 3.741251351308e-01 > 5 KSP Residual norm 3.449669992123e-01 > 6 KSP Residual norm 3.220670285389e-01 > 7 KSP Residual norm 2.974753633724e-01 > 8 KSP Residual norm 2.786493949211e-01 > 9 KSP Residual norm 2.666365822398e-01 > 10 KSP Residual norm 2.615313007214e-01 > 11 KSP Residual norm 2.580757152907e-01 > 12 KSP Residual norm 2.549505151556e-01 > 13 KSP Residual norm 2.509670840632e-01 > 14 KSP Residual norm 2.464315316006e-01 > 15 KSP Residual norm 2.416198261586e-01 > 16 KSP Residual norm 2.359854861997e-01 > 17 KSP Residual norm 2.298769116813e-01 > 18 KSP Residual norm 2.232203276735e-01 > 19 KSP Residual norm 2.153461917946e-01 > 20 KSP Residual norm 2.072492142809e-01 > 21 KSP Residual norm 1.983342705665e-01 > 22 KSP Residual norm 1.894778760982e-01 > 23 KSP Residual norm 1.803772952130e-01 > 24 KSP Residual norm 1.718394016178e-01 > 25 KSP Residual norm 1.639952865611e-01 > 26 KSP Residual norm 1.571843225056e-01 > 27 KSP Residual norm 1.512651261054e-01 > 28 KSP Residual norm 1.459688114323e-01 > 29 KSP Residual norm 1.410245577886e-01 > 30 KSP Residual norm 1.364936912778e-01 > 0 KSP Residual norm 2.864424649239e+00 > 1 KSP Residual norm 2.719941822422e+00 > 2 KSP Residual norm 2.486679927440e+00 > 3 KSP Residual norm 2.176182306332e+00 > 0 KSP preconditioned resid norm 3.133018965496e+02 true resid norm > 5.229629619561e+01 ||Ae||/||Ax|| 1.000000000000e+00 > > = > ====================================================================== > From mossaiby at yahoo.com Wed Dec 10 11:00:46 2008 From: mossaiby at yahoo.com (Farshid Mossaiby) Date: Wed, 10 Dec 2008 09:00:46 -0800 (PST) Subject: Non-local values Message-ID: <447671.28315.qm@web52208.mail.re2.yahoo.com> Hi all, I know that this has been probably asked zilions of times, just to make sure, what is the way to go if a process needs some values from a vec/mat that is not local to that process? Is there any way? For example each process provides an array of global indices, and in a (of course collective) call, gets the related values from appropriate processes? Thank you, Farshid Mossaiby From knepley at gmail.com Wed Dec 10 11:28:26 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 10 Dec 2008 11:28:26 -0600 Subject: Non-local values In-Reply-To: <447671.28315.qm@web52208.mail.re2.yahoo.com> References: <447671.28315.qm@web52208.mail.re2.yahoo.com> Message-ID: On Wed, Dec 10, 2008 at 11:00 AM, Farshid Mossaiby wrote: > Hi all, > > I know that this has been probably asked zilions of times, just to make > sure, what is the way to go if a process needs some values from a vec/mat > that is not local to that process? Is there any way? For example each > process provides an array of global indices, and in a (of course collective) > call, gets the related values from appropriate processes? VecScatter is the correct way to structure this communication. Matt > > Thank you, > Farshid Mossaiby > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mossaiby at yahoo.com Wed Dec 10 12:32:07 2008 From: mossaiby at yahoo.com (Farshid Mossaiby) Date: Wed, 10 Dec 2008 10:32:07 -0800 (PST) Subject: Non-local values In-Reply-To: Message-ID: <288299.61540.qm@web52202.mail.re2.yahoo.com> Thanks for your reply. Of course this is a way, but I just need some of the values (probably few non-locals) on each process. I do not want to reproduce the same vector on all machines. Am I wrong? Farshid Mossaiby --- On Wed, 12/10/08, Matthew Knepley wrote: > From: Matthew Knepley > Subject: Re: Non-local values > To: petsc-users at mcs.anl.gov > Date: Wednesday, December 10, 2008, 8:58 PM > On Wed, Dec 10, 2008 at 11:00 AM, Farshid Mossaiby > wrote: > > > Hi all, > > > > I know that this has been probably asked zilions of > times, just to make > > sure, what is the way to go if a process needs some > values from a vec/mat > > that is not local to that process? Is there any way? > For example each > > process provides an array of global indices, and in a > (of course collective) > > call, gets the related values from appropriate > processes? > > > VecScatter is the correct way to structure this > communication. > > Matt > > > > > > Thank you, > > Farshid Mossaiby > > > -- > What most experimenters take for granted before they begin > their experiments > is infinitely more interesting than any results to which > their experiments > lead. > -- Norbert Wiener From knepley at gmail.com Wed Dec 10 12:36:20 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 10 Dec 2008 12:36:20 -0600 Subject: Non-local values In-Reply-To: <288299.61540.qm@web52202.mail.re2.yahoo.com> References: <288299.61540.qm@web52202.mail.re2.yahoo.com> Message-ID: On Wed, Dec 10, 2008 at 12:32 PM, Farshid Mossaiby wrote: > Thanks for your reply. > > Of course this is a way, but I just need some of the values (probably few > non-locals) on each process. I do not want to reproduce the same vector on > all machines. Am I wrong? VecScatter is a general mechanism. If you want to transfer 3 values, make a vector length 3 and a VecScatter from the global vector to that small vector. Matt > > Farshid Mossaiby > > --- On Wed, 12/10/08, Matthew Knepley wrote: > > > From: Matthew Knepley > > Subject: Re: Non-local values > > To: petsc-users at mcs.anl.gov > > Date: Wednesday, December 10, 2008, 8:58 PM > > On Wed, Dec 10, 2008 at 11:00 AM, Farshid Mossaiby > > wrote: > > > > > Hi all, > > > > > > I know that this has been probably asked zilions of > > times, just to make > > > sure, what is the way to go if a process needs some > > values from a vec/mat > > > that is not local to that process? Is there any way? > > For example each > > > process provides an array of global indices, and in a > > (of course collective) > > > call, gets the related values from appropriate > > processes? > > > > > > VecScatter is the correct way to structure this > > communication. > > > > Matt > > > > > > > > > > Thank you, > > > Farshid Mossaiby > > > > > -- > > What most experimenters take for granted before they begin > > their experiments > > is infinitely more interesting than any results to which > > their experiments > > lead. > > -- Norbert Wiener > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jarunan.Panyasantisuk at eleves.ec-nantes.fr Fri Dec 12 09:43:32 2008 From: Jarunan.Panyasantisuk at eleves.ec-nantes.fr (Panyasantisuk Jarunan) Date: Fri, 12 Dec 2008 16:43:32 +0100 Subject: Memory Usage Analysis Message-ID: <20081212164332.jzb0v2njsbcgssoc@webmail.ec-nantes.fr> Hello, I would like to do memory usage analysis by using these command. call PetscMemorySetGetMaximumUsage(ierr) call PetscMemoryGetMaximumUsage(mem,ierr) But when I compiled, it said undefined reference to `petscmemorysetgetmaximumusage_' I use Fortran and have included petsc.h, petscvec.h, petscmat.h, petscpc.h, petscksp.h, petscsys.h. In makefile I use ${PETSC_KSP_LIB}. So now I use the command "call PetscMemoryGetCurrentUsage(mem,ierr)" to get memory from 2 points. I do not know if it is efficient. Could you please give me some advice? Thank you -- Jarunan PANYASANTISUK MSc. in Computational Mechanics Erasmus Mundus Master Program Ecole Centrale de Nantes 1, rue de la no?, 44321 NANTES, FRANCE From Jarunan.Panyasantisuk at eleves.ec-nantes.fr Fri Dec 12 09:46:02 2008 From: Jarunan.Panyasantisuk at eleves.ec-nantes.fr (Panyasantisuk Jarunan) Date: Fri, 12 Dec 2008 16:46:02 +0100 Subject: BlockSolve95 error In-Reply-To: <925346A443D4E340BEB20248BAFCDBDF088166EA@CFEVS1-IP.americas.cray.com> References: <20081126154353.rrn4jl3k440sco4w@webmail.ec-nantes.fr> <925346A443D4E340BEB20248BAFCDBDF088166EA@CFEVS1-IP.americas.cray.com> Message-ID: <20081212164602.ioyooqyfk3s4gwo8@webmail.ec-nantes.fr> Thanks, it still does not work in a big case, as you adviced. I will try to investigate. Regards, Jarunan -- Jarunan PANYASANTISUK MSc. in Computational Mechanics Erasmus Mundus Master Program Ecole Centrale de Nantes 1, rue de la no?, 44321 NANTES, FRANCE Keita Teranishi a ??crit? : > Jarunan, > > Make the value for MAX_NUM_MSGS bigger in BlockSolve95/include/BSprivate.h. > >> #define MAX_NUM_MSGS 10000000 > > You will need to change the values of other "MAX_XXXX" variables in > the header files because computers nowadays have much bigger > capacity/capability than those in mid 90's. > > > ================================ > Keita Teranishi > Math Software Group > Cray, Inc. > keita at cray.com > ================================ > > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Barry Smith > Sent: Wednesday, November 26, 2008 3:20 PM > To: petsc-users at mcs.anl.gov > Subject: Re: BlockSolve95 error > > > I have no idea. BlockSolve has not been supported by its > developers for like > 10 years. Those messages are coming from inside BlockSolve which we > don't > support. > > I'm sorry for the terse response but we cannot support all the > orphaned parallel > numerical software in the world. > > Barry > > On Nov 26, 2008, at 8:43 AM, Panyasantisuk Jarunan wrote: > >> Hello, >> >> Could you please explain the error I got from running code with >> blocksolve95?: >> ERROR: Code -30 occured at Line 103 in File BMcomp_msg.c: Too many >> messages in comp_msg >> ERROR: Code -30 occured at Line 48 in File BSsetup_factor.c >> >> I cannot find BMcomp_msg.c and BSsetup_factor.c anywhere. >> >> For running I use: mpirun -np 3 -machinefile machines ./test2 >> It works with other test cases though. >> >> In the fortran code: >> >> call MatCreate(PETSC_COMM_WORLD,D,ierr) >> call MatSetSizes(D,N,N,PETSC_DETERMINE,PETSC_DETERMINE, >> $ ierr) >> call MatSetType(D,MATMPIROWBS,ierr) >> call MatSetFromOptions(D,ierr) >> call MatMPIRowbsSetPreallocation(D,nz,PETSC_NULL_INTEGER,ierr) >> ... >> ... >> >> call KSPCreate(PETSC_COMM_WORLD,ksp,ierr) >> call KSPSetOperators(ksp,D,D,DIFFERENT_NONZERO_PATTERN,ierr) >> call KSPGetPC(ksp,pc,ierr) >> call PCSetType(pc,PCILU,ierr) >> >> Thank you before hand >> >> Regards, >> Jarunan >> >> -- >> Jarunan PANYASANTISUK >> MSc. in Computational Mechanics >> Erasmus Mundus Master Program >> Ecole Centrale de Nantes >> 1, rue de la no?, 44321 NANTES, FRANCE >> >> >> > > From knepley at gmail.com Fri Dec 12 10:56:38 2008 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 12 Dec 2008 10:56:38 -0600 Subject: Memory Usage Analysis In-Reply-To: <20081212164332.jzb0v2njsbcgssoc@webmail.ec-nantes.fr> References: <20081212164332.jzb0v2njsbcgssoc@webmail.ec-nantes.fr> Message-ID: These have changed to PetscMemoryGetMaximumUsage PetscMemorySetGetMaximumUsage Matt On Fri, Dec 12, 2008 at 9:43 AM, Panyasantisuk Jarunan < Jarunan.Panyasantisuk at eleves.ec-nantes.fr> wrote: > Hello, > > I would like to do memory usage analysis by using these command. > > call PetscMemorySetGetMaximumUsage(ierr) > call PetscMemoryGetMaximumUsage(mem,ierr) > > But when I compiled, it said > > undefined reference to `petscmemorysetgetmaximumusage_' > > I use Fortran and have included petsc.h, petscvec.h, petscmat.h, petscpc.h, > petscksp.h, petscsys.h. In makefile I use ${PETSC_KSP_LIB}. > > So now I use the command "call PetscMemoryGetCurrentUsage(mem,ierr)" to get > memory from 2 points. I do not know if it is efficient. > > Could you please give me some advice? > > Thank you > > > > > -- > Jarunan PANYASANTISUK > MSc. in Computational Mechanics > Erasmus Mundus Master Program > Ecole Centrale de Nantes > 1, rue de la no?, 44321 NANTES, FRANCE > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From achatter at cse.psu.edu Fri Dec 12 11:10:30 2008 From: achatter at cse.psu.edu (Anirban Chatterjee) Date: Fri, 12 Dec 2008 12:10:30 -0500 Subject: HYPRE_BoomerAMG and FEI interface Message-ID: <49429B06.5010205@cse.psu.edu> Hi, I want to use hypre from petsc. Specifically, how do I use the BoomerAMG solver in hypre with the Finite Element Interface (FEI) through petsc? I have installed petsc along with the hypre package as an external package. Thanks, Anirban From knepley at gmail.com Fri Dec 12 11:17:11 2008 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 12 Dec 2008 11:17:11 -0600 Subject: HYPRE_BoomerAMG and FEI interface In-Reply-To: <49429B06.5010205@cse.psu.edu> References: <49429B06.5010205@cse.psu.edu> Message-ID: -pc_type hypre -pc_hypre_type boomeramg Matt On Fri, Dec 12, 2008 at 11:10 AM, Anirban Chatterjee wrote: > Hi, > > I want to use hypre from petsc. Specifically, how do I use the BoomerAMG > solver in hypre with the Finite Element Interface (FEI) through petsc? > I have installed petsc along with the hypre package as an external package. > > Thanks, > Anirban > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 12 11:52:28 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 12 Dec 2008 11:52:28 -0600 Subject: Memory Usage Analysis In-Reply-To: <20081212164332.jzb0v2njsbcgssoc@webmail.ec-nantes.fr> References: <20081212164332.jzb0v2njsbcgssoc@webmail.ec-nantes.fr> Message-ID: Sorry, the Fortran stub for this function is missing. If you are using petsc-dev you can just do hg pull; hg update to get my fix. If you are using petsc-2.3.3 I have attached a replacement src/sys/memory/ftn-custom/zmtrf.c You can just stick it in and run make in that one directory. Barry On Dec 12, 2008, at 9:43 AM, Panyasantisuk Jarunan wrote: > Hello, > > I would like to do memory usage analysis by using these command. > > call PetscMemorySetGetMaximumUsage(ierr) > call PetscMemoryGetMaximumUsage(mem,ierr) > > But when I compiled, it said > > undefined reference to `petscmemorysetgetmaximumusage_' > > I use Fortran and have included petsc.h, petscvec.h, petscmat.h, > petscpc.h, petscksp.h, petscsys.h. In makefile I use ${PETSC_KSP_LIB}. > > So now I use the command "call PetscMemoryGetCurrentUsage(mem,ierr)" > to get memory from 2 points. I do not know if it is efficient. > > Could you please give me some advice? > > Thank you > > > > > -- > Jarunan PANYASANTISUK > MSc. in Computational Mechanics > Erasmus Mundus Master Program > Ecole Centrale de Nantes > 1, rue de la no?, 44321 NANTES, FRANCE > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: zmtrf.c Type: application/octet-stream Size: 1235 bytes Desc: not available URL: From keita at cray.com Fri Dec 12 12:48:47 2008 From: keita at cray.com (Keita Teranishi) Date: Fri, 12 Dec 2008 12:48:47 -0600 Subject: HYPRE_BoomerAMG and FEI interface In-Reply-To: References: <49429B06.5010205@cse.psu.edu> Message-ID: <925346A443D4E340BEB20248BAFCDBDF08B390AF@CFEVS1-IP.americas.cray.com> Anirban, The file src/mat/impls/hypre/mhyp.c indicates that PETSc only supports Linear algebra system (IJ) interface. Why don't you work directly on hypre's FEI interface test program? Thanks, ================================ Keita Teranishi Math Software Group Cray, Inc. keita at cray.com ================================ From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Matthew Knepley Sent: Friday, December 12, 2008 11:17 AM To: petsc-users at mcs.anl.gov Subject: Re: HYPRE_BoomerAMG and FEI interface -pc_type hypre -pc_hypre_type boomeramg Matt On Fri, Dec 12, 2008 at 11:10 AM, Anirban Chatterjee wrote: Hi, I want to use hypre from petsc. Specifically, how do I use the BoomerAMG solver in hypre with the Finite Element Interface (FEI) through petsc? I have installed petsc along with the hypre package as an external package. Thanks, Anirban -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From achatter at cse.psu.edu Fri Dec 12 13:43:31 2008 From: achatter at cse.psu.edu (Anirban Chatterjee) Date: Fri, 12 Dec 2008 14:43:31 -0500 Subject: HYPRE_BoomerAMG and FEI interface In-Reply-To: References: <49429B06.5010205@cse.psu.edu> Message-ID: <4942BEE3.90001@cse.psu.edu> Does the data have to be in a special format? Because I want to use Hypre's Finite Element Interface to input unstructured meshes. Basically, if I format the data in the Hypre FEI format and send it through Petsc with these command line options then will it work. Thanks, Anirban Matthew Knepley wrote: > -pc_type hypre -pc_hypre_type boomeramg > > Matt > > On Fri, Dec 12, 2008 at 11:10 AM, Anirban Chatterjee > > wrote: > > Hi, > > I want to use hypre from petsc. Specifically, how do I use the > BoomerAMG solver in hypre with the Finite Element Interface (FEI) > through petsc? > I have installed petsc along with the hypre package as an external > package. > > Thanks, > Anirban > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener From achatter at cse.psu.edu Fri Dec 12 14:04:28 2008 From: achatter at cse.psu.edu (Anirban Chatterjee) Date: Fri, 12 Dec 2008 15:04:28 -0500 Subject: HYPRE_BoomerAMG and FEI interface In-Reply-To: <925346A443D4E340BEB20248BAFCDBDF08B390AF@CFEVS1-IP.americas.cray.com> References: <49429B06.5010205@cse.psu.edu> <925346A443D4E340BEB20248BAFCDBDF08B390AF@CFEVS1-IP.americas.cray.com> Message-ID: <4942C3CC.9050909@cse.psu.edu> Thanks Keita. That is what I am trying to do now. Thanks, Anirban Keita Teranishi wrote: > > Anirban, > > The file src/mat/impls/hypre/mhyp.c indicates that PETSc only supports > Linear algebra system (IJ) interface. > > Why don?t you work directly on hypre?s FEI interface test program? > > Thanks, > > ================================ > Keita Teranishi > Math Software Group > Cray, Inc. > keita at cray.com > ================================ > > *From:* owner-petsc-users at mcs.anl.gov > [mailto:owner-petsc-users at mcs.anl.gov] *On Behalf Of *Matthew Knepley > *Sent:* Friday, December 12, 2008 11:17 AM > *To:* petsc-users at mcs.anl.gov > *Subject:* Re: HYPRE_BoomerAMG and FEI interface > > -pc_type hypre -pc_hypre_type boomeramg > > Matt > > On Fri, Dec 12, 2008 at 11:10 AM, Anirban Chatterjee > > wrote: > > Hi, > > I want to use hypre from petsc. Specifically, how do I use the > BoomerAMG solver in hypre with the Finite Element Interface (FEI) > through petsc? > I have installed petsc along with the hypre package as an external > package. > > Thanks, > Anirban > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > From bsmith at mcs.anl.gov Fri Dec 12 14:25:29 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 12 Dec 2008 14:25:29 -0600 Subject: HYPRE_BoomerAMG and FEI interface In-Reply-To: <4942BEE3.90001@cse.psu.edu> References: <49429B06.5010205@cse.psu.edu> <4942BEE3.90001@cse.psu.edu> Message-ID: <149D318B-1433-4AAD-AA1B-8311CBD8CC53@mcs.anl.gov> On Dec 12, 2008, at 1:43 PM, Anirban Chatterjee wrote: > Does the data have to be in a special format? Because I want to use > Hypre's Finite Element Interface to input unstructured meshes. > Basically, if I format the data in the Hypre FEI format and send it > through Petsc with these command line options then will it work. If you put the data directly into Hypre via the FEI then there is no PETSc involved in the process and so PETSc's command line options won't work. Barry > > > Thanks, > Anirban > > Matthew Knepley wrote: >> -pc_type hypre -pc_hypre_type boomeramg >> >> Matt >> >> On Fri, Dec 12, 2008 at 11:10 AM, Anirban Chatterjee > > wrote: >> >> Hi, >> >> I want to use hypre from petsc. Specifically, how do I use the >> BoomerAMG solver in hypre with the Finite Element Interface (FEI) >> through petsc? >> I have installed petsc along with the hypre package as an external >> package. >> >> Thanks, >> Anirban >> >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > From recrusader at gmail.com Wed Dec 17 15:02:17 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 17 Dec 2008 13:02:17 -0800 Subject: how to sum two matrices? Message-ID: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> Hi, I have matrices A1 and A2. It is distributed in the cluster. Their distribution is same I want to do A=a1A1+a2A2. How to do it? I can't find the suitble function. thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Dec 17 15:17:24 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 17 Dec 2008 15:17:24 -0600 Subject: how to sum two matrices? In-Reply-To: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> References: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> Message-ID: MatAXPY() On Dec 17, 2008, at 3:02 PM, Yujie wrote: > Hi, > > I have matrices A1 and A2. It is distributed in the cluster. Their > distribution is same > > I want to do A=a1A1+a2A2. How to do it? I can't find the suitble > function. thanks a lot. > > Regards, > > Yujie > From recrusader at gmail.com Wed Dec 17 15:19:22 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 17 Dec 2008 13:19:22 -0800 Subject: how to sum two matrices? In-Reply-To: References: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> Message-ID: <7ff0ee010812171319u764fbb92k374fd2c6c7d2e83e@mail.gmail.com> Thanks, Barry. I got it :). Regards, Yujie On Wed, Dec 17, 2008 at 1:17 PM, Barry Smith wrote: > > MatAXPY() > > On Dec 17, 2008, at 3:02 PM, Yujie wrote: > > Hi, >> >> I have matrices A1 and A2. It is distributed in the cluster. Their >> distribution is same >> >> I want to do A=a1A1+a2A2. How to do it? I can't find the suitble function. >> thanks a lot. >> >> Regards, >> >> Yujie >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Wed Dec 17 15:33:09 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 17 Dec 2008 13:33:09 -0800 Subject: how to sum two matrices? In-Reply-To: References: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> Message-ID: <7ff0ee010812171333q2845e8b2j68cce39acc7d9639@mail.gmail.com> Hi, Barry I want to do A=a1A1+a2A2. MatAXPY is just for Y = a*X + Y. Though I can realize my application, it is not good. Do you have direct function for my application? thanks a lot. Regards, Yujie On Wed, Dec 17, 2008 at 1:17 PM, Barry Smith wrote: > > MatAXPY() > > On Dec 17, 2008, at 3:02 PM, Yujie wrote: > > Hi, >> >> I have matrices A1 and A2. It is distributed in the cluster. Their >> distribution is same >> >> I want to do A=a1A1+a2A2. How to do it? I can't find the suitble function. >> thanks a lot. >> >> Regards, >> >> Yujie >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Dec 17 15:37:06 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 17 Dec 2008 15:37:06 -0600 Subject: how to sum two matrices? In-Reply-To: <7ff0ee010812171333q2845e8b2j68cce39acc7d9639@mail.gmail.com> References: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> <7ff0ee010812171333q2845e8b2j68cce39acc7d9639@mail.gmail.com> Message-ID: If PETSc had a function that does a direct sum as you requested I would have told you. Just do a copy, a scale and then an MatAXPY() Barry On Dec 17, 2008, at 3:33 PM, Yujie wrote: > Hi, Barry > > I want to do A=a1A1+a2A2. MatAXPY is just for Y = a*X + Y. Though I > can realize my application, it is not good. Do you have direct > function for my application? thanks a lot. > > Regards, > > Yujie > > > On Wed, Dec 17, 2008 at 1:17 PM, Barry Smith > wrote: > > MatAXPY() > > > On Dec 17, 2008, at 3:02 PM, Yujie wrote: > > Hi, > > I have matrices A1 and A2. It is distributed in the cluster. Their > distribution is same > > I want to do A=a1A1+a2A2. How to do it? I can't find the suitble > function. thanks a lot. > > Regards, > > Yujie > > > From recrusader at gmail.com Wed Dec 17 15:39:09 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 17 Dec 2008 13:39:09 -0800 Subject: how to sum two matrices? In-Reply-To: References: <7ff0ee010812171302y5d1b9e4an50f9f392f179c443@mail.gmail.com> <7ff0ee010812171333q2845e8b2j68cce39acc7d9639@mail.gmail.com> Message-ID: <7ff0ee010812171339u44326498qb89a03abacfd9d17@mail.gmail.com> :). thanks, Barry On Wed, Dec 17, 2008 at 1:37 PM, Barry Smith wrote: > > If PETSc had a function that does a direct sum as you requested I would > have told you. > Just do a copy, a scale and then an MatAXPY() > > Barry > > On Dec 17, 2008, at 3:33 PM, Yujie wrote: > > Hi, Barry >> >> I want to do A=a1A1+a2A2. MatAXPY is just for Y = a*X + Y. Though I can >> realize my application, it is not good. Do you have direct function for my >> application? thanks a lot. >> >> Regards, >> >> Yujie >> >> >> On Wed, Dec 17, 2008 at 1:17 PM, Barry Smith wrote: >> >> MatAXPY() >> >> >> On Dec 17, 2008, at 3:02 PM, Yujie wrote: >> >> Hi, >> >> I have matrices A1 and A2. It is distributed in the cluster. Their >> distribution is same >> >> I want to do A=a1A1+a2A2. How to do it? I can't find the suitble function. >> thanks a lot. >> >> Regards, >> >> Yujie >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naromero at alcf.anl.gov Wed Dec 17 16:20:41 2008 From: naromero at alcf.anl.gov (Nichols A. Romero) Date: Wed, 17 Dec 2008 16:20:41 -0600 (CST) Subject: petsc-2.3.3-p15 In-Reply-To: <26746210.711361229551883039.JavaMail.root@zimbra> Message-ID: <21668470.712061229552441938.JavaMail.root@zimbra> Hi, This is regarding both surveyor.alcf.anl.gov and intrepid.alcf.anl.gov. Would it be possible for someone to bring PETSc 2.3.3 up to the latest patch level? Right now they are only at -p13. Nichols A. Romero, Ph.D. Argonne Leadership Computing Facility Argonne National Laboratory Building 360 Room L-146 9700 South Cass Avenue Argonne, IL 60490 (630) 252-3441 From balay at mcs.anl.gov Wed Dec 17 17:38:34 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 17 Dec 2008 17:38:34 -0600 (CST) Subject: petsc-2.3.3-p15 In-Reply-To: <21668470.712061229552441938.JavaMail.root@zimbra> References: <21668470.712061229552441938.JavaMail.root@zimbra> Message-ID: I should be able to this. However we are in the process of a new release - so I should be testing that on bgp first.. Satish On Wed, 17 Dec 2008, Nichols A. Romero wrote: > Hi, > > This is regarding both surveyor.alcf.anl.gov and intrepid.alcf.anl.gov. > > Would it be possible for someone to bring PETSc 2.3.3 up to the latest patch > level? > > Right now they are only at -p13. > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 > > From naromero at alcf.anl.gov Wed Dec 17 21:36:25 2008 From: naromero at alcf.anl.gov (Nichols A. Romero) Date: Wed, 17 Dec 2008 21:36:25 -0600 (CST) Subject: external solvers - scalapack In-Reply-To: <2896387.719641229571061941.JavaMail.root@zimbra> Message-ID: <28702257.719761229571385757.JavaMail.root@zimbra> Hi, I looked through the mailing lists to see if this question was already asked but could not find anything. In the PETSc TACC2008 tutorial, it states that PETSc interfaces with Scalapack? I would like to know what is supported from Scalapack. I am particularly interested in: 1. Cholesky factorization (I know there are a couple of alternative options here, e.g. PLAPACK). 2. Dense eigenvalue solver Nichols A. Romero, Ph.D. Argonne Leadership Computing Facility Argonne National Laboratory Building 360 Room L-146 9700 South Cass Avenue Argonne, IL 60490 (630) 252-3441 From hzhang at mcs.anl.gov Wed Dec 17 22:08:00 2008 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Wed, 17 Dec 2008 22:08:00 -0600 (CST) Subject: external solvers - scalapack In-Reply-To: <28702257.719761229571385757.JavaMail.root@zimbra> References: <28702257.719761229571385757.JavaMail.root@zimbra> Message-ID: On Wed, 17 Dec 2008, Nichols A. Romero wrote: > Hi, > > I looked through the mailing lists to see if this question > was already asked but could not find anything. > > In the PETSc TACC2008 tutorial, it states that PETSc interfaces > with Scalapack? No, petsc is not interfaced with Scalapack. Petsc is interfaced with MUMPS, which requires Scalapack. > > I would like to know what is supported from Scalapack. I am particularly > interested in: > 1. Cholesky factorization (I know there are a couple of alternative options here, e.g. PLAPACK). Yes, you can get parallel dense Cholesky factorization through petsc/PLAPACK interface. > 2. Dense eigenvalue solver You may take a look at slepc. Hong > > > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 > > From bsmith at mcs.anl.gov Wed Dec 17 22:09:42 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 17 Dec 2008 22:09:42 -0600 Subject: external solvers - scalapack In-Reply-To: <28702257.719761229571385757.JavaMail.root@zimbra> References: <28702257.719761229571385757.JavaMail.root@zimbra> Message-ID: Whoever wrote a PETSc TACC2008 tutorial that says PETSc interfaces to scalapack is a fool. It does not interface to scalapack in any way (since we support building mumps which needs scalapack we support building scalapack but PETSc itself makes no calls to scalapack.)* We do have support for using Cholesky in PLAPACK with our dense matrices. You should use petsc-dev or wait a couple of days for the petsc-3.0 release before using it. Barry * One reason we don't screw with scalapack is the complicated messy way the dense matrix has to be spread across processes which is painful. On Dec 17, 2008, at 9:36 PM, Nichols A. Romero wrote: > Hi, > > I looked through the mailing lists to see if this question > was already asked but could not find anything. > > In the PETSc TACC2008 tutorial, it states that PETSc interfaces > with Scalapack? > > I would like to know what is supported from Scalapack. I am > particularly > interested in: > 1. Cholesky factorization (I know there are a couple of alternative > options here, e.g. PLAPACK). > 2. Dense eigenvalue solver > > > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 > From Lars.Rindorf at teknologisk.dk Thu Dec 18 02:02:36 2008 From: Lars.Rindorf at teknologisk.dk (Lars Rindorf) Date: Thu, 18 Dec 2008 09:02:36 +0100 Subject: MPI questions In-Reply-To: Message-ID: Dear all I have some questions regarding MPI and PETSc. I have browsed the documentation and the internet, and I have not found the answers to them. But if they are trivial, then please accept my apologies :-) First of all, I'm running petsc on a quad core xeon and I'm using MUMPS direct solver (iterative solvers not an option for my purposes). I compile PETSc with g++ and gfortran under redhat linux. My program using petsc is compiled with mpiCC (a C++ program). The questions are: 1. Does petsc need to be compiled with mpi (mpicc, mpiCC etc) to be efficient? If not, is it faster when compiled with mpi compilers? 2. Using a CPU with four cores then does MPI provide any significant speed-up? And if so, can it be optimized? 3. I'm having problem with a problem that should scale (approx.) linear in time as function of DOF. But it only scales linearly for small number of DOF for large number of DOF it scales quadraticly. This is before the RAM is even 1/4 full. (Direct solvers use a lot of RAM.). I guess what is limiting the speed is the communication between the CPU and the RAM, but I don't know Does anybody has experience with this problem? Or maybe a small test program that I can test? Thanks! KR, Lars From bsmith at mcs.anl.gov Thu Dec 18 08:20:01 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 18 Dec 2008 08:20:01 -0600 Subject: MPI questions In-Reply-To: References: Message-ID: <8F1268EF-5B5C-4D22-AD93-5E105AC4F098@mcs.anl.gov> On Dec 18, 2008, at 2:02 AM, Lars Rindorf wrote: > Dear all > > I have some questions regarding MPI and PETSc. I have browsed the > documentation and the internet, and I have not found the answers to > them. But if they are trivial, then please accept my apologies :-) > > First of all, I'm running petsc on a quad core xeon and I'm using > MUMPS > direct solver (iterative solvers not an option for my purposes). I > compile PETSc with g++ and gfortran under redhat linux. My program > using > petsc is compiled with mpiCC (a C++ program). > > > The questions are: > > 1. Does petsc need to be compiled with mpi (mpicc, mpiCC etc) to be > efficient? If not, is it faster when compiled with mpi compilers? You should compile PETSc with the exact same compilers as your code (or similarly compile your code with the exact same compilers as PETSc), this is not because of speed but because not all compilers work mixed with other compilers. We highly recommend using the mpicc/mpiCC etc compilers to build PETSc rather than directly g++; otherwise people end up mixing different compilers. It won't make any difference in terms of speed, in the end they ALWAYS just use the underlying compiler to compile the code. For performance you should build PETSc with the config/configure.py option --with-debugging=0 > > > 2. Using a CPU with four cores then does MPI provide any significant > speed-up? And if so, can it be optimized? Make sure you use --with-debugging=0; there is no particular "optimization" you can do. You will not likely see speedup in going from 1 process to 2 with MUMPS (the parallelism introduces a lot of overhead that is not there when running on one process). You could see some improvement in going from 2 to 4 processes. > > > 3. I'm having problem with a problem that should scale (approx.) > linear > in time as function of DOF. But it only scales linearly for small > number > of DOF for large number of DOF it scales quadraticly. This is before > the > RAM is even 1/4 full. (Direct solvers use a lot of RAM.). I guess what > is limiting the speed is the communication between the CPU and the > RAM, > but I don't know Does anybody has experience with this problem? Or > maybe > a small test program that I can test? Where is the sparse matrix coming from? If it is coming from a 3d grid problem (like a PDE with finite elements) you could easily see quadratic growth with problem size with a direct solver. How do you know it should scale linearly? If you run the PETSc program with -log_summary you can see where the run is taking the time and see if that makes sense. What you should see is that the large bulk of the time is in the LU numerical factorization. Barry > > > Thanks! > > KR, Lars > From naromero at alcf.anl.gov Thu Dec 18 10:50:48 2008 From: naromero at alcf.anl.gov (Nichols A. Romero) Date: Thu, 18 Dec 2008 10:50:48 -0600 (CST) Subject: external solvers - scalapack In-Reply-To: <10334632.743401229618686921.JavaMail.root@zimbra> Message-ID: <30366106.743851229619048052.JavaMail.root@zimbra> Barry, PLAPACK has some dense eigensolvers. The public release version has QR (which is probably too slow for us), but a developmental version (available by contacting the authors) implements MR^3. Does petsc have support for PLAPACK QR? If so, would it be possible to put support for MR^3? Nichols A. Romero, Ph.D. Argonne Leadership Computing Facility Argonne National Laboratory Building 360 Room L-146 9700 South Cass Avenue Argonne, IL 60490 (630) 252-3441 ----- Original Message ----- From: "Barry Smith" To: petsc-users at mcs.anl.gov Sent: Wednesday, December 17, 2008 10:09:42 PM GMT -06:00 US/Canada Central Subject: Re: external solvers - scalapack Whoever wrote a PETSc TACC2008 tutorial that says PETSc interfaces to scalapack is a fool. It does not interface to scalapack in any way (since we support building mumps which needs scalapack we support building scalapack but PETSc itself makes no calls to scalapack.)* We do have support for using Cholesky in PLAPACK with our dense matrices. You should use petsc-dev or wait a couple of days for the petsc-3.0 release before using it. Barry * One reason we don't screw with scalapack is the complicated messy way the dense matrix has to be spread across processes which is painful. On Dec 17, 2008, at 9:36 PM, Nichols A. Romero wrote: > Hi, > > I looked through the mailing lists to see if this question > was already asked but could not find anything. > > In the PETSc TACC2008 tutorial, it states that PETSc interfaces > with Scalapack? > > I would like to know what is supported from Scalapack. I am > particularly > interested in: > 1. Cholesky factorization (I know there are a couple of alternative > options here, e.g. PLAPACK). > 2. Dense eigenvalue solver > > > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 > From bsmith at mcs.anl.gov Thu Dec 18 11:13:14 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 18 Dec 2008 11:13:14 -0600 Subject: external solvers - scalapack In-Reply-To: <30366106.743851229619048052.JavaMail.root@zimbra> References: <30366106.743851229619048052.JavaMail.root@zimbra> Message-ID: <187D4688-9C9A-458D-97E9-0DE232F94CE7@mcs.anl.gov> We don't really do eigensolvers; that is handled by SLEPc that has an infrastructure just for that. If it does not support plapack, that is where it could be added. We could contract to add it to slepc if it is not currently there: the contract would be $64,000. Barry On Dec 18, 2008, at 10:50 AM, Nichols A. Romero wrote: > Barry, > > PLAPACK has some dense eigensolvers. The public release version has QR > (which is probably too slow for us), but a developmental version > (available > by contacting the authors) implements MR^3. > > Does petsc have support for PLAPACK QR? If so, would it be possible > to put support > for MR^3? > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 > > > ----- Original Message ----- > From: "Barry Smith" > To: petsc-users at mcs.anl.gov > Sent: Wednesday, December 17, 2008 10:09:42 PM GMT -06:00 US/Canada > Central > Subject: Re: external solvers - scalapack > > > Whoever wrote a PETSc TACC2008 tutorial that says PETSc interfaces > to scalapack > is a fool. It does not interface to scalapack in any way (since we > support building > mumps which needs scalapack we support building scalapack but PETSc > itself makes > no calls to scalapack.)* > > We do have support for using Cholesky in PLAPACK with our dense > matrices. > You should use petsc-dev or wait a couple of days for the petsc-3.0 > release > before using it. > > Barry > > * One reason we don't screw with scalapack is the complicated messy > way the dense > matrix has to be spread across processes which is painful. > > > On Dec 17, 2008, at 9:36 PM, Nichols A. Romero wrote: > >> Hi, >> >> I looked through the mailing lists to see if this question >> was already asked but could not find anything. >> >> In the PETSc TACC2008 tutorial, it states that PETSc interfaces >> with Scalapack? >> >> I would like to know what is supported from Scalapack. I am >> particularly >> interested in: >> 1. Cholesky factorization (I know there are a couple of alternative >> options here, e.g. PLAPACK). >> 2. Dense eigenvalue solver >> >> >> >> Nichols A. Romero, Ph.D. >> Argonne Leadership Computing Facility >> Argonne National Laboratory >> Building 360 Room L-146 >> 9700 South Cass Avenue >> Argonne, IL 60490 >> (630) 252-3441 >> > From jroman at dsic.upv.es Thu Dec 18 12:41:17 2008 From: jroman at dsic.upv.es (Jose E. Roman) Date: Thu, 18 Dec 2008 19:41:17 +0100 Subject: external solvers - scalapack In-Reply-To: <30366106.743851229619048052.JavaMail.root@zimbra> References: <30366106.743851229619048052.JavaMail.root@zimbra> Message-ID: <51FDB888-A491-4427-9BD1-E4C6D847F79B@dsic.upv.es> On 18/12/2008, Nichols A. Romero wrote: > Barry, > > PLAPACK has some dense eigensolvers. The public release version has QR > (which is probably too slow for us), but a developmental version > (available > by contacting the authors) implements MR^3. > > Does petsc have support for PLAPACK QR? If so, would it be possible > to put support > for MR^3? > > Nichols A. Romero, Ph.D. > Argonne Leadership Computing Facility > Argonne National Laboratory > Building 360 Room L-146 > 9700 South Cass Avenue > Argonne, IL 60490 > (630) 252-3441 With SLEPc you can reliably compute up to a few thousand eigenpairs of a huge matrix. This is sufficient for most applications. We are working for optimizing memory usage in the case that many eigenpairs are requested. Have a look at http://www.grycap.upv.es/slepc and if you need more details, contact us at slepc-maint at grycap.upv.es Jose From rafaelsantoscoelho at gmail.com Fri Dec 19 12:33:30 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 16:33:30 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> Message-ID: <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> Oh, and one more thing: lambda = 4.0 was the Bratu parameter chosen for all tests. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafaelsantoscoelho at gmail.com Fri Dec 19 12:27:20 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 16:27:20 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module Message-ID: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> Hello to everyone, Weeks ago I did some experiments with PETSc's SNESMF module using the Bratu nonlinear PDE in 2Dexample. I ran a bunch of tests (varying the number of processors and the mesh size) with the "-log_summary" command-line option to collect several measures, namely: max runtime, total memory usage, floating point operations, linear iterations count and MPI messages sent. My intention was to compare SNESMF (the jacobian-free Newton-Krylov method, which I will call JFNK) with SNES ("simple" Newton-Krylov method, which I will call NK). In theory, we basically know that: 1. JFNK is supposed to consume less memory than NK since the true elements of the jacobian matrix are never actually stored; 2. JFNK is supposed to do less or roughly the same amount of floating point operations than NK since it does not calculate the jacobian matrix entries in every Newton iteration. Well, I have to admit that I was pretty surprised by the results. Shortly, except in few cases, NK outperformed JFNK with regard to each one of the metrics mentioned above. Before anything, some clarifications: 1. Each test was repeated 5 times and I summarized the results using the arithmetic mean in order to attenuate possible fluctuations; 2. The tests were run on a Beowulf cluster with 30 processing nodes, and each node is a Intel(R) Core(TM)2 CPU 4300 1.80GHz with 2MB of cache, 2GB of RAM, Fedora Core 6 GNU/Linux (kernel version 2.6.26.6-49), PETSc version 2.3.3-p15 and LAM/MPI version 6.41; 3. The Walker-Perniceformula was chosen to compute the differencing parameter "h" used with the finite difference based matrix-free jacobian; 4. No preconditioners or matrix reorderings were employed. Now here come my questions: - How come JFNK used more memory than NK? Looking at the log files at the end of this message, there were 4 matrix creations in JFNK. Why 4?! And also why did JFNK create one vector more (46) than NK (45)? Where does that particular vector come from? Is that vector the h parameter history? How can I figure that out? - Why did JFNK perform worse than NK in terms of max runtime, total linear iterations count, MPI messages sent, MPI reductions and flops? My answer: JFNK had a much higher linear iterations count than NK, and that's probably why it turned out to be worse than NK in all those aspects. As for the MPI reductions, I believe that JFNK topped NK because of all the vector norm operations needed to compute the "h" parameter. - Why did JFNK's KSP solver (GMRES) iterate way more per Newton iteration than NK? My answer: That probably has to do with the fact that JFNK approximately computes the product J(x)a. If the precision of that approximation is poor (and that's closely related to which approach was used to calculate the differencing parameter, I think), then the linear solver must iterate more to produce a "good" Newton correction. I'd really appreciate if you guys could comment on my observations and conclusions and help me out with my questions. I've pasted below some excerpts of two log files generated by the "-log_summary" option. Thanks in advance, Rafael -------------------------------------------------------------------------------------------------------- NK KSP SOLVER: GMRES MESH SIZE : 512 x 512 unknows NUMBER OF PROCESSORS : 24 Linear solve converged due to CONVERGED_RTOL iterations 25038 Linear solve converged due to CONVERGED_RTOL iterations 25995 Linear solve converged due to CONVERGED_RTOL iterations 26769 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE Max Max/Min Avg Total *Time (sec): 4.597e+02 1.00008 4.597e+02* Objects: 6.200e+01 1.00000 6.200e+01 *Flops: 6.553e+10 1.01230 6.501e+10 1.560e+12* Flops/sec: 1.425e+08 1.01233 1.414e+08 3.394e+09 *Memory: 4.989e+06 1.01590 1.186e+08* *MPI Messages: 3.216e+05 2.00000 2.546e+05 6.111e+06 * MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e+09 *MPI Reductions: 6.704e+03 1.00000* Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage SNES 1 1 124 0 Krylov Solver 1 1 16880 0 Preconditioner 1 1 0 0 Distributed array 1 1 46568 0 Index Set 6 6 135976 0 *Vec 45 45 3812684 0* Vec Scatter 3 3 0 0 IS L to G Mapping 1 1 45092 0 *Matrix 3 3 1011036 0* -------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------- JFNK KSP SOLVER: GMRES MESH SIZE : 512 x 512 unknows NUMBER OF PROCESSORS : 24 Linear solve converged due to CONVERGED_RTOL iterations 25042 Linear solve converged due to CONVERGED_RTOL iterations 33804 Linear solve converged due to CONVERGED_RTOL iterations 33047 Linear solve converged due to CONVERGED_RTOL iterations 21219 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE Max Max/Min Avg Total *Time (sec): 1.076e+03 1.00009 1.076e+03* Objects: 6.500e+01 1.00000 6.500e+01 *Flops: 1.044e+11 1.01176 1.036e+11 2.485e+12* Flops/sec: 9.702e+07 1.01185 9.626e+07 2.310e+09 *Memory: 5.076e+06 1.01530 1.207e+08* *MPI Messages: 4.676e+05 2.00000 3.702e+05 8.884e+06* MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e+09 *MPI Reductions: 9.901e+03 1.00000* Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage SNES 1 1 124 0 Krylov Solver 1 1 16880 0 Preconditioner 1 1 0 0 Distributed array 1 1 46568 0 Index Set 6 6 135976 0 *Vec 46 46 3901252 0* Vec Scatter 3 3 0 0 IS L to G Mapping 1 1 45092 0 MatMFFD 1 1 0 0 *Matrix 4 4 1011036 0* -------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Fri Dec 19 13:15:02 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 19 Dec 2008 17:15:02 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> Message-ID: Hi, Rafael. Did you make sure to pass '-pc_type none' for both JFNK and NK?? If you do not explicitely pass it to NK, you will end-up by default using a preconditioner: ILU(0) in uniprocessors runs and Block-Jacobi with ILU(0) on each local block with two or more processors.... In short, pass always pass '-pc_type none' when comparing matrix-free and and non-matrix-free iterations .... If not, the comparison will not be fair ... On Fri, Dec 19, 2008 at 4:33 PM, Rafael Santos Coelho wrote: > Oh, and one more thing: lambda = 4.0 was the Bratu parameter chosen for all > tests. > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From rafaelsantoscoelho at gmail.com Fri Dec 19 13:54:40 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 17:54:40 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> Message-ID: <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> Hi, Lisandro, Did you make sure to pass '-pc_type none' for both JFNK and NK?? Yes, I did. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Fri Dec 19 14:10:18 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 19 Dec 2008 18:10:18 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> Message-ID: Then the problem is restarted GMRES, the default restart of 30 is not enough.... But before you go on trying... do this... Add: -ksp_type cg and let me know your results ;-) On Fri, Dec 19, 2008 at 5:54 PM, Rafael Santos Coelho wrote: > Hi, Lisandro, > >> Did you make sure to pass '-pc_type none' for both JFNK and NK?? > > Yes, I did. > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From knepley at gmail.com Fri Dec 19 14:10:24 2008 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 19 Dec 2008 14:10:24 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> Message-ID: On Fri, Dec 19, 2008 at 1:54 PM, Rafael Santos Coelho < rafaelsantoscoelho at gmail.com> wrote: > Hi, Lisandro, > > Did you make sure to pass '-pc_type none' for both JFNK and NK?? > > > Yes, I did. If this is the case, please explain why more than 1 matrix is created, or in the case of JFNK, any matrix is created. Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 19 14:16:43 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 19 Dec 2008 14:16:43 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191033t637c5cf0p5695cff5b5efb369@mail.gmail.com> <3b6f83d40812191154p6e986a4ehe37e3f5ea66e4461@mail.gmail.com> Message-ID: <891CC7CF-0027-4ACC-B5D6-B5EDBFAD7B0C@mcs.anl.gov> On Dec 19, 2008, at 2:10 PM, Matthew Knepley wrote: > On Fri, Dec 19, 2008 at 1:54 PM, Rafael Santos Coelho > wrote: > Hi, Lisandro, > > > Did you make sure to pass '-pc_type none' for both JFNK and NK?? > > Yes, I did. > > If this is the case, please explain why more than 1 matrix is > created, or in the > case of JFNK, any matrix is created. Matt, Even in the matrix free a Mat is created; it just does not have a sparse storage representation. Barry > > > Thanks, > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener From bsmith at mcs.anl.gov Fri Dec 19 14:21:54 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 19 Dec 2008 14:21:54 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> Message-ID: Send the output from using -snes_view -options_table for each run; it may be you are not running what you think you are running. Barry On Dec 19, 2008, at 12:27 PM, Rafael Santos Coelho wrote: > Hello to everyone, > > Weeks ago I did some experiments with PETSc's SNESMF module using > the Bratu nonlinear PDE in 2D example. I ran a bunch of tests > (varying the number of processors and the mesh size) with the "- > log_summary" command-line option to collect several measures, > namely: max runtime, total memory usage, floating point operations, > linear iterations count and MPI messages sent. My intention was to > compare SNESMF (the jacobian-free Newton-Krylov method, which I will > call JFNK) with SNES ("simple" Newton-Krylov method, which I will > call NK). In theory, we basically know that: > > ? JFNK is supposed to consume less memory than NK since the true > elements of the jacobian matrix are never actually stored; > ? JFNK is supposed to do less or roughly the same amount of > floating point operations than NK since it does not calculate the > jacobian matrix entries in every Newton iteration. > Well, I have to admit that I was pretty surprised by the results. > Shortly, except in few cases, NK outperformed JFNK with regard to > each one of the metrics mentioned above. Before anything, some > clarifications: > > ? Each test was repeated 5 times and I summarized the results using > the arithmetic mean in order to attenuate possible fluctuations; > ? The tests were run on a Beowulf cluster with 30 processing nodes, > and each node is a Intel(R) Core(TM)2 CPU 4300 1.80GHz with 2MB of > cache, 2GB of RAM, Fedora Core 6 GNU/Linux (kernel version > 2.6.26.6-49), PETSc version 2.3.3-p15 and LAM/MPI version 6.41; > ? The Walker-Pernice formula was chosen to compute the differencing > parameter "h" used with the finite difference based matrix-free > jacobian; > ? No preconditioners or matrix reorderings were employed. > Now here come my questions: > > ? How come JFNK used more memory than NK? Looking at the log files > at the end of this message, there were 4 matrix creations in JFNK. > Why 4?! And also why did JFNK create one vector more (46) than NK > (45)? Where does that particular vector come from? Is that vector > the h parameter history? How can I figure that out? > ? Why did JFNK perform worse than NK in terms of max runtime, total > linear iterations count, MPI messages sent, MPI reductions and flops? > My answer: JFNK had a much higher linear iterations count than NK, > and that's probably why it turned out to be worse than NK in all > those aspects. As for the MPI reductions, I believe that JFNK topped > NK because of all the vector norm operations needed to compute the > "h" parameter. > > ? Why did JFNK's KSP solver (GMRES) iterate way more per Newton > iteration than NK? > My answer: That probably has to do with the fact that JFNK > approximately computes the product J(x)a. If the precision of that > approximation is poor (and that's closely related to which approach > was used to calculate the differencing parameter, I think), then the > linear solver must iterate more to produce a "good" Newton correction. > > I'd really appreciate if you guys could comment on my observations > and conclusions and help me out with my questions. I've pasted below > some excerpts of two log files generated by the "-log_summary" option. > > Thanks in advance, > > Rafael > > -------------------------------------------------------------------------------------------------------- > NK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25038 > Linear solve converged due to CONVERGED_RTOL iterations 25995 > Linear solve converged due to CONVERGED_RTOL iterations 26769 > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/Min > Avg Total > Time (sec): 4.597e+02 1.00008 4.597e+02 > Objects: 6.200e+01 1.00000 6.200e+01 > Flops: 6.553e+10 1.01230 6.501e > +10 1.560e+12 > Flops/sec: 1.425e+08 1.01233 1.414e > +08 3.394e+09 > Memory: 4.989e+06 > 1.01590 1.186e+08 > MPI Messages: 3.216e+05 2.00000 2.546e+05 > 6.111e+06 > MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e > +09 > MPI Reductions: 6.704e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 124 0 > Krylov Solver 1 1 16880 0 > Preconditioner 1 1 0 0 > Distributed array 1 1 46568 0 > Index Set 6 6 135976 0 > Vec 45 45 3812684 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 45092 0 > Matrix 3 3 1011036 0 > -------------------------------------------------------------------------------------------------------- > > -------------------------------------------------------------------------------------------------------- > JFNK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25042 > Linear solve converged due to CONVERGED_RTOL iterations 33804 > Linear solve converged due to CONVERGED_RTOL iterations 33047 > Linear solve converged due to CONVERGED_RTOL iterations 21219 > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/Min > Avg Total > Time (sec): 1.076e+03 1.00009 1.076e+03 > Objects: 6.500e+01 1.00000 6.500e+01 > Flops: 1.044e+11 1.01176 1.036e > +11 2.485e+12 > Flops/sec: 9.702e+07 1.01185 9.626e > +07 2.310e+09 > Memory: 5.076e+06 > 1.01530 1.207e+08 > MPI Messages: 4.676e+05 2.00000 3.702e+05 > 8.884e+06 > MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e > +09 > MPI Reductions: 9.901e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 > 124 0 > Krylov Solver 1 1 > 16880 0 > Preconditioner 1 1 > 0 0 > Distributed array 1 1 > 46568 0 > Index Set 6 6 > 135976 0 > Vec 46 46 > 3901252 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 > 45092 0 > MatMFFD 1 1 > 0 0 > Matrix 4 4 > 1011036 0 > -------------------------------------------------------------------------------------------------------- > > From rafaelsantoscoelho at gmail.com Fri Dec 19 16:39:19 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 20:39:19 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> Message-ID: <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> Hey, Barry, here's what you asked: -------------------------------------------------------------------------------------------------------- JFNK KSP SOLVER: GMRES MESH SIZE : 512 x 512 unknows NUMBER OF PROCESSORS : 24 Linear solve converged due to CONVERGED_RTOL iterations 25042 Linear solve converged due to CONVERGED_RTOL iterations 33804 Linear solve converged due to CONVERGED_RTOL iterations 33047 Linear solve converged due to CONVERGED_RTOL iterations 21219 SNES Object: type: ls line search variant: SNESLineSearchCubic alpha=0.0001, maxstep=1e+08, steptol=1e-12 maximum iterations=100, maximum function evaluations=1000000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=113112 total number of function evaluations=116893 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=1000000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: none linear system matrix = precond matrix: Matrix Object: type=mffd, rows=262144, cols=262144 matrix-free approximation: err=1e-07 (relative error in function evaluation) Using wp compute h routine Computes normA Does not compute normU Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE Max Max/Min Avg Total Time (sec): 8.080e+02 1.00009 8.080e+02 Objects: 6.600e+01 1.00000 6.600e+01 Flops: 1.044e+11 1.01176 1.036e+11 2.485e+12 Flops/sec: 1.292e+08 1.01184 1.282e+08 3.076e+09 Memory: 5.076e+06 1.01530 1.207e+08 MPI Messages: 4.676e+05 2.00000 3.702e+05 8.884e+06 MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e+09 MPI Reductions: 9.901e+03 1.00000 Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage SNES 1 1 124 0 Krylov Solver 1 1 16880 0 Preconditioner 1 1 0 0 Distributed array 1 1 46568 0 Index Set 6 6 135976 0 Vec 46 46 3901252 0 Vec Scatter 3 3 0 0 IS L to G Mapping 1 1 45092 0 MatMFFD 1 1 0 0 Matrix 4 4 1011036 0 Viewer 1 1 0 0 OptionTable: -ksp_converged_reason OptionTable: -ksp_max_it 1000000 OptionTable: -ksp_type gmres OptionTable: -log_summary OptionTable: -option_table OptionTable: -par 4.0 OptionTable: -pc_type none OptionTable: -snes_converged_reason OptionTable: -snes_max_funcs 1000000 OptionTable: -snes_max_it 100 OptionTable: -snes_mf OptionTable: -snes_view OptionTable: -xdiv 512 OptionTable: -ydiv 512 -------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------- NK KSP SOLVER: GMRES MESH SIZE : 512 x 512 unknows NUMBER OF PROCESSORS : 24 Linear solve converged due to CONVERGED_RTOL iterations 25038 Linear solve converged due to CONVERGED_RTOL iterations 25995 Linear solve converged due to CONVERGED_RTOL iterations 26769 SNES Object: type: ls line search variant: SNESLineSearchCubic alpha=0.0001, maxstep=1e+08, steptol=1e-12 maximum iterations=100, maximum function evaluations=1000000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=77802 total number of function evaluations=4 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=1000000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: none linear system matrix = precond matrix: Matrix Object: type=mpiaij, rows=262144, cols=262144 total: nonzeros=1308672, allocated nonzeros=1308672 not using I-node (on process 0) routines Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE Max Max/Min Avg Total Time (sec): 5.329e+02 1.00008 5.329e+02 Objects: 6.300e+01 1.00000 6.300e+01 Flops: 6.553e+10 1.01230 6.501e+10 1.560e+12 Flops/sec: 1.230e+08 1.01230 1.220e+08 2.928e+09 Memory: 4.989e+06 1.01589 1.186e+08 MPI Messages: 3.216e+05 1.99999 2.546e+05 6.111e+06 MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e+09 MPI Reductions: 6.704e+03 1.00000 Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage SNES 1 1 124 0 Krylov Solver 1 1 16880 0 Preconditioner 1 1 0 0 Distributed array 1 1 46568 0 Index Set 6 6 135976 0 Vec 45 45 3812684 0 Vec Scatter 3 3 0 0 IS L to G Mapping 1 1 45092 0 Matrix 3 3 1011036 0 Viewer 1 1 0 0 OptionTable: -ksp_converged_reason OptionTable: -ksp_max_it 1000000 OptionTable: -ksp_type gmres OptionTable: -log_summary OptionTable: -option_table OptionTable: -par 4.0 OptionTable: -pc_type none OptionTable: -snes_converged_reason OptionTable: -snes_max_funcs 1000000 OptionTable: -snes_max_it 100 OptionTable: -snes_view OptionTable: -xdiv 512 OptionTable: -ydiv 512 -------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Dec 19 16:53:30 2008 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 19 Dec 2008 16:53:30 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> Message-ID: On Fri, Dec 19, 2008 at 4:39 PM, Rafael Santos Coelho < rafaelsantoscoelho at gmail.com> wrote: > Hey, Barry, > > here's what you asked: You are taking 25K + iterations of GMRES(30). This I think is probably pretty sensitive to the exact nature of the FD approximation. Did you try varying the h parameter? Matt > > > -------------------------------------------------------------------------------------------------------- > JFNK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25042 > Linear solve converged due to CONVERGED_RTOL iterations 33804 > Linear solve converged due to CONVERGED_RTOL iterations 33047 > Linear solve converged due to CONVERGED_RTOL iterations 21219 > SNES Object: > type: ls > line search variant: SNESLineSearchCubic > alpha=0.0001, maxstep=1e+08, steptol=1e-12 > maximum iterations=100, maximum function evaluations=1000000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=113112 > total number of function evaluations=116893 > KSP Object: > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=1000000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: none > linear system matrix = precond matrix: > Matrix Object: > type=mffd, rows=262144, cols=262144 > matrix-free approximation: > err=1e-07 (relative error in function evaluation) > Using wp compute h routine > Computes normA > Does not compute normU > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/Min > Avg Total > Time (sec): 8.080e+02 1.00009 8.080e+02 > Objects: 6.600e+01 1.00000 6.600e+01 > Flops: 1.044e+11 1.01176 1.036e+11 > 2.485e+12 > Flops/sec: 1.292e+08 1.01184 1.282e+08 > 3.076e+09 > Memory: 5.076e+06 1.01530 > 1.207e+08 > MPI Messages: 4.676e+05 2.00000 3.702e+05 8.884e+06 > MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e+09 > MPI Reductions: 9.901e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 124 > 0 > Krylov Solver 1 1 16880 0 > Preconditioner 1 1 0 > 0 > Distributed array 1 1 46568 0 > Index Set 6 6 135976 0 > Vec 46 46 3901252 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 45092 0 > MatMFFD 1 1 0 > 0 > Matrix 4 4 1011036 > 0 > Viewer 1 1 > 0 0 > > OptionTable: -ksp_converged_reason > OptionTable: -ksp_max_it 1000000 > OptionTable: -ksp_type gmres > OptionTable: -log_summary > OptionTable: -option_table > OptionTable: -par 4.0 > OptionTable: -pc_type none > OptionTable: -snes_converged_reason > OptionTable: -snes_max_funcs 1000000 > OptionTable: -snes_max_it 100 > OptionTable: -snes_mf > OptionTable: -snes_view > OptionTable: -xdiv 512 > OptionTable: -ydiv 512 > > -------------------------------------------------------------------------------------------------------- > > > -------------------------------------------------------------------------------------------------------- > NK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25038 > Linear solve converged due to CONVERGED_RTOL iterations 25995 > Linear solve converged due to CONVERGED_RTOL iterations 26769 > SNES Object: > type: ls > line search variant: SNESLineSearchCubic > alpha=0.0001, maxstep=1e+08, steptol=1e-12 > maximum iterations=100, maximum function evaluations=1000000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=77802 > total number of function evaluations=4 > KSP Object: > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=1000000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: none > linear system matrix = precond matrix: > Matrix Object: > type=mpiaij, rows=262144, cols=262144 > total: nonzeros=1308672, allocated nonzeros=1308672 > not using I-node (on process 0) routines > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/Min > Avg Total > Time (sec): 5.329e+02 1.00008 5.329e+02 > Objects: 6.300e+01 1.00000 6.300e+01 > Flops: 6.553e+10 1.01230 6.501e+10 > 1.560e+12 > Flops/sec: 1.230e+08 1.01230 1.220e+08 > 2.928e+09 > Memory: 4.989e+06 1.01589 > 1.186e+08 > MPI Messages: 3.216e+05 1.99999 2.546e+05 6.111e+06 > MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e+09 > MPI Reductions: 6.704e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 > 124 0 > Krylov Solver 1 1 16880 > 0 > Preconditioner 1 1 > 0 0 > Distributed array 1 1 46568 > 0 > Index Set 6 6 135976 > 0 > Vec 45 45 3812684 > 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 45092 0 > Matrix 3 3 > 1011036 0 > Viewer 1 1 > 0 0 > > OptionTable: -ksp_converged_reason > OptionTable: -ksp_max_it 1000000 > OptionTable: -ksp_type gmres > OptionTable: -log_summary > OptionTable: -option_table > OptionTable: -par 4.0 > OptionTable: -pc_type none > OptionTable: -snes_converged_reason > OptionTable: -snes_max_funcs 1000000 > OptionTable: -snes_max_it 100 > OptionTable: -snes_view > OptionTable: -xdiv 512 > OptionTable: -ydiv 512 > > -------------------------------------------------------------------------------------------------------- > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafaelsantoscoelho at gmail.com Fri Dec 19 17:27:05 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 21:27:05 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> Message-ID: <3b6f83d40812191527i4628525ax512fbf40c27ab62a@mail.gmail.com> Hey, Matthew, What do you mean by "varying the h parameter"? Using another approach to compute it (such as the DS)? Changing its "calculation period"? Anyhow, I didn't try those things yet. In fact, that's what I plan on doing next this weekend, and also maybe implementing new ways of calculating the h parameter (who knows, that could be a contribution of mine to PETSc's code base :D). After some googling, I found this articlethat demonstrates at least two other ways to compute the differencing parameter. Thank you guys for your replies! -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Fri Dec 19 17:28:36 2008 From: recrusader at gmail.com (Yujie) Date: Fri, 19 Dec 2008 15:28:36 -0800 Subject: about MatSetValue() and VecSetValue() Message-ID: <7ff0ee010812191528k2354aa51nc3778364cf3ba5f@mail.gmail.com> Hi, PETSc developer I am wondering if several cpus simultaneously insert the values in the same position using MatSetValue() or VecSetValue(), how do the functions deal with it? thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 19 17:32:48 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 19 Dec 2008 17:32:48 -0600 Subject: about MatSetValue() and VecSetValue() In-Reply-To: <7ff0ee010812191528k2354aa51nc3778364cf3ba5f@mail.gmail.com> References: <7ff0ee010812191528k2354aa51nc3778364cf3ba5f@mail.gmail.com> Message-ID: <9EA767D6-D594-4E98-BF9A-D7F0FBA47D73@mcs.anl.gov> If INSERT_VALUE is being used then ONE of the values you put in ends up being the value; which one is not under your control. If ADD_VALUES is used then the sum of all the values is used. On Dec 19, 2008, at 5:28 PM, Yujie wrote: > Hi, PETSc developer > > I am wondering if several cpus simultaneously insert the values in > the same position using MatSetValue() or VecSetValue(), how do the > functions deal with it? thanks a lot. > > Regards, > > Yujie > From recrusader at gmail.com Fri Dec 19 17:41:59 2008 From: recrusader at gmail.com (Yujie) Date: Fri, 19 Dec 2008 15:41:59 -0800 Subject: about MatSetValue() and VecSetValue() In-Reply-To: <9EA767D6-D594-4E98-BF9A-D7F0FBA47D73@mcs.anl.gov> References: <7ff0ee010812191528k2354aa51nc3778364cf3ba5f@mail.gmail.com> <9EA767D6-D594-4E98-BF9A-D7F0FBA47D73@mcs.anl.gov> Message-ID: <7ff0ee010812191541v4e69ecbfq22905a69134e80f0@mail.gmail.com> thanks a lot, Barry. Merry Xmas:) ! Yujie On Fri, Dec 19, 2008 at 3:32 PM, Barry Smith wrote: > > If INSERT_VALUE is being used then ONE of the values you put in ends up > being the value; which one is > not under your control. If ADD_VALUES is used then the sum of all the > values is used. > > > On Dec 19, 2008, at 5:28 PM, Yujie wrote: > > Hi, PETSc developer >> >> I am wondering if several cpus simultaneously insert the values in the >> same position using MatSetValue() or VecSetValue(), how do the functions >> deal with it? thanks a lot. >> >> Regards, >> >> Yujie >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 19 17:51:44 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 19 Dec 2008 17:51:44 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> Message-ID: <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> When I was a kid I was guided to be fair and to not exaggerate. When I was a teenager I found out that adults don't actually act that way, it was a disappointment to me. What was worse was once I became an adult I learned that scientists don't act that way: when they advocate something they exaggerate its positive features and minimize its negative features. This is what the great priests of Jacobian-Free Newton Krylov do, after you see one of their presentations you know that JFNK is better in every possible way from any other approach. In the practical world outside of their lecture halls there is no such thing as "one algorithm being better than another" it is only "one algorithm is better than another in this specific circumstance." This is why PETSc was written the way it is, to provide a toolbox that allows the selection of many possibilities, NOT to have the library writers decide what algorithm is better. Their is actually nothing weird about the results you see, they are actually what a practitioner would expect :-) Run both for 40 iterations with -ksp_monitor_true_residual >My answer: That probably has to do with the fact that JFNK approximately computes the product J(x)a. If the precision >of that approximation is poor (and that's closely related to which approach was used to calculate the differencing parameter, >I think), then the linear solver must iterate more to produce a "good" Newton correction. This is kind of right, but the answer really has nothing to do with Newton. It is only a problem with the linear solve. Krylov methods depend fundamentally on the fact that they are used for LINEAR operators; when you do the differencing to apply J this is no longer exactly a linear operator thus the "Krylov" space being generated starts to become garbage. So matrix-free and non-matrix-free for the first few iterations will be generally similar but after some iterations the matrix-free will start to drift off into near garbage. Monkeying around with the h that is used can help reduce the garbage to some degree, but I doubt there is one "great" way to do it that always works well. In fact, when people publish on "my way of computing h is better than yours", they will, of course, exaggerate the benefits of theirs and minimize the problems with it. Because of this "drifting problems" JFNK rarely makes sense without a very good preconditioner, that reduces the number of iterations to a handful. With regard to why particular things happen in one code and not in another (like an extra vector), you will have to look at the code and run in the debugger to answer the questions. That is how we would answer these questions. BTW: the extra vector is because one applies J(u)*a, the extra vector is where the u values are stored. Barry If you poke around with the code you will sometimes find there are better ways to do things then we have done them, this is why PETSc is open source, we don't expect that we can do things better than everyone else. On Dec 19, 2008, at 4:39 PM, Rafael Santos Coelho wrote: > Hey, Barry, > > here's what you asked: > > -------------------------------------------------------------------------------------------------------- > JFNK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25042 > Linear solve converged due to CONVERGED_RTOL iterations 33804 > Linear solve converged due to CONVERGED_RTOL iterations 33047 > Linear solve converged due to CONVERGED_RTOL iterations 21219 > SNES Object: > type: ls > line search variant: SNESLineSearchCubic > alpha=0.0001, maxstep=1e+08, steptol=1e-12 > maximum iterations=100, maximum function evaluations=1000000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=113112 > total number of function evaluations=116893 > KSP Object: > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=1000000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: none > linear system matrix = precond matrix: > Matrix Object: > type=mffd, rows=262144, cols=262144 > matrix-free approximation: > err=1e-07 (relative error in function evaluation) > Using wp compute h routine > Computes normA > Does not compute normU > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/ > Min Avg Total > Time (sec): 8.080e+02 1.00009 8.080e+02 > Objects: 6.600e+01 1.00000 6.600e+01 > Flops: 1.044e+11 1.01176 1.036e > +11 2.485e+12 > Flops/sec: 1.292e+08 1.01184 1.282e > +08 3.076e+09 > Memory: 5.076e+06 > 1.01530 1.207e+08 > MPI Messages: 4.676e+05 2.00000 3.702e+05 > 8.884e+06 > MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e > +09 > MPI Reductions: 9.901e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' > Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 > 124 0 > Krylov Solver 1 1 > 16880 0 > Preconditioner 1 1 > 0 0 > Distributed array 1 1 > 46568 0 > Index Set 6 6 > 135976 0 > Vec 46 46 > 3901252 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 > 45092 0 > MatMFFD 1 1 > 0 0 > Matrix 4 4 > 1011036 0 > Viewer 1 1 > 0 0 > > OptionTable: -ksp_converged_reason > OptionTable: -ksp_max_it 1000000 > OptionTable: -ksp_type gmres > OptionTable: -log_summary > OptionTable: -option_table > OptionTable: -par 4.0 > OptionTable: -pc_type none > OptionTable: -snes_converged_reason > OptionTable: -snes_max_funcs 1000000 > OptionTable: -snes_max_it 100 > OptionTable: -snes_mf > OptionTable: -snes_view > OptionTable: -xdiv 512 > OptionTable: -ydiv 512 > -------------------------------------------------------------------------------------------------------- > > -------------------------------------------------------------------------------------------------------- > NK > KSP SOLVER: GMRES > MESH SIZE : 512 x 512 unknows > NUMBER OF PROCESSORS : 24 > > Linear solve converged due to CONVERGED_RTOL iterations 25038 > Linear solve converged due to CONVERGED_RTOL iterations 25995 > Linear solve converged due to CONVERGED_RTOL iterations 26769 > SNES Object: > type: ls > line search variant: SNESLineSearchCubic > alpha=0.0001, maxstep=1e+08, steptol=1e-12 > maximum iterations=100, maximum function evaluations=1000000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=77802 > total number of function evaluations=4 > KSP Object: > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=1000000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: none > linear system matrix = precond matrix: > Matrix Object: > type=mpiaij, rows=262144, cols=262144 > total: nonzeros=1308672, allocated nonzeros=1308672 > not using I-node (on process 0) routines > Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE > > Max Max/Min > Avg Total > Time (sec): 5.329e+02 1.00008 5.329e+02 > Objects: 6.300e+01 1.00000 6.300e+01 > Flops: 6.553e+10 1.01230 6.501e > +10 1.560e+12 > Flops/sec: 1.230e+08 1.01230 1.220e+08 > 2.928e+09 > Memory: 4.989e+06 > 1.01589 1.186e+08 > MPI Messages: 3.216e+05 1.99999 2.546e+05 > 6.111e+06 > MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e+09 > MPI Reductions: 6.704e+03 1.00000 > > Object Type Creations Destructions Memory Descendants' Mem. > > --- Event Stage 0: Main Stage > > SNES 1 1 > 124 0 > Krylov Solver 1 1 > 16880 0 > Preconditioner 1 1 > 0 0 > Distributed array 1 1 > 46568 0 > Index Set 6 6 > 135976 0 > Vec 45 45 > 3812684 0 > Vec Scatter 3 3 > 0 0 > IS L to G Mapping 1 1 > 45092 0 > Matrix 3 3 > 1011036 0 > Viewer 1 1 > 0 0 > > OptionTable: -ksp_converged_reason > OptionTable: -ksp_max_it 1000000 > OptionTable: -ksp_type gmres > OptionTable: -log_summary > OptionTable: -option_table > OptionTable: -par 4.0 > OptionTable: -pc_type none > OptionTable: -snes_converged_reason > OptionTable: -snes_max_funcs 1000000 > OptionTable: -snes_max_it 100 > OptionTable: -snes_view > OptionTable: -xdiv 512 > OptionTable: -ydiv 512 > -------------------------------------------------------------------------------------------------------- > From rafaelsantoscoelho at gmail.com Fri Dec 19 18:13:34 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Fri, 19 Dec 2008 22:13:34 -0200 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> Message-ID: <3b6f83d40812191613r3ecfc6canbe5850dd4a83f5e8@mail.gmail.com> Hey, Barry, thank you so much for your help and sincerity. I'll do some more tests tomorrow with different problems (using finite elements to discretize the PDE) and I see how it goes. Just one thing that wasn't directly answered (maybe you forgot to or maybe you don't know): I still don't get why the JFNK created 4 matrix objects. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Fri Dec 19 18:20:45 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 19 Dec 2008 21:20:45 -0300 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> Message-ID: As Barry said (please correct me if I'm wrong), GMRES is sometimes sensitive to roundoff error, and VERY sensitive is this error is the noise when computing an approximate linear operator via FD, and this could get even worse for large problems if you use GMRES(30), with is the default restart value in PETSc... Rafael, please follow my advice, give a try to plain CG, and please come back with your comments... On Fri, Dec 19, 2008 at 8:51 PM, Barry Smith wrote: > > When I was a kid I was guided to be fair and to not exaggerate. When I > was > a teenager I found out that adults don't actually act that way, it was a > disappointment to > me. What was worse was once I became an adult I learned that scientists > don't act > that way: when they advocate something they exaggerate its positive features > and > minimize its negative features. This is what the great priests of > Jacobian-Free Newton Krylov > do, after you see one of their presentations you know that JFNK is better in > every possible > way from any other approach. In the practical world outside of their lecture > halls there is > no such thing as "one algorithm being better than another" it is only "one > algorithm is better > than another in this specific circumstance." This is why PETSc was written > the way it > is, to provide a toolbox that allows the selection of many possibilities, > NOT to have the > library writers decide what algorithm is better. > > Their is actually nothing weird about the results you see, they are > actually what a > practitioner would expect :-) > > > Run both for 40 iterations with -ksp_monitor_true_residual > >>My answer: That probably has to do with the fact that JFNK approximately >> computes the product J(x)a. If the precision >>of that approximation is poor (and that's closely related to which approach >> was used to calculate the differencing parameter, >>I think), then the linear solver must iterate more to produce a "good" >> Newton correction. > > This is kind of right, but the answer really has nothing to do with Newton. > It is only a > problem with the linear solve. Krylov methods depend fundamentally on the > fact that > they are used for LINEAR operators; when you do the differencing to apply J > this is > no longer exactly a linear operator thus the "Krylov" space being generated > starts > to become garbage. So matrix-free and non-matrix-free for the first few > iterations will > be generally similar but after some iterations the matrix-free will start to > drift off into > near garbage. Monkeying around with the h that is used can help reduce the > garbage to some > degree, but I doubt there is one "great" way to do it that always works > well. In fact, when > people publish on "my way of computing h is better than yours", they will, > of course, exaggerate > the benefits of theirs and minimize the problems with it. > > Because of this "drifting problems" JFNK rarely makes sense without a very > good preconditioner, > that reduces the number of iterations to a handful. > > With regard to why particular things happen in one code and not in another > (like an extra vector), you > will have to look at the code and run in the debugger to answer the > questions. That is how we > would answer these questions. BTW: the extra vector is because one applies > J(u)*a, the extra vector > is where the u values are stored. > > Barry > > If you poke around with the code you will sometimes find there are better > ways to do things then > we have done them, this is why PETSc is open source, we don't expect that we > can do things better > than everyone else. > > > > On Dec 19, 2008, at 4:39 PM, Rafael Santos Coelho wrote: > >> Hey, Barry, >> >> here's what you asked: >> >> >> -------------------------------------------------------------------------------------------------------- >> JFNK >> KSP SOLVER: GMRES >> MESH SIZE : 512 x 512 unknows >> NUMBER OF PROCESSORS : 24 >> >> Linear solve converged due to CONVERGED_RTOL iterations 25042 >> Linear solve converged due to CONVERGED_RTOL iterations 33804 >> Linear solve converged due to CONVERGED_RTOL iterations 33047 >> Linear solve converged due to CONVERGED_RTOL iterations 21219 >> SNES Object: >> type: ls >> line search variant: SNESLineSearchCubic >> alpha=0.0001, maxstep=1e+08, steptol=1e-12 >> maximum iterations=100, maximum function evaluations=1000000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of linear solver iterations=113112 >> total number of function evaluations=116893 >> KSP Object: >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=1000000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> PC Object: >> type: none >> linear system matrix = precond matrix: >> Matrix Object: >> type=mffd, rows=262144, cols=262144 >> matrix-free approximation: >> err=1e-07 (relative error in function evaluation) >> Using wp compute h routine >> Computes normA >> Does not compute normU >> Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE >> >> Max Max/Min Avg >> Total >> Time (sec): 8.080e+02 1.00009 8.080e+02 >> Objects: 6.600e+01 1.00000 6.600e+01 >> Flops: 1.044e+11 1.01176 1.036e+11 >> 2.485e+12 >> Flops/sec: 1.292e+08 1.01184 1.282e+08 >> 3.076e+09 >> Memory: 5.076e+06 1.01530 >> 1.207e+08 >> MPI Messages: 4.676e+05 2.00000 3.702e+05 >> 8.884e+06 >> MPI Message Lengths: 4.002e+08 2.00939 8.623e+02 7.661e+09 >> MPI Reductions: 9.901e+03 1.00000 >> >> Object Type Creations Destructions Memory Descendants' Mem. >> >> --- Event Stage 0: Main Stage >> >> SNES 1 1 124 >> 0 >> Krylov Solver 1 1 16880 >> 0 >> Preconditioner 1 1 0 >> 0 >> Distributed array 1 1 46568 0 >> Index Set 6 6 135976 >> 0 >> Vec 46 46 3901252 >> 0 >> Vec Scatter 3 3 0 >> 0 >> IS L to G Mapping 1 1 45092 0 >> MatMFFD 1 1 0 >> 0 >> Matrix 4 4 1011036 >> 0 >> Viewer 1 1 0 >> 0 >> >> OptionTable: -ksp_converged_reason >> OptionTable: -ksp_max_it 1000000 >> OptionTable: -ksp_type gmres >> OptionTable: -log_summary >> OptionTable: -option_table >> OptionTable: -par 4.0 >> OptionTable: -pc_type none >> OptionTable: -snes_converged_reason >> OptionTable: -snes_max_funcs 1000000 >> OptionTable: -snes_max_it 100 >> OptionTable: -snes_mf >> OptionTable: -snes_view >> OptionTable: -xdiv 512 >> OptionTable: -ydiv 512 >> >> -------------------------------------------------------------------------------------------------------- >> >> >> -------------------------------------------------------------------------------------------------------- >> NK >> KSP SOLVER: GMRES >> MESH SIZE : 512 x 512 unknows >> NUMBER OF PROCESSORS : 24 >> >> Linear solve converged due to CONVERGED_RTOL iterations 25038 >> Linear solve converged due to CONVERGED_RTOL iterations 25995 >> Linear solve converged due to CONVERGED_RTOL iterations 26769 >> SNES Object: >> type: ls >> line search variant: SNESLineSearchCubic >> alpha=0.0001, maxstep=1e+08, steptol=1e-12 >> maximum iterations=100, maximum function evaluations=1000000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of linear solver iterations=77802 >> total number of function evaluations=4 >> KSP Object: >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=1000000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> PC Object: >> type: none >> linear system matrix = precond matrix: >> Matrix Object: >> type=mpiaij, rows=262144, cols=262144 >> total: nonzeros=1308672, allocated nonzeros=1308672 >> not using I-node (on process 0) routines >> Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE >> >> Max Max/Min Avg >> Total >> Time (sec): 5.329e+02 1.00008 5.329e+02 >> Objects: 6.300e+01 1.00000 6.300e+01 >> Flops: 6.553e+10 1.01230 6.501e+10 >> 1.560e+12 >> Flops/sec: 1.230e+08 1.01230 1.220e+08 >> 2.928e+09 >> Memory: 4.989e+06 1.01589 >> 1.186e+08 >> MPI Messages: 3.216e+05 1.99999 2.546e+05 6.111e+06 >> MPI Message Lengths: 2.753e+08 2.00939 8.623e+02 5.269e+09 >> MPI Reductions: 6.704e+03 1.00000 >> >> Object Type Creations Destructions Memory Descendants' Mem. >> >> --- Event Stage 0: Main Stage >> >> SNES 1 1 124 >> 0 >> Krylov Solver 1 1 16880 >> 0 >> Preconditioner 1 1 0 >> 0 >> Distributed array 1 1 46568 >> 0 >> Index Set 6 6 135976 >> 0 >> Vec 45 45 3812684 >> 0 >> Vec Scatter 3 3 0 >> 0 >> IS L to G Mapping 1 1 45092 0 >> Matrix 3 3 1011036 >> 0 >> Viewer 1 1 0 >> 0 >> >> OptionTable: -ksp_converged_reason >> OptionTable: -ksp_max_it 1000000 >> OptionTable: -ksp_type gmres >> OptionTable: -log_summary >> OptionTable: -option_table >> OptionTable: -par 4.0 >> OptionTable: -pc_type none >> OptionTable: -snes_converged_reason >> OptionTable: -snes_max_funcs 1000000 >> OptionTable: -snes_max_it 100 >> OptionTable: -snes_view >> OptionTable: -xdiv 512 >> OptionTable: -ydiv 512 >> >> -------------------------------------------------------------------------------------------------------- >> > > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From bsmith at mcs.anl.gov Fri Dec 19 18:34:17 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 19 Dec 2008 18:34:17 -0600 Subject: Weird (?) results obtained from experiments with the SNESMF module In-Reply-To: <3b6f83d40812191613r3ecfc6canbe5850dd4a83f5e8@mail.gmail.com> References: <3b6f83d40812191027v86339c3pd20aeb7457d3184e@mail.gmail.com> <3b6f83d40812191439o10d156a9q99cf307b7685ce24@mail.gmail.com> <41A960AD-3C1E-4551-BADE-6153D7D0C199@mcs.anl.gov> <3b6f83d40812191613r3ecfc6canbe5850dd4a83f5e8@mail.gmail.com> Message-ID: <342CBDF9-FB5A-4E1B-9980-467553B14286@mcs.anl.gov> On Dec 19, 2008, at 6:13 PM, Rafael Santos Coelho wrote: > Hey, Barry, > > thank you so much for your help and sincerity. I'll do some more > tests tomorrow with different problems (using finite elements to > discretize the PDE) and I see how it goes. > > Just one thing that wasn't directly answered (maybe you forgot to or > maybe you don't know): I still don't get why the JFNK created 4 > matrix objects. I don't know. If you run in the debugger and put a break point in MatCreate() you will see exactly where each of the four is being created. BTW: just in case you don't know, the matrix-free operator in PETSc is also a Mat object. Barry From bsmith at mcs.anl.gov Sun Dec 21 20:11:08 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 21 Dec 2008 20:11:08 -0600 Subject: PETSc 3.0 release References: <59883ADF-B80C-416E-8C01-6D9EE235706B@mcs.anl.gov> Message-ID: <943FFBD5-2F7F-443D-AF5D-48B854050496@mcs.anl.gov> > ------------------ Portable, Extensible Toolkit for Scientific Computation (PETSc)-------------------- We are pleased to announce the release of PETSc 3.0; parallel software libraries for the implicit solution of PDEs and related problems involving sparse matrices. Major improvements include * PETSc modules and interface definitions for use from Fortran 90 * A much improved interface for using external direct solver packages include: MUMPS, SuperLU_DIST, PLAPACK and Spooles * A brand new Python interface (developed by Lisandro Dalcin) * An implementation of the Yand and Brent variant of Bi-CG-stab that requires a single parallel synchronization step per iteration * Preliminary support for unstructured meshes. The Sieve interfaces allows creation, manipulation, and output of unstructured meshes in any dimension (developed by Matt Knepley and Dmitry Karpeev) The software is available at http://www.mcs.anl.gov/petsc. Changes and additions are listed at http://www.mcs.anl.gov/petsc/petsc-as/documentation/changes/300.html . As always, please send bug reports, questions, and requests for new features to petsc-maint at mcs.anl.gov Thanks for your continued support. The PETSc developers, Satish, Lisandro, Matt, Dmitry, Victor, Barry and Hong From recrusader at gmail.com Mon Dec 22 11:47:31 2008 From: recrusader at gmail.com (Yujie) Date: Mon, 22 Dec 2008 09:47:31 -0800 Subject: about MatGetSubMatrices() Message-ID: <7ff0ee010812220947k46a9480i1c984d37a494d5f1@mail.gmail.com> Hi, PETSc developer Now, when using MatGetSubMatrices(), the submatrices are saved in one cpu. Sometimes, the dimension of the submatrices are huge, it is impossible to save them in one cpu. I am wondering whether the submatrices can be saved in parallel mode when using MatGetSubMatrices(). thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.mayhem23 at gmail.com Mon Dec 22 13:00:21 2008 From: dave.mayhem23 at gmail.com (Dave May) Date: Tue, 23 Dec 2008 06:00:21 +1100 Subject: about MatGetSubMatrices() In-Reply-To: <7ff0ee010812220947k46a9480i1c984d37a494d5f1@mail.gmail.com> References: <7ff0ee010812220947k46a9480i1c984d37a494d5f1@mail.gmail.com> Message-ID: <956373f0812221100g128f5bb9g4437e7c12346b126@mail.gmail.com> No. If you want to extract a parallel sub matrix, use MatGetSubMatrix(). http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html Cheers, Dave On Tue, Dec 23, 2008 at 4:47 AM, Yujie wrote: > Hi, PETSc developer > > Now, when using MatGetSubMatrices(), the submatrices are saved in one cpu. > Sometimes, the dimension of the submatrices are huge, it is impossible to > save them in one cpu. > > I am wondering whether the submatrices can be saved in parallel mode when > using MatGetSubMatrices(). thanks a lot. > > Regards, > > Yujie > -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Tue Dec 23 11:35:49 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 23 Dec 2008 09:35:49 -0800 Subject: further about MatGetSubMatrices() Message-ID: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> Hi, PETSc developer I am using MatGetSubMatrices() for my application. Because I need to exact several matrices from the old one according to my index sets. These index sets are not sorted. To MPIDense matrix, it didn't work. could you give me some help? thanks a lot. Happy holiday:). Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Dec 23 16:06:11 2008 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 23 Dec 2008 17:06:11 -0500 Subject: further about MatGetSubMatrices() In-Reply-To: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> References: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> Message-ID: You can sort them with ISSort. Matt On Tue, Dec 23, 2008 at 12:35 PM, Yujie wrote: > Hi, PETSc developer > > I am using MatGetSubMatrices() for my application. Because I need to exact > several matrices from the old one according to my index sets. These index > sets are not sorted. To MPIDense matrix, it didn't work. could you give me > some help? thanks a lot. > > Happy holiday:). > > Regards, > > Yujie > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Tue Dec 23 16:36:43 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 23 Dec 2008 14:36:43 -0800 Subject: further about MatGetSubMatrices() In-Reply-To: References: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> Message-ID: <7ff0ee010812231436h4c1cbd8bwc222c463b1660e0b@mail.gmail.com> Dear Matthew: The question is that I can't sort the IS. Because I need extract the matrix according to the index of IS. I have tried to use MatPermute(). However, there is not MPIDense-based. I am wondering why MatGetSubMatrices() need use the sorted IS. Thanks a lot. Regards, Yujie On Tue, Dec 23, 2008 at 2:06 PM, Matthew Knepley wrote: > You can sort them with ISSort. > > Matt > > On Tue, Dec 23, 2008 at 12:35 PM, Yujie wrote: > >> Hi, PETSc developer >> >> I am using MatGetSubMatrices() for my application. Because I need to exact >> several matrices from the old one according to my index sets. These index >> sets are not sorted. To MPIDense matrix, it didn't work. could you give me >> some help? thanks a lot. >> >> Happy holiday:). >> >> Regards, >> >> Yujie >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Tue Dec 23 18:49:12 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 23 Dec 2008 16:49:12 -0800 Subject: exact MPI-based matrix using MatGetSubMatrix() Message-ID: <7ff0ee010812231649t16ee6895g991dd4ee5a979711@mail.gmail.com> Hi, PETSc Developer Now, I have two matrices, A and B. A (its dimension is 2N*2N) is MPIDense format. B (N*N) is MPIAIJ. I want to exact four matrices from A, like this A11 A12 A= A21 A22 each submatrices is multplied by B. My question is if A11, A12, A21, A22 have same distribution with B, how to do it with MatGetSubMatrix()? Thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Dec 24 13:15:44 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 24 Dec 2008 14:15:44 -0500 Subject: exact MPI-based matrix using MatGetSubMatrix() In-Reply-To: <7ff0ee010812231649t16ee6895g991dd4ee5a979711@mail.gmail.com> References: <7ff0ee010812231649t16ee6895g991dd4ee5a979711@mail.gmail.com> Message-ID: <420DA49E-10D3-4BCB-A62A-555805F0ABA6@mcs.anl.gov> If you want the resulting A11, etc to have the same distribution as B then when you construct A you will have to construct it with the rows/columns from the 1 block interlaced with the 2 block. This means you would form A it would look a00 a0,n a0,1 a0,n+1 ... an,0 an,n an,1 an,n+1 .. a1,0 a1,n ... .... Barry On Dec 23, 2008, at 7:49 PM, Yujie wrote: > Hi, PETSc Developer > > Now, I have two matrices, A and B. A (its dimension is 2N*2N) is > MPIDense format. B (N*N) is MPIAIJ. > > I want to exact four matrices from A, like this > > A11 A12 > > A= > > A21 A22 > > each submatrices is multplied by B. My question is if A11, A12, A21, > A22 have same distribution with B, how to do it with > MatGetSubMatrix()? > > Thanks a lot. > > Regards, > > Yujie > From bsmith at mcs.anl.gov Wed Dec 24 13:18:03 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 24 Dec 2008 14:18:03 -0500 Subject: further about MatGetSubMatrices() In-Reply-To: <7ff0ee010812231436h4c1cbd8bwc222c463b1660e0b@mail.gmail.com> References: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> <7ff0ee010812231436h4c1cbd8bwc222c463b1660e0b@mail.gmail.com> Message-ID: <44275305-64DA-4D9C-9844-1E4F9ABADB18@mcs.anl.gov> On Dec 23, 2008, at 5:36 PM, Yujie wrote: > Dear Matthew: > > The question is that I can't sort the IS. Because I need extract the > matrix according to the index of IS. I have tried to use > MatPermute(). However, there is not MPIDense-based. I am wondering > why MatGetSubMatrices() need use the sorted IS. > Because its damn hard to do the getsubmatrices if they are not sorted. Why not sort the IS in your call and then use that renumbering when it is needed in your code? An ordering is just a naming of each value so once it is sorted that is just giving you a new naming and you can use that new naming everywhere. Barry > Thanks a lot. > Regards, > > Yujie > > > On Tue, Dec 23, 2008 at 2:06 PM, Matthew Knepley > wrote: > You can sort them with ISSort. > > Matt > > On Tue, Dec 23, 2008 at 12:35 PM, Yujie wrote: > Hi, PETSc developer > > I am using MatGetSubMatrices() for my application. Because I need to > exact several matrices from the old one according to my index sets. > These index sets are not sorted. To MPIDense matrix, it didn't work. > could you give me some help? thanks a lot. > > Happy holiday:). > > Regards, > > Yujie > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > From recrusader at gmail.com Wed Dec 24 14:10:58 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 24 Dec 2008 12:10:58 -0800 Subject: exact MPI-based matrix using MatGetSubMatrix() In-Reply-To: <420DA49E-10D3-4BCB-A62A-555805F0ABA6@mcs.anl.gov> References: <7ff0ee010812231649t16ee6895g991dd4ee5a979711@mail.gmail.com> <420DA49E-10D3-4BCB-A62A-555805F0ABA6@mcs.anl.gov> Message-ID: <7ff0ee010812241210i56c9062fx60c332684c4f6320@mail.gmail.com> Dear Barry: thank you for your reply. if the distribution of A11 and B is different, it will not work if I do B*A11? Happy holiday:). Regards, Yujie On Wed, Dec 24, 2008 at 11:15 AM, Barry Smith wrote: > > If you want the resulting A11, etc to have the same distribution as B then > when you construct A you will > have to construct it with the rows/columns from the 1 block interlaced with > the 2 block. This means > you would form A it would look > a00 a0,n a0,1 a0,n+1 ... > an,0 an,n an,1 an,n+1 .. > a1,0 a1,n ... > .... > > Barry > > > On Dec 23, 2008, at 7:49 PM, Yujie wrote: > > Hi, PETSc Developer >> >> Now, I have two matrices, A and B. A (its dimension is 2N*2N) is MPIDense >> format. B (N*N) is MPIAIJ. >> >> I want to exact four matrices from A, like this >> >> A11 A12 >> >> A= >> >> A21 A22 >> >> each submatrices is multplied by B. My question is if A11, A12, A21, A22 >> have same distribution with B, how to do it with MatGetSubMatrix()? >> >> Thanks a lot. >> >> Regards, >> >> Yujie >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Wed Dec 24 14:14:13 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 24 Dec 2008 12:14:13 -0800 Subject: further about MatGetSubMatrices() In-Reply-To: <44275305-64DA-4D9C-9844-1E4F9ABADB18@mcs.anl.gov> References: <7ff0ee010812230935l66c402d3u43e2f2be23688414@mail.gmail.com> <7ff0ee010812231436h4c1cbd8bwc222c463b1660e0b@mail.gmail.com> <44275305-64DA-4D9C-9844-1E4F9ABADB18@mcs.anl.gov> Message-ID: <7ff0ee010812241214y33735238g9bab982f9a90e35a@mail.gmail.com> Dear Barry: I have solved my problem. If I need to do it according to my previous idea, because the matrix is MPIDense, I can't find MatPermute() for MPIDense. I have thought it didn't work in PETSc according to your suggestion. Happy holiday! Regards, Yujie On Wed, Dec 24, 2008 at 11:18 AM, Barry Smith wrote: > > On Dec 23, 2008, at 5:36 PM, Yujie wrote: > > Dear Matthew: >> >> The question is that I can't sort the IS. Because I need extract the >> matrix according to the index of IS. I have tried to use MatPermute(). >> However, there is not MPIDense-based. I am wondering why MatGetSubMatrices() >> need use the sorted IS. >> >> > Because its damn hard to do the getsubmatrices if they are not sorted. > > Why not sort the IS in your call and then use that renumbering when it is > needed in your code? An ordering is just a naming > of each value so once it is sorted that is just giving you a new naming and > you can use that new naming everywhere. > > Barry > > Thanks a lot. >> Regards, >> >> Yujie >> >> >> On Tue, Dec 23, 2008 at 2:06 PM, Matthew Knepley >> wrote: >> You can sort them with ISSort. >> >> Matt >> >> On Tue, Dec 23, 2008 at 12:35 PM, Yujie wrote: >> Hi, PETSc developer >> >> I am using MatGetSubMatrices() for my application. Because I need to exact >> several matrices from the old one according to my index sets. These index >> sets are not sorted. To MPIDense matrix, it didn't work. could you give me >> some help? thanks a lot. >> >> Happy holiday:). >> >> Regards, >> >> Yujie >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomjan at jay.au.poznan.pl Sat Dec 27 17:28:01 2008 From: tomjan at jay.au.poznan.pl (Tomasz Jankowski) Date: Sun, 28 Dec 2008 00:28:01 +0100 (CET) Subject: PETSC in the future... Message-ID: Hello, I'm not even beginner user of PETSC (about three years ago I have only successfully passed by - quick installation of petsc + quick diskless cluster setup + few nights of old code adjustment + few months of computations), but I have question to PETSC developers (which is result of Christmas meditations or rather Christmas surfeit ;-) ). Two weeks ago I have taken place in cell processor programming workshop. It seems that NEW is coming to us. It seems that never ending cpu clock picking up has just finished. It seems that if we will need speed up of computations in hpc we will need to switch - start writing & new rewrite our old soft! - to such specialized processors like powerxcell or developed by clearspeed math coprocessors. So my question is: could someone of PETSC developers comment this subject? Are you going to port PETSC to one of such emerging architectures or maybe to all? Is it fast&easy or another missionimpossible? or maybe it is only commercial propaganda of guys from ibm and NEW is NOT comming to us...? tom ######################################################## # tomjan at jay.au.poznan.pl # # jay.au.poznan.pl/~tomjan/ # ######################################################## From knepley at gmail.com Sat Dec 27 18:13:11 2008 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 27 Dec 2008 18:13:11 -0600 Subject: PETSC in the future... In-Reply-To: References: Message-ID: On Sat, Dec 27, 2008 at 5:28 PM, Tomasz Jankowski wrote: > Hello, > > I'm not even beginner user of PETSC (about three years ago I have only > successfully passed by - quick installation of petsc + quick diskless > cluster setup + few nights of old code adjustment + few months of > computations), but I have question to PETSC developers (which is result of > Christmas meditations or rather Christmas surfeit ;-) ). > > Two weeks ago I have taken place in cell processor programming workshop. It > seems that NEW is coming to us. It seems that never ending cpu clock picking > up has just finished. It seems that if we will need speed up of computations > in hpc we will need to switch - start writing & new rewrite our old soft! - > to such specialized processors like powerxcell or developed by > clearspeed math coprocessors. So my question is: could someone of PETSC > developers comment this subject? > Are you going to port PETSC to one of such emerging architectures or maybe > to all? Is it fast&easy or another missionimpossible? or maybe it is only > commercial propaganda of guys from ibm and NEW is NOT comming to us...? My feeling is that these architectures will indeed to important, however this view is not universal. Notice that the coprocessor idea has been with us since the beginning of computing, and each time it is sold as the new revolution, only to disappoint its adherents. I think this time is different because I have seen real performance benefit and new thinking about software design, mainly from Robert van de Geijn at UT Austin (FLAME project). I see a lot of benefit to running hierarchical, block algorithms on a GPU. However, sparse MatMult() is a lost cause. Thus if the major cost is sparse matvec, change your algorithm. I truly believe that Krylov accelerators will fade back into the background as we get high quality implementations of FFT, MG, FMM, Wavelets, and other order N methods. These can be fixed up by relatively solver Krylov wrappers on the outside. Matt > > tom > > ######################################################## > # tomjan at jay.au.poznan.pl # > # jay.au.poznan.pl/~tomjan/ # > ######################################################## > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinzishuai at yahoo.com Mon Dec 29 03:34:37 2008 From: jinzishuai at yahoo.com (Shi Jin) Date: Mon, 29 Dec 2008 01:34:37 -0800 (PST) Subject: Problem installing petsc on a Blue Gene/L Message-ID: <452872.38590.qm@web36206.mail.mud.yahoo.com> Hi there, I have been trying to install petsc-3.0 on a Blue Gene/L system. Here is what I used to configure: ./configure --CC=blrts_xlc --CFLAGS="-I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts" --with-mpi-compilers=0 --with-fortran=0 I got an error as following: TESTING: checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) ********************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------- Unable to determine sizeof_char ********************************************************************************* Here is the relevant configure.log part: TEST checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) TESTING: checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) Determines the size of type "typeName", and defines SIZEOF_"typeName" to be the size Checking for size of type: char Pushing language C sh: blrts_xlc -c -o conftest.o -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c Executing: blrts_xlc -c -o conftest.o -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c sh: Pushing language C Popping language C sh: blrts_xlc -o conftest -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o Executing: blrts_xlc -o conftest -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o sh: Executing: ./conftest sh: ./conftest Executing: ./conftest sh: ERROR while running executable: Could not execute './conftest': ********************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------- Unable to determine sizeof_char ********************************************************************************* File "./configure", line 217, in petsc_configure framework.configure(out = sys.stdout) File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/framework.py", line 884, in configure child.configure() File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", line 353, in configure map(lambda type: self.executeTest(self.checkSizeof, type), ['char','void *', 'short', 'int', 'long', 'long long', 'float', 'double', 'size_t']) File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", line 353, in map(lambda type: self.executeTest(self.checkSizeof, type), ['char','void *', 'short', 'int', 'long', 'long long', 'float', 'double', 'size_t']) File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/base.py", line 97, in executeTest ret = apply(test, args,kargs) File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", line 275, in checkSizeof raise RuntimeError('Unable to determine '+typename) Do you know what could be my possible cause of the error? I wrote a simple C test code and was able to show that sizeof(char)=1. Is there a way to get the conftest.c code and try it out manually? Thanks a lot. -- Shi Jin, PhD From bsmith at mcs.anl.gov Mon Dec 29 09:34:50 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 29 Dec 2008 10:34:50 -0500 Subject: Problem installing petsc on a Blue Gene/L In-Reply-To: <452872.38590.qm@web36206.mail.mud.yahoo.com> References: <452872.38590.qm@web36206.mail.mud.yahoo.com> Message-ID: Since the BlueGene is a batch system you need the additional option --with-batch and follow the directions. Barry On Dec 29, 2008, at 4:34 AM, Shi Jin wrote: > Hi there, > > I have been trying to install petsc-3.0 on a Blue Gene/L system. > Here is what I used to configure: > ./configure --CC=blrts_xlc --CFLAGS="-I/bgl/BlueLight/ppcfloor/ > bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/ > BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts - > ldevices.rts" --with-mpi-compilers=0 --with-fortran=0 > > I got an error as following: > TESTING: checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/ > petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > ********************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for details): > --------------------------------------------------------------------------------------- > Unable to determine sizeof_char > ********************************************************************************* > > Here is the relevant configure.log part: > TEST checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/ > petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > TESTING: checkSizeof from config.types(/gpfs/bglscratch/pi/sjin/src/ > petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > Determines the size of type "typeName", and defines > SIZEOF_"typeName" to be the size > Checking for size of type: char > Pushing language C > sh: blrts_xlc -c -o conftest.o -I/bgl/BlueLight/ppcfloor/bglsys/ > include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/ > BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts - > ldevices.rts -g conftest.c > Executing: blrts_xlc -c -o conftest.o -I/bgl/BlueLight/ppcfloor/ > bglsys/include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/ > BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts - > ldevices.rts -g conftest.c > sh: > Pushing language C > Popping language C > sh: blrts_xlc -o conftest -I/bgl/BlueLight/ppcfloor/bglsys/ > include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/ > BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts - > ldevices.rts -g conftest.o > Executing: blrts_xlc -o conftest -I/bgl/BlueLight/ppcfloor/bglsys/ > include -qtune=440 -qarch=440d -qlist -qsource -O3 -L/bgl/ > BlueLight/ppcfloor/bglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts - > ldevices.rts -g conftest.o > sh: > Executing: ./conftest > sh: ./conftest > Executing: ./conftest > sh: > ERROR while running executable: Could not execute './conftest': > ********************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for details): > --------------------------------------------------------------------------------------- > Unable to determine sizeof_char > ********************************************************************************* > File "./configure", line 217, in petsc_configure > framework.configure(out = sys.stdout) > File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/ > BuildSystem/config/framework.py", line 884, in configure > child.configure() > File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/ > BuildSystem/config/types.py", line 353, in configure > map(lambda type: self.executeTest(self.checkSizeof, type), > ['char','void *', 'short', 'int', 'long', 'long long', 'float', > 'double', 'size_t']) > File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/ > BuildSystem/config/types.py", line 353, in > map(lambda type: self.executeTest(self.checkSizeof, type), > ['char','void *', 'short', 'int', 'long', 'long long', 'float', > 'double', 'size_t']) > File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/ > BuildSystem/config/base.py", line 97, in executeTest > ret = apply(test, args,kargs) > File "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/ > BuildSystem/config/types.py", line 275, in checkSizeof > raise RuntimeError('Unable to determine '+typename) > > Do you know what could be my possible cause of the error? > I wrote a simple C test code and was able to show that > sizeof(char)=1. Is there a way to get the conftest.c code and try it > out manually? > Thanks a lot. > > -- > Shi Jin, PhD > > > > From recrusader at gmail.com Mon Dec 29 13:55:40 2008 From: recrusader at gmail.com (Yujie) Date: Mon, 29 Dec 2008 11:55:40 -0800 Subject: MPIDense*MPIDense Message-ID: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> Hi, I am wondering whether it have been realized in PETSc to do MPIDense*MPIDense? thanks a lot. Happy holiday:). Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinzishuai at yahoo.com Mon Dec 29 21:43:37 2008 From: jinzishuai at yahoo.com (Shi Jin) Date: Mon, 29 Dec 2008 19:43:37 -0800 (PST) Subject: Problem installing petsc on a Blue Gene/L In-Reply-To: Message-ID: <945406.85120.qm@web36204.mail.mud.yahoo.com> Thank you very much. I did 1. ./configure --CC=mpixlc --FC=mpixlf77 --with-batch --download-f-blas-lapack=1 ================================================================================= Since your compute nodes require use of a batch system or mpiexec you must: 1) Submit ./conftest to 1 processor of your batch system or system you are cross-compiling for; this will generate the file reconfigure.py 2) Run ./reconfigure.py (to complete the configure process). ================================================================================= 2. mpirun -partition R001 -n 1 -mode vn -cwd $PWD -exe $PWD/conftest But it exited with error during the run. I see the output file sjin at fen1 petsc-3.0.0-p0 $ cat reconfigure.py #!/usr/bin/python configure_options = [ '--with-memcmp-ok', '--sizeof_char=1', '--sizeof_void_p=4', '--sizeof_short=2', '--sizeof_int=4', '--sizeof_long=4', '--sizeof_long_long=8', '--sizeof_float=4', '--sizeof_double=8', '--sizeof_size_t=4', '--bits_per_byte=8', '--sizeof_MPI_Comm=4', '--sizeof_MPI_Fint=4', sjin at fen1 petsc-3.0.0-p0 $ It is not complete and error must have happened after sizeof_MPI_Fint. The good news is that the sizeof(char) error is no longer there once we do the batch mode. I am wondering if there is a way to keep the conftest.c source and relevant header files (I think they are created temporarily and deleted after the built). This way, it would be fairly easy to figure out which part is having problem. Please help. Thank you very much. -- Shi Jin, PhD --- On Mon, 12/29/08, Barry Smith wrote: > From: Barry Smith > Subject: Re: Problem installing petsc on a Blue Gene/L > To: petsc-users at mcs.anl.gov > Date: Monday, December 29, 2008, 10:34 AM > Since the BlueGene is a batch system you need the additional > option --with-batch and follow > the directions. > > Barry > > On Dec 29, 2008, at 4:34 AM, Shi Jin wrote: > > > Hi there, > > > > I have been trying to install petsc-3.0 on a Blue > Gene/L system. > > Here is what I used to configure: > > ./configure --CC=blrts_xlc > --CFLAGS="-I/bgl/BlueLight/ppcfloor/bglsys/include > -qtune=440 -qarch=440d -qlist -qsource -O3 > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > -lmsglayer.rts -lrts.rts -ldevices.rts" > --with-mpi-compilers=0 --with-fortran=0 > > > > I got an error as following: > > TESTING: checkSizeof from > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > > ********************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > configure.log for details): > > > --------------------------------------------------------------------------------------- > > Unable to determine sizeof_char > > > ********************************************************************************* > > > > Here is the relevant configure.log part: > > TEST checkSizeof from > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > TESTING: checkSizeof from > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > Determines the size of type "typeName", and > defines SIZEOF_"typeName" to be the size > > Checking for size of type: char > > Pushing language C > > sh: blrts_xlc -c -o conftest.o > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > -qarch=440d -qlist -qsource -O3 > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c > > Executing: blrts_xlc -c -o conftest.o > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > -qarch=440d -qlist -qsource -O3 > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c > > sh: > > Pushing language C > > Popping language C > > sh: blrts_xlc -o conftest > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > -qarch=440d -qlist -qsource -O3 > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o > > Executing: blrts_xlc -o conftest > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > -qarch=440d -qlist -qsource -O3 > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o > > sh: > > Executing: ./conftest > > sh: ./conftest > > Executing: ./conftest > > sh: > > ERROR while running executable: Could not execute > './conftest': > > > ********************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > configure.log for details): > > > --------------------------------------------------------------------------------------- > > Unable to determine sizeof_char > > > ********************************************************************************* > > File "./configure", line 217, in > petsc_configure > > framework.configure(out = sys.stdout) > > File > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/framework.py", > line 884, in configure > > child.configure() > > File > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > line 353, in configure > > map(lambda type: self.executeTest(self.checkSizeof, > type), ['char','void *', 'short', > 'int', 'long', 'long long', > 'float', 'double', 'size_t']) > > File > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > line 353, in > > map(lambda type: self.executeTest(self.checkSizeof, > type), ['char','void *', 'short', > 'int', 'long', 'long long', > 'float', 'double', 'size_t']) > > File > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/base.py", > line 97, in executeTest > > ret = apply(test, args,kargs) > > File > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > line 275, in checkSizeof > > raise RuntimeError('Unable to determine > '+typename) > > > > Do you know what could be my possible cause of the > error? > > I wrote a simple C test code and was able to show that > sizeof(char)=1. Is there a way to get the conftest.c code > and try it out manually? > > Thanks a lot. > > > > -- > > Shi Jin, PhD > > > > > > > > From balay at mcs.anl.gov Tue Dec 30 00:52:14 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 30 Dec 2008 00:52:14 -0600 (CST) Subject: Problem installing petsc on a Blue Gene/L In-Reply-To: <945406.85120.qm@web36204.mail.mud.yahoo.com> References: <945406.85120.qm@web36204.mail.mud.yahoo.com> Message-ID: On Mon, 29 Dec 2008, Shi Jin wrote: > Thank you very much. > I did > 1. ./configure --CC=mpixlc --FC=mpixlf77 --with-batch --download-f-blas-lapack=1 > ================================================================================= > Since your compute nodes require use of a batch system or mpiexec you must: > 1) Submit ./conftest to 1 processor of your batch system or system you are > cross-compiling for; this will generate the file reconfigure.py > 2) Run ./reconfigure.py (to complete the configure process). > ================================================================================= > 2. mpirun -partition R001 -n 1 -mode vn -cwd $PWD -exe $PWD/conftest > But it exited with error during the run. I see the output file > sjin at fen1 petsc-3.0.0-p0 $ cat reconfigure.py > #!/usr/bin/python > > configure_options = [ > '--with-memcmp-ok', '--sizeof_char=1', > '--sizeof_void_p=4', > '--sizeof_short=2', > '--sizeof_int=4', > '--sizeof_long=4', > '--sizeof_long_long=8', > '--sizeof_float=4', > '--sizeof_double=8', > '--sizeof_size_t=4', > '--bits_per_byte=8', > '--sizeof_MPI_Comm=4', > '--sizeof_MPI_Fint=4', > sjin at fen1 petsc-3.0.0-p0 $ > > It is not complete and error must have happened after sizeof_MPI_Fint. The good news is that the sizeof(char) error is no longer there once we do the batch mode. I am wondering if there is a way to keep the conftest.c source and relevant header files (I think they are created temporarily and deleted after the built). This way, it would be fairly easy to figure out which part is having problem. Yes- preserving conftest.c would be useful. Currently I've managed to grab this code by introducing a compile bug into the generated code, and then looking at configure.log [since the compile fails - configure spits out the code into configure.log] For eg: - the following change will do this.. asterix:/home/balay/tmp/petsc-3.0.0-p0>hg diff diff -r 91983f07319f config/BuildSystem/config/packages/MPI.py --- a/config/BuildSystem/config/packages/MPI.py Tue Dec 30 00:49:45 2008 -0600 +++ b/config/BuildSystem/config/packages/MPI.py Tue Dec 30 00:50:01 2008 -0600 @@ -233,7 +233,7 @@ self.framework.addBatchBody(''' { MPI_Aint size; - int ierr = MPI_Type_extent(%s, &size); + int ierr = MPI_Type_extent(%s, &size) if(!ierr && (size != 0)) { fprintf(output, " \'--have-mpi-%s=1\',\\n"); } else { asterix:/home/balay/tmp/petsc-3.0.0-p0> However it is likely the attached patch will fix this problem [it is scheduled for the next patch update] cd config/BuildSystem patch -Np1 < mpi-check-cray.patch Satish > Please help. > Thank you very much. > -- > Shi Jin, PhD > > > --- On Mon, 12/29/08, Barry Smith wrote: > > > From: Barry Smith > > Subject: Re: Problem installing petsc on a Blue Gene/L > > To: petsc-users at mcs.anl.gov > > Date: Monday, December 29, 2008, 10:34 AM > > Since the BlueGene is a batch system you need the additional > > option --with-batch and follow > > the directions. > > > > Barry > > > > On Dec 29, 2008, at 4:34 AM, Shi Jin wrote: > > > > > Hi there, > > > > > > I have been trying to install petsc-3.0 on a Blue > > Gene/L system. > > > Here is what I used to configure: > > > ./configure --CC=blrts_xlc > > --CFLAGS="-I/bgl/BlueLight/ppcfloor/bglsys/include > > -qtune=440 -qarch=440d -qlist -qsource -O3 > > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > > -lmsglayer.rts -lrts.rts -ldevices.rts" > > --with-mpi-compilers=0 --with-fortran=0 > > > > > > I got an error as following: > > > TESTING: checkSizeof from > > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > > > > ********************************************************************************* > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > > configure.log for details): > > > > > --------------------------------------------------------------------------------------- > > > Unable to determine sizeof_char > > > > > ********************************************************************************* > > > > > > Here is the relevant configure.log part: > > > TEST checkSizeof from > > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > > TESTING: checkSizeof from > > config.types(/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py:251) > > > Determines the size of type "typeName", and > > defines SIZEOF_"typeName" to be the size > > > Checking for size of type: char > > > Pushing language C > > > sh: blrts_xlc -c -o conftest.o > > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > > -qarch=440d -qlist -qsource -O3 > > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c > > > Executing: blrts_xlc -c -o conftest.o > > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > > -qarch=440d -qlist -qsource -O3 > > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.c > > > sh: > > > Pushing language C > > > Popping language C > > > sh: blrts_xlc -o conftest > > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > > -qarch=440d -qlist -qsource -O3 > > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o > > > Executing: blrts_xlc -o conftest > > -I/bgl/BlueLight/ppcfloor/bglsys/include -qtune=440 > > -qarch=440d -qlist -qsource -O3 > > -L/bgl/BlueLight/ppcfloor/bglsys/lib -lmpich.rts > > -lmsglayer.rts -lrts.rts -ldevices.rts -g conftest.o > > > sh: > > > Executing: ./conftest > > > sh: ./conftest > > > Executing: ./conftest > > > sh: > > > ERROR while running executable: Could not execute > > './conftest': > > > > > ********************************************************************************* > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > > configure.log for details): > > > > > --------------------------------------------------------------------------------------- > > > Unable to determine sizeof_char > > > > > ********************************************************************************* > > > File "./configure", line 217, in > > petsc_configure > > > framework.configure(out = sys.stdout) > > > File > > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/framework.py", > > line 884, in configure > > > child.configure() > > > File > > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > > line 353, in configure > > > map(lambda type: self.executeTest(self.checkSizeof, > > type), ['char','void *', 'short', > > 'int', 'long', 'long long', > > 'float', 'double', 'size_t']) > > > File > > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > > line 353, in > > > map(lambda type: self.executeTest(self.checkSizeof, > > type), ['char','void *', 'short', > > 'int', 'long', 'long long', > > 'float', 'double', 'size_t']) > > > File > > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/base.py", > > line 97, in executeTest > > > ret = apply(test, args,kargs) > > > File > > "/gpfs/bglscratch/pi/sjin/src/petsc-3.0.0-p0/config/BuildSystem/config/types.py", > > line 275, in checkSizeof > > > raise RuntimeError('Unable to determine > > '+typename) > > > > > > Do you know what could be my possible cause of the > > error? > > > I wrote a simple C test code and was able to show that > > sizeof(char)=1. Is there a way to get the conftest.c code > > and try it out manually? > > > Thanks a lot. > > > > > > -- > > > Shi Jin, PhD > > > > > > > > > > > > > > > > > -------------- next part -------------- tree 1573f683f67d parent 0d13e9fc4001 author Matt Knepley 1230131724 21600 committer Matt Knepley 1230131724 21600 revision 2207 branch default Disable testing code diff --git a/config/packages/MPI.py b/config/packages/MPI.py --- a/config/packages/MPI.py +++ b/config/packages/MPI.py @@ -218,7 +218,8 @@ oldLibs = self.compilers.LIBS self.compilers.CPPFLAGS += ' '+self.headers.toString(self.include) self.compilers.LIBS = self.libraries.toString(self.lib)+' '+self.compilers.LIBS - for datatype, name in [('MPI_LONG_DOUBLE', 'long-double'), ('MPI_ENORMOUS_DOUBLE', 'enormous-double'), ('MPI_UNBELIEVABLE_DOUBLE', 'unbelievable-double')]: + #for datatype, name in [('MPI_LONG_DOUBLE', 'long-double'), ('MPI_ENORMOUS_DOUBLE', 'enormous-double'), ('MPI_UNBELIEVABLE_DOUBLE', 'unbelievable-double')]: + for datatype, name in [('MPI_LONG_DOUBLE', 'long-double')]: if self.checkCompile('#ifdef PETSC_HAVE_STDLIB_H\n #include \n#endif\n#include \n', 'MPI_Aint size;\nint ierr = MPI_Type_extent('+datatype+', &size);\nif(ierr || (size == 0)) exit(1);\n'): if 'have-mpi-'+name in self.argDB: if int(self.argDB['have-mpi-'+name]): tree e3bac313a1d7 parent 5c1db22fa3b5 author Matt Knepley 1230134339 21600 committer Matt Knepley 1230134339 21600 revision 2208 branch default Added setup/cleanup section for batch executable and made main() take arguments - Put in MPI_Init/Finalize() for batch run diff --git a/config/framework.py b/config/framework.py --- a/config/framework.py +++ b/config/framework.py @@ -83,8 +83,10 @@ self.argDB['CPPFLAGS'] = '' if not 'LDFLAGS' in self.argDB: self.argDB['LDFLAGS'] = '' + self.batchSetup = [] self.batchIncludes = [] self.batchBodies = [] + self.batchCleanup = [] self.batchIncludeDirs = [] self.dependencies = {} self.configureParent = None @@ -817,6 +819,13 @@ self.externalPackagesDir = None return + def addBatchSetup(self, setup): + '''Add code to be run before batch tests execute''' + if not isinstance(setup, list): + setup = [setup] + self.batchSetup.extend(setup) + return + def addBatchInclude(self, includes): '''Add an include or a list of includes to the batch run''' if not isinstance(includes, list): @@ -831,6 +840,13 @@ self.batchBodies.extend(statements) return + def addBatchCleanup(self, cleanup): + '''Add code to be run after batch tests execute''' + if not isinstance(cleanup, list): + cleanup = [cleanup] + self.batchCleanup.extend(cleanup) + return + def configureBatch(self): '''F''' if self.batchBodies: @@ -841,7 +857,9 @@ body = ['FILE *output = fopen("reconfigure.py","w");'] body.append('fprintf(output, "#!'+sys.executable+'\\n");') body.append('fprintf(output, "\\nconfigure_options = [\\n");') + body.extend(self.batchSetup) body.extend(self.batchBodies) + body.extend(self.batchCleanup) body.append('fprintf(output, " '+repr(args)[1:-1]+'\\n]");') driver = ['fprintf(output, "\\nif __name__ == \'__main__\':', ' import os', @@ -856,7 +874,7 @@ oldFlags = self.compilers.CPPFLAGS self.compilers.CPPFLAGS += ' '.join(self.batchIncludeDirs) self.batchIncludes.insert(0, '#include \n#include \n#include ') - if not self.checkLink('\n'.join(self.batchIncludes)+'\n', '\n'.join(body), cleanup = 0): + if not self.checkLink('\n'.join(self.batchIncludes)+'\n', '\n'.join(body), cleanup = 0, codeBegin = '\nint main(int argc, char **argv) {\n'): sys.exit('Unable to generate test file for cross-compilers/batch-system\n') self.compilers.CPPFLAGS = oldFlags self.logClear() diff --git a/config/packages/MPI.py b/config/packages/MPI.py --- a/config/packages/MPI.py +++ b/config/packages/MPI.py @@ -46,6 +46,7 @@ # local state self.commf2c = 0 self.commc2f = 0 + self.needBatchMPI = 1 return def setupHelp(self, help): @@ -230,6 +231,10 @@ self.addDefine('HAVE_'+datatype, 1) self.popLanguage() else: + if self.needBatchMPI: + self.framework.addBatchSetup('if (MPI_Init(&argc, &argv));') + self.framework.addBatchCleanup('if (MPI_Finalize());') + self.needBatchMPI = 0 self.framework.addBatchInclude(['#include ', '#include ']) self.framework.addBatchBody(''' { diff -r 97d5249b8bbd config/packages/MPI.py --- a/config/packages/MPI.py Wed Dec 24 09:58:59 2008 -0600 +++ b/config/packages/MPI.py Sat Dec 27 09:00:46 2008 -0600 @@ -238,8 +238,11 @@ self.framework.addBatchInclude(['#include ', '#include ']) self.framework.addBatchBody(''' { - MPI_Aint size; - int ierr = MPI_Type_extent(%s, &size); + MPI_Aint size=0; + int ierr=0; + if (MPI_LONG_DOUBLE != MPI_DATATYPE_NULL) { + ierr = MPI_Type_extent(%s, &size); + } if(!ierr && (size != 0)) { fprintf(output, " \'--have-mpi-%s=1\',\\n"); } else { From bsmith at mcs.anl.gov Tue Dec 30 14:28:46 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 30 Dec 2008 14:28:46 -0600 Subject: MPIDense*MPIDense In-Reply-To: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> Message-ID: In theory it is all there. Just do MatMatMult() with your dense matrices Barry On Dec 29, 2008, at 1:55 PM, Yujie wrote: > Hi, > > I am wondering whether it have been realized in PETSc to do > MPIDense*MPIDense? thanks a lot. Happy holiday:). > > Regards, > > Yujie > From recrusader at gmail.com Tue Dec 30 15:27:33 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 30 Dec 2008 13:27:33 -0800 Subject: MPIDense*MPIDense In-Reply-To: References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> Message-ID: <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> Dear Barry: thank you for your reply. MatMatMult() uses Plapcak package for Dense matrices? In addition, about MatGetSubMatrix() and submatrix multiplcation problems, could you give me some advice? "if the distribution of A11 and B is different, it will not work if I do B*A11? if it work, what is the distribution of B*A11?" thanks a lot. Happy holiday:). Regards, Yujie On Tue, Dec 30, 2008 at 12:28 PM, Barry Smith wrote: > > In theory it is all there. Just do MatMatMult() with your dense matrices > > Barry > > On Dec 29, 2008, at 1:55 PM, Yujie wrote: > > Hi, >> >> I am wondering whether it have been realized in PETSc to do >> MPIDense*MPIDense? thanks a lot. Happy holiday:). >> >> Regards, >> >> Yujie >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Dec 30 15:54:20 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 30 Dec 2008 15:54:20 -0600 Subject: MPIDense*MPIDense In-Reply-To: <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> Message-ID: On Dec 30, 2008, at 3:27 PM, Yujie wrote: > Dear Barry: > > thank you for your reply. MatMatMult() uses Plapcak package for > Dense matrices? > In the parallel case, for sequential it uses LAPACK > > In addition, about MatGetSubMatrix() and submatrix multiplcation > problems, could you give me some advice? > > "if the distribution of A11 and B is different, it will not work if > I do B*A11? if it work, what is the distribution of B*A11?" > The row ownership of B and the column ownership of A11 > thanks a lot. Happy holiday:). > > Regards, > > Yujie > > > > > > On Tue, Dec 30, 2008 at 12:28 PM, Barry Smith > wrote: > > In theory it is all there. Just do MatMatMult() with your dense > matrices > > Barry > > > On Dec 29, 2008, at 1:55 PM, Yujie wrote: > > Hi, > > I am wondering whether it have been realized in PETSc to do > MPIDense*MPIDense? thanks a lot. Happy holiday:). > > Regards, > > Yujie > > > From recrusader at gmail.com Tue Dec 30 17:04:00 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 30 Dec 2008 15:04:00 -0800 Subject: MPIDense*MPIDense In-Reply-To: References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> Message-ID: <7ff0ee010812301504g534abd61vc523e63c6a01af4b@mail.gmail.com> Dear Barry: You mean B*A11 works in parallel mode if B and A11 have different distribution? Its distribution is decided by the row ownership of B and the column ownership of A11? thanks a lot. Regards, Yujie On Tue, Dec 30, 2008 at 1:54 PM, Barry Smith wrote: > > On Dec 30, 2008, at 3:27 PM, Yujie wrote: > > Dear Barry: >> >> thank you for your reply. MatMatMult() uses Plapcak package for Dense >> matrices? >> >> In the parallel case, for sequential it uses LAPACK > >> >> In addition, about MatGetSubMatrix() and submatrix multiplcation problems, >> could you give me some advice? >> >> "if the distribution of A11 and B is different, it will not work if I do >> B*A11? if it work, what is the distribution of B*A11?" >> >> > The row ownership of B and the column ownership of A11 > >> thanks a lot. Happy holiday:). >> >> Regards, >> >> Yujie >> >> >> >> >> >> On Tue, Dec 30, 2008 at 12:28 PM, Barry Smith wrote: >> >> In theory it is all there. Just do MatMatMult() with your dense matrices >> >> Barry >> >> >> On Dec 29, 2008, at 1:55 PM, Yujie wrote: >> >> Hi, >> >> I am wondering whether it have been realized in PETSc to do >> MPIDense*MPIDense? thanks a lot. Happy holiday:). >> >> Regards, >> >> Yujie >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Dec 30 17:17:17 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 30 Dec 2008 17:17:17 -0600 Subject: MPIDense*MPIDense In-Reply-To: <7ff0ee010812301504g534abd61vc523e63c6a01af4b@mail.gmail.com> References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> <7ff0ee010812301504g534abd61vc523e63c6a01af4b@mail.gmail.com> Message-ID: <84887409-9289-405B-B812-BDFAEFDDC3D2@mcs.anl.gov> They need to have the same distributions. Barry On Dec 30, 2008, at 5:04 PM, Yujie wrote: > Dear Barry: > > You mean B*A11 works in parallel mode if B and A11 have different > distribution? Its distribution is decided by the row ownership of B > and the column ownership of A11? thanks a lot. > > Regards, > > Yujie > > > On Tue, Dec 30, 2008 at 1:54 PM, Barry Smith > wrote: > > On Dec 30, 2008, at 3:27 PM, Yujie wrote: > > Dear Barry: > > thank you for your reply. MatMatMult() uses Plapcak package for > Dense matrices? > > In the parallel case, for sequential it uses LAPACK > > > In addition, about MatGetSubMatrix() and submatrix multiplcation > problems, could you give me some advice? > > "if the distribution of A11 and B is different, it will not work if > I do B*A11? if it work, what is the distribution of B*A11?" > > > The row ownership of B and the column ownership of A11 > > thanks a lot. Happy holiday:). > > Regards, > > Yujie > > > > > > On Tue, Dec 30, 2008 at 12:28 PM, Barry Smith > wrote: > > In theory it is all there. Just do MatMatMult() with your dense > matrices > > Barry > > > On Dec 29, 2008, at 1:55 PM, Yujie wrote: > > Hi, > > I am wondering whether it have been realized in PETSc to do > MPIDense*MPIDense? thanks a lot. Happy holiday:). > > Regards, > > Yujie > > > > > From recrusader at gmail.com Tue Dec 30 17:39:38 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 30 Dec 2008 15:39:38 -0800 Subject: MPIDense*MPIDense In-Reply-To: <84887409-9289-405B-B812-BDFAEFDDC3D2@mcs.anl.gov> References: <7ff0ee010812291155t6afbe91el19de4d2a35624abe@mail.gmail.com> <7ff0ee010812301327r32a0a683p91d8d8d23f1ce67b@mail.gmail.com> <7ff0ee010812301504g534abd61vc523e63c6a01af4b@mail.gmail.com> <84887409-9289-405B-B812-BDFAEFDDC3D2@mcs.anl.gov> Message-ID: <7ff0ee010812301539x5634da2du659bd7243f1aecb6@mail.gmail.com> Whether is there a method to repartition the matrix to get the same distribtutions? thanks. Yujie On Tue, Dec 30, 2008 at 3:17 PM, Barry Smith wrote: > > They need to have the same distributions. > > Barry > > On Dec 30, 2008, at 5:04 PM, Yujie wrote: > > Dear Barry: >> >> You mean B*A11 works in parallel mode if B and A11 have different >> distribution? Its distribution is decided by the row ownership of B and the >> column ownership of A11? thanks a lot. >> >> Regards, >> >> Yujie >> >> >> On Tue, Dec 30, 2008 at 1:54 PM, Barry Smith wrote: >> >> On Dec 30, 2008, at 3:27 PM, Yujie wrote: >> >> Dear Barry: >> >> thank you for your reply. MatMatMult() uses Plapcak package for Dense >> matrices? >> >> In the parallel case, for sequential it uses LAPACK >> >> >> In addition, about MatGetSubMatrix() and submatrix multiplcation problems, >> could you give me some advice? >> >> "if the distribution of A11 and B is different, it will not work if I do >> B*A11? if it work, what is the distribution of B*A11?" >> >> >> The row ownership of B and the column ownership of A11 >> >> thanks a lot. Happy holiday:). >> >> Regards, >> >> Yujie >> >> >> >> >> >> On Tue, Dec 30, 2008 at 12:28 PM, Barry Smith wrote: >> >> In theory it is all there. Just do MatMatMult() with your dense matrices >> >> Barry >> >> >> On Dec 29, 2008, at 1:55 PM, Yujie wrote: >> >> Hi, >> >> I am wondering whether it have been realized in PETSc to do >> MPIDense*MPIDense? thanks a lot. Happy holiday:). >> >> Regards, >> >> Yujie >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinzishuai at yahoo.com Tue Dec 30 22:47:37 2008 From: jinzishuai at yahoo.com (Shi Jin) Date: Tue, 30 Dec 2008 20:47:37 -0800 (PST) Subject: Problem installing petsc on a Blue Gene/L In-Reply-To: Message-ID: <535919.87151.qm@web36201.mail.mud.yahoo.com> > Yes- preserving conftest.c would be useful. Currently > I've managed to > grab this code by introducing a compile bug into the > generated code, > and then looking at configure.log [since the compile fails > - configure > spits out the code into configure.log] > > For eg: - the following change will do this.. > asterix:/home/balay/tmp/petsc-3.0.0-p0>hg diff > diff -r 91983f07319f > config/BuildSystem/config/packages/MPI.py > --- a/config/BuildSystem/config/packages/MPI.py Tue Dec 30 > 00:49:45 2008 -0600 > +++ b/config/BuildSystem/config/packages/MPI.py Tue Dec 30 > 00:50:01 2008 -0600 > @@ -233,7 +233,7 @@ > self.framework.addBatchBody(''' > { > MPI_Aint size; > - int ierr = MPI_Type_extent(%s, &size); > + int ierr = MPI_Type_extent(%s, &size) > if(!ierr && (size != 0)) { > fprintf(output, " > \'--have-mpi-%s=1\',\\n"); > } else { > asterix:/home/balay/tmp/petsc-3.0.0-p0> > > > However it is likely the attached patch will fix this > problem [it is > scheduled for the next patch update] > > cd config/BuildSystem > patch -Np1 < mpi-check-cray.patch > > Satish > Thank you very much. Actually I just used the petsc-dev code and it worked without any problem. Thank you very much for your help. By the way, could you please suggest a petsc test case of KSP solver on the Blue Gene? There are many tests coming with PETSc. It would be great if I could just focus on one test. Thanks again. Shi From recrusader at gmail.com Tue Dec 30 23:21:36 2008 From: recrusader at gmail.com (Yujie) Date: Tue, 30 Dec 2008 21:21:36 -0800 Subject: MatAXPY() for MPIDense Message-ID: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> Hi, PETSc Developers I have checked the codes in PETSc3.0.0. I cannot find there is MatAXPY() for MPIDense. Now, MatAXPY() supports MPIDense format? if not, how to do it? thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Dec 31 01:08:45 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 31 Dec 2008 01:08:45 -0600 (CST) Subject: Problem installing petsc on a Blue Gene/L In-Reply-To: <535919.87151.qm@web36201.mail.mud.yahoo.com> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> Message-ID: On Tue, 30 Dec 2008, Shi Jin wrote: > > However it is likely the attached patch will fix this > > problem [it is > > scheduled for the next patch update] > > > > cd config/BuildSystem > > patch -Np1 < mpi-check-cray.patch > > > > Satish > > > Thank you very much. Actually I just used the petsc-dev code and it worked without any problem. Thank you very much for your help. Ok - that above patch is in current petsc-dev - so it must have fixed the problem. > > By the way, could you please suggest a petsc test case of KSP solver on the Blue Gene? There are many tests coming with PETSc. It would be great if I could just focus on one test. Perhaps you can check src/ksp/ksp/examples/tutorials/ex2.c Satish > > Thanks again. > Shi > > > > > From recrusader at gmail.com Wed Dec 31 10:21:17 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 31 Dec 2008 08:21:17 -0800 Subject: get several submatrices from several matrices and combine submatrices into another matrix Message-ID: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> Hi, PETSc Developers In parallel mode, I have 3 MPIDense matrices, A1, A2, A3. Now, I need to exact submatrices B1, B2, B3 in parallel mode from A1, A2, A3 respectively. I know I should use MatGetSubMatrix(). My problem is in MatGetSubMatrix(Mat mat,IS isrow,IS iscol,PetscInt csize,MatReuse cll,Mat *newmat) , the parameter "isrow" means "rows this processor should obtain". if Bi in some nodes (cpus) of the cluster (Ai have rows in them) don't have rows, is it work? After getting Bi, I will create MPIDense-based C and combine them into C like B1 C= B2 B3 MatGetRow(); MatSetValues() should work, right? Could you give me some comments about these operations? thanks a lot. Happy new year! Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From xy2102 at columbia.edu Wed Dec 31 10:24:30 2008 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Wed, 31 Dec 2008 11:24:30 -0500 Subject: PetscFOpen with mode "a+" In-Reply-To: References: <535919.87151.qm@web36201.mail.mud.yahoo.com> Message-ID: <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> Hi,All, I tried to use PetscFOpen to write the numerical result to a file, and I would like to use the mode "a+" to add my result to the existing content in my file. However, whenever the new result comes out, it will overwrite the existing content in the file. Is there any problem here? Thanks a lot! Rebecca From knepley at gmail.com Wed Dec 31 10:54:32 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 31 Dec 2008 10:54:32 -0600 Subject: PetscFOpen with mode "a+" In-Reply-To: <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> Message-ID: You use a ViewerCreate() and then http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerFileSetMode.html to set append, and then FileSetName() to open the file. Matt On Wed, Dec 31, 2008 at 10:24 AM, (Rebecca) Xuefei YUAN wrote: > Hi,All, > > I tried to use PetscFOpen to write the numerical result to a file, and I > would like to use the mode "a+" to add my result to the existing content in > my file. However, whenever the new result comes out, it will overwrite the > existing content in the file. Is there any problem here? > > Thanks a lot! > > Rebecca > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Dec 31 10:56:01 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 31 Dec 2008 10:56:01 -0600 Subject: get several submatrices from several matrices and combine submatrices into another matrix In-Reply-To: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> References: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> Message-ID: Yes, you can have 0 length IS arguments. Matt On Wed, Dec 31, 2008 at 10:21 AM, Yujie wrote: > Hi, PETSc Developers > > In parallel mode, I have 3 MPIDense matrices, A1, A2, A3. Now, I need to > exact submatrices B1, B2, B3 in parallel mode from A1, A2, A3 respectively. > I know I should use MatGetSubMatrix(). My problem is in > > MatGetSubMatrix(Mat mat,IS isrow,IS iscol,PetscInt csize,MatReuse cll,Mat *newmat) > > , the parameter "isrow" means "rows this processor should obtain". if Bi in > some nodes (cpus) of the cluster (Ai have rows in them) don't have rows, is > it work? > > After getting Bi, I will create MPIDense-based C and combine them into C > like > > B1 > C= B2 > B3 > MatGetRow(); > MatSetValues() should work, right? > > Could you give me some comments about these operations? thanks a lot. Happy > new year! > > Regards, > Yujie > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Wed Dec 31 11:25:26 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 31 Dec 2008 09:25:26 -0800 Subject: get several submatrices from several matrices and combine submatrices into another matrix In-Reply-To: References: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> Message-ID: <7ff0ee010812310925u20872680i62d85a8cbf8139f0@mail.gmail.com> Dear Matthew: thank you very much. Regarding my problem about MatAXPY() for MPIDense, I have checked SEQDense has this function. To my knowledge, in each node, MPIDense's format is SEQDense, right? if it is, I should ba able to write a MPIDense-based function using SEQDense? thanks a lot. Regards, Yujie On Wed, Dec 31, 2008 at 8:56 AM, Matthew Knepley wrote: > Yes, you can have 0 length IS arguments. > > Matt > > > On Wed, Dec 31, 2008 at 10:21 AM, Yujie wrote: > >> Hi, PETSc Developers >> >> In parallel mode, I have 3 MPIDense matrices, A1, A2, A3. Now, I need to >> exact submatrices B1, B2, B3 in parallel mode from A1, A2, A3 respectively. >> I know I should use MatGetSubMatrix(). My problem is in >> >> MatGetSubMatrix(Mat mat,IS isrow,IS iscol,PetscInt csize,MatReuse cll,Mat *newmat) >> >> , the parameter "isrow" means "rows this processor should obtain". if Bi >> in some nodes (cpus) of the cluster (Ai have rows in them) don't have rows, >> is it work? >> >> After getting Bi, I will create MPIDense-based C and combine them into C >> like >> >> B1 >> C= B2 >> B3 >> MatGetRow(); >> MatSetValues() should work, right? >> >> Could you give me some comments about these operations? thanks a lot. >> Happy new year! >> >> Regards, >> Yujie >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Dec 31 11:29:16 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 31 Dec 2008 11:29:16 -0600 Subject: get several submatrices from several matrices and combine submatrices into another matrix In-Reply-To: <7ff0ee010812310925u20872680i62d85a8cbf8139f0@mail.gmail.com> References: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> <7ff0ee010812310925u20872680i62d85a8cbf8139f0@mail.gmail.com> Message-ID: Yes. Matt On Wed, Dec 31, 2008 at 11:25 AM, Yujie wrote: > Dear Matthew: > > thank you very much. Regarding my problem about MatAXPY() for MPIDense, I > have checked SEQDense has this function. To my knowledge, in each node, > MPIDense's format is SEQDense, right? if it is, I should ba able to write a > MPIDense-based function using SEQDense? thanks a lot. > > Regards, > Yujie > > > On Wed, Dec 31, 2008 at 8:56 AM, Matthew Knepley wrote: > >> Yes, you can have 0 length IS arguments. >> >> Matt >> >> >> On Wed, Dec 31, 2008 at 10:21 AM, Yujie wrote: >> >>> Hi, PETSc Developers >>> >>> In parallel mode, I have 3 MPIDense matrices, A1, A2, A3. Now, I need to >>> exact submatrices B1, B2, B3 in parallel mode from A1, A2, A3 respectively. >>> I know I should use MatGetSubMatrix(). My problem is in >>> >>> MatGetSubMatrix(Mat mat,IS isrow,IS iscol,PetscInt csize,MatReuse cll,Mat *newmat) >>> >>> , the parameter "isrow" means "rows this processor should obtain". if Bi >>> in some nodes (cpus) of the cluster (Ai have rows in them) don't have rows, >>> is it work? >>> >>> After getting Bi, I will create MPIDense-based C and combine them into C >>> like >>> >>> B1 >>> C= B2 >>> B3 >>> MatGetRow(); >>> MatSetValues() should work, right? >>> >>> Could you give me some comments about these operations? thanks a lot. >>> Happy new year! >>> >>> Regards, >>> Yujie >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Wed Dec 31 12:24:36 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 31 Dec 2008 10:24:36 -0800 Subject: get several submatrices from several matrices and combine submatrices into another matrix In-Reply-To: References: <7ff0ee010812310821v8f3c9feyf9597ca92c011506@mail.gmail.com> <7ff0ee010812310925u20872680i62d85a8cbf8139f0@mail.gmail.com> Message-ID: <7ff0ee010812311024m2ce1e288j810ab34d9d36b1d8@mail.gmail.com> thanks :) On Wed, Dec 31, 2008 at 9:29 AM, Matthew Knepley wrote: > Yes. > > Matt > > On Wed, Dec 31, 2008 at 11:25 AM, Yujie wrote: > >> Dear Matthew: >> >> thank you very much. Regarding my problem about MatAXPY() for MPIDense, I >> have checked SEQDense has this function. To my knowledge, in each node, >> MPIDense's format is SEQDense, right? if it is, I should ba able to write a >> MPIDense-based function using SEQDense? thanks a lot. >> >> Regards, >> Yujie >> >> On Wed, Dec 31, 2008 at 8:56 AM, Matthew Knepley wrote: >> >>> Yes, you can have 0 length IS arguments. >>> >>> Matt >>> >>> On Wed, Dec 31, 2008 at 10:21 AM, Yujie wrote: >>> >>>> Hi, PETSc Developers >>>> >>>> In parallel mode, I have 3 MPIDense matrices, A1, A2, A3. Now, I need to >>>> exact submatrices B1, B2, B3 in parallel mode from A1, A2, A3 respectively. >>>> I know I should use MatGetSubMatrix(). My problem is in >>>> >>>> MatGetSubMatrix(Mat mat,IS isrow,IS iscol,PetscInt csize,MatReuse cll,Mat *newmat) >>>> >>>> , the parameter "isrow" means "rows this processor should obtain". if Bi >>>> in some nodes (cpus) of the cluster (Ai have rows in them) don't have rows, >>>> is it work? >>>> >>>> After getting Bi, I will create MPIDense-based C and combine them into C >>>> like >>>> >>>> B1 >>>> C= B2 >>>> B3 >>>> MatGetRow(); >>>> MatSetValues() should work, right? >>>> >>>> Could you give me some comments about these operations? thanks a lot. >>>> Happy new year! >>>> >>>> Regards, >>>> Yujie >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xy2102 at columbia.edu Wed Dec 31 14:30:36 2008 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Wed, 31 Dec 2008 15:30:36 -0500 Subject: PetscFOpen with mode "a+" In-Reply-To: References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> Message-ID: <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> I tried PetscViewerFileSetMode and use the "FILE_MODE_APPEND", but it seems that the file has been overwritten,too. I was not able to find any examples in tutorials, is there one using it? Thanks a lot! Quoting Matthew Knepley : > You use a ViewerCreate() and then > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerFileSetMode.html > > to set append, and then FileSetName() to open the file. > > Matt > > On Wed, Dec 31, 2008 at 10:24 AM, (Rebecca) Xuefei YUAN > wrote: > >> Hi,All, >> >> I tried to use PetscFOpen to write the numerical result to a file, and I >> would like to use the mode "a+" to add my result to the existing content in >> my file. However, whenever the new result comes out, it will overwrite the >> existing content in the file. Is there any problem here? >> >> Thanks a lot! >> >> Rebecca >> >> > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From bsmith at mcs.anl.gov Wed Dec 31 16:32:17 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 31 Dec 2008 16:32:17 -0600 Subject: PetscFOpen with mode "a+" In-Reply-To: <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> Message-ID: Here is the relevant code PetscErrorCode PETSC_DLLEXPORT PetscFOpen(MPI_Comm comm,const char name[],const char mode[],FILE **fp) { PetscErrorCode ierr; PetscMPIInt rank; FILE *fd; char fname[PETSC_MAX_PATH_LEN],tname[PETSC_MAX_PATH_LEN]; PetscFunctionBegin; ierr = MPI_Comm_rank(comm,&rank);CHKERRQ(ierr); if (!rank) { PetscTruth isstdout,isstderr; ierr = PetscStrcmp(name,"stdout",&isstdout);CHKERRQ(ierr); ierr = PetscStrcmp(name,"stderr",&isstderr);CHKERRQ(ierr); if (isstdout || !name) { fd = PETSC_STDOUT; } else if (isstderr) { fd = PETSC_STDERR; } else { ierr = PetscStrreplace (PETSC_COMM_SELF,name,tname,PETSC_MAX_PATH_LEN);CHKERRQ(ierr); ierr = PetscFixFilename(tname,fname);CHKERRQ(ierr); ierr = PetscInfo1(0,"Opening file %s\n",fname);CHKERRQ(ierr); fd = fopen(fname,mode); if (!fd) SETERRQ1(PETSC_ERR_FILE_OPEN,"Unable to open file %s \n",fname); It appears the mode that you pass in is passed into fopen() unchanged; thus if you pass in an 'a' or 'a+' that should be passed into fopen() you can confirm this in the debugger. Start up your program (with one process) with the option -start_in_debugger once it is in the debugger do "break fopen" then "cont" (without the quotes). When it gets to the relevant fopen() check the value of mode. Use the debug version of PETSc (that is do not config/configure.py with the option --with-debugging=0) Please report back to petsc-maint at mcs.anl.gov Barry On Dec 31, 2008, at 2:30 PM, (Rebecca) Xuefei YUAN wrote: > > I tried PetscViewerFileSetMode and use the "FILE_MODE_APPEND", but > it seems that the file has been overwritten,too. > > I was not able to find any examples in tutorials, is there one using > it? > > Thanks a lot! > > > Quoting Matthew Knepley : > >> You use a ViewerCreate() and then >> >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerFileSetMode.html >> >> to set append, and then FileSetName() to open the file. >> >> Matt >> >> On Wed, Dec 31, 2008 at 10:24 AM, (Rebecca) Xuefei YUAN >> wrote: >> >>> Hi,All, >>> >>> I tried to use PetscFOpen to write the numerical result to a file, >>> and I >>> would like to use the mode "a+" to add my result to the existing >>> content in >>> my file. However, whenever the new result comes out, it will >>> overwrite the >>> existing content in the file. Is there any problem here? >>> >>> Thanks a lot! >>> >>> Rebecca >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments >> is infinitely more interesting than any results to which their >> experiments >> lead. >> -- Norbert Wiener >> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > From knepley at gmail.com Wed Dec 31 16:38:45 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 31 Dec 2008 16:38:45 -0600 Subject: PetscFOpen with mode "a+" In-Reply-To: <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> Message-ID: On Wed, Dec 31, 2008 at 2:30 PM, (Rebecca) Xuefei YUAN wrote: > > I tried PetscViewerFileSetMode and use the "FILE_MODE_APPEND", but it seems > that the file has been overwritten,too. > > I was not able to find any examples in tutorials, is there one using it? Here is an example that appends to a file. It works for me. Matt > > Thanks a lot! > > > > Quoting Matthew Knepley : > > You use a ViewerCreate() and then >> >> >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerFileSetMode.html >> >> to set append, and then FileSetName() to open the file. >> >> Matt >> >> On Wed, Dec 31, 2008 at 10:24 AM, (Rebecca) Xuefei YUAN < >> xy2102 at columbia.edu >> >>> wrote: >>> >> >> Hi,All, >>> >>> I tried to use PetscFOpen to write the numerical result to a file, and I >>> would like to use the mode "a+" to add my result to the existing content >>> in >>> my file. However, whenever the new result comes out, it will overwrite >>> the >>> existing content in the file. Is there any problem here? >>> >>> Thanks a lot! >>> >>> Rebecca >>> >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> >> > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex1.c Type: text/x-csrc Size: 828 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Dec 31 16:47:15 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 31 Dec 2008 16:47:15 -0600 Subject: MatAXPY() for MPIDense In-Reply-To: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> References: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> Message-ID: <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> I have added MatAXPY_MPIDense() to petsc-dev; it will also be in the next patch for petsc-3.0.0 Barry On Dec 30, 2008, at 11:21 PM, Yujie wrote: > Hi, PETSc Developers > > I have checked the codes in PETSc3.0.0. I cannot find there is > MatAXPY() for MPIDense. Now, MatAXPY() supports MPIDense format? if > not, how to do it? thanks a lot. > > Regards, > Yujie From recrusader at gmail.com Wed Dec 31 17:07:18 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 31 Dec 2008 15:07:18 -0800 Subject: MatAXPY() for MPIDense In-Reply-To: <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> References: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> Message-ID: <7ff0ee010812311507p27d086fdsbd19c33db75355fb@mail.gmail.com> Dear Barry: Thank you very much. I am considering to realize it:). Regards, Yujie On Wed, Dec 31, 2008 at 2:47 PM, Barry Smith wrote: > > I have added MatAXPY_MPIDense() to petsc-dev; it will also be in the next > patch for petsc-3.0.0 > > Barry > > On Dec 30, 2008, at 11:21 PM, Yujie wrote: > > Hi, PETSc Developers >> >> I have checked the codes in PETSc3.0.0. I cannot find there is MatAXPY() >> for MPIDense. Now, MatAXPY() supports MPIDense format? if not, how to do it? >> thanks a lot. >> >> Regards, >> Yujie >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Wed Dec 31 17:11:46 2008 From: recrusader at gmail.com (Yujie) Date: Wed, 31 Dec 2008 15:11:46 -0800 Subject: MatAXPY() for MPIDense In-Reply-To: <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> References: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> Message-ID: <7ff0ee010812311511w4b20cc12rb5d09dc699cf6e5f@mail.gmail.com> Dear Barry: I have checked petsc-dev ( http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/mat/impls/dense/mpi/mpidense.c.html). However, I can't find MatAXPY_MPIDense(). did you upload it? thanks. Regards, Yujie On Wed, Dec 31, 2008 at 2:47 PM, Barry Smith wrote: > > I have added MatAXPY_MPIDense() to petsc-dev; it will also be in the next > patch for petsc-3.0.0 > > Barry > > On Dec 30, 2008, at 11:21 PM, Yujie wrote: > > Hi, PETSc Developers >> >> I have checked the codes in PETSc3.0.0. I cannot find there is MatAXPY() >> for MPIDense. Now, MatAXPY() supports MPIDense format? if not, how to do it? >> thanks a lot. >> >> Regards, >> Yujie >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Dec 31 17:22:27 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 31 Dec 2008 17:22:27 -0600 Subject: MatAXPY() for MPIDense In-Reply-To: <7ff0ee010812311511w4b20cc12rb5d09dc699cf6e5f@mail.gmail.com> References: <7ff0ee010812302121g575dde5eqab5d8f1fe2f1e17d@mail.gmail.com> <619D73C0-709D-4A82-85F7-1EAADD013759@mcs.anl.gov> <7ff0ee010812311511w4b20cc12rb5d09dc699cf6e5f@mail.gmail.com> Message-ID: The html files, manual pages for petsc-dev are only generated once a day at roughly 2 am Chicago time. The new source code is in petsc-dev if you do an hg pull; but it won't be in the generated html until tommorow. Barry On Dec 31, 2008, at 5:11 PM, Yujie wrote: > Dear Barry: > > I have checked petsc-dev (http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/mat/impls/dense/mpi/mpidense.c.html > ). However, I can't find MatAXPY_MPIDense(). did you upload it? > thanks. > > Regards, > > Yujie > > > On Wed, Dec 31, 2008 at 2:47 PM, Barry Smith > wrote: > > I have added MatAXPY_MPIDense() to petsc-dev; it will also be in > the next patch for petsc-3.0.0 > > Barry > > > On Dec 30, 2008, at 11:21 PM, Yujie wrote: > > Hi, PETSc Developers > > I have checked the codes in PETSc3.0.0. I cannot find there is > MatAXPY() for MPIDense. Now, MatAXPY() supports MPIDense format? if > not, how to do it? thanks a lot. > > Regards, > Yujie > > From xy2102 at columbia.edu Wed Dec 31 18:09:06 2008 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Wed, 31 Dec 2008 19:09:06 -0500 Subject: PetscFOpen with mode "a+" In-Reply-To: References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> Message-ID: <20081231190906.g6ph2cnqko4ckkgg@cubmail.cc.columbia.edu> Hi,Matt, Thanks very much and it works in my code then. Happy New Year! Rebecca Quoting Matthew Knepley : > On Wed, Dec 31, 2008 at 2:30 PM, (Rebecca) Xuefei YUAN > wrote: > >> >> I tried PetscViewerFileSetMode and use the "FILE_MODE_APPEND", but it seems >> that the file has been overwritten,too. >> >> I was not able to find any examples in tutorials, is there one using it? > > > Here is an example that appends to a file. It works for me. > > Matt > > >> >> Thanks a lot! >> >> >> >> Quoting Matthew Knepley : >> >> You use a ViewerCreate() and then >>> >>> >>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerFileSetMode.html >>> >>> to set append, and then FileSetName() to open the file. >>> >>> Matt >>> >>> On Wed, Dec 31, 2008 at 10:24 AM, (Rebecca) Xuefei YUAN < >>> xy2102 at columbia.edu >>> >>>> wrote: >>>> >>> >>> Hi,All, >>>> >>>> I tried to use PetscFOpen to write the numerical result to a file, and I >>>> would like to use the mode "a+" to add my result to the existing content >>>> in >>>> my file. However, whenever the new result comes out, it will overwrite >>>> the >>>> existing content in the file. Is there any problem here? >>>> >>>> Thanks a lot! >>>> >>>> Rebecca >>>> >>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments >>> is infinitely more interesting than any results to which their experiments >>> lead. >>> -- Norbert Wiener >>> >>> >> >> >> -- >> (Rebecca) Xuefei YUAN >> Department of Applied Physics and Applied Mathematics >> Columbia University >> Tel:917-399-8032 >> www.columbia.edu/~xy2102 >> >> > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From xy2102 at columbia.edu Wed Dec 31 21:14:47 2008 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Wed, 31 Dec 2008 22:14:47 -0500 Subject: Math function "ln" in Petsc In-Reply-To: <20081231190906.g6ph2cnqko4ckkgg@cubmail.cc.columbia.edu> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> <20081231190906.g6ph2cnqko4ckkgg@cubmail.cc.columbia.edu> Message-ID: <20081231221447.azw0svwiisgcks8g@cubmail.cc.columbia.edu> I know that there is a PetscExpScalar, but it seems that there is no such a function like PetscLnScalar. Thanks! From bsmith at mcs.anl.gov Wed Dec 31 21:17:57 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 31 Dec 2008 21:17:57 -0600 Subject: Math function "ln" in Petsc In-Reply-To: <20081231221447.azw0svwiisgcks8g@cubmail.cc.columbia.edu> References: <535919.87151.qm@web36201.mail.mud.yahoo.com> <20081231112430.mbh33xdc04s0o0w0@cubmail.cc.columbia.edu> <20081231153036.ex23wo4l280o8skg@cubmail.cc.columbia.edu> <20081231190906.g6ph2cnqko4ckkgg@cubmail.cc.columbia.edu> <20081231221447.azw0svwiisgcks8g@cubmail.cc.columbia.edu> Message-ID: It is called PetscLogScalar() Barry On Dec 31, 2008, at 9:14 PM, (Rebecca) Xuefei YUAN wrote: > I know that there is a PetscExpScalar, but it seems that there is no > such a function like PetscLnScalar. > > Thanks! >