[petsc-users] Accessing Global Vectors

[Andrew Cramer]

On 20 May 2014 12:27, Matthew Knepley <knepley at gmail.com> wrote:

> On Mon, May 19, 2014 at 8:50 PM, Andrew Cramer <andrewdalecramer at gmail.com
> > wrote:
>
>> Hi All,
>>
>> I'm new to PETSc and would like to use it as my linear elasticity solver
>> within a structural optimisation program. Originally I was using GP-GPUs
>> and CUDA for my solver but I would like to shift to using PETSc to leverage
>> it's breadth of trustworthy solvers. We have some SMP servers and a couple
>> compute clusters (one with GPUs, one without). I've been digging through
>> the docs and I'd like some feedback on my plan and perhaps some pointers if
>> at all possible.
>>
>> The plan is to keep the 6000 lines or so of current code and try as much
>> as possible to use PETSc as a 'drop-in'. This would require giving one
>> field (array) of densities and receiving a 3d field (array) of
>> displacements back. Providing the density field would be easy with the
>> usual array construction functions on one node/process but pulling the
>> displacements back to the 'controlling' node would be difficult.
>>
>> I understand that this goes against the ethos of PETSc which is
>> distributed all the way. My code is highly modular with differing objective
>> functions and optimisers (some of which are written by other research
>> groups) that I drop in and pull out. I don't want to throw all that away. I
>> would need to relearn object oriented programming within PETSc (currently I
>> use c++) and rewrite my entire code base. In terms of performance the
>> optimisers typically rely heavily on tight loops of reductions once the
>> solve is completed so I'm not sure that the speed-up would be too great
>> rewriting them as distributed anyway.
>>
>> Sorry for the long winded post but I'm just not sure how to move forward,
>> I'm sick of implementing every solver I want to try in CUDA especially
>> knowing that people have done it better than I can in PETSc. But it's a
>> framework that I don't know how to interface with, all the examples seem to
>> have the solve as the main thing rather than one part of a broader program.
>>
>
> 1) PETSc can do a good job on linear elasticity. GAMG is particularly
> effective, and we have an example:
>
>
> http://www.mcs.anl.gov/petsc/petsc-dev/src/ksp/ksp/examples/tutorials/ex56.c.html
>
> 2) You can use this function to go back and forth from 1 process
>
>
> http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Vec/VecScatterCreateToZero.html
>
> 3) The expense of pushing all that data to nodes can large. You might be
> better off just using GAMG on 1 process, which is how I would start.
>
>     Matt
>
>
>> Andrew Cramer
>> University of Queensland, Australia
>> PhD Candidate
>>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>


Thanks for your help, I was eyeing off ksp/ex29 as it uses DMDA which I
thought would simplify things. I'll take a look at ex56 instead and see
what I can do.

Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140520/9eab4149/attachment.html>