<div dir="ltr"><div><div><div><div>HI,<br><br></div>I extensively tested and used the BDDC/FETI-DP code on two different linux clusters and on BlueGene/Q and it worked fine for me.<br></div><div>The example (src/ksp/ksp/examples/tutorials/ex59.c)
needs to be adapted in its LAPACK calls to get it working on
BlueGene/Q; namely, LAPACKsterf_ should be added to petscblaslapack
header files (Jed, can you do this?). Also, I remember that some time
ago Jed experienced some runtime problems with this example. Can you
test it now? It should work.
</div><div><br>Yes, the complex case is not yet supported; also, there is some additional work to do to get it working with 64 bit indices.<br><br></div>Regarding
PARMETIS dependency: now the code makes explicit calls to METIS
functions and uses METIS types. Using MatPartitioning interface is on my
todo list to remove such an explicit dependency. <br>
</div><div><br></div>In the dev repo, the BDDC/FETI-DP code is all
contained in bddc.c; I have an improved version on my laptop and on
BlueGene/Q with source codes split in different files (for BDDC, FETI-DP
and the BDDCGraph structure which is used to analyze the interface
between subdomains) and with the 64 bit indices issue solved, but I
cannot push it because it contains an advanced scaling operator (see
e.g. p. 9 bottom at <a href="http://cs.nyu.edu/csweb/Research/TechReports/TR2013-951/TR2013-951.pdf" target="_blank">http://cs.nyu.edu/csweb/Research/TechReports/TR2013-951/TR2013-951.pdf</a>)
which I'm extensively using in my research. Also, the new code needs
the Colpack library to compute distance 1 coloring of graphs (there's a
preliminary version of interface to Colpack at <a href="https://bitbucket.org/petsc/petsc-dev/issue/25/add-support-for-1-coloring-instead-of-only" target="_blank">https://bitbucket.org/petsc/petsc-dev/issue/25/add-support-for-1-coloring-instead-of-only</a>)<br>
<br></div>In summary: I'm fine with you adding the guards for
PARMETIS, reals and 64 bit indices. I'm the only developer of BDDC code
in PETSc, and now I don't have time to push changes since I mostly work
on it in my spare time (and now I don't have spare time.. Maybe by the
end of March)<div class=""><div id=":1c" class="" tabindex="0"><img class="" src="https://mail.google.com/mail/u/0/images/cleardot.gif"></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/3/5 Stefano Zampini <span dir="ltr"><<a href="mailto:s.zampini@cineca.it" target="_blank">s.zampini@cineca.it</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>HI,<br><br></div>I extensively tested and used the BDDC/FETI-DP code on two different linux clusters and on BlueGene/Q and it worked fine for me.<br>
</div><div>The example (src/ksp/ksp/examples/tutorials/ex59.c) needs to be adapted in its LAPACK calls to get it working on BlueGene/Q; namely, LAPACKsterf_ should be added to petscblaslapack header files (Jed, can you do this?). Also, I remember that some time ago Jed experienced some runtime problems with this example. Can you test it now? It should work.
</div><div><br>Yes, the complex case is not yet supported; also, there is some additional work to do to get it working with 64 bit indices.<br><br></div>Regarding PARMETIS dependency: now the code makes explicit calls to METIS functions and uses METIS types. Using MatPartitioning interface is on my todo list to remove such an explicit dependency. <br>
</div><div><br></div>In the dev repo, the BDDC/FETI-DP code is all contained in bddc.c; I have an improved version on my laptop and on BlueGene/Q with source codes split in different files (for BDDC, FETI-DP and the BDDCGraph structure which is used to analyze the interface between subdomains) and with the 64 bit indices issue solved, but I cannot push it because it contains an advanced scaling operator (see e.g. p. 9 bottom at <a href="http://cs.nyu.edu/csweb/Research/TechReports/TR2013-951/TR2013-951.pdf" target="_blank">http://cs.nyu.edu/csweb/Research/TechReports/TR2013-951/TR2013-951.pdf</a>) which I'm extensively using in my research. Also, the new code needs the Colpack library to compute distance 1 coloring of graphs (there's a preliminary version of interface to Colpack at <a href="https://bitbucket.org/petsc/petsc-dev/issue/25/add-support-for-1-coloring-instead-of-only" target="_blank">https://bitbucket.org/petsc/petsc-dev/issue/25/add-support-for-1-coloring-instead-of-only</a>)<br>
<br></div><div>In summary: I'm fine with you adding the guards for PARMETIS, reals and 64 bit indices. I'm the only developer of BDDC code in PETSc, and now I don't have time to push changes since I mostly work on it in my spare time (and now I don't have spare time.. Maybe by the end of March)<span class="HOEnZb"><font color="#888888"><br>
<br></font></span></div><span class="HOEnZb"><font color="#888888">Stefano<br></font></span></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">2013/3/5 Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote"><div>On Mon, Mar 4, 2013 at 6:25 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div> The makefile should be marked as not supporting complex<br>
<br>
> It was put into the repository mostly because we don't have a good system for supporting forks.\<br>
<br>
I thought bitbucket now gives a system for handling forks?<br></div></blockquote><div><br></div></div><div>Yeah, this was before we used bitbucket, but we still haven't established a workflow conducive to supporting downstream forks.</div>
<div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
> It was off by default because the code was unstable and untested. There is an example now (src/ksp/ksp/examples/tutorials/ex59.c) so I think we can remove the --with-pcbddc option. Stefano, does this sound right to you?<br>
<br>
</div> Or it could be a good test example for a PETSc plugin repository?</div></blockquote></div></div><br>Yes, either way. Stefano, if you feel it is reasonably stable, can you remove the --with-pcbddc guard and put in the necessary restrictions (#requirespackage 'PETSC_HAVE_PARMETIS', #requiresscalar real)?</div>
</div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div><br clear="all"><br>-- <br>Stefano