[petsc-dev] Question about Binary-IO in READ mode with POSIX APIs

[Lisandro Dalcin]

On Mon, 16 Mar 2020 at 16:35, Jed Brown <jed at jedbrown.org> wrote:

> Lisandro Dalcin <dalcinl at gmail.com> writes:
>
> > Currently, binary viewers using POSIX file descriptors with READ mode
> open
> > the file in ALL processes in the communicator. For WRITE mode, only
> process
> > zero opens the file.
> >
> > The current PetscViewerBynaryXXX APIs make it really unnecessary to open
> > the file in all processes for READ. I would like to get rid of that and
> > always open on rank 0 for both READ or WROTE.
>
> I think we should use MPI-IO by default, and advise that people use it
> whenever they can.


OK, let me look again to the details, I think a few minor things should be
done before using MPI-IO as a default (like a proper subviewer
implementation)


>   I'm not sure of this suggested change, in that a
> "bad for MPI-IO" workload (like each rank randomly seeking around a big
> file) might not be better with rank 0 acting as a service rank.
>

Please note my main question is unrelated to MPI-IO. It is about the
original POSIX-based implementation of binary viewers. For mode READ, all
processes open the file (with the open() system call), but in the current
implementation, only process zero ever reads the file (unless the user gets
the file descriptor and start issuing low-level PetscBinaryRead() calls).
So I do not see the point of opening the file on all processes (and then
stress metadata servers on parallel filesystem), if we are not going to
ever read from rank != 0. Let's just fix things to open the file at rank==0
only. If users ever need to read in rank != 0, then can very well create
the viewer on COMM_SELF, or whatever.


-- 
Lisandro Dalcin
============
Research Scientist
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20200316/b0884160/attachment.html>