[mpich-discuss] reading Ascii files using parallel I/o

[Brock Palen]

Sreenivas,

Note using binary IO tends to be faster as it takes less space for the same amount of data.

You may also want to look at the hdf5/*-parallel modules on Nyx/Flux on campus. I find using hdf5 parallel IO to be simpler and more interoperable (modifiable and readable in matlab/idl with little effort) than invoking MPI-IO directly.

Just my own $0.02 others with more parallel IO experience may have other input.


Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp at umich.edu
(734)936-1985



On May 23, 2011, at 1:00 AM, Rajeev Thakur wrote:

> MPI-IO doesn't support formatted I/O, i.e., the equivalent of fprintf and fscanf. However you can read/write unformatted character buffers.
> 
> Rajeev
> 
> 
>> Date: Sat, 21 May 2011 23:48:57 -0400
>> From: <vvvnew at umich.edu>
>> Subject: [mpich-discuss] reading Ascii files using parallel I/o
>> To: mpich-discuss at mcs.anl.gov
>> Message-ID: <7f8c0ee756294fb644ab193ff0338e1c at umich.edu>
>> Content-Type: text/plain; charset="UTF-8"
>> 
>> 
>> Hi,
>> I want to know if there's a general way to read and write ASCII data
>> using parallel input/output. I know there are only 3 Mpi data formats
>> 'Native', 'Internal','External32', none of which support Ascii data. Since
>> all my input data is in ASCII and i also need the output to be in ASCII to
>> increase interoperability with other software, can someone suggest an
>> efficient way to get around this problem. Right now iam sending all the
>> data in and out through one node and makes the scaling pretty bad for large
>> datasets, so parallel i/o with ascii would make my life a whole lot easier.
>> 
>> Thanks
>> 
>> Sreenivas
> 
> _______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> 
>