Methods to deal with the problem of "Size of I/O request exceeds INT_MAX"

Wei-Keng Liao wkliao at northwestern.edu
Wed Sep 30 22:36:55 CDT 2020


Instead of "--enable-large-req”, try "--enable-large-single-req”
Please note this feature requires the underneath MPI-IO to support
large single requests (each > 2GiB), which I am not aware of any
MPI-IO implementation today supports that. Your best bet is to break
large requests into smaller ones.

PnetCDF may implement this in the future, but we have no immediate
plan to make it happen.

Wei-keng

> On Oct 1, 2020, at 10:14 AM, 刘壮 <liuzhuang at lsec.cc.ac.cn> wrote:
> 
> Hi,
> 
>     I have got a problem when using pnetcdf to output a large variable, and the error message is "Size of I/O request exceeds INT_MAX". I have tried the compiling option "--enable-large-req", but it does not work. Breaking the large variable into several parts and using "nf90mpi_iput_var" and "nf90mpi_wait_all" several times may solve this problem, but the code is inelegant. Is there a better way to deal with this problem?
> 
>     Thank you very much!
> 
> Zhuang
> 



More information about the parallel-netcdf mailing list