[petsc-users] Unable to create >4GB sized HDF5 files on Cray XC30

[Jed Brown]

Juha Jäykkä <juhaj at iki.fi> writes:

> Actually, I didn't ask for it. I only asked for a bug to be fixed. A bug which 
> means > 4 GB Vec cannot be saved into a HDF5 file using PETSc VecView, because 
> chunking *was* introduced, but with insane chunk sizes (i.e. the *global* 
> instead of local Vec size for each chunk).

Ah, right.  I would note that using the local size is also flawed
because we can only have 65k chunks, but we sometimes run jobs with more
than that number of processes.  Maybe we need something like this?

  chunk_size = min(vec_size, max(avg_local_vec_size, vec_size/65k, 10 MiB), 4 GiB)

The combination of the number of chunks (65k) and chunk size (4GiB)
makes for messy constraints.

> But what I really meant by timetable was that when is the next 3.4.x due and 
> when do you need the fix if it is to go to the next release?

I think we're planning to tag 3.4.3 in the next couple weeks.  There
might be a 3.4.4 as well, but I could see going straight to 3.5.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131005/553908e6/attachment.pgp>