[petsc-users] Skipping data when reading from binary file

[Matthew Knepley]

On Mon, Oct 4, 2021 at 11:27 AM Peder Jørgensgaard Olesen via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> In theory it should be relatively simple to write up a routine to skip a
> given number of objects. I'm not getting PetscBinarySeek() to work even on
> a header-less array of integers, however. I suppose that
>
> > PetscBinarySeek(fd, PETSC_BINARY_INT_SIZE*array_size,
> PETSC_BINARY_SEEK_CUR, NULL);
>
> The VecView code is here:

  https://gitlab.com/petsc/petsc/-/blob/main/src/vec/vec/utils/vecio.c#L32

you can see that it writes 2 integers and then the scalar data. So, first
you skip the 2 integers, and then you skip the array_size scalars (so you
want PETSC_BINARY_SCALAR_SIZE).

  Thanks,

     Matt

> ought to do the trick, but that does not seem to be the case.
>
>
> Am I somehow using this routine incorrectly?
>
>
> Peder
> ------------------------------
> *Fra:* Barry Smith <bsmith at petsc.dev>
> *Sendt:* 4. oktober 2021 16:38:10
> *Til:* Peder Jørgensgaard Olesen
> *Cc:* petsc-users at mcs.anl.gov
> *Emne:* Re: [petsc-users] Skipping data when reading from binary file
>
>
>   To minimize code changes you could add a PETSc viewer format that caused
> skipping reading in an object. Then each object load would need a skip-read
> method that mimicked the reading but actually just skipped over the parts
> of the data of the object (using the correct sizes). For vectors this is
> trivial since you just skip the single known array. For sparse matrices it
> is not difficult but you will need to read in the number of nonzeros so you
> know how much to skip etc.
>
>   Barry
>
>
> On Oct 4, 2021, at 9:58 AM, Peder Jørgensgaard Olesen via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
> Thank you for your quick reply.
>
>
> I've had to change away from HDF5 to Binary format at an earlier stage in
> my work due to the former not working well with what I needed, so I would
> prefer to stick with the binary format.
>
> I had a quick view at the code for some of the Viewer-routines, but I'm
> not well versed in gleaning information about header sizes from that. Hints
> about what I'm looking for there would be appreciated.
>
>
> Best regards
>
> Peder
> ------------------------------
> *Fra:* Matthew Knepley <knepley at gmail.com>
> *Sendt:* 4. oktober 2021 15:37:08
> *Til:* Peder Jørgensgaard Olesen
> *Cc:* petsc-users at mcs.anl.gov
> *Emne:* Re: [petsc-users] Skipping data when reading from binary file
>
> On Mon, Oct 4, 2021 at 9:26 AM Peder Jørgensgaard Olesen via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Hello
>>
>> I have a binary file in which a mix of different objects is stored (Vecs,
>> Mats, PetscInts). I can read each element just fine using VecLoad, MatLoad,
>> and PetscIntView, provided they're read in the order in which they were put
>> in the binary. What I would like to do is to skip the reading of any
>> unneeded element, instead proceeding directly to the next one. I tried
>> using PetscBinarySeek() for this, as shown in the attached code. This
>> produces segmentation faults, suggesting that the file pointer isn't going
>> where I want it to.
>>
> There is header information you also have to skip for each object.
>
> We can go over the sizes  for that (it is best just to look at the code),
> but that is fragile. A more robust way to achieve this random
> access is to use HDF5 and name the objects.
>
>   Thanks,
>
>      Matt
>
>> Any suggestions as to what I'm doing wrong here?
>>
>>
>> Best regards,
>>
>> Peder
>>
>>
>>
>> Peder Jørgensgaard Olesen
>> PhD student
>> Department of Mechanical Engineering
>>
>> pjool at mek.dtu.dk
>> Koppels Allé
>> Building 403, room 105
>> 2800 Kgs. Lyngby
>> www.dtu.dk/english
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211004/ccb5a959/attachment-0001.html>