[petsc-users] Rather different matrix product results on multiple processes

[Peder Jørgensgaard Olesen]

Hello,


When computing a matrix product of the type R = D.DT using MatMatTransposeMult() I find I get rather different results depending on the number of processes. In one example using a data set that is small compared to the application I get Frobenius norms |R| = 1.047e3 on a single process, 1.0363e3 on a single HPC node (40 cores), and 9.7307e2 on two nodes.


I have ascertained that the single process result is indeed the correct one (i.e., eigenvectors of R form a proper basis for the columns of D), so naturally I'd love to be able to reproduce this result across different parallel setups. How might I achieve this?


I'm attaching MWE code and the data set used for the example.


Thanks in advance!


Best Regards


Peder Jørgensgaard Olesen

PhD Student, Turbulence Research Lab

Dept. of Mechanical Engineering

Technical University of Denmark

Niels Koppels Allé

Bygning 403, Rum 105

DK-2800 Kgs. Lyngby
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210419/3f09853f/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acorr_mwe.c
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210419/3f09853f/attachment-0001.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sample_data.h5
Type: application/octet-stream
Size: 201840 bytes
Desc: sample_data.h5
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210419/3f09853f/attachment-0001.obj>