[petsc-dev] [Minor issue] Lag during PetscInitialize on Fedora 36 (never seen before with other OS)

[Satish Balay]

"sleep 30; strace ./ex39" shows an extra pause while reading the following file (on my laptop):

>>>>>>>>
balay at p1 /home/balay
$ sleep 10; time cat /sys/bus/pci/devices/0000:00:01.0/config; time cat /sys/bus/pci/devices/0000:00:01.0/config
�00�������
real	0m1.271s
user	0m0.000s
sys	0m0.008s
�00�������
real	0m0.003s
user	0m0.001s
sys	0m0.002s

balay at p1 /home/balay
$ lspci -v -s 0000:00:01.0
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 122, IOMMU group 1
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff [size=4K]
	Memory behind bridge: ed000000-ee0fffff [size=17M]
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff [size=288M]
	Capabilities: <access denied>
	Kernel driver in use: pcieport
<<<<<<<

Maybe should try without hwloc...

Satish


On Wed, 27 Jul 2022, LEDAC Pierre wrote:

> 
> Ok thanks all, so with Satish test, we know it is MPI_Init call from PetscInitialize.
> 
> Probably, as Matt said, something related to gethostbyname() ...
> 
> 
> I will do some other test, contact MPICH support and will report any workaround,
> 
> 
> Thanks again for your reactivity !
> 
> 
> Pierre LEDAC
> Commissariat à l’énergie atomique et aux énergies alternatives
> Centre de SACLAY
> DES/ISAS/DM2S/STMF/LGLS
> Bâtiment 451 – point courrier n°43
> F-91191 Gif-sur-Yvette
> +33 1 69 08 04 03
> +33 6 83 42 05 79
> ________________________________
> De : Satish Balay <balay at mcs.anl.gov>
> Envoyé : mercredi 27 juillet 2022 18:57:16
> À : LEDAC Pierre
> Cc : petsc-dev at mcs.anl.gov
> Objet : Re: [petsc-dev] [Minor issue] Lag during PetscInitialize on Fedora 36 (never seen before with other OS)
> 
> I'm seeing this behavior on a laptop - but not desktop [both F36] (with both mpich and openmpi buids)
> 
> >>>> laptop/mpich <<<<
> balay at p1 /home/balay/petsc/src/ksp/ksp/tests (main =)
> $ sleep 30; time ./ex39; time ./ex39
> 
> real    0m1.709s
> user    0m0.079s
> sys     0m0.042s
> 
> real    0m0.116s
> user    0m0.077s
> sys     0m0.014s
> <<<<
> 
> However --with-mpi=0 is fine
> 
> >>>>
> balay at p1 /home/balay/petsc/src/ksp/ksp/tests (main =)
> $ sleep 30; time ./ex39; time ./ex39
> 
> real    0m0.116s
> user    0m0.108s
> sys     0m0.008s
> 
> real    0m0.081s
> user    0m0.072s
> sys     0m0.008s
> <<<<<
> 
> Here is the desktop run (with mpich)
> 
> >>>
> [balay at pj01 tests]$ sleep 30; time ./ex39; time ./ex39
> 
> real    0m0.065s
> user    0m0.033s
> sys     0m0.019s
> 
> real    0m0.053s
> user    0m0.029s
> sys     0m0.013s
> <<<<
> 
> And --with-mpi=0
> 
> >>>
> [balay at pj01 tests]$ sleep 30; time ./ex39; time ./ex39
> 
> real    0m0.087s
> user    0m0.046s
> sys     0m0.015s
> 
> real    0m0.024s
> user    0m0.018s
> sys     0m0.006s
> <<<
> 
> But don't know why its behaving this way [on the laptop]
> 
> Satish
> 
> On Wed, 27 Jul 2022, LEDAC Pierre wrote:
> 
> > Hello,
> >
> >
> > Recently migrated from Fedora34 to Fedora36, using PETSc I have some lag during the PetscInitialize, which
> >
> > disappeared if I run again immediately the binary. But after few seconds, the lag happens again (see below).
> >
> >
> > I suspected MPICH 4.0.2 on Fedora36, but a small reproducer indicated the issue is not in MPI_Init but really during PetscInitialize
> >
> >
> > Just annoying during testing, not very important, but did someone already see this ?
> >
> >
> > Thanks,
> >
> >
> > portable: /volatile/ledacp/petsc/src/ksp/ksp/tests (main) > time ./ex39
> >
> > real    0m3,593s
> > user    0m0,872s
> > sys    0m0,433s
> > portable: /volatile/ledacp/petsc/src/ksp/ksp/tests (main) > time ./ex39
> >
> > real    0m0,310s
> > user    0m0,884s
> > sys    0m0,023s
> >
> > # Wait ~20 seconds then again:
> >
> > portable: /volatile/ledacp/petsc/src/ksp/ksp/tests (main) > time ./ex39
> >
> > real    0m3,507s
> > user    0m0,897s
> > sys    0m0,025s
> > portable: /volatile/ledacp/petsc/src/ksp/ksp/tests (main) > time ./ex39
> >
> > real    0m0,154s
> > user    0m0,725s
> > sys    0m0,015s
> > portable: /volatile/ledacp/petsc/src/ksp/ksp/tests (main) > time ./ex39
> >
> > real    0m0,176s
> > user    0m0,875s
> > sys    0m0,021s
> >
> >
> >
> > Pierre LEDAC
> > Commissariat à l’énergie atomique et aux énergies alternatives
> > Centre de SACLAY
> > DES/ISAS/DM2S/STMF/LGLS
> > Bâtiment 451 – point courrier n°43
> > F-91191 Gif-sur-Yvette
> > +33 1 69 08 04 03
> > +33 6 83 42 05 79
> >
>