[Mochi-devel] running mochi examples

[Latham, Robert J.]

On Tue, 2019-11-12 at 13:42 +0000, Carns, Philip H. via mochi-devel
wrote:
> Hi Abid,
> 
> It still sounds like your libfabric build does not include the verbs
> and rxm providers.  If you want to confirm this, then load the
> libfabric package and (on a compute node, within a job script) run
> "fi_info" and save the output.

OH! Abid, are you running from the login nodes?  that's not going to
work.

A more succint way to get the information Phil requested: 'fi_info -l'
will list the providers it knows:

Here's the output from the summit login node:

% fi_info -l
ofi_rxm:
    version: 1.0
shm:
    version: 1.1
ofi_perf_hook:
    version: 1.0
ofi_noop_hook:
    version: 1.0
ofi_mrail:
    version: 1.0

Summit login node doesn't have any infiband cards, so it does not
advertise the infiniband (verbs) provider.

But how about from a compute node?  Here's what I see:

% bsub -W 0:10 -nnodes 2 -P CSC332 -Is /bin/bash
Job <732804> is submitted to default queue <batch>.
<<Waiting for dispatch ...>>
<<Starting on batch1>>
robl at batch1:$ fi_info -l
verbs:
    version: 1.0
ofi_rxm:
    version: 1.0
shm:
    version: 1.1
ofi_perf_hook:
    version: 1.0
ofi_noop_hook:
    version: 1.0
ofi_mrail:
    version: 1.0

The rest of Phil's message is spot-on -- if you don't see 'verbs' from
the compute node, you'll have to rebuild libfabric.


==rob