[MPICH] FreeBSD and the ch3:smm channel?

[Darius Buntinas]

Great!  I'm glad it's working.
-d

On Wed, 31 Jan 2007, Steve Kargl wrote:

> On Wed, Jan 31, 2007 at 04:49:59PM -0600, Darius Buntinas wrote:
>>
>> First, what's the execution time of some of the codes you generally run on
>> those machines for the different configurations?
>
> My single processor rip code take roughly 15 minutes of CPU time
> to perform a computation.  The ripmp code does that same computation
> with mpich2 and using ch3:sock reports about 39 seconds when spread
> across the 24 effective cpus.  Wall clock is roughly 41-42 seconds.
> There is very little communication (ie master sends initial data
> to slaves and then sits and waits for a single send from each slave).
> It may be just some start-up latency that I now see with nemesis.
>
>> Just to check message latency and bandwidth you can run netpipe that comes
>> with the MPICH2 distibution:
>>
>> #substitute ${MPICH2_SRCDIR} with the location of the source directory
>> cp ${MPICH2_SRCDIR}/test/mpi/basic/GetOpt.* .
>> cp ${MPICH2_SRCDIR}/test/mpi/basic/netmpi.c .
>>
>> #assuming MPICH2 bin is in your path
>> mpicc Getopt.c netmpi.c -o netmpi
>>
>> #just to see whether its running on the same node or not
>> mpiexec -n 2 hostname
>
> node10:kargl[246] mpiexec -n 2 hostname
> node10.cimu.org
> node10.cimu.org
>
>> mpiexec -n 2 ./netmpi
>
> Some selective output:
>
> 0: node10.cimu.org
> 1: node10.cimu.org
> Latency: 0.000000749
> Sync Time: 0.000002478
> Now starting main loop
>  0:         0 bytes 1048576 times -->    0.00 Mbps in 0.000000754 sec
>  1:         1 bytes 1048576 times -->    8.44 Mbps in 0.000000904 sec
> 10:        20 bytes 163950 times -->  181.48 Mbps in 0.000000841 sec
> 20:        63 bytes 157433 times -->  534.17 Mbps in 0.000000900 sec
> 30:       194 bytes 175328 times -->  1430.46 Mbps in 0.000001035 sec
> 40:       764 bytes 93751 times -->  3780.93 Mbps in 0.000001542 sec
> 50:      2047 bytes 43727 times -->  5420.73 Mbps in 0.000002881 sec
> 60:      6146 bytes 26032 times -->  7221.15 Mbps in 0.000006493 sec
> 70:     24572 bytes 8300 times -->  7731.53 Mbps in 0.000024247 sec
> 80:     65535 bytes 1847 times -->  7298.06 Mbps in 0.000068510 sec
> 90:    196610 bytes  914 times -->  8200.11 Mbps in 0.000182926 sec
> 100:    786428 bytes  256 times -->  6540.57 Mbps in 0.000917347 sec
> 110:   2097151 bytes   51 times -->  6542.41 Mbps in 0.002445581 sec
> 120:   6291458 bytes   22 times -->  6638.42 Mbps in 0.007230639 sec
> 130:  25165820 bytes    6 times -->  6914.70 Mbps in 0.027766923 sec
> 140:  67108863 bytes    3 times -->  6939.06 Mbps in 0.073785186 sec
>
>
>> Try this for the different configurations, and with both processes on the
>> same node and on different nodes.
>
> netmpi may be the app I need for some network.
>
>> It's possible that your codes aren't that sensitive to latency or
>> bandwidth, so you won't see much of a difference.  Also, if certain
>> processes communicate more than others, it would be beneficial, especially
>> with nemesis, to put them on the same node.  By default, mpd will
>> distribute them in round-robin, so process 0 and 1 will be on separate
>> nodes.
>
> My colleague's code is the heavier communicator.  After my last email,
> I received the following from him concerning his code and nemesis.
>
> --------------------------------------------------------------------------
> Steve,
>
> recompiling seems to have worked well! Now it runs much faster on multiple
> cores on the same node!  It seems to be working, Thanks a lot.
>
> F
>
> |
> |Franco,
> |
> |Can you recompile all of your codes to test the shared memory
> |method?  I think you may need to link against the new
> |libraries to get this to work.
> |
> |--
> |Steve
> --------------------------------------------------------------------------
>
> If he's happy, I'm happy.
> Thanks for your help.  If you would like some future testing
> of nemesis in a development version of mpich2, feel free to
> drop me a note.
>
>