[mpich-discuss] cryptic (to me) error

SULLIVAN David (AREVA) David.Sullivan at areva.com
Fri Sep 3 12:40:46 CDT 2010


 Oh, Ok. Well it just hangs so I aborted. Checked each machine with top
and neither are doing anything when it hangs. Attached is the screen
output.

Thanks again.


Dave

-----Original Message-----
From: mpich-discuss-bounces at mcs.anl.gov
[mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Dave Goodell
Sent: Friday, September 03, 2010 12:14 PM
To: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] cryptic (to me) error

Give the full path to your host file: /path/to/node_list

When the test script changes directories to the different test
directories, a relative path isn't correct anymore.

-Dave

On Sep 3, 2010, at 9:48 AM CDT, SULLIVAN David (AREVA) wrote:

> Wow, that made a difference. Not a good one. But a big one. I stopped 
> the test since each process resulted in the attached errors. It looks 
> like it is trying to access files that are not there to parse the 
> HYDRA_HOST_FILE. I checked if there were there and it was just a 
> permissions thing, but that was not the case.
> 
> Dave
> 
> -----Original Message-----
> From: mpich-discuss-bounces at mcs.anl.gov 
> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Dave Goodell
> Sent: Friday, September 03, 2010 10:36 AM
> To: mpich-discuss at mcs.anl.gov
> Subject: Re: [mpich-discuss] cryptic (to me) error
> 
> I was unclear.  I meant to use HYDRA_HOST_FILE when running the MPICH2

> test suite in order to make it run across the same set of machines 
> that your MCNP code runs on.
> 
> -Dave
> 
> On Sep 3, 2010, at 9:25 AM CDT, SULLIVAN David (AREVA) wrote:
> 
>> Same result, mpi doesn't even get to call mcnp5.mpi. It just returns 
>> the
>> 
>> Fatal error in PMPI_Comm_dup: Other MPI error, error stack: error. 
>> 
>> -----Original Message-----
>> From: mpich-discuss-bounces at mcs.anl.gov 
>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Dave Goodell
>> Sent: Friday, September 03, 2010 10:21 AM
>> To: mpich-discuss at mcs.anl.gov
>> Subject: Re: [mpich-discuss] cryptic (to me) error
>> 
>> Try setting HYDRA_HOST_FILE=your_machine_file_here
>> 
>> This will make hydra act as though you passed "-f your 
>> machine_file_here" to every mpiexec.
>> 
>> http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Mana
>> g
>> er
>> #Environment_Settings
>> 
>> -Dave
>> 
>> On Sep 3, 2010, at 5:00 AM CDT, SULLIVAN David (AREVA) wrote:
>> 
>>> I was wondering about that. Is there a configuration file that sets 
>>> up
>> the cluster and defines which node to run on? Would that make the 
>> issue any clearer?
>>> 
>>> Thanks,
>>> 
>>> Dave
>>> 
>>> 
>>> -----Original Message-----
>>> From: mpich-discuss-bounces at mcs.anl.gov on behalf of Rajeev Thakur
>>> Sent: Thu 9/2/2010 10:22 PM
>>> To: mpich-discuss at mcs.anl.gov
>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>> 
>>> There might be some connection issues between the two machines. The
>> MPICH2 test suite that you ran with "make testing" probably ran on a 
>> single machine.
>>> 
>>> On Sep 2, 2010, at 6:27 PM, SULLIVAN David (AREVA) wrote:
>>> 
>>>> The error occurs immediately- I don't think it even starts the
>> executable. It does work on the single machine with 4 processes.
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: mpich-discuss-bounces at mcs.anl.gov on behalf of Rajeev Thakur
>>>> Sent: Thu 9/2/2010 4:34 PM
>>>> To: mpich-discuss at mcs.anl.gov
>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>> 
>>>> Does it run with 2 processes on a single machine?
>>>> 
>>>> 
>>>> On Sep 2, 2010, at 2:38 PM, SULLIVAN David (AREVA) wrote:
>>>> 
>>>>> That fixed the compile. Thanks!
>>>>> 
>>>>> The latest release does not fix the issues I am having though. Cpi

>>>>> works fine, the test suit is certainly improved (see summary.xml
>>>>> output) though when I try to use mcnp it still crashes in the same

>>>>> way (see
>>>>> error.txt)
>>>>> 
>>>>> -----Original Message-----
>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Anthony 
>>>>> Chan
>>>>> Sent: Thursday, September 02, 2010 1:38 PM
>>>>> To: mpich-discuss at mcs.anl.gov
>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>> 
>>>>> 
>>>>> There is a bug in 1.3b1 about the option --enable-fc.  Since 
>>>>> Fortran
>> 
>>>>> 90 is enabled by default, so remove the --enable-fc from your 
>>>>> configure command and try again.  If there is error again, send us

>>>>> the configure output as you seen on your screen (See README) 
>>>>> instead
>> of config.log.
>>>>> 
>>>>> A.Chan
>>>>> 
>>>>> ----- "SULLIVAN David (AREVA)" <David.Sullivan at areva.com> wrote:
>>>>> 
>>>>>> Failure again.
>>>>>> The 1.3 beta version will not compile with Intel 10.1. It bombs 
>>>>>> at
> 
>>>>>> the
>>>>> 
>>>>>> configuration script:
>>>>>> 
>>>>>> checking for Fortran flag needed to allow free-form source... 
>>>>>> unknown
>>>>>> configure: WARNING: Fortran 90 test being disabled because the 
>>>>>> /home/dfs/mpich2-1.3b1/bin/mpif90 compiler does not accept a .f90

>>>>>> extension
>>>>>> configure: error: Fortran does not accept free-form source
>>>>>> configure: error: ./configure failed for test/mpi
>>>>>> 
>>>>>> I have attached to config.log
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Rajeev 
>>>>>> Thakur
>>>>>> Sent: Thursday, September 02, 2010 12:11 PM
>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>> 
>>>>>> Just try relinking with the new library at first.
>>>>>> 
>>>>>> Rajeev
>>>>>> 
>>>>>> On Sep 2, 2010, at 9:32 AM, SULLIVAN David (AREVA) wrote:
>>>>>> 
>>>>>>> I saw that there was a newer beta. I was really hoping to find I
>>>>>> just
>>>>>>> configured something incorrectly. Will this not require me to
>>>>>> re-build
>>>>>> 
>>>>>>> mcnp (the only program I run that uses mpi for parallel) if I 
>>>>>>> change
>>>>>> 
>>>>>>> the mpi version? If so, this is a bit of a hardship, requiring 
>>>>>>> codes
>>>>>> 
>>>>>>> to be revalidated. If not- I will try it in a second.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> Dave
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Dave
>>>>>> Goodell
>>>>>>> Sent: Thursday, September 02, 2010 10:27 AM
>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>> 
>>>>>>> Can you try the latest release (1.3b1) to see if that fixes the 
>>>>>>> problems you are seeing with your application?
>>>>>>> 
>>>>>>> 
>>>>>> http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.p
>>>>>> h
>>>>>> p
>>>>>> ?s=
>>>>>>> do
>>>>>>> wnloads
>>>>>>> 
>>>>>>> -Dave
>>>>>>> 
>>>>>>> On Sep 2, 2010, at 9:15 AM CDT, SULLIVAN David (AREVA) wrote:
>>>>>>> 
>>>>>>>> Another output file, hopefully of use. 
>>>>>>>> 
>>>>>>>> Thanks again
>>>>>>>> 
>>>>>>>> Dave
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of 
>>>>>>>> SULLIVAN
> 
>>>>>>>> David
>>>>>>>> (AREVA)
>>>>>>>> Sent: Thursday, September 02, 2010 8:20 AM
>>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>>> 
>>>>>>>> First my apologies for the delay in continuing this thread.
>>>>>>>> Unfortunately I have not resolved it so if I can indulge the 
>>>>>>>> gurus
>>>>>> 
>>>>>>>> and
>>>>>>> 
>>>>>>>> developers once again...
>>>>>>>> 
>>>>>>>> As suggested by Rajeev I ran the testing suit in the source
>>>>>> directory.
>>>>>>>> The output of errors, which are similar to what I was seeing 
>>>>>>>> when
>> 
>>>>>>>> I
>>>>>> 
>>>>>>>> ran
>>>>>>>> mcnp5 (v. 1.40 and 1.51), is attached. 
>>>>>>>> 
>>>>>>>> Any insights would be greatly appreciated,
>>>>>>>> 
>>>>>>>> Dave
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Rajeev
>>>>>> Thakur
>>>>>>>> Sent: Wednesday, August 04, 2010 3:06 PM
>>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>>> 
>>>>>>>> Then one level above that directory (in the main MPICH2 source 
>>>>>>>> directory), type make testing, which will run through the 
>>>>>>>> entire
>>>>>>>> MPICH2 test suite.
>>>>>>>> 
>>>>>>>> Rajeev
>>>>>>>> 
>>>>>>>> On Aug 4, 2010, at 2:04 PM, SULLIVAN David (AREVA) wrote:
>>>>>>>> 
>>>>>>>>> Oh. That's  embarrassing. Yea. I have those examples. It runs
>>>>>> with
>>>>>>>>> no
>>>>>>>>> problems:
>>>>>>>>> 
>>>>>>>>> [dfs at aramis examples]$ mpiexec -host aramis -n 4 ./cpi Process
>>>>>>>>> 2
>>>>>> of
>>>>>>>>> 4
>>>>>>> 
>>>>>>>>> is on aramis Process 3 of 4 is on aramis Process 0 of 4 is on
>>>>>> aramis
>>>>>> 
>>>>>>>>> Process 1 of 4 is on aramis pi is approximately
>>>>>> 3.1415926544231239,
>>>>>>>>> Error is 0.0000000008333307 wall clock time = 0.000652
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Gus
>>>>>> Correa
>>>>>>>>> Sent: Wednesday, August 04, 2010 1:13 PM
>>>>>>>>> To: Mpich Discuss
>>>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>>>> 
>>>>>>>>> Hi David
>>>>>>>>> 
>>>>>>>>> I think the "examples" dir is not copied to the installation
>>>>>>>> directory.
>>>>>>>>> You may find it where you decompressed the MPICH2 tarball, in 
>>>>>>>>> case
>>>>>> 
>>>>>>>>> you
>>>>>>>> 
>>>>>>>>> installed it from source.
>>>>>>>>> At least, this is what I have here.
>>>>>>>>> 
>>>>>>>>> Gus Correa
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> SULLIVAN David (AREVA) wrote:
>>>>>>>>>> Yea, that always bothered me.  There is no such folder.
>>>>>>>>>> There are :
>>>>>>>>>> bin
>>>>>>>>>> etc
>>>>>>>>>> include
>>>>>>>>>> lib
>>>>>>>>>> sbin
>>>>>>>>>> share
>>>>>>>>>> 
>>>>>>>>>> The  only examples I found were in the  share folder,  where
>>>>>> there
>>>>>>>>> are
>>>>>>>>>> examples for collchk,  graphics and  logging.   
>>>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of 
>>>>>>>>>> Rajeev
> 
>>>>>>>>>> Thakur
>>>>>>>>>> Sent: Wednesday, August 04, 2010 12:45 PM
>>>>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>>>>> 
>>>>>>>>>> Not cpilog. Can you run just cpi from the mpich2/examples
>>>>>> directory.
>>>>>>>>>> 
>>>>>>>>>> Rajeev
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Aug 4, 2010, at 11:37 AM, SULLIVAN David (AREVA) wrote:
>>>>>>>>>> 
>>>>>>>>>>> Rajeev,  Darius,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for your response.
>>>>>>>>>>> cpi yields  the  following-
>>>>>>>>>>> 
>>>>>>>>>>> [dfs at aramis examples_logging]$ mpiexec -host aramis -n 12
>>>>>> ./cpilog
>>>>>> 
>>>>>>>>>>> Process 0 running on aramis Process 2 running on aramis 
>>>>>>>>>>> Process
>>>>>> 3
>>>>>>>>>>> running on aramis Process 1 running on aramis Process 6 
>>>>>>>>>>> running
>>>>>> on
>>>>>> 
>>>>>>>>>>> aramis Process 7 running on aramis Process 8 running on 
>>>>>>>>>>> aramis
>> 
>>>>>>>>>>> Process
>>>>>>>>>> 
>>>>>>>>>>> 4 running on aramis Process 5 running on aramis Process 9
>>>>>> running
>>>>>>>>>>> on
>>>>>>>> 
>>>>>>>>>>> aramis Process 10 running on aramis Process 11 running on 
>>>>>>>>>>> aramis
>>>>>> 
>>>>>>>>>>> pi
>>>>>>> 
>>>>>>>>>>> is
>>>>>>>>>> 
>>>>>>>>>>> approximately 3.1415926535898762, Error is 
>>>>>>>>>>> 0.0000000000000830
>>>>>> wall
>>>>>> 
>>>>>>>>>>> clock time = 0.058131 Writing logfile....
>>>>>>>>>>> Enabling the Default clock synchronization...
>>>>>>>>>>> clog_merger.c:CLOG_Merger_init() -  Could not open file
>>>>>>>>>>> ./cpilog.clog2 for merging!
>>>>>>>>>>> Backtrace of the callstack at rank 0:
>>>>>>>>>>> At [0]: ./cpilog(CLOG_Util_abort+0x92)[0x456326]
>>>>>>>>>>> At [1]: ./cpilog(CLOG_Merger_init+0x11f)[0x45db7c]
>>>>>>>>>>> At [2]: ./cpilog(CLOG_Converge_init+0x8e)[0x45a691]
>>>>>>>>>>> At [3]: ./cpilog(MPE_Finish_log+0xea)[0x4560aa]
>>>>>>>>>>> At [4]: ./cpilog(MPI_Finalize+0x50c)[0x4268af]
>>>>>>>>>>> At [5]: ./cpilog(main+0x428)[0x415963]  At [6]:
>>>>>> /lib64/libc.so.6(__libc_start_main+0xf4)[0x3c1881d994]
>>>>>>>>>>> At [7]: ./cpilog[0x415449]
>>>>>>>>>>> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 
>>>>>>>>>>> APPLICATION TERMINATED WITH THE EXIT STRING: Terminated 
>>>>>>>>>>> (signal
>>>>>> 
>>>>>>>>>>> 15)
>>>>>>>>>>> 
>>>>>>>>>>> So  it looks like it works  with some issues.
>>>>>>>>>>> 
>>>>>>>>>>> When does  it fail? Immediately
>>>>>>>>>>> 
>>>>>>>>>>> Is there  a  bug? Many sucessfully use the aplication 
>>>>>>>>>>> (MCNP5,
> 
>>>>>>>>>>> from
>>>>>>>>>>> LANL) with  mpi,  so  think that  a  bug there is  unlikely.
>>>>>>>>>>> 
>>>>>>>>>>> Core files, unfortunately reveals some ignorance on my part.
>>>>>> Were
>>>>>>>>>>> exactly should I be looking for them?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks again,
>>>>>>>>>>> 
>>>>>>>>>>> Dave
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of 
>>>>>>>>>>> Darius
>> 
>>>>>>>>>>> Buntinas
>>>>>>>>>>> Sent: Wednesday, August 04, 2010 12:19 PM
>>>>>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>>>>>> Subject: Re: [mpich-discuss] cryptic (to me) error
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> This error message says that two processes terminated 
>>>>>>>>>>> because
>>>>>> they
>>>>>> 
>>>>>>>>>>> were unable to communicate with another (or two other)
>> process.
>>>>>>>> It's
>>>>>>>>> 
>>>>>>>>>>> possible that another process died, so the others got errors

>>>>>>>>>>> trying
>>>>>>> 
>>>>>>>>>>> to
>>>>>>>>>> 
>>>>>>>>>>> communicate with them.  It's also possible that there is
>>>>>> something
>>>>>> 
>>>>>>>>>>> preventing some processes from communicating with each
other.
>>>>>>>>>>> 
>>>>>>>>>>> Are you able to run cpi from the examples directory with 12
>>>>>>>>> processes?
>>>>>>>>>>> 
>>>>>>>>>>> At what point in your code does this fail?  Are there any 
>>>>>>>>>>> other
>>>>>> 
>>>>>>>>>>> communication operations before the MPI_Comm_dup?
>>>>>>>>>>> 
>>>>>>>>>>> Enable core files (add "ulimit -c unlimited" to your .bashrc

>>>>>>>>>>> or
>>>>>>>>>>> .tcshrc) then run your app and look for core files.  If 
>>>>>>>>>>> there
> 
>>>>>>>>>>> is
>>>>>> a
>>>>>> 
>>>>>>>>>>> bug
>>>>>>>>>> 
>>>>>>>>>>> in your application that causes a process to die this might 
>>>>>>>>>>> tell
>>>>>> 
>>>>>>>>>>> you
>>>>>>>> 
>>>>>>>>>>> which one and why.
>>>>>>>>>>> 
>>>>>>>>>>> Let us know how this goes.
>>>>>>>>>>> 
>>>>>>>>>>> -d
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Aug 4, 2010, at 11:03 AM, SULLIVAN David (AREVA) wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Since I have  had  no responses, is  there any other
>>>>>> additional
>>>>>>>>>>> information could I provide to solicit some direction for 
>>>>>>>>>>> overcoming
>>>>>>>> 
>>>>>>>>>>> these latest string of mpi errors?
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> 
>>>>>>>>>>>> Dave
>>>>>>>>>>>> 
>>>>>>>>>>>> From: mpich-discuss-bounces at mcs.anl.gov 
>>>>>>>>>>>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of
>>>>>> SULLIVAN
>>>>>>>>>>>> David F (AREVA NP INC)
>>>>>>>>>>>> Sent: Friday, July 23, 2010 4:29 PM
>>>>>>>>>>>> To: mpich-discuss at mcs.anl.gov
>>>>>>>>>>>> Subject: [mpich-discuss] cryptic (to me) error
>>>>>>>>>>>> 
>>>>>>>>>>>> With my firewall issues firmly behind me, I have a new 
>>>>>>>>>>>> problem
>>>>>> 
>>>>>>>>>>>> for
>>>>>>> 
>>>>>>>>>>>> the
>>>>>>>>>>> collective wisdom. I am attempting to run a program to which

>>>>>>>>>>> the
>>>>>> 
>>>>>>>>>>> response is as follows:
>>>>>>>>>>>> [mcnp5_1-4 at athos ~]$ mpiexec -f nodes -n 12 mcnp5.mpi 
>>>>>>>>>>>> i=TN04
> 
>>>>>>>>>>>> o=TN04.o
>>>>>>>>>> 
>>>>>>>>>>>> Fatal error in MPI_Comm_dup: Other MPI error, error stack:
>>>>>>>>>>>> MPI_Comm_dup(168).................:
>>>>>> MPI_Comm_dup(MPI_COMM_WORLD,
>>>>>>>>>>>> new_comm=0x7fff58edb450) failed
>>>>>>>>>>>> MPIR_Comm_copy(923)...............:
>>>>>>>>>>>> MPIR_Get_contextid(639)...........:
>>>>>>>>>>>> MPI_Allreduce(773)................:
>>>>>>>> MPI_Allreduce(sbuf=MPI_IN_PLACE,
>>>>>>>>>>> rbuf=0x7fff
>>>>>>>>>>> 58edb1a0, count=64, MPI_INT, MPI_BAND, MPI_COMM_WORLD) 
>>>>>>>>>>> failed
>>>>>>>>>>>> MPIR_Allreduce(228)...............:
>>>>>>>>>>>> MPIC_Send(41).....................:
>>>>>>>>>>>> MPIC_Wait(513)....................:
>>>>>>>>>>>> MPIDI_CH3I_Progress(150)..........:
>>>>>>>>>>>> MPID_nem_mpich2_blocking_recv(933):
>>>>>>>>>>>> MPID_nem_tcp_connpoll(1709).......: Communication error 
>>>>>>>>>>>> Fatal
>> 
>>>>>>>>>>>> error
>>>>>>>> 
>>>>>>>>>>>> in
>>>>>>>>>>>> MPI_Comm_dup: Other MPI error, error stack:
>>>>>>>>>>>> MPI_Comm_dup(168).................:
>>>>>> MPI_Comm_dup(MPI_COMM_WORLD,
>>>>>>>>>>> new_comm=0x7fff
>>>>>>>>>>> 97dca620) failed
>>>>>>>>>>>> MPIR_Comm_copy(923)...............:
>>>>>>>>>>>> MPIR_Get_contextid(639)...........:
>>>>>>>>>>>> MPI_Allreduce(773)................:
>>>>>>>> MPI_Allreduce(sbuf=MPI_IN_PLACE,
>>>>>>>>>>> rbuf=0x7fff
>>>>>>>>>>> 97dca370, count=64, MPI_INT, MPI_BAND, MPI_COMM_WORLD) 
>>>>>>>>>>> failed
>>>>>>>>>>>> MPIR_Allreduce(289)...............:
>>>>>>>>>>>> MPIC_Sendrecv(161)................:
>>>>>>>>>>>> MPIC_Wait(513)....................:
>>>>>>>>>>>> MPIDI_CH3I_Progress(150)..........:
>>>>>>>>>>>> MPID_nem_mpich2_blocking_recv(948):
>>>>>>>>>>>> MPID_nem_tcp_connpoll(1709).......: Communication error 
>>>>>>>>>>>> Killed
>>>>>> by
>>>>>> 
>>>>>>>>>>>> signal 2.
>>>>>>>>>>>> Ctrl-C caught... cleaning up processes [mpiexec at athos] 
>>>>>>>>>>>> HYDT_dmx_deregister_fd (./tools/demux/demux.c:142): could 
>>>>>>>>>>>> not
>> 
>>>>>>>>>>>> find
>>>>>>> 
>>>>>>>>>>>> fd
>>>>>>>>>> 
>>>>>>>>>>>> to deregister: -2 [mpiexec at athos] HYD_pmcd_pmiserv_cleanup
>>>>>>>>>>>> (./pm/pmiserv/pmiserv_cb.c:401): error deregistering fd 
>>>>>>>>>>>> [press
>>>>>> 
>>>>>>>>>>>> Ctrl-C
>>>>>>>>>> 
>>>>>>>>>>>> again to force abort] APPLICATION TERMINATED WITH THE EXIT
>>>>>> STRING:
>>>>>>>>>>>> Killed (signal 9) [mcnp5_1-4 at athos ~]$ Any ideas?
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks in advance,
>>>>>>>>>>>> 
>>>>>>>>>>>> David Sullivan
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> AREVA NP INC
>>>>>>>>>>>> 400 Donald Lynch Boulevard
>>>>>>>>>>>> Marlborough, MA, 01752
>>>>>>>>>>>> Phone: (508) 573-6721
>>>>>>>>>>>> Fax: (434) 382-5597
>>>>>>>>>>>> David.Sullivan at AREVA.com
>>>>>>>>>>>> 
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> mpich-discuss mailing list
>>>>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> mpich-discuss mailing list
>>>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> mpich-discuss mailing list
>>>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> mpich-discuss mailing list
>>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>>> _______________________________________________
>>>>>>>>>> mpich-discuss mailing list
>>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> mpich-discuss mailing list
>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>>> _______________________________________________
>>>>>>>>> mpich-discuss mailing list
>>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> mpich-discuss mailing list
>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>>> <summary.xml>_______________________________________________
>>>>>>>> mpich-discuss mailing list
>>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> mpich-discuss mailing list
>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>>> _______________________________________________
>>>>>>> mpich-discuss mailing list
>>>>>>> mpich-discuss at mcs.anl.gov
>>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>> 
>>>>>> _______________________________________________
>>>>>> mpich-discuss mailing list
>>>>>> mpich-discuss at mcs.anl.gov
>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>>> 
>>>>>> _______________________________________________
>>>>>> mpich-discuss mailing list
>>>>>> mpich-discuss at mcs.anl.gov
>>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>> _______________________________________________
>>>>> mpich-discuss mailing list
>>>>> mpich-discuss at mcs.anl.gov
>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>> <summary.xml><error.txt>__________________________________________
>>>>> _
>>>>> _
>>>>> ___
>>>>> mpich-discuss mailing list
>>>>> mpich-discuss at mcs.anl.gov
>>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>> 
>>>> _______________________________________________
>>>> mpich-discuss mailing list
>>>> mpich-discuss at mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>> 
>>>> <winmail.dat>_______________________________________________
>>>> mpich-discuss mailing list
>>>> mpich-discuss at mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>> 
>>> _______________________________________________
>>> mpich-discuss mailing list
>>> mpich-discuss at mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>> 
>>> <winmail.dat>_______________________________________________
>>> mpich-discuss mailing list
>>> mpich-discuss at mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>> 
>> _______________________________________________
>> mpich-discuss mailing list
>> mpich-discuss at mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>> _______________________________________________
>> mpich-discuss mailing list
>> mpich-discuss at mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> 
> _______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> <testing_screen.txt>_______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: testing_screen.txt
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20100903/40f97ef0/attachment.txt>


More information about the mpich-discuss mailing list