[mpich-discuss] mpiexec kills the remote login shell
Yu-Cheng Chou
cycchou at ucdavis.edu
Tue Feb 3 19:52:05 CST 2009
> # Can you run non-MPI programs using mpiexec (mpiexec -n 2 hostname) ?
Yes.
> # Can you compile and run the hello world program (examples/hellow.c)
> provided with MPICH2 (mpiexec -n 2 ./hellow)?
Yes.
> # How did you start smpd (the command used to start smpd) ? How did you run
> your MPI job (the command used to run your job)?
I have a ".smpd" file containing one line of information, which is "phrase=123".
Thus, I started smpd using "smpd -s".
Then I used "mpiexec -n 1 hellow" to run hellow on Korebot.
> # How did you find that mpiexec kills the sshd process (We typically ssh to
> unix machines and run mpiexec without any problems) ?
I logged in Korebot with two terminals.
>From #1 terminal, I checked all the processes running on Korebot.
>From #2 terminal, I started smpd and run hellow using the commands
mentioned above.
After hellow was finished, the connection to Korebot via #2 terminal was closed.
>From #1 terminal, I knew that the sshd process associated with #2
terminal was gone.
> Can you run smpd/mpiexec in debug mode and provide us with the outputs
> (smpd -d / mpiexec -n 2 -verbose hostname) ?
The first attached text file is the output from running hellow in
mpiexec's verbose mode.
There is another issue.
This time, I used two machines. One is Korebot as mentioned above, and
the other is a laptop running Ubuntu Linux OS.
I started smpd with the same ".smpd" file and command as mentioned
above both on Korebot and the lap top.
There is a machine file called "hostfile" on Korebot. The file
contains the following information about the name of the two machines.
korebot
shrimp
Then from Korebot, I ran cpi using the following command.
mpiexec -machinefile ./hostfile -verbose -n 2 cpi
But the value of pi is a huge number. I think it is related to "double
type variables" being transferred between processes running on an
ARM-based Linux and a general Linux machines.
The second attached text file is the output from running cpi in
mpiexec's verbose mode.
>
> I am cross-compiling mpich2-1.0.8 with smpd for Khepera III mobile robot.
>
> This mobile robot has a Korebot board which is an ARM-based computer with a
> Linux operating system.
>
> The cross-compilation was fine.
>
> Firstly, I logged in to Korebot through ssh.
> Secondly, I started smpd.
> Thirdly, I ran mpiexec to execute an MPI program (cpi) that comes with the
> package.
>
> The result was correct, but when mpiexec was finished, the ssh connection to
> the Korebot was closed.
> I found that mpiexec kills the sshd process through which I was remotely
> connected to Korebot.
>
> I've been looking for the cause, but still have not found any clues.
>
> Could you give me any ideas to solve this problem?
>
> Thank you,
>
> Yu-Cheng
>
-------------- next part --------------
~ # mpiexec -n 1 -verbose hello
..\smpd_get_full_path_name
...\smpd_search_path
.../smpd_search_path
../smpd_get_full_path_name
..handling executable:
hello
..\smpd_get_next_host
...\smpd_get_next_hostname
.../smpd_get_next_hostname
...\smpd_get_host_id
.../smpd_get_host_id
../smpd_get_next_host
..\smpd_create_cliques
...\next_launch_node
.../next_launch_node
...\next_launch_node
.../next_launch_node
../smpd_create_cliques
..\smpd_fix_up_host_tree
../smpd_fix_up_host_tree
./mp_parse_command_args
.host tree:
. host: korebot, parent: 0, id: 1
.launch nodes:
. iproc: 0, id: 1, exe: hello
.\smpd_get_smpd_data
..\smpd_hide_string_arg
...\first_token
.../first_token
...\compare_token
.../compare_token
...\next_token
....\first_token
..../first_token
....\first_token
..../first_token
.../next_token
../smpd_hide_string_arg
../smpd_hide_string_arg
..\smpd_get_smpd_data_from_environment
../smpd_get_smpd_data_from_environment
..\smpd_parse_smpd_file
...\smpd_open_smpd_file
.../smpd_open_smpd_file
...\str_replace
.../str_replace
../smpd_parse_smpd_file
..\smpd_hide_string_arg
...\first_token
.../first_token
...\compare_token
.../compare_token
...\next_token
....\first_token
..../first_token
....\first_token
..../first_token
.../next_token
../smpd_hide_string_arg
../smpd_hide_string_arg
./smpd_get_smpd_data
.\smpd_create_context
..\smpd_init_context
...\smpd_init_command
.../smpd_init_command
../smpd_init_context
./smpd_create_context
.\smpd_enter_at_state
..sock_waiting for the next event.
..SOCK_OP_CONNECT
..\smpd_handle_op_connect
...connect succeeded, posting read of the challenge string
../smpd_handle_op_connect
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_challenge_string
....read challenge string: '1.0.8 1312222903'
....\smpd_verify_version
..../smpd_verify_version
....\smpd_hash
..../smpd_hash
.../smpd_state_reading_challenge_string
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_challenge_response
....wrote challenge response: 'c67e96ae02ed68f3c2a606452c593643'
.../smpd_state_writing_challenge_response
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_connect_result
....read connect result: 'SUCCESS'
.../smpd_state_reading_connect_result
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_process_session_request
....wrote process session request: 'process'
.../smpd_state_writing_process_session_request
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cred_request
....read cred request: 'nocredentials'
.../smpd_state_reading_cred_request
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_reconnect_request
....read re-connect request: '-1'
....\smpd_generate_session_header
.....session header: (id=1 parent=0 level=0)
..../smpd_generate_session_header
.../smpd_state_reading_reconnect_request
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_session_header
....wrote session header: 'id=1 parent=0 level=0'
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 47 bytes for command: "cmd=start_dbs src=0 dest=1 tag=0 "
..../smpd_post_write_command
.../smpd_state_writing_session_header
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=start_dbs src=0 dest=1 tag=0 "
....moving 'start_dbs' command to the wait_list.
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 160 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=1 dest=0 tag=0 cmd_tag=0 cmd_orig=start_dbs kvs_name=BC072445846E77170A40A377FC0A56 domain_name=17E32A536AECDA9A16BBEB2210E350F0 result=SUCCESS "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = result
..... tag = 0
..... ctx = left
..... len = 160
..... str = cmd=result src=1 dest=0 tag=0 cmd_tag=0 cmd_orig=start_dbs kvs_name=BC072445846E77170A40A377FC0A56 domain_name=17E32A536AECDA9A16BBEB2210E350F0 result=SUCCESS
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......start_dbs succeeded, kvs_name: 'BC072445846E77170A40A377FC0A56', domain_name: '17E32A536AECDA9A16BBEB2210E350F0'
......\smpd_launch_processes
.......creating a process group of size 1 on node 0 called BC072445846E77170A40A377FC0A56
.......\create_process_group
......./create_process_group
.......launching the processes.
.......\smpd_create_command
........\smpd_init_command
......../smpd_init_command
......./smpd_create_command
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_post_write_command
........\smpd_package_command
......../smpd_package_command
........smpd_post_write_command on the left context sock 16777216: 149 bytes for command: "cmd=launch src=0 dest=1 tag=1 c=hello s=0 a=0 d=/root q=0 i=0 n=1 k=BC072445846E77170A40A377FC0A56 kd=17E32A536AECDA9A16BBEB2210E350F0 "
......./smpd_post_write_command
....../smpd_launch_processes
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=launch src=0 dest=1 tag=1 c=hello s=0 a=0 d=/root q=0 i=0 n=1 k=BC072445846E77170A40A377FC0A56 kd=17E32A536AECDA9A16BBEB2210E350F0 "
....moving 'launch' command to the wait_list.
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 128 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=1 dest=0 tag=1 cmd_tag=1 cmd_orig=launch result=SUCCESS pg_id=BC072445846E77170A40A377FC0A56 pg_rank=0 pg_ctx=0 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = result
..... tag = 1
..... ctx = left
..... len = 128
..... str = cmd=result src=1 dest=0 tag=1 cmd_tag=1 cmd_orig=launch result=SUCCESS pg_id=BC072445846E77170A40A377FC0A56 pg_rank=0 pg_ctx=0
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......successfully launched: 'cmd=launch src=0 dest=1 tag=1 c=hello s=0 a=0 d=/root q=0 i=0 n=1 k=BC072445846E77170A40A377FC0A56 kd=17E32A536AECDA9A16BBEB2210E350F0 '
......root process launched, starting stdin redirection.
......\smpd_create_context
.......\smpd_init_context
........\smpd_init_command
......../smpd_init_command
......./smpd_init_context
....../smpd_create_context
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 123 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=abort src=1 dest=0 tag=1 result=FAIL error="Unable to exec 'hello' on korebot. Error 2 - No such file or directory
" "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = abort
..... tag = 1
..... ctx = left
..... len = 123
..... str = cmd=abort src=1 dest=0 tag=1 result=FAIL error="Unable to exec 'hello' on korebot. Error 2 - No such file or directory
"
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_abort_command
......abort: Unable to exec 'hello' on korebot. Error 2 - No such file or directory
...../smpd_handle_abort_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 43 bytes for command: "cmd=close src=0 dest=1 tag=2 "
..../smpd_post_write_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=close src=0 dest=1 tag=2 "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 80 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=exit src=1 dest=0 tag=2 rank=0 code=123 kvs=BC072445846E77170A40A377FC0A56 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = exit
..... tag = 2
..... ctx = left
..... len = 80
..... str = cmd=exit src=1 dest=0 tag=2 rank=0 code=123 kvs=BC072445846E77170A40A377FC0A56
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_exit_command
......saving exit code: rank 0, exitcode 123, pg <BC072445846E77170A40A377FC0A56>
......process exited without calling init.
......process exited before anyone has called init.
......last process exited, returning SMPD_EXIT.
...../smpd_handle_exit_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 43 bytes for command: "cmd=close src=0 dest=1 tag=3 "
..../smpd_post_write_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 31 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=close src=0 dest=1 tag=3 "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=closed src=1 dest=0 tag=3 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = closed
..... tag = 3
..... ctx = left
..... len = 31
..... str = cmd=closed src=1 dest=0 tag=3
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_closed_command
......closed command received from left child, closing sock.
......MPIDU_Sock_post_close(16777216)
......received a closed at node with no parent context, assuming root, returning SMPD_EXITING.
...../smpd_handle_closed_command
..../smpd_handle_command
....not posting read for another command because SMPD_EXITING returned
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_CLOSE
..\smpd_handle_op_close
...\smpd_get_state_string
.../smpd_get_state_string
...op_close received - SMPD_EXITING state.
...\smpd_free_context
....freeing left context.
....\smpd_init_context
.....\smpd_init_command
...../smpd_init_command
..../smpd_init_context
.../smpd_free_context
../smpd_handle_op_close
./smpd_enter_at_state
/main
\smpd_exit
.\smpd_kill_all_processes
./smpd_kill_all_processes
.\smpd_dbs_finalize
./smpd_dbs_finalize
~ # Connection to korebot closed.
ycchou at shrimp:~$
-------------- next part --------------
~ # mpiexec -machinefile ./hostfile -verbose -n 2 cpi
..\smpd_get_full_path_name
...\smpd_search_path
.../smpd_search_path
../smpd_get_full_path_name
..handling executable:
cpi
..\smpd_get_next_host
...\smpd_get_next_hostname
.../smpd_get_next_hostname
...\smpd_get_host_id
.../smpd_get_host_id
../smpd_get_next_host
..\smpd_get_next_host
...\smpd_get_next_hostname
.../smpd_get_next_hostname
...\smpd_get_host_id
.../smpd_get_host_id
../smpd_get_next_host
..\smpd_create_cliques
...\prev_launch_node
.../prev_launch_node
...\prev_launch_node
.../prev_launch_node
...\prev_launch_node
.../prev_launch_node
...\prev_launch_node
.../prev_launch_node
../smpd_create_cliques
..\smpd_fix_up_host_tree
../smpd_fix_up_host_tree
./mp_parse_command_args
.host tree:
. host: korebot, parent: 0, id: 1
. host: shrimp, parent: 1, id: 2
.launch nodes:
. iproc: 1, id: 2, exe: cpi
. iproc: 0, id: 1, exe: cpi
.\smpd_get_smpd_data
..\smpd_hide_string_arg
...\first_token
.../first_token
...\compare_token
.../compare_token
...\next_token
....\first_token
..../first_token
....\first_token
..../first_token
.../next_token
../smpd_hide_string_arg
../smpd_hide_string_arg
..\smpd_get_smpd_data_from_environment
../smpd_get_smpd_data_from_environment
..\smpd_parse_smpd_file
...\smpd_open_smpd_file
.../smpd_open_smpd_file
...\str_replace
.../str_replace
../smpd_parse_smpd_file
..\smpd_hide_string_arg
...\first_token
.../first_token
...\compare_token
.../compare_token
...\next_token
....\first_token
..../first_token
....\first_token
..../first_token
.../next_token
../smpd_hide_string_arg
../smpd_hide_string_arg
./smpd_get_smpd_data
.\smpd_create_context
..\smpd_init_context
...\smpd_init_command
.../smpd_init_command
../smpd_init_context
./smpd_create_context
.\smpd_enter_at_state
..sock_waiting for the next event.
..SOCK_OP_CONNECT
..\smpd_handle_op_connect
...connect succeeded, posting read of the challenge string
../smpd_handle_op_connect
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_challenge_string
....read challenge string: '1.0.8 747711126'
....\smpd_verify_version
..../smpd_verify_version
....\smpd_hash
..../smpd_hash
.../smpd_state_reading_challenge_string
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_challenge_response
....wrote challenge response: '80fd4c4e51f13e13e5bd6fc287293622'
.../smpd_state_writing_challenge_response
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_connect_result
....read connect result: 'SUCCESS'
.../smpd_state_reading_connect_result
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_process_session_request
....wrote process session request: 'process'
.../smpd_state_writing_process_session_request
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cred_request
....read cred request: 'nocredentials'
.../smpd_state_reading_cred_request
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_reconnect_request
....read re-connect request: '-1'
....\smpd_generate_session_header
.....session header: (id=1 parent=0 level=0)
..../smpd_generate_session_header
.../smpd_state_reading_reconnect_request
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_session_header
....wrote session header: 'id=1 parent=0 level=0'
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....creating connect command for left node
....creating connect command to 'shrimp'
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_add_command_arg
..../smpd_add_command_arg
....\smpd_add_command_int_arg
..../smpd_add_command_int_arg
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 62 bytes for command: "cmd=connect src=0 dest=1 tag=0 host=shrimp id=2 "
..../smpd_post_write_command
....not connected yet: shrimp not connected
.../smpd_state_writing_session_header
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=connect src=0 dest=1 tag=0 host=shrimp id=2 "
....moving 'connect' command to the wait_list.
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 73 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=1 dest=0 tag=0 cmd_tag=0 cmd_orig=connect result=SUCCESS "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = result
..... tag = 0
..... ctx = left
..... len = 73
..... str = cmd=result src=1 dest=0 tag=0 cmd_tag=0 cmd_orig=connect result=SUCCESS
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......successful connect, state: connecting tree.
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 47 bytes for command: "cmd=start_dbs src=0 dest=1 tag=1 "
..../smpd_post_write_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=start_dbs src=0 dest=1 tag=1 "
....moving 'start_dbs' command to the wait_list.
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 160 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=1 dest=0 tag=1 cmd_tag=1 cmd_orig=start_dbs kvs_name=5176BB76E7E408E1A7314B9D6FF4CF domain_name=17E32A536AECDA9A16BBEB2210E350F0 result=SUCCESS "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = result
..... tag = 1
..... ctx = left
..... len = 160
..... str = cmd=result src=1 dest=0 tag=1 cmd_tag=1 cmd_orig=start_dbs kvs_name=5176BB76E7E408E1A7314B9D6FF4CF domain_name=17E32A536AECDA9A16BBEB2210E350F0 result=SUCCESS
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......start_dbs succeeded, kvs_name: '5176BB76E7E408E1A7314B9D6FF4CF', domain_name: '17E32A536AECDA9A16BBEB2210E350F0'
......\smpd_launch_processes
.......creating a process group of size 2 on node 0 called 5176BB76E7E408E1A7314B9D6FF4CF
.......\create_process_group
......./create_process_group
.......launching the processes.
.......\smpd_create_command
........\smpd_init_command
......../smpd_init_command
......./smpd_create_command
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_post_write_command
........\smpd_package_command
......../smpd_package_command
........smpd_post_write_command on the left context sock 16777216: 154 bytes for command: "cmd=launch src=0 dest=2 tag=2 c=cpi s=0 a=0 d=/root p=/bin q=1 i=1 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 "
......./smpd_post_write_command
.......\smpd_create_command
........\smpd_init_command
......../smpd_init_command
......./smpd_create_command
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_int_arg
......./smpd_add_command_int_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_add_command_arg
......./smpd_add_command_arg
.......\smpd_post_write_command
........\smpd_package_command
......../smpd_package_command
........enqueueing write at the end of the list.
......./smpd_post_write_command
....../smpd_launch_processes
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=launch src=0 dest=2 tag=2 c=cpi s=0 a=0 d=/root p=/bin q=1 i=1 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 "
....moving 'launch' command to the wait_list.
....smpd_handle_written: posting write(154 bytes) for command: "cmd=launch src=0 dest=1 tag=3 c=cpi s=0 a=0 d=/root p=/bin q=0 i=0 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 "
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=launch src=0 dest=1 tag=3 c=cpi s=0 a=0 d=/root p=/bin q=0 i=0 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 "
....moving 'launch' command to the wait_list.
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 128 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=1 dest=0 tag=2 cmd_tag=3 cmd_orig=launch result=SUCCESS pg_id=5176BB76E7E408E1A7314B9D6FF4CF pg_rank=0 pg_ctx=0 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = result
..... tag = 2
..... ctx = left
..... len = 128
..... str = cmd=result src=1 dest=0 tag=2 cmd_tag=3 cmd_orig=launch result=SUCCESS pg_id=5176BB76E7E408E1A7314B9D6FF4CF pg_rank=0 pg_ctx=0
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......tag 2 != match_tag 3
......successfully launched: 'cmd=launch src=0 dest=1 tag=3 c=cpi s=0 a=0 d=/root p=/bin q=0 i=0 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 '
......root process launched, starting stdin redirection.
......\smpd_create_context
.......\smpd_init_context
........\smpd_init_command
......../smpd_init_command
......./smpd_init_context
....../smpd_create_context
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 128 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=result src=2 dest=0 tag=0 cmd_tag=2 cmd_orig=launch result=SUCCESS pg_id=5176BB76E7E408E1A7314B9D6FF4CF pg_rank=1 pg_ctx=0 "
....\smpd_handle_command
.....handling command:
..... src = 2
..... dest = 0
..... cmd = result
..... tag = 0
..... ctx = left
..... len = 128
..... str = cmd=result src=2 dest=0 tag=0 cmd_tag=2 cmd_orig=launch result=SUCCESS pg_id=5176BB76E7E408E1A7314B9D6FF4CF pg_rank=1 pg_ctx=0
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_result
......successfully launched: 'cmd=launch src=0 dest=2 tag=2 c=cpi s=0 a=0 d=/root p=/bin q=1 i=1 n=2 k=5176BB76E7E408E1A7314B9D6FF4CF kd=17E32A536AECDA9A16BBEB2210E350F0 '
......\smpd_free_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_free_command
...../smpd_handle_result
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 99 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=init src=2 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 value=2 node_id=2 "
....\smpd_handle_command
.....handling command:
..... src = 2
..... dest = 0
..... cmd = init
..... tag = 0
..... ctx = left
..... len = 99
..... str = cmd=init src=2 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 value=2 node_id=2
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_init_command
......\smpd_create_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_create_command
......\smpd_add_command_int_arg
....../smpd_add_command_int_arg
......\smpd_add_command_arg
....../smpd_add_command_arg
......\get_name_key_value
....../get_name_key_value
......init: 1:2:5176BB76E7E408E1A7314B9D6FF4CF
......sending reply to init command 'cmd=init src=2 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 value=2 node_id=2 '.
......\smpd_add_command_arg
....../smpd_add_command_arg
......sending result command to left context: "cmd=result src=0 dest=2 tag=4 cmd_tag=0 ctx_key=0 result=SUCCESS "
......\smpd_post_write_command
.......\smpd_package_command
......./smpd_package_command
.......smpd_post_write_command on the left context sock 16777216: 79 bytes for command: "cmd=result src=0 dest=2 tag=4 cmd_tag=0 ctx_key=0 result=SUCCESS "
....../smpd_post_write_command
...../smpd_handle_init_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=result src=0 dest=2 tag=4 cmd_tag=0 ctx_key=0 result=SUCCESS "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 99 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=init src=1 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=0 value=2 node_id=1 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = init
..... tag = 0
..... ctx = left
..... len = 99
..... str = cmd=init src=1 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=0 value=2 node_id=1
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_init_command
......\smpd_create_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_create_command
......\smpd_add_command_int_arg
....../smpd_add_command_int_arg
......\smpd_add_command_arg
....../smpd_add_command_arg
......\get_name_key_value
....../get_name_key_value
......init: 0:2:5176BB76E7E408E1A7314B9D6FF4CF
......sending reply to init command 'cmd=init src=1 dest=0 tag=0 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=0 value=2 node_id=1 '.
......\smpd_add_command_arg
....../smpd_add_command_arg
......sending result command to left context: "cmd=result src=0 dest=1 tag=5 cmd_tag=0 ctx_key=0 result=SUCCESS "
......\smpd_post_write_command
.......\smpd_package_command
......./smpd_package_command
.......smpd_post_write_command on the left context sock 16777216: 79 bytes for command: "cmd=result src=0 dest=1 tag=5 cmd_tag=0 ctx_key=0 result=SUCCESS "
....../smpd_post_write_command
...../smpd_handle_init_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=result src=0 dest=1 tag=5 cmd_tag=0 ctx_key=0 result=SUCCESS "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 110 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=stdout src=1 dest=0 tag=7 rank=0 first=1 data=50726F636573732030206F662032206973206F6E206B6F7265626F740A "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = stdout
..... tag = 7
..... ctx = left
..... len = 110
..... str = cmd=stdout src=1 dest=0 tag=7 rank=0 first=1 data=50726F636573732030206F662032206973206F6E206B6F7265626F740A
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_stdout_command
......\write_to_stdout
Process 0 of 2 is on korebot
....../write_to_stdout
...../smpd_handle_stdout_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 108 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=stdout src=2 dest=0 tag=1 rank=1 first=1 data=50726F636573732031206F662032206973206F6E20736872696D700A "
....\smpd_handle_command
.....handling command:
..... src = 2
..... dest = 0
..... cmd = stdout
..... tag = 1
..... ctx = left
..... len = 108
..... str = cmd=stdout src=2 dest=0 tag=1 rank=1 first=1 data=50726F636573732031206F662032206973206F6E20736872696D700A
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_stdout_command
......\write_to_stdout
Process 1 of 2 is on shrimp
....../write_to_stdout
...../smpd_handle_stdout_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 1036 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=stdout src=1 dest=0 tag=9 rank=0 first=1 data=6D7969643A203020206D7970693A20312E3537303834360A706920697320617070726F78696D6174656C79202D313833323239303433373831383937323937323632353739373535373939323838313938303831383233323936383334373438303339383834303336303837393237333332373137323637353535323333353832333836333633373131323938353537353133333537323231333531363835383533313038363035373039313733383336333836393537323536393530333933363931323539373636303830363937333730363533343834353231353930323039393537303638382E303030303030303030303030303030302C204572726F7220697320313833323239303433373831383937323937323632353739373535373939323838313938303831383233323936383334373438303339383834303336303837393237333332373137323637353535323333353832333836333633373131323938353537353133333537323231333531363835383533313038363035373039313733383336333836393537323536393530333933363931323539373636303830363937333730363533343834353231353930323039393537303638382E303030303030303030303030303030300A77616C6C20636C6F636B2074696D65203D20302E3038313630370A "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = stdout
..... tag = 9
..... ctx = left
..... len = 1036
..... str = cmd=stdout src=1 dest=0 tag=9 rank=0 first=1 data=6D7969643A203020206D7970693A20312E3537303834360A706920697320617070726F78696D6174656C79202D313833323239303433373831383937323937323632353739373535373939323838313938303831383233323936383334373438303339383834303336303837393237333332373137323637353535323333353832333836333633373131323938353537353133333537323231333531363835383533313038363035373039313733383336333836393537323536393530333933363931323539373636303830363937333730363533343834353231353930323039393537303638382E303030303030303030303030303030302C204572726F7220697320313833323239303433373831383937323937323632353739373535373939323838313938303831383233323936383334373438303339383834303336303837393237333332373137323637353535323333353832333836333633373131323938353537353133333537323231333531363835383533313038363035373039313733383336333836393537323536393530333933363931323539373636303830363937333730363533343834353231353930323039393537303638382E303030303030303030303030303030300A77616C6C20636C6F636B2074696D65203D20302E3038313630370A
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_stdout_command
......\write_to_stdout
myid: 0 mypi: 1.570846
pi is approximately -1832290437818972972625797557992881980818232968347480398840360879273327172675552335823863637112985575133572213516858531086057091738363869572569503936912597660806973706534845215902099570688.0000000000000000, Error is 1832290437818972972625797557992881980818232968347480398840360879273327172675552335823863637112985575133572213516858531086057091738363869572569503936912597660806973706534845215902099570688.0000000000000000
wall clock time = 0.081607
....../write_to_stdout
...../smpd_handle_stdout_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 95 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=finalize src=1 dest=0 tag=4 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=0 n......\get_name_key_value
....../get_name_key_value
......finalize: 0:5176BB76E7E408E1A7314B9D6FF4CF
......sending reply to finalize command 'cmd=finalize src=1 dest=0 tag=4 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=0 node_id=1 '.
......\smpd_add_command_arg
....../smpd_add_command_arg
......sending result command to left context: "cmd=result src=0 dest=1 tag=6 cmd_tag=4 ctx_key=0 result=SUCCESS "
......\smpd_post_write_command
.......\smpd_package_command
......./smpd_package_command
.......smpd_post_write_command on the left context sock 16777216: 79 bytes for command: "cmd=result src=0 dest=1 tag=6 cmd_tag=4 ctx_key=0 result=SUCCESS "
....../smpd_post_write_command
...../smpd_handle_finalize_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 95 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=result src=0 dest=1 tag=6 cmd_tag=4 ctx_key=0 result=SUCCESS "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=finalize src=2 dest=0 tag=3 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 node_id=2 "
....\smpd_handle_command
.....handling command:
..... src = 2
..... dest = 0
..... cmd = finalize
..... tag = 3
..... ctx = left
..... len = 95
..... str = cmd=finalize src=2 dest=0 tag=3 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 node_id=2
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_finalize_command
......\smpd_create_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_create_command
......\smpd_add_command_int_arg
....../smpd_add_command_int_arg
......\smpd_add_command_arg
....../smpd_add_command_arg
......\get_name_key_value
....../get_name_key_value
......finalize: 1:5176BB76E7E408E1A7314B9D6FF4CF
......sending reply to finalize command 'cmd=finalize src=2 dest=0 tag=3 ctx_key=0 name=5176BB76E7E408E1A7314B9D6FF4CF key=1 node_id=2 '.
......\smpd_add_command_arg
....../smpd_add_command_arg
......sending result command to left context: "cmd=result src=0 dest=2 tag=7 cmd_tag=3 ctx_key=0 result=SUCCESS "
......\smpd_post_write_command
.......\smpd_package_command
......./smpd_package_command
.......smpd_post_write_command on the left context sock 16777216: 79 bytes for command: "cmd=result src=0 dest=2 tag=7 cmd_tag=3 ctx_key=0 result=SUCCESS "
....../smpd_post_write_command
...../smpd_handle_finalize_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=result src=0 dest=2 tag=7 cmd_tag=3 ctx_key=0 result=SUCCESS "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 79 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=exit src=1 dest=0 tag=12 rank=0 code=0 kvs=5176BB76E7E408E1A7314B9D6FF4CF "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = exit
..... tag = 12
..... ctx = left
..... len = 79
..... str = cmd=exit src=1 dest=0 tag=12 rank=0 code=0 kvs=5176BB76E7E408E1A7314B9D6FF4CF
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_exit_command
......saving exit code: rank 0, exitcode 0, pg <5176BB76E7E408E1A7314B9D6FF4CF>
...../smpd_handle_exit_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 78 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=exit src=2 dest=0 tag=2 rank=1 code=0 kvs=5176BB76E7E408E1A7314B9D6FF4CF "
....\smpd_handle_command
.....handling command:
..... src = 2
..... dest = 0
..... cmd = exit
..... tag = 2
..... ctx = left
..... len = 78
..... str = cmd=exit src=2 dest=0 tag=2 rank=1 code=0 kvs=5176BB76E7E408E1A7314B9D6FF4CF
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_exit_command
......saving exit code: rank 1, exitcode 0, pg <5176BB76E7E408E1A7314B9D6FF4CF>
......last process exited, returning SMPD_EXIT.
...../smpd_handle_exit_command
..../smpd_handle_command
....\smpd_post_read_command
.....posting a read for a command header on the left context, sock 16777216
..../smpd_post_read_command
....\smpd_create_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_create_command
....\smpd_post_write_command
.....\smpd_package_command
...../smpd_package_command
.....smpd_post_write_command on the left context sock 16777216: 43 bytes for command: "cmd=close src=0 dest=1 tag=8 "
..../smpd_post_write_command
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=close src=0 dest=1 tag=8 "
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd_header
....read command header
....command header read, posting read for data: 40 bytes
.../smpd_state_reading_cmd_header
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_READ
..\smpd_handle_op_read
...\smpd_state_reading_cmd
....read command
....\smpd_parse_command
..../smpd_parse_command
....read command: "cmd=closed_request src=1 dest=0 tag=14 "
....\smpd_handle_command
.....handling command:
..... src = 1
..... dest = 0
..... cmd = closed_request
..... tag = 14
..... ctx = left
..... len = 40
..... str = cmd=closed_request src=1 dest=0 tag=14
.....\smpd_command_destination
......0 -> 0 : returning NULL context
...../smpd_command_destination
.....\smpd_handle_closed_request_command
......\smpd_create_command
.......\smpd_init_command
......./smpd_init_command
....../smpd_create_command
......sending closed command to context: "cmd=closed src=0 dest=1 tag=9 "
......\smpd_post_write_command
.......\smpd_package_command
......./smpd_package_command
.......smpd_post_write_command on the left context sock 16777216: 44 bytes for command: "cmd=closed src=0 dest=1 tag=9 "
....../smpd_post_write_command
......posted closed command to context.
...../smpd_handle_closed_request_command
..../smpd_handle_command
....not posting read for another command because SMPD_CLOSE returned
.../smpd_state_reading_cmd
../smpd_handle_op_read
..sock_waiting for the next event.
..SOCK_OP_WRITE
..\smpd_handle_op_write
...\smpd_state_writing_cmd
....wrote command
....command written to left: "cmd=closed src=0 dest=1 tag=9 "
....closed command written, posting close of the sock.
....MPIDU_Sock_post_close(16777216)
....\smpd_free_command
.....\smpd_init_command
...../smpd_init_command
..../smpd_free_command
.../smpd_state_writing_cmd
../smpd_handle_op_write
..sock_waiting for the next event.
..SOCK_OP_CLOSE
..\smpd_handle_op_close
...\smpd_get_state_string
.../smpd_get_state_string
...op_close received - SMPD_CLOSING state.
...Unaffiliated left context closing.
...\smpd_free_context
....freeing left context.
....\smpd_init_context
.....\smpd_init_command
...../smpd_init_command
..../smpd_init_context
.../smpd_free_context
...all contexts closed, exiting state machine.
../smpd_handle_op_close
./smpd_enter_at_state
/main
\smpd_exit
.\smpd_kill_all_processes
./smpd_kill_all_processes
.\smpd_dbs_finalize
./smpd_dbs_finalize
~ # Connection to korebot closed.
More information about the mpich-discuss
mailing list