[mpich-discuss] program in system() executes before prior onefinishes

[Jayesh Krishna]

Hi,
 What is the error message that you get ? Can you try adding the command ("start /w swat_Edit.exe") into a batch file (say "swat_edit_batch.bat") and invoking the batch file from R (system("swat_edit_batch.bat"))?

Regards,
Jayesh
----- Original Message -----
From: John Joseph <John.Joseph at utsa.edu>
To: Jayesh Krishna <jayesh at mcs.anl.gov>
Cc: mpich-discuss at mcs.anl.gov
Sent: Fri, 03 Dec 2010 09:59:10 -0600 (CST)
Subject: RE: [mpich-discuss] program in system() executes before prior onefinishes

Thanks for your response, Jayesh.  The lines of code I've presented are actually written and executed in R, while the executables "swat_Edit.exe" and "swat2009.exe" are written and compiled in Fortran.  I used the latest Intel Fortran compiler to compile swat2009.exe, but cannot alter swat_Edit.exe, as I do not have the source code.  
 
system("start /w swat_Edit.exe") does not work in R.  As far as Fortran, I'm sure you are more of an expert in it than I am.  You seem to be saying that I might be able to write some Fortran code that that forces swat2009.exe to wait until swat_Edit.exe completely finishes.  Is that right?
 
I appreciate even your guesses because I've been stumped by this for so long.  And that goes for everyone else out there, too...even guesses/brainstorming are appreciated.
 
John Joseph

________________________________

From: Jayesh Krishna [mailto:jayesh at mcs.anl.gov]
Sent: Fri 12/3/2010 9:17 AM
To: John Joseph
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] program in system() executes before prior onefinishes



Hi,
 I am not a FORTRAN expert. Is system(...,...,...) (The version with 3 args) available with your compiler ? Which compiler are you using ?
 If you are running on Windows you can try,

=====================================

 system("start /w swat_Edit.exe")
 system("swat2009.exe")

=====================================

Regards,
Jayesh

----- Original Message -----
From: John Joseph <John.Joseph at utsa.edu>
To: mpich-discuss at mcs.anl.gov
Sent: Fri, 03 Dec 2010 07:03:08 -0600 (CST)
Subject: [mpich-discuss] program in system() executes before prior one  finishes


Hello,

I have a problem I've been struggling with for months, and I'm thinking that folks that deal with parallel processing are the most likely ones to be able to solve it. 

I have this simple code that I run in R on various Windows XP and Windows 7:

system("swat_Edit.exe",wait=FALSE,invisible=FALSE)
system("swat2009.exe")

(Both swat_Edit.exe and swat2009.exe are compiled in Fortran.  I have access to the swat2009.exe source code, but not to the swat_Edit.exe source code.)

In the first of the above two lines, swat_Edit.exe opens hundreds of ASCII files having data descriptive of water basin characteristics, edits them, and then closes them.  In the second line, swat2009.exe reads those files to calculate streamflow at the water basin outlet.   These two code lines are actually part of a loop in which the set of ASCII files is revised and the corresponding streamflow is calculated, and the loop typically cycles about 10,000 times.  In each new cycle the streamflow calculated from the previous cycle is needed to direct swat_Edit.exe in revising the ASCII files so that swat2009.exe can calculate the streamflow for the new cycle.

Now, the problem is this: swat2009.exe sometimes begins executing BEFORE swat_Edit.exe finishes revising and closing all the ASCII files.  As a partial remedy, I insert a Sys.sleep() line like this:

system("swat_Edit.exe",wait=FALSE,invisible=FALSE)
Sys.sleep(7)
system("swat2009.exe")

But this doesn't always work.  It might work for, say, 7,553 cycles, but then on the 7,554th swat2009.exe starts before swat_Edit.exe is quite finished, and then the whole set of simulations, which takes a day or two, must start all over again.  Also, Sys.sleep() adds time to the whole process.  The higher the argument in Sys.sleep(), the less likely the crash, but the more the slow down - and the risk of swat2009.exe starting too soon is never completely eliminated even for Sys.sleep() times that far exceed the average time it takes swat_Edit.exe to complete its task.

The common sense reader might say, "Why not change the 'wait=FALSE' argument in the first line to 'wait=TRUE'?"  Well, if I do that the command won't execute.  I can change 'invisible=FALSE' to 'invisible=TRUE', but that doesn't seem to make a difference.

The problem is worst on my Core-i7, but also happens on an old single-processor computer I have.

MPICH and Rmpi are installed on my computers, and I'm hoping that I can somehow use them to positively prevent swat2009.exe from starting before swat_Edit.exe finishes. 

I'm a newbie as far as mpi goes.  Any ideas?  Your help would be much appreciated.

John Joseph