[mpich-discuss] MPI application design

[ronald at audaces.com.br]

Hi all,

This is my first message to this mailing list. Im just entering this world of
MPI and MPICH seems to be a wonderful implementation from what Ive seen till
now.

My subject is not really into MPICH itself buf a more general MPI application
design. So... I hope you experts at MPI/MPICH can help me.

Well... lets go to the problem:

In our company, we a have a CPU intesive algorithm that we already run on a
multi-process "manually controlled" (not MPI) environment. The core algorithm
is a thirdy party library (not parallel) so, we dont have access to the code in
order to implement paralellism on the algorithm itself. 

But we can see it as a candidate for MPI if we look at the problem at an higher
level: A set of problems that can be solved concurrently by this algorithm...
The paralellismm consists of solving each problem in parallel. Each "run" of
the serial algorithm in a process. And of course we get all the message passing
mechanism (that is a big problem regarding parallel applications and MPI handles
it very well).

We plan to use it like this: Every machine on the network will provide its
processors to the Cluster (one machine gives 1, others 2, others 4 processors)
in every processor we should run one problem at a time. A central processing
unit will coordinate (and collect) the results and manage the processes (using
MPI).

An example: My problem consists of 100 problems to be solved and... on my
network I have 10 processing unit (processors on slave machines). Lets say we
want to run our algorith (an heuristic procedure) for 10 minutes for each
problem. Our controller will for each problem:
- See if there is a free Processor.
- Send the problem to this processor so that it can compute its solution
- Recieve back its solution and keep it.
- Repeat till the 100 problems are solved.
In this case it will take 100 min. to solve all the 100 problems. (10 mins for
each 10 parallel workers, if it weren't in parallel it would take 1000 min.)

This is the kind of paralellism that we need. I though of a lot of ways to solve
it but I would like to hear what do you guys think.

At this time of my studies I'm tending to do it this way:
- The controlling unit is a server that accepts connections.
- For every new process it needs, it launches (mpiexec) a new process to
proccess it, passing the connection as a parameter to the executor.
- the executor communicates the progress through mpi to the server.
- The it goes till the end;


What do you guys think. Any help will be appreciated... Im really stuck by my
lack of experience on choosing the way to go.

Thanks in advance.

Ronald