[ExM Users] best server:worker ratio

[Tim Armstrong]

ADLB_SERVERS.  You're probably looking at the 0.5.0 documentation.  Here is
the up-to-date 0.6.x documentation, which doesn't have any references to
TURBINE_ENGINE: http://swift-lang.org/Swift-T/guide.html

The ratio just depends on your application characteristics: I can't give
you a universal answer.  I'll give some general thoughts.

If the application isn't too demanding, you'll probably have a wide
envelope of settings where you'll get decent results.  There are a few
factors to consider to guesstimate what this envelope might be:

   1. Whether you've allocated enough servers to keep pace with your
   workload.  This depends on many factors including task rate, burstiness of
   the tasks, complexity of data flow, load imbalance, etc.  The maximum
   throughput ofa server is up to 500k task/data operations a second.  You
   want servers to be loaded much more lightly than that to allow for low
   latency responses, however.  For a typical application you might want to
   aim for, say, below 10k tasks/sec/server, to allow for data operations and
   various confounding factors.
   2. The % of processes you're using as servers and that therefore can't
   execute work tasks.  This is a fairly minor factor, since it would be
   unusual to devote more than 5% of processes to servers.
   3. Overhead of additional server processes for task distribution, etc.
   This isn't too important with the 0.6.x ADLB server, but I thought I'd at
   least mention that there were some downsides to allocating many servers.

If you're really not sure, I'd choose something like 1:32 or 1:64, since
you're then only using 2-3% of processes.  If your workload has
long-running tasks (100ms+) and not too much dataflow going on you could
safely reduce the number of servers from that.
- Tim


On Wed, Jul 30, 2014 at 1:15 PM, Ketan Maheshwari <ketan at mcs.anl.gov> wrote:

> In the current state T where ADLB and Turbine servers are combined into
> one server, what is the best server:worker ratio for load such as 1 rack
> over BG/Q, ie. ~16K tasks.
>
> How to set this in light of this combination of ADLB, Turbine?
> Documentation says:
>
> TURBINE_ENGINES
> Number of Turbine engines
>
> ADLB_SERVERS
> Number of ADLB servers
>
> Which one is effective?
>
> Thanks,
> Ketan
>
> _______________________________________________
> ExM-user mailing list
> ExM-user at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/exm-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/exm-user/attachments/20140730/8c0e0613/attachment.html>