[codes-ross-users] Help with Storage Simulation

[Mubarak, Misbah]

Hi Harsh,

Have you looked at the codes-storage-server model? There is a wiki at:

https://xgitlab.cels.anl.gov/codes/codes-storage-server/wikis/home

The wiki has an example case study here that models storage nodes on the dragonfly network. The clients (compute nodes) interact with the storage nodes via an API that does RDMA style communication. You can find the example on the API usage in tests/test-client-checkpoint.c [Look for the function: send_req_to_store].  The API itself can be found at: codes-store-lp.h.

The repo can be cloned from:
https://xgitlab.cels.anl.gov/codes/codes-storage-server

The CODES storage model makes use of the local-storage-model in CODES. Attached is a diagram showing the sequence of steps for a write operation (read operation is also implemented, you have to specify a flag indicating whether it is a read or a write).

Hope that helps.

Regards,
Misbah
From: <codes-ross-users-bounces at lists.mcs.anl.gov<mailto:codes-ross-users-bounces at lists.mcs.anl.gov>> on behalf of Harsh Khetawat <hkhetaw at ncsu.edu<mailto:hkhetaw at ncsu.edu>>
Date: Thursday, July 6, 2017 at 3:37 PM
To: "codes-ross-users at lists.mcs.anl.gov<mailto:codes-ross-users at lists.mcs.anl.gov>" <codes-ross-users at lists.mcs.anl.gov<mailto:codes-ross-users at lists.mcs.anl.gov>>
Cc: Harsh Khetawat <hkhetaw at ncsu.edu<mailto:hkhetaw at ncsu.edu>>
Subject: [codes-ross-users] Help with Storage Simulation

Hi,

I have set-up CODES on my cluster and would like to use it to simulate different storage strategies.

I wanted to use the darshan logs I have collected along with the local-storage-model in CODES to simulate I/O behavior of the application.

My simulation involves not having just one global storage LP, but having: (i) one storage LP per terminal, or (ii) one storage LP per Modelnet_group, or (iii) other strategies.
Would the local-storage-model be the correct choice for this?
Should I write my own storage model for each scenario?
How would I go about incorporating this with the fattree or dragonfly network models?

Thanks for the help.

Regards,
Harsh Khetawat

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/codes-ross-users/attachments/20170706/640f29d6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bb-model-fig.pdf
Type: application/pdf
Size: 109296 bytes
Desc: bb-model-fig.pdf
URL: <http://lists.mcs.anl.gov/pipermail/codes-ross-users/attachments/20170706/640f29d6/attachment-0001.pdf>