<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Hi Jon and All,</span></div><div><br></div><div>MosaStore+Swift integration is done successfully. This is really a great news.<br></div><div><br></div><div><span>I successfully ran the setup provided by Jon with MosaStore+Swift on Surveyor (on 1, 3 , 5 and 64 nodes with different number of jobs varying from 1 to 1024).</span></div><div><br><span></span></div><div><span>Jon and Mike thank you again for your time in helping me to get this far.</span></div><div><br><span></span></div><div><span style="font-weight: bold;">A quick question:</span> The setup creates the swift working directory in GPFS but stores the intermediate files in MosaStore. Where should we actually have the swift-working directory (in GPFS / Mosa)? and Does this matter at all in case if I am going to use Mosa as an intermediate storage?
<br><span></span></div><span><br>(As a next step I will be scripting our pipeline benchmark on swift and run at large scale to get some initial performance numbers. Also I will be parallely working to get Montage and ModFTDock running on BG/P with swift-CDM and MosaStore.)<br><br>Regards<br>Emalayan<br></span><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jonathan Monette <jonmon@mcs.anl.gov><br> <b><span style="font-weight: bold;">To:</span></b> Emalayan Vairavanathan <svemalayan@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> "mosastore@googlegroups.com" <mosastore@googlegroups.com>; Justin M Wozniak <wozniak@mcs.anl.gov>; "swift-devel@ci.uchicago.edu Devel"
<swift-devel@ci.uchicago.edu> <br> <b><span style="font-weight: bold;">Sent:</span></b> Wednesday, 14 March 2012 2:29 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Swift-devel] Mosaswift update<br> </font> </div> <br><div id="yiv1518615791"><div>The working example is located in /home/jonmon/Workspace/Swift/mosaswift-test<div><br></div><div>There is a README there that provides a little bit of information.</div><div><br></div><div>Quick version to run it:</div><div>1) start your workers</div><div>2) ./setup.sh</div><div>3) ./run.sh</div><div><br></div><div>The directory final will have 5 files if it completed without error. The content of the files will look like:</div><div>Hello world</div><div>Hello from <number></div><div><br><div><div>On Mar 14, 2012, at 4:03 PM, Emalayan Vairavanathan wrote:</div><br class="yiv1518615791Apple-interchange-newline"><blockquote type="cite"><div><div
style="color:#000;background-color:#fff;font-family:times new roman, new york, times, serif;font-size:12pt;"><div><span>Hi Jon,</span></div><div><br><span></span></div><div><span>Great, Thank you very much. Could you please send me the setup ? <br></span></div><div><br><span></span></div><div><span>Thank you</span></div><div><span>Emalayan<br></span></div><div><br></div> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jonathan Monette <<a rel="nofollow" ymailto="mailto:jonmon@mcs.anl.gov" target="_blank" href="mailto:jonmon@mcs.anl.gov">jonmon@mcs.anl.gov</a>><br> <b><span style="font-weight:bold;">To:</span></b> Justin M Wozniak <<a rel="nofollow" ymailto="mailto:wozniak@mcs.anl.gov" target="_blank"
href="mailto:wozniak@mcs.anl.gov">wozniak@mcs.anl.gov</a>> <br><b><span style="font-weight:bold;">Cc:</span></b> "<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a> Devel" <<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>>; MosaStore
<<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>> <br> <b><span style="font-weight:bold;">Sent:</span></b> Wednesday, 14 March 2012 2:00 PM<br> <b><span style="font-weight:bold;">Subject:</span></b> Re: [Swift-devel] Mosaswift update<br> </font> </div> <br>So that actually fixed the issue. So we have a working example of using CDM and intermediate file storage that Emalayan can use for testing his mosa store set up.<br><br>On Mar 14, 2012, at 3:05 PM, Jonathan Monette wrote:<br><br>> Oh, thanks, didn't catch that. Trying another run right now but I think someone is using all of surveyor right now, but manual coaster service job is stuck in the queue state.<br>> <br>> <br>> On Mar 14, 2012, at 3:02 PM, Justin M Wozniak wrote:<br>> <br>>> <br>>> Ok, then it's a typo on readlinE .<br>>> <br>>> On
Wed, 14 Mar 2012, Jonathan Monette wrote:<br>>> <br>>>> I am not sure. I thought readlink returned the
absolute path of the link which is what I wanted to use. I am not sure how readlink behaves on the compute nodes. This was my first attempt at getting around the link but I do not think this is working.<br>>>> <br>>>> On Mar 14, 2012, at 2:43 PM, Justin M Wozniak wrote:<br>>>> <br>>>>> On Wed, 14 Mar 2012, Jonathan Monette wrote:<br>>>>> <br>>>>>> I say semi working because I intended the script to build up output during the stages on the swift run and the final output would show that it was stored temporarily in /tmp. This is not happening as the final output for the file contains only the last line(a Hello from <id> line). I believe this is due to how CDM using symlinks for the files that is matched, so a symlink is being overwritten instead of at the file it points to. I ran into this problem in the past(but this is documented in the user guide) and just
need verify that this is indeed the issue.<br>>>>> <br>>>>> In the past, the application unlinked the link. It looks like you are appending to the link.<br>>>>> <br>>>>> infile=`readlink $1`<br>>>>> outfile=`readline $2`<br>>>>> cat $infile >> $outfile<br>>>>> <br>>>>> Is there a case in which the readlink affects the behavior of this sequence?<br>>>>> <br>>>>> Does readline work correctly on the BG/P compute node?<br>>>>> <br>>>>> Justin<br>>>>> <br>>>>> --<br>>>>> Justin M Wozniak<br>>>> <br>>>> <br>>> <br>>> -- <br>>> Justin M Wozniak<br>> <br>> _______________________________________________<br>> Swift-devel mailing list<br>> <a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu"
target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>> <a rel="nofollow" target="_blank" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br><br>-- <br>You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>To post to this group, send email to <a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>.<br>To unsubscribe from this group, send email to mosastore+<a rel="nofollow" ymailto="mailto:unsubscribe@googlegroups.com" target="_blank" href="mailto:unsubscribe@googlegroups.com">unsubscribe@googlegroups.com</a>.<br>For more options, visit this group at <a rel="nofollow" target="_blank"
href="http://groups.google.com/group/mosastore?hl=en">http://groups.google.com/group/mosastore?hl=en</a>.<br><br><br><br> </div> </div> </div></div>_______________________________________________<br>Swift-devel mailing list<br><a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu" target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel<br></blockquote></div><br></div></div>
-- <br>
You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>
To post to this group, send email to mosastore@googlegroups.com.<br>
To unsubscribe from this group, send email to mosastore+unsubscribe@googlegroups.com.<br>
For more options, visit this group at http://groups.google.com/group/mosastore?hl=en.<br>
</div><br><br> </div> </div> </div></body></html>