[Swift-devel] LQCD meeting at Fermi

[Ian Foster]

Nika:

Thanks for the summary. I am very eager to see some results for 
executions of real application problems. Did they agree to a timeline 
for that?

Ian.

Veronika Nefedova wrote:
> Hi,
>
> Yong and I have met with Xian-He and his team today to talk over their 
> current problems with the production swift code.
>
> Some of the major issues we talked about:
>
> - Sperate of concern: SwiftScript could be made to just describe the
> abstract interfaces and data flows, and the app blocks could be pushed
> into some separate specifications ( in a repository or something ), in
> which other scripting lanugages can be used (e.g. python) to specify how
> to invoke an actual application.
>
> - Dealing with absolute path:
>   LQCD uses dcache, which requires copying to/from some absolute path.
>
> - Run clean up jobs outside pbs (i.e. using the fork manager instead)
>
> - parameter problem: need to override things in tc.data, sites.xml, like
> number of nodes for MPI jobs
>   possible solution: put profile specification back in. (but we do not
> have derivations, in which we were able to put some profiles).
>   template based sites.xml and tc.data (generate the actual config files
> using some templates and user supplied values at runtime)
>
> - DB-mapper: users have an elaborate input data structures, keep it in 
> the DB, so it would be nice to have a mapper that would read the input 
> from the DB. This feature is in the works (?)
>
> -intermediate results problem -- the same as MolDyn: need to have an 
> ability to specify which file to keep and which file not.
>
> - quoting problem:
>   MPIrun does not deal correctly with "" that are passed to wrapper.sh
> I remember there was also quoting issue with condor queues.
>
> We also talked about using Falkon. But since LQCD uses dedicated 
> resources
> (600 or more nodes) and pbs queue checking time is set to around 10s, it
> is not a big issue for them to run large number of jobs.
>
> None of these except for the absolute path problem is a show-stoppers, 
> next
> we'll try to get their swiftscript running, and push some of the requests
> into 0.3 features.
>
> Yong and Nika
>
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>

-- 

   Ian Foster, Director, Computation Institute
Argonne National Laboratory & University of Chicago
Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
      Globus Alliance: www.globus.org.