[Swift-user] Re: Copy of parallel scripting paper
Allan Espinosa
yecartes at gmail.com
Tue Nov 17 03:38:42 CST 2009
the first line in the app() function should be installed in tc.data.
typically we deploy at systems level the programs we want. but this in
itself is an interesting approach in running stuff. Also if you are
load balancing the the application to difference architecture systems
on the second stage, the other site will not be utilized since it will
create a lot of failed jobs.
-Allan
2009/11/17 William Emmanuel Yu <wyu at ateneo.edu>:
>
> Hey,
>
> Here is an intermediate swift question. How do I run a program that I
> just recently compiled. I would like to do something like what is
> intended by the program below. Of course, this program does not work
> because I can't run the generated exefile.
>
> --- start script ---
>
> type sourcefile;
> type exefile;
> type outputfile;
>
> (exefile e) compile(sourcefile s) {
> app {
> gcc "-o" @filename(e) @filename(s);
> }
> }
>
> (outputfile o) run(exefile e) {
> app {
> @filename(e) stdout=@filename(o)
> }
> }
>
> exefile efile <"hello.exe">;
> sourcefile sfile <"hello.c">;
> outputfile ofile <"hello.out">;
>
> efile = compile(sfile);
> ofile = run(efile);
>
> --- end script ---
>
> Of course, the main idea is that I won't be compiling and running one
> file per script but a full directly of C source files that I want to
> compile and run.
>
> Thanks!
>
> On Mon, 2009-11-16 at 02:14 -0600, Allan Espinosa wrote:
>> I guess that works. But if you have a scheduler, you should use it.
>> we have support for pbs, cobalt and condor. any unsupported scheduler
>> can be interfaced with Globus GRAM2 or GRAM4. From my perspective,
>> the cluster sysads should make it easy for its users to submit jobs :)
>>
>> 2009/11/15 William Emmanuel Yu <wyu at ateneo.edu>:
>> >
>> > After reading the article, I think I have a better appreciate of the
>> > problems you are trying to solve. Let me try to read about SWIFT more.
>> >
>> > What is the easier way to run SWIFT on a small cluster (say 64 cores on
>> > 16 nodes)? Can I just do an NFS and passwordless-SSH thing? I think I
>> > will try to help a buddy of mine with his thesis on aquatic migrations
>> > in UP as practice.
>> >
>> > Thanks.
>> >
>> > On Thu, 2009-11-12 at 20:11 -0600, Allan Espinosa wrote:
>> >> you can try the swift tutorial designed for localhost:
>> >>
>> >> http://www.ci.uchicago.edu/swift/guides/tutorial.php
>> >>
>> >> 2009/11/12 William Emmanuel Yu <wyu at ateneo.edu>:
>> >> > let me review first if this is easier to teach ... but, if you have a
>> >> > cookbook for a quick local install then that would also be cool.
>> >> >
>> >> > On Thu, 2009-11-12 at 19:08 -0600, Allan Espinosa wrote:
>> >> >> its user and setup easy when you use a local scheduler (run from
>> >> >> localhost). I can help you on the setup on local host or via ssh.
>> >> >> Some of my colleagues have also tried using this over Amazon EC2.
>> >> >>
>> >> >> -Allan
>> >> >>
>> >> >> 2009/11/12 William Emmanuel Yu <wyu at ateneo.edu>:
>> >> >> >
>> >> >> > interesting.. but this is setup heavy but user easy right?
>> >> >> >
>> >> >> > On Thu, 2009-11-12 at 18:58 -0600, Allan Espinosa wrote:
>> >> >> >> hi william
>> >> >> >>
>> >> >> >> see attached file.
>> >> >> >>
>> >> >> >> We are using Swift (http://www.ci.uchicago.edu/swift). It has
>> >> >> >> adapters to cluster schedulers like PBS, GRAM2, GRAM4, Cobalt and
>> >> >> >> Condor. But you can use ssh and local (fork) for a start.
>> >> >> >>
>> >> >> >> We couple it to Falkon (http://dev.globus.org/wiki/Incubator/Falkon)
>> >> >> >> for faster job throughput in short-time jobs and deployments on
>> >> >> >> supercomputers.
>> >> >> >>
>> >> >> >> -Allan
>> >> >> >>
>> >> >> >> 2009/11/12 William Emmanuel Yu <wyu at ateneo.edu>:
>> >> >> >> >
>> >> >> >> > Can I get a copy of this paper? What tools are you using now?
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > -------------------------------------------------------
>> >> >> >> > William Emmanuel S. Yu (杨怀义)
>> >> >> >> > Department of Information Systems and Computer Science
>> >> >> >> > Ateneo de Manila University
>> >> >> >> > email : wyu at ateneo dot edu
>> >> >> >> > blog : http://hip2b2.yutivo.org/
>> >> >> >> > web : http://CNG.ateneo.edu/cng/wyu/
>> >> >> >> > phone : +63(2)4266001 loc. 4186
>> >> >> >> > GPG : http://CNG.ateneo.net/cng/wyu/wyy.pgp
>> >> >> >> >
>> >> >> >> > Confidentiality Issue: This message is intended only for the use of the
>> >> >> >> > addressee and may contain information that is privileged and
>> >> >> >> > confidential. If you are not the intended recipient, you are hereby
>> >> >> >> > notified that any use or dissemination of this communication is strictly
>> >> >> >> > prohibited. If you have received this communication in error, please
>> >> >> >> > notify us immediately by reply and delete this message from your system.
>> >> >> >> >
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> > --
>> >> >> > -------------------------------------------------------
>> >> >> > William Emmanuel S. Yu (杨怀义)
>> >> >> > Department of Information Systems and Computer Science
>> >> >> > Ateneo de Manila University
>> >> >> > email : wyu at ateneo dot edu
>> >> >> > blog : http://hip2b2.yutivo.org/
>> >> >> > web : http://CNG.ateneo.edu/cng/wyu/
>> >> >> > phone : +63(2)4266001 loc. 4186
>> >> >> > GPG : http://CNG.ateneo.net/cng/wyu/wyy.pgp
>> >> >> >
>> >> >> > Confidentiality Issue: This message is intended only for the use of the
>> >> >> > addressee and may contain information that is privileged and
>> >> >> > confidential. If you are not the intended recipient, you are hereby
>> >> >> > notified that any use or dissemination of this communication is strictly
>> >> >> > prohibited. If you have received this communication in error, please
>> >> >> > notify us immediately by reply and delete this message from your system.
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> > --
>> >> > -------------------------------------------------------
>> >> > William Emmanuel S. Yu (杨怀义)
>> >> > Department of Information Systems and Computer Science
>> >> > Ateneo de Manila University
>> >> > email : wyu at ateneo dot edu
>> >> > blog : http://hip2b2.yutivo.org/
>> >> > web : http://CNG.ateneo.edu/cng/wyu/
>> >> > phone : +63(2)4266001 loc. 4186
>> >> > GPG : http://CNG.ateneo.net/cng/wyu/wyy.pgp
>> >> >
>> >> > Confidentiality Issue: This message is intended only for the use of the
>> >> > addressee and may contain information that is privileged and
>> >> > confidential. If you are not the intended recipient, you are hereby
>> >> > notified that any use or dissemination of this communication is strictly
>> >> > prohibited. If you have received this communication in error, please
>> >> > notify us immediately by reply and delete this message from your system.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> > --
>> > -------------------------------------------------------
>> > William Emmanuel S. Yu (杨怀义)
>> > Department of Information Systems and Computer Science
>> > Ateneo de Manila University
>> > email : wyu at ateneo dot edu
>> > blog : http://hip2b2.yutivo.org/
>> > web : http://CNG.ateneo.edu/cng/wyu/
>> > phone : +63(2)4266001 loc. 4186
>> > GPG : http://CNG.ateneo.net/cng/wyu/wyy.pgp
>> >
>> > Confidentiality Issue: This message is intended only for the use of the
>> > addressee and may contain information that is privileged and
>> > confidential. If you are not the intended recipient, you are hereby
>> > notified that any use or dissemination of this communication is strictly
>> > prohibited. If you have received this communication in error, please
>> > notify us immediately by reply and delete this message from your system.
>> >
>> >
>>
>>
>>
> --
> -------------------------------------------------------
> William Emmanuel S. Yu (杨怀义)
> Department of Information Systems and Computer Science
> Ateneo de Manila University
> email : wyu at ateneo dot edu
> blog : http://hip2b2.yutivo.org/
> web : http://CNG.ateneo.edu/cng/wyu/
> phone : +63(2)4266001 loc. 4186
> GPG : http://CNG.ateneo.net/cng/wyu/wyy.pgp
>
> Confidentiality Issue: This message is intended only for the use of the
> addressee and may contain information that is privileged and
> confidential. If you are not the intended recipient, you are hereby
> notified that any use or dissemination of this communication is strictly
> prohibited. If you have received this communication in error, please
> notify us immediately by reply and delete this message from your system.
>
>
--
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
More information about the Swift-user
mailing list