From wilde at mcs.anl.gov  Thu Dec  1 07:36:17 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 1 Dec 2011 07:36:17 -0600 (CST)
Subject: [Swift-devel] How to update wwwdev web?
Message-ID: <615439439.16648.1322746577007.JavaMail.root@zimbra.anl.gov>

Hi David,

I posted an update to the new swift main page about a week ago, and it didnt show up on wwwdev. Should it have?  And whats the procedure to push updates manually?

Is there an up to date page somewhere on how to maintain the old and new webs?

Thanks,

- Mike


From davidk at ci.uchicago.edu  Thu Dec  1 08:53:29 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Thu, 1 Dec 2011 08:53:29 -0600 (CST)
Subject: [Swift-devel] How to update wwwdev web?
In-Reply-To: <615439439.16648.1322746577007.JavaMail.root@zimbra.anl.gov>
Message-ID: <548401697.7775.1322751209664.JavaMail.root@zimbra-mb2.anl.gov>

Mike,

To update content on the website, run the push_to.sh script and point it to the wwwdev directory:

./push_to.sh /ci/www/projects/swift/wwwdev/

There is no cron job for automating this, so this has to be run after every update. The script contains an explicit list of files which get copied, so if you add new files, the script should also be updated. I ran the update this morning, so your changes should be reflected on the site now.

David

----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, December 1, 2011 7:36:17 AM
> Subject: How to update wwwdev web?
> Hi David,
> 
> I posted an update to the new swift main page about a week ago, and it
> didnt show up on wwwdev. Should it have? And whats the procedure to
> push updates manually?
> 
> Is there an up to date page somewhere on how to maintain the old and
> new webs?
> 
> Thanks,
> 
> - Mike


From wilde at mcs.anl.gov  Thu Dec  1 09:44:47 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 1 Dec 2011 09:44:47 -0600 (CST)
Subject: [Swift-devel] How to update wwwdev web?
In-Reply-To: <548401697.7775.1322751209664.JavaMail.root@zimbra-mb2.anl.gov>
Message-ID: <1742975557.17238.1322754287090.JavaMail.root@zimbra.anl.gov>

Thanks, David.  But the pushed main page doesnt look right - its missing icon graphics for the 3rd and 4th bullet points. Maybe because, as you say below, they are new files just added and need to get added to the push script? Or maybe I didnt commit them?

Can you compare the main page to:
  http://www.ci.uchicago.edu/~wilde/swift/main/

and fix it up to match?  Please let me know if I forgot to commit some files.

Thanks,

- Mike

----- Original Message -----
> From: "David Kelly" <davidk at ci.uchicago.edu>
> To: "Michael Wilde" <wilde at mcs.anl.gov>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, December 1, 2011 8:53:29 AM
> Subject: Re: How to update wwwdev web?
> Mike,
> 
> To update content on the website, run the push_to.sh script and point
> it to the wwwdev directory:
> 
> ./push_to.sh /ci/www/projects/swift/wwwdev/
> 
> There is no cron job for automating this, so this has to be run after
> every update. The script contains an explicit list of files which get
> copied, so if you add new files, the script should also be updated. I
> ran the update this morning, so your changes should be reflected on
> the site now.
> 
> David
> 
> ----- Original Message -----
> > From: "Michael Wilde" <wilde at mcs.anl.gov>
> > To: "David Kelly" <davidk at ci.uchicago.edu>
> > Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > Sent: Thursday, December 1, 2011 7:36:17 AM
> > Subject: How to update wwwdev web?
> > Hi David,
> >
> > I posted an update to the new swift main page about a week ago, and
> > it
> > didnt show up on wwwdev. Should it have? And whats the procedure to
> > push updates manually?
> >
> > Is there an up to date page somewhere on how to maintain the old and
> > new webs?
> >
> > Thanks,
> >
> > - Mike

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From davidk at ci.uchicago.edu  Thu Dec  1 10:51:06 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Thu, 1 Dec 2011 10:51:06 -0600 (CST)
Subject: [Swift-devel] How to update wwwdev web?
In-Reply-To: <1742975557.17238.1322754287090.JavaMail.root@zimbra.anl.gov>
Message-ID: <562926915.8054.1322758266155.JavaMail.root@zimbra-mb2.anl.gov>


Yep, that was the case. Should be updated now.

David


----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, December 1, 2011 9:44:47 AM
> Subject: Re: How to update wwwdev web?
> Thanks, David. But the pushed main page doesnt look right - its
> missing icon graphics for the 3rd and 4th bullet points. Maybe
> because, as you say below, they are new files just added and need to
> get added to the push script? Or maybe I didnt commit them?
> 
> Can you compare the main page to:
> http://www.ci.uchicago.edu/~wilde/swift/main/
> 
> and fix it up to match? Please let me know if I forgot to commit some
> files.
> 
> Thanks,
> 
> - Mike
> 
> ----- Original Message -----
> > From: "David Kelly" <davidk at ci.uchicago.edu>
> > To: "Michael Wilde" <wilde at mcs.anl.gov>
> > Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > Sent: Thursday, December 1, 2011 8:53:29 AM
> > Subject: Re: How to update wwwdev web?
> > Mike,
> >
> > To update content on the website, run the push_to.sh script and
> > point
> > it to the wwwdev directory:
> >
> > ./push_to.sh /ci/www/projects/swift/wwwdev/
> >
> > There is no cron job for automating this, so this has to be run
> > after
> > every update. The script contains an explicit list of files which
> > get
> > copied, so if you add new files, the script should also be updated.
> > I
> > ran the update this morning, so your changes should be reflected on
> > the site now.
> >
> > David
> >
> > ----- Original Message -----
> > > From: "Michael Wilde" <wilde at mcs.anl.gov>
> > > To: "David Kelly" <davidk at ci.uchicago.edu>
> > > Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > > Sent: Thursday, December 1, 2011 7:36:17 AM
> > > Subject: How to update wwwdev web?
> > > Hi David,
> > >
> > > I posted an update to the new swift main page about a week ago,
> > > and
> > > it
> > > didnt show up on wwwdev. Should it have? And whats the procedure
> > > to
> > > push updates manually?
> > >
> > > Is there an up to date page somewhere on how to maintain the old
> > > and
> > > new webs?
> > >
> > > Thanks,
> > >
> > > - Mike
> 
> --
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory


From ketancmaheshwari at gmail.com  Thu Dec  1 16:49:12 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Thu, 1 Dec 2011 16:49:12 -0600
Subject: [Swift-devel] modftdock - all 3 stages
In-Reply-To: <1322756231.28210.YahooMailNeo@web39505.mail.mud.yahoo.com>
References: <1322424798.29302.YahooMailNeo@web39502.mail.mud.yahoo.com>
	<CAMUuvirp9JHrFDF9sHmPUbg4Ns+kps=Pe0cG=xSKOO_+Bbu6Wg@mail.gmail.com>
	<1322756231.28210.YahooMailNeo@web39505.mail.mud.yahoo.com>
Message-ID: <CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>

On Thu, Dec 1, 2011 at 10:17 AM, Emalayan Vairavanathan <
svemalayan at yahoo.com> wrote:

> Hi Ketan,
>
> I have a quick question.
>
> We are trying to configure MosaStore for an application with multiple data
> flow patterns. Currently we cannot have some of these optimizations
> together in MosaStore. So we decided have two MosaStore instances with
> optimizations in our experiments.
>
> I am curious to know whether is it possible to run swift programs with the
> multiple Mosa installations ? ( may be is there a way to specify multiple
> working directories in swift ? / May be can I dynamically change the swift
> working directory ?)
>

cc'ing swift-devel, for more input. My answer below:

Having one working directory should not be an issue with what you want to
achieve, as long as that workdir is accessible to and from the mosastore
instaces you mentioned above. All you need to do is to change the mappings
in the Swift source to the places you want swift to access/write data. So,
for instance, for one part you can use one instance of mosastore while for
the other part you can use another mosastore instance.


>
>
> Thank you
> Emalayan
>
>   ------------------------------
> *From:* Ketan Maheshwari <ketancmaheshwari at gmail.com>
> *To:* Emalayan Vairavanathan <svemalayan at yahoo.com>
> *Sent:* Monday, 28 November 2011 6:47 AM
> *Subject:* Re: modftdock - all 3 stages
>
> Emalayan,
>
> Do you have access to this svn:
> https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/modFTDock
>
> Try to check out, you will find the three stage modFTDock Swift script and
> the related binaries.
>
> Regards,
> Ketan
>
> On Sun, Nov 27, 2011 at 2:13 PM, Emalayan Vairavanathan <
> svemalayan at yahoo.com> wrote:
>
> Hi Ketan,
>
> We are trying to run  all the stages of ModFTDock with Mosastore.  Could
> you please let me know how I can run the modemerge and score stages ?
>
> In case if you already have a working installation of modftdock, we will
> be very happy to try that.
>
> Thank you
> Emalayan
>
>
>
>
>
>
> --
> Ketan
>
>
>
>
>


-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111201/cde0d5c5/attachment.html>

From wozniak at mcs.anl.gov  Thu Dec  1 17:48:15 2011
From: wozniak at mcs.anl.gov (Justin M Wozniak)
Date: Thu, 1 Dec 2011 17:48:15 -0600 (Central Standard Time)
Subject: [Swift-devel] modftdock - all 3 stages
In-Reply-To: <CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>
References: <1322424798.29302.YahooMailNeo@web39502.mail.mud.yahoo.com>
	<CAMUuvirp9JHrFDF9sHmPUbg4Ns+kps=Pe0cG=xSKOO_+Bbu6Wg@mail.gmail.com>
	<1322756231.28210.YahooMailNeo@web39505.mail.mud.yahoo.com>
	<CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>
Message-ID: <alpine.WNT.2.00.1112011745580.4204@JWOZNIAK-DESK>

On Thu, 1 Dec 2011, Ketan Maheshwari wrote:

> On Thu, Dec 1, 2011 at 10:17 AM, Emalayan Vairavanathan <
> svemalayan at yahoo.com> wrote:
>
>> I am curious to know whether is it possible to run swift programs with the
>> multiple Mosa installations ? ( may be is there a way to specify multiple
>> working directories in swift ? / May be can I dynamically change the swift
>> working directory ?)
>
> Having one working directory should not be an issue with what you want to
> achieve, as long as that workdir is accessible to and from the mosastore
> instaces you mentioned above. All you need to do is to change the mappings
> in the Swift source to the places you want swift to access/write data. So,
> for instance, for one part you can use one instance of mosastore while for
> the other part you can use another mosastore instance.

Another way to do this without modifying the Swift script would be to use 
CDM DIRECT to rename the directory in which the file is found.  However, 
this relies on the ability of _swiftwrap to soft link to files in the 
Mosa file system.

-- 
Justin M Wozniak


From ketancmaheshwari at gmail.com  Fri Dec  2 15:40:53 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Fri, 2 Dec 2011 15:40:53 -0600
Subject: [Swift-devel] Could not convert value to boolean
Message-ID: <CAMUuvirXf6aD25ojo+ELB=zJEhKOkMw-kPn0T+id0T3+i1MUMg@mail.gmail.com>

Hello,

Today, I saw this error in a simple catsn script running from engage submit
host to OSG via GWMS.

The run involved 98 tasks out of which 2 failed with the error message:

Caused by: org.globus.cog.karajan.workflow.KarajanRuntimeException: Could
not convert value to boolean: null
at org.globus.cog.karajan.util.TypeUtil.toBoolean(TypeUtil.java:127)
at org.griphyn.vdl.karajan.lib.Mark.function(Mark.java:30)
at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:62)
at
org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194)
at
org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214)
at
org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
at
org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
at
org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29)
at
org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20)
at
org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
at org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139)
at org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197)
at
org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227)
at org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:104)
at
org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

This does not happen in each run. Out of 4 such runs that I tried, it
happened in one run.

The complete log is here:
http://www.mcs.anl.gov/~ketan/catsn-20111202-1627-0e9196p8.log

In the log, I see a couple of "Broken pipe" messages but am not sure if the
two are related in any ways.

I had 50 workers running.

The coaster service log is:  http://www.mcs.anl.gov/~ketan/swift.log


Regards,
-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111202/02d2595d/attachment.html>

From wilde at mcs.anl.gov  Fri Dec  2 16:30:01 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Fri, 2 Dec 2011 16:30:01 -0600 (CST)
Subject: [Swift-devel] Could not convert value to boolean
In-Reply-To: <CAMUuvirXf6aD25ojo+ELB=zJEhKOkMw-kPn0T+id0T3+i1MUMg@mail.gmail.com>
Message-ID: <1513072700.26109.1322865001164.JavaMail.root@zimbra.anl.gov>

David and I just discussed this. Its bug 585, currently assigned to Mihael.

Its also holding up people on Sarah's team at UCI.

David tells me that if you run the simple Swift script in bug 85 100X, its almost guaranteed to happen.

Mihael, do you know the cause and/or need anything from us to fix this?  Any extra logging we should insert?

- Mike


----- Original Message -----
> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> To: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Friday, December 2, 2011 3:40:53 PM
> Subject: [Swift-devel] Could not convert value to boolean
> Hello,
> 
> 
> Today, I saw this error in a simple catsn script running from engage
> submit host to OSG via GWMS.
> 
> 
> The run involved 98 tasks out of which 2 failed with the error
> message:
> 
> 
> 
> Caused by: org.globus.cog.karajan.workflow.KarajanRuntimeException:
> Could not convert value to boolean: null
> at org.globus.cog.karajan.util.TypeUtil.toBoolean(TypeUtil.java:127)
> at org.griphyn.vdl.karajan.lib.Mark.function(Mark.java:30)
> at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:62)
> at
> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194)
> at
> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214)
> at
> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> at
> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
> at
> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29)
> at
> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20)
> at
> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
> at
> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139)
> at
> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197)
> at
> org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227)
> at
> org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:104)
> at
> org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> 
> 
> This does not happen in each run. Out of 4 such runs that I tried, it
> happened in one run.
> 
> 
> 
> The complete log is here:
> http://www.mcs.anl.gov/~ketan/catsn-20111202-1627-0e9196p8.log
> 
> 
> In the log, I see a couple of "Broken pipe" messages but am not sure
> if the two are related in any ways.
> 
> 
> I had 50 workers running.
> 
> 
> The coaster service log is: http://www.mcs.anl.gov/~ketan/swift.log
> 
> 
> 
> 
> Regards, --
> Ketan
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From hategan at mcs.anl.gov  Fri Dec  2 16:44:35 2011
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Fri, 02 Dec 2011 14:44:35 -0800
Subject: [Swift-devel] Could not convert value to boolean
In-Reply-To: <1513072700.26109.1322865001164.JavaMail.root@zimbra.anl.gov>
References: <1513072700.26109.1322865001164.JavaMail.root@zimbra.anl.gov>
Message-ID: <1322865875.14513.4.camel@blabla>

On Fri, 2011-12-02 at 16:30 -0600, Michael Wilde wrote:
> David and I just discussed this. Its bug 585, currently assigned to Mihael.
> 
> Its also holding up people on Sarah's team at UCI.
> 
> David tells me that if you run the simple Swift script in bug 85 100X,
> its almost guaranteed to happen.

That's good to know.

> 
> Mihael, do you know the cause and/or need anything from us to fix
> this?  Any extra logging we should insert?

Not really. It's a weird one. And I was trying to focus on what seemed
like more important problems.

I'll have a month of vacation after next week, and that seems like a
good time to fix all the weird things that have accumulated.



From svemalayan at yahoo.com  Fri Dec  2 13:20:44 2011
From: svemalayan at yahoo.com (Emalayan Vairavanathan)
Date: Fri, 2 Dec 2011 11:20:44 -0800 (PST)
Subject: [Swift-devel] modftdock - all 3 stages
In-Reply-To: <CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>
References: <1322424798.29302.YahooMailNeo@web39502.mail.mud.yahoo.com>
	<CAMUuvirp9JHrFDF9sHmPUbg4Ns+kps=Pe0cG=xSKOO_+Bbu6Wg@mail.gmail.com>
	<1322756231.28210.YahooMailNeo@web39505.mail.mud.yahoo.com>
	<CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>
Message-ID: <1322853644.99275.YahooMailNeo@web39506.mail.mud.yahoo.com>

Hi Ketan,

Thank you very much for the reply. I will ask you if I have more questions.


Regards
Emalayan



________________________________
 From: Ketan Maheshwari <ketancmaheshwari at gmail.com>
To: Emalayan Vairavanathan <svemalayan at yahoo.com> 
Cc: Swift Devel <swift-devel at ci.uchicago.edu> 
Sent: Thursday, 1 December 2011 2:49 PM
Subject: Re: modftdock - all 3 stages
 

On Thu, Dec 1, 2011 at 10:17 AM, Emalayan Vairavanathan <svemalayan at yahoo.com> wrote:

Hi Ketan,
>
>
>I have a quick question. 
>
>
>
>We are trying to configure MosaStore for an application with multiple data flow patterns. Currently we cannot have some of these optimizations together in MosaStore. So we decided have two MosaStore instances with optimizations in our experiments.
>
>
>I am curious to know whether is it possible to run swift programs with the multiple Mosa installations ? ( may be is there a way to specify multiple working directories in swift ? / May be can I dynamically change the swift working directory ?)

cc'ing swift-devel, for more input. My answer below:

Having one working directory should not be an issue with what you want to achieve, as long as that workdir is accessible to and from the mosastore instaces you mentioned above. All you need to do is to change the mappings in the Swift source to the places you want swift to access/write data. So, for instance, for one part you can use one instance of mosastore while for the other part you can use another mosastore instance.
?

>
>
>
>Thank you
>Emalayan
>
>
>
>________________________________
> From: Ketan Maheshwari <ketancmaheshwari at gmail.com>
>To: Emalayan Vairavanathan <svemalayan at yahoo.com> 
>Sent: Monday, 28 November 2011 6:47 AM
>Subject: Re: modftdock - all 3 stages
> 
>
>Emalayan,
>
>
>Do you have access to this svn:?https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/modFTDock
>
>
>Try to check out, you will find the three stage modFTDock Swift script and the related binaries.
>
>
>Regards,
>Ketan
>
>
>On Sun, Nov 27, 2011 at 2:13 PM, Emalayan Vairavanathan <svemalayan at yahoo.com> wrote:
>
>Hi Ketan,
>>
>>
>>We are trying to run? all the stages of ModFTDock with Mosastore.? Could you please let me know how I can run the modemerge and score stages ?
>>
>>
>>In case if you already have a working installation of modftdock, we will be very happy to try that.
>>
>>
>>
>>Thank you
>>Emalayan
>>
>>
>>
>>? 
>
>
>
>-- 
>Ketan
>
>
>
>
>


-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111202/43b9864e/attachment.html>

From svemalayan at yahoo.com  Fri Dec  2 13:47:16 2011
From: svemalayan at yahoo.com (Emalayan Vairavanathan)
Date: Fri, 2 Dec 2011 11:47:16 -0800 (PST)
Subject: [Swift-devel] modftdock - all 3 stages
In-Reply-To: <alpine.WNT.2.00.1112011745580.4204@JWOZNIAK-DESK>
References: <1322424798.29302.YahooMailNeo@web39502.mail.mud.yahoo.com>
	<CAMUuvirp9JHrFDF9sHmPUbg4Ns+kps=Pe0cG=xSKOO_+Bbu6Wg@mail.gmail.com>
	<1322756231.28210.YahooMailNeo@web39505.mail.mud.yahoo.com>
	<CAMUuvioDhVZjFJbLFYPsM9V+R2SyoJoXC=F8nDuf=p4Ke6LFnQ@mail.gmail.com>
	<alpine.WNT.2.00.1112011745580.4204@JWOZNIAK-DESK>
Message-ID: <1322855236.52572.YahooMailNeo@web39505.mail.mud.yahoo.com>

Thank you Justin, but this approach wont work for now since current MosaStore does not support symbolic links. May be I can try this in future. 


Regards
Emalayan


________________________________
 From: Justin M Wozniak <wozniak at mcs.anl.gov>
To: Ketan Maheshwari <ketancmaheshwari at gmail.com> 
Cc: Emalayan Vairavanathan <svemalayan at yahoo.com>; Swift Devel <swift-devel at ci.uchicago.edu> 
Sent: Thursday, 1 December 2011 3:48 PM
Subject: Re: [Swift-devel] modftdock - all 3 stages
 
On Thu, 1 Dec 2011, Ketan Maheshwari wrote:

> On Thu, Dec 1, 2011 at 10:17 AM, Emalayan Vairavanathan <
> svemalayan at yahoo.com> wrote:
> 
>> I am curious to know whether is it possible to run swift programs with the
>> multiple Mosa installations ? ( may be is there a way to specify multiple
>> working directories in swift ? / May be can I dynamically change the swift
>> working directory ?)
> 
> Having one working directory should not be an issue with what you want to
> achieve, as long as that workdir is accessible to and from the mosastore
> instaces you mentioned above. All you need to do is to change the mappings
> in the Swift source to the places you want swift to access/write data. So,
> for instance, for one part you can use one instance of mosastore while for
> the other part you can use another mosastore instance.

Another way to do this without modifying the Swift script would be to use CDM DIRECT to rename the directory in which the file is found.? However, this relies on the ability of _swiftwrap to soft link to files in the Mosa file system.

-- Justin M Wozniak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111202/40354a89/attachment.html>

From skenny at uci.edu  Fri Dec  2 17:40:52 2011
From: skenny at uci.edu (Sarah Kenny)
Date: Fri, 2 Dec 2011 15:40:52 -0800
Subject: [Swift-devel] Could not convert value to boolean
In-Reply-To: <1322865875.14513.4.camel@blabla>
References: <1513072700.26109.1322865001164.JavaMail.root@zimbra.anl.gov>
	<1322865875.14513.4.camel@blabla>
Message-ID: <CAE50RW27K+r1DyG7A1NSGb2iaZoXZUQ-jm9=9UGHf1owjsHmZQ@mail.gmail.com>

if it helps at all, we were getting the error on ranger yesterday when they
were apparently having trouble with their filesystem. we were then able to
get things running once it was back up...but i do see it frequently.

On Fri, Dec 2, 2011 at 2:44 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:

> On Fri, 2011-12-02 at 16:30 -0600, Michael Wilde wrote:
> > David and I just discussed this. Its bug 585, currently assigned to
> Mihael.
> >
> > Its also holding up people on Sarah's team at UCI.
> >
> > David tells me that if you run the simple Swift script in bug 85 100X,
> > its almost guaranteed to happen.
>
> That's good to know.
>
> >
> > Mihael, do you know the cause and/or need anything from us to fix
> > this?  Any extra logging we should insert?
>
> Not really. It's a weird one. And I was trying to focus on what seemed
> like more important problems.
>
> I'll have a month of vacation after next week, and that seems like a
> good time to fix all the weird things that have accumulated.
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>



-- 
Sarah Kenny
Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
University of California Irvine, Dept. of Neurology ~ 773-818-8300
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111202/eba7e5da/attachment.html>

From davidk at ci.uchicago.edu  Tue Dec  6 13:52:09 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Tue, 6 Dec 2011 13:52:09 -0600 (CST)
Subject: [Swift-devel] Swift Licensing
In-Reply-To: <1238717116.16123.1323200627286.JavaMail.root@zimbra-mb2.anl.gov>
Message-ID: <1481066739.16139.1323201129384.JavaMail.root@zimbra-mb2.anl.gov>

Hello,

I added a few pages to wwwdev relating to Swift licensing:

http://www.ci.uchicago.edu/swift/wwwdev/downloads/index.php
http://www.ci.uchicago.edu/swift/wwwdev/docs/license.php
http://www.ci.uchicago.edu/swift/wwwdev/docs/tracking_policy.php

I can add the standard Apache licensing info to the head of each source file (.java files first, but possibly shell scripts and others later). This is the same style that is used in the Globus Toolkit source code:

---
Copyright 2011 University of Chicago

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

Mike thought I should discuss on swift-devel before submitting the changes. Thoughts?


From ketancmaheshwari at gmail.com  Tue Dec  6 20:27:15 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Tue, 6 Dec 2011 20:27:15 -0600
Subject: [Swift-devel] buffer allocation release exception
Message-ID: <CAMUuvipWFEKxDX7BxHDmD8=Dp9nnOd4CdXAn+GLKwytLa2QhDA@mail.gmail.com>

Hi,

I am seeing this error message with coasters based provider-staging script:

java.lang.Exception
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.Buffers.free(Buffers.java:152)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.ReadBuffer.deallocateBuffers(ReadBuffer.java:117)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.ReadBuffer.close(ReadBuffer.java:123)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.NIOChannelReadBuffer.close(NIOChannelReadBuffer.java:57)
        at
org.globus.cog.abstraction.impl.file.coaster.handlers.providers.LocalIOProvider$Reader.closeBuffer(LocalIOProvider.java:191)
        at
org.globus.cog.abstraction.impl.file.coaster.handlers.providers.LocalIOProvider$Reader.close(LocalIOProvider.java:204)
        at
org.globus.cog.abstraction.impl.file.coaster.handlers.providers.LocalIOProvider$Reader.error(LocalIOProvider.java:200)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.ReadBuffer.error(ReadBuffer.java:93)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.NIOChannelReadBuffer.doStuff(NIOChannelReadBuffer.java:52)
        at
org.globus.cog.abstraction.impl.file.coaster.buffers.Buffers.run(Buffers.java:188)
I trying to release buffer allocation twice

This is swift 0.93.1, I am also seeing the following message:

Command(616, HEARTBEAT): handling reply timeout;
sendReqTime=111206-201635.533, sendTime=691231-180000.000,
now=111206-201835.619, channel=SC-null

The log is:
http://ci.uchicago.edu/~ketan/postproc-20111206-1710-elho0y40.log

Filed as bug 654: https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=654

Regards,
-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111206/9151f9b1/attachment.html>

From wilde at mcs.anl.gov  Thu Dec  8 08:44:16 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 8 Dec 2011 08:44:16 -0600 (CST)
Subject: [Swift-devel] Web site cutover
Message-ID: <1131432397.45269.1323355456030.JavaMail.root@zimbra.anl.gov>

David, can you plan on doing the cutover tomorrow morning?

Can we do the cutover by renaming directories?

- Mike


From davidk at ci.uchicago.edu  Thu Dec  8 09:36:51 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Thu, 8 Dec 2011 09:36:51 -0600 (CST)
Subject: [Swift-devel] Web site cutover
In-Reply-To: <1131432397.45269.1323355456030.JavaMail.root@zimbra.anl.gov>
Message-ID: <1532637489.19783.1323358611105.JavaMail.root@zimbra-mb2.anl.gov>


Yep, it should just be a matter of moving directories around a bit.

----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, December 8, 2011 8:44:16 AM
> Subject: Web site cutover
> David, can you plan on doing the cutover tomorrow morning?
> 
> Can we do the cutover by renaming directories?
> 
> - Mike


From wilde at mcs.anl.gov  Fri Dec  9 16:15:22 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Fri, 9 Dec 2011 16:15:22 -0600 (CST)
Subject: [Swift-devel] New Swift Web site is live
Message-ID: <1698351710.51974.1323468922639.JavaMail.root@zimbra.anl.gov>

The new Swift web went live today around 4PM central.

We have a backup of the old site in case we missed anything, but that is not web accessible at the moment. (We may make it accessible for reference, shortly).

Thanks, everyone for comments and suggestions. Many of the comments were acted on, and the rest have been saved for consideration as we tune up the site.

Please send any new web site problems to swift-devel.

- Mike


From ketancmaheshwari at gmail.com  Tue Dec 13 20:04:32 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Tue, 13 Dec 2011 20:04:32 -0600
Subject: [Swift-devel] location of worker logs on automatic coasters
Message-ID: <CAMUuvirVF6e+R1g=tFpeDO4uFUWMfC9C3EuUCYgVLFQ47BYyKA@mail.gmail.com>

Hi,

Quick question on setting up the location for the worker logs when using
automatic coasters. Currently they are sent to ~/.globus/coasters directory
which is taking too much space on /home. Any clues as to how could I
redirect the worker logs to /scratch or /tmp?

-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111213/b6344e30/attachment.html>

From hategan at mcs.anl.gov  Tue Dec 13 20:10:38 2011
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Tue, 13 Dec 2011 18:10:38 -0800
Subject: [Swift-devel] location of worker logs on automatic coasters
In-Reply-To: <CAMUuvirVF6e+R1g=tFpeDO4uFUWMfC9C3EuUCYgVLFQ47BYyKA@mail.gmail.com>
References: <CAMUuvirVF6e+R1g=tFpeDO4uFUWMfC9C3EuUCYgVLFQ47BYyKA@mail.gmail.com>
Message-ID: <1323828638.10169.1.camel@blabla>

globus:workerLoggingDirectory profile.

On Tue, 2011-12-13 at 20:04 -0600, Ketan Maheshwari wrote:
> Hi,
> 
> 
> Quick question on setting up the location for the worker logs when
> using automatic coasters. Currently they are sent to
> ~/.globus/coasters directory which is taking too much space on /home.
> Any clues as to how could I redirect the worker logs to /scratch
> or /tmp?
> 
> 
> -- 
> Ketan
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel




From iraicu at cs.iit.edu  Thu Dec 15 07:45:48 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Thu, 15 Dec 2011 07:45:48 -0600
Subject: [Swift-devel] CFP: ACM Symposium on High-Performance Parallel and
 Distributed Computing (HPDC'12)
Message-ID: <4EE9FA0C.40006@cs.iit.edu>

                       **** CALL FOR PAPERS ****

                 The 21st International ACM Symposium on
          High-Performance Parallel and Distributed Computing
                             (HPDC'12)

           Delft University of Technology, Delft, the Netherlands

                          June 18-22, 2012

                     http://www.hpdc.org/2012


The ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC)
is the premier annual conference on the design, the implementation, the evaluation, and
the use of parallel and distributed systems for high-end computing. HPDC'12 will take place
in Delft, the Netherlands, a historical, picturesque city that is less than one hour away
from Amsterdam-Schiphol airport. The conference will be held on June 20-22 (Wednesday to
Friday), with affiliated workshops taking place on June 18-19 (Monday and Tuesday).


**** SUBMISSION DEADLINES ****
Abstracts:    16 January 2012
Papers:       23 January 2012 (No extensions!)


**** HPDC'12 GENERAL CHAIR ****
Dick Epema, Delft University of Technology, Delft, the Netherlands


**** HPDC'12 PROGRAM CO-CHAIRS ****
Thilo Kielmann, Vrije Universiteit, Amsterdam, the Netherlands
Matei Ripeanu, The University of British Columbia, Vancouver, Canada


**** HPDC'12 WORKSHOPS CHAIR ****
Alexandru Iosup, Delft University of Technology, Delft, the Netherlands


**** SCOPE AND TOPICS ****
Submissions are welcomed on all forms of high-performance parallel and distributed computing,
including but not limited to clusters, clouds, grids, utility computing, data-intensive
computing, and massively multicore systems. Submissions that explore solutions to estimate
and reduce the energy footprint of such systems are particularly encouraged. All papers
will be evaluated for their originality, potential impact, correctness, quality of
presentation, appropriate presentation of related work, and relevance to the conference,
with a strong preference for rigorous results obtained in operational parallel and
distributed systems.

The topics of interest of the conference include, but are not limited to, the following,
in the context of high-performance parallel and distributed computing:

    - Systems, networks, and architectures for high-end computing
    - Massively multicore systems
    - Virtualization of machines, networks, and storage
    - Programming languages and environments
    - I/O, storage systems, and data management
    - Resource management, energy and cost minimizations
    - Performance modeling and analysis
    - Fault tolerance, reliability, and availability
    - Data-intensive computing
    - Applications of parallel and distributed computing


**** PAPER SUBMISSION GUIDELINES ****
Authors are invited to submit technical papers of at most 12 pages in PDF format, including
figures and references. Papers should be formatted in the ACM Proceedings Style and
submitted via the conference web site. No changes to the margins, spacing, or font sizes as
specified by the style file are allowed. Accepted papers will appear in the conference
proceedings, and will be incorporated into the ACM Digital Library. A limited number of
papers will be accepted as posters.

Papers must be self-contained and provide the technical substance required for the program
committee to evaluate their contributions. Submitted papers must be original work that has
not appeared in and is not under consideration for another conference or a journal. See the
ACM Prior Publication Policy for more details.


**** IMPORTANT DATES ****
Abstracts Due:				16 January 2012
Papers Due:				23 January 2012 (No extensions!)
Reviews Released to Authors:		8 March 2012
Author Rebuttals Due:			12 March 2012
Author Notifications:			19 March 2012
Final Papers Due:			16 April 2012
Conference Dates:			18-22 June 2012

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================




From iraicu at cs.iit.edu  Fri Dec 16 16:54:14 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Fri, 16 Dec 2011 16:54:14 -0600
Subject: [Swift-devel] CFP: Workshops at ACM HPDC 2012
Message-ID: <4EEBCC16.9040300@cs.iit.edu>

**** WORKSHOPS at ACM HPDC 2012 ****
http://www.hpdc.org/2012/workshops/

The ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC)
is the premier annual conference on the design, the implementation, the evaluation, and the
use of parallel and distributed systems for high-end computing. HPDC'12 will take place in
Delft, the Netherlands, a historical, picturesque city that is less than one hour away
from Amsterdam-Schiphol airport. The conference will be held on June 20-22 (Wednesday to
Friday), with affiliated workshops taking place on June 18-19 (Monday and Tuesday).

**** HPDC'12 WORKSHOPS CHAIR ****
Alexandru Iosup, Delft University of Technology, Delft, the Netherlands

The workshops include:
    - Astro-HPC: Workshop on High-Performance Computing for Astronomy
      http://www.pds.ewi.tudelft.nl/~varbanescu/AstroHPC/index.html
    - ECMLS2012: 3rd Int'l Emerging Computational Methods for the Life Sciences Workshop
      http://salsahpc.indiana.edu/ECMLS2012
    - SocMP'12: First Workshop on Social Media Processing
      http://www.pds.ewi.tudelft.nl/~iosup/socmp2012
    - ScienceCloud: 3rd Workshop on Scientific Cloud Computing
      http://ceng.usc.edu/~simmhan/ScienceCloud2012
    - DIDC: Fifth Int'l Workshop on Data-Intensive Distributed Computing
      http://www.cse.buffalo.edu/faculty/tkosar/didc2012
    - ISDP: In-Situ Data Processing technologies
      http://users.nccs.gov/~sklasky/isdp
    - LSAP2012: 4th Workshop on Large-scale Systems and Applications Performance
      http://www.lsap2012.org/
    - MapReduce'12: The Third Int'l Workshop on MapReduce and its Applications
      http://graal.ens-lyon.fr/mapreduce
    - VTDC-2012: 6th Int'l Workshop on Virtualization Technologies in Distributed Computing
      http://www.grid-appliance.org/wiki/index.php/VTDC12



-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================




From iraicu at cs.iit.edu  Tue Dec 20 18:21:16 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Tue, 20 Dec 2011 18:21:16 -0600
Subject: [Swift-devel] CFP: 8th IEEE Int. Conf. on eScience 2012,
	Chicago IL USA
Message-ID: <4EF1267C.8070101@cs.iit.edu>

CALL FOR PAPERS

8th IEEE International Conference on eScience
http://www.ci.uchicago.edu/escience2012/
October 8-12, 2012
Chicago, IL, USA

Researchers in all disciplines are increasingly adopting digital tools, techniques and practices, often in
communities and projects that span disciplines, laboratories, organizations, and national boundaries.
The eScience 2012 conference is designed to bring together leading international and interdisciplinary
research communities, developers, and users of eScience applications and enabling IT technologies. The
conference serves as a forum to present the results of the latest applications research and product/tool
developments and to highlight related activities from around the world. Also, we are now entering the
second decade of eScience and the 2012 conference gives an opportunity to take stock of what has
been achieved so far and look forward to the challenges and opportunities the next decade will bring.

A special emphasis of the 2012 conference is on advances in the application of technology in a
particular discipline. Accordingly, significant advances in applications science and technology will be
considered as important as the development of new technologies themselves. Further, we welcome
contributions in educational activities under any of these disciplines.

As a result, the conference will be structured around two e-Science tracks:

	? eScience Algorithms and Applications
		? eScience application areas, including:
			? Physical sciences
			? Biomedical sciences
			? Social sciences and humanities
		? Data-oriented approaches and applications
		? Compute-oriented approaches and applications
		? Extreme scale approaches and applications
	? Cyberinfrastructure to support eScience
		? Novel hardware
		? Novel uses of production infrastructure
		? Software and services
		? Tools
The conference proceedings will be published by the IEEE Computer Society Press, USA and will be made
available online through the IEEE Digital Library. Selected papers will be invited to submit extended
versions to a special issue of the Future Generation Computer Systems (FGCS)journal.

SUBMISSION PROCESS
Authors are invited to submit papers with unpublished, original work of not more than 8 pages of
double column text using single spaced 10 point size on 8.5 x 11 inch pages, as per IEEE 8.5 x 11
manuscript guidelines. (Up to 2 additional pages may be purchased for US$150/page)

Templates are available
from http://www.ieee.org/conferences_events/conferences/publishing/templates.html.

Authors should submit a PDF file that will print on a PostScript printer
to https://www.easychair.org/conferences/?conf=escience2012

(Note that paper submitters also must submit an abstract in advance of the paper deadline. This should
be done through the same site where papers are submitted.)

It is a requirement that at least one author of each accepted paper attend the conference.

IMPORTANT DATES

Abstract submission (required): 4 July 2012
Paper submission: 11 July 2012
Paper author notification: 22 August 2012
Camera-ready papers due: 10 September 2012
Conference: 8-12 October 2012

CONFERENCE ORGANIZATION

General Chair
	? Ian Foster, University of Chicago&  Argonne National Laboratory, USA
Program Co-Chairs
	? Daniel S. Katz, University of Chicago&  Argonne National Laboratory, USA
	? Heinz Stockinger, SIB Swiss Institute of Bioinformatics, Switzerland
Program Vice Co-Chairs
	? eScience Algorithms and Applications Track
		? David Abramson, Monash University, Australia
		? Gabrielle Allen, Louisiana State University, USA
	? Cyberinfrastructure to support eScience Track
		? Rosa M. Badia, Barcelona Supercomputing Center / CSIC, Spain
		? Geoffrey Fox, Indiana University, USA
Early Results and Works-in-Progress Posters Chair
	? Roger Barga, Microsoft, USA
Workshops Chair
	? Ruth Pordes, FNAL, USA
Sponsorship Chair
	? Charlie Catlett, Argonne National Laboratory, USA
Conference Manager and Finance Chair
	? Julie Wulf-Knoerzer, University of Chicago&  Argonne National Laboratory, USA
Publicity Chairs
	? Kento Aida, National Institute of Informatics, Japan
	? Ioan Raicu, Illinois Institute of Technology, USA
	? David Wallom, Oxford e-Research Centre, UK
Local Organizing Committee
	? Ninfa Mayorga, University of Chicago, USA
	? Evelyn Rayburn, University of Chicago, USA
	? Lynn Valentini, Argonne National Laboratory, USA
Program Committee
	? eScience Algorithms and Applications Track
		? Srinivas Aluru, Iowa State University, USA
		? Ashiq Anjum, University of Derby, UK
		? David A. Bader, Georgia Institute of Technology, USA
		? Jon Blower, University of Reading, UK
		? Paul Bonnington, Monash University, Australia
		? Simon Cox, University of Southampton, UK
		? David De Roure, Oxford e-Research Centre, UK
		? George Djorgovski, California Institute of Technology, USA
		? Anshu Dubey, University of Chicago&  Argonne National Laboratory, USA
		? Yuri Estrin, Monash University, Australia
		? Dan Fay, Microsoft, USA
		? Jeremy Frey, University of Southampton, UK
		? Wolfgang Gentzsch, HPC Consultant, Germany
		? Lutz Gross, The University of Queensland, Austrialia
		? Sverker Holmgren, Uppsala University, Sweden
		? Bill Howe, University of Washington, USA
		? Marina Jirotka, University of Oxford, UK
		? Timoleon Kipouros, University of Cambridge, UK
		? Kerstin Kleese van Dam, Pacific Northwest National Laboratory, USA
		? Arun S. Konagurthu, Monash University, Australia
		? Peter Kunszt, SystemsX.ch, Switzerland
		? Alexey Lastovetsky, University College Dublin, Ireland
		? Andrew Lewis, Griffith University, Australia
		? Sergio Maffioletti, University of Zurich, Switzerland
		? Amitava Majumdar, San Diego Supercomputer Center, University of California at San Diego,
USA
		? Rui Mao, Shenzhen University, China
		? Madhav V. Marathe, Virginia Tech, USA
		? Maryann Martone, University of California at San Diego, USA
		? Louis Moresi, Monash University, Australia
		? Riccardo Murri, University of Zurich, Switzerland
		? Silvia D. Olabarriaga, Academic Medical Center of the University of Amsterdam, Netherlands
		? Enrique S. Quintana-Ort?, Universidad Jaume I, Spain
		? Abani Patra, University at Buffalo, USA
		? Rob Pennington, NSF, USA
		? Andrew Perry, Monash University, Australia
		? Beth Plale, Indiana University, USA
		? Michael Resch, University of Stuttgart, Germany
		? Adrian Sandu, Virginia Tech, USA
		? Mark Savill, Cranfield University, UK
		? Erik Schnetter, Perimeter Institute for Theoretical Physics, Canada
		? Edward Seidel, Louisiana State University, USA
		? Suzanne M. Shontz, The Pennsylvania State University, USA
		? David Skinner, Lawrence Berkeley National Laboratory, USA
		? Alan Sussman, University of Maryland, USA
		? Alex Szalay, Johns Hopkins University, USA
		? Domenico Talia, ICAR-CNR&  University of Calabria, Italy
		? Jian Tao, Louisiana State University, USA
		? David Wallom, Oxford e-Research Centre, UK
		? Shaowen Wang, University of Illinois at Urbana-Champaign, USA
		? Michael Wilde, Argonne National Laboratory&  University of Chicago, USA
		? Nancy Wilkins-Diehr, San Diego Supercomputer Center, University of California at San
Diego, USA
		? Wu Zhang, Shanghai University, China
		? Yunquan Zhang, Chinese Academy of Sciences, China
	? Cyberinfrastructure to support eScience Track
		? Deb Agarwal, Lawrence Berkeley National Laboratory, USA
		? Ilkay Altintas, San Diego Supercomputer Center, University of California at San Diego, USA
		? Henri Bal, Vrije Universiteit, Netherlands
		? Roger Barga, Microsoft, USA
		? Martin Berzins, University of Utah, USA
		? John Brooke, University of Manchester, UK
		? Thomas Fahringer, University of Innsbruck, Austria
		? Gilles Fedak, INRIA, France
		? Jos? A. B. Fortes, University of Florida, USA
		? Yolanda Gil, ISI/USC, USA
		? Madhusudhan Govindaraju, SUNY Binghamton, USA
		? Thomas Hacker, Purdue University, USA
		? Ken Hawick, Massey University, New Zealand
		? Marty Humphrey, University of Virginia, USA
		? Hai Jin, Huazhong University of Science and Technology, China
		? Thilo Kielmann, Vrije Universiteit, Netherlands
		? Scott Klasky, Oak Ridge National Laboratory, USA
		? Isao Kojima, AIST, Japan
		? Tevfik Kosar, University at Buffalo, USA
		? Dieter Kranzlmueller, LMU&  LRZ Munich, Germany
		? Erwin Laure, KTH, Sweden
		? Jysoo Lee, KISTI, Korea
		? Li Xiaoming, Peking University, China
		? Bertram Lud?scher, University of California, Davis, USA
		? Andrew Lumsdaine, Indiana University, USA
		? Tanu Malik, University of Chicago, USA
		? Satoshi Matsuoka, Tokyo Institute of Technology, Japan
		? Reagan Moore, University of North Carolina at Chapel Hill, USA
		? Shirley Moore, University of Kentucky, USA
		? Steven Newhouse, EGI, Netherlands
		? Dhabaleswar K. (DK) Panda, The Ohio State University, USA
		? Manish Parashar, Rutgers University, USA
		? Ron Perrott, University of Oxford, UK
		? Depei Qian, Beihang University, China
		? Judy Qui, Indiana University, USA
		? Ioan Raicu, Illinois Institute of Technology, USA
		? Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory, USA
		? Omer Rana, Cardiff University, UK
		? Paul Roe, Queensland University of Technology, Australia
		? Bruno Schulze, LNCC, Brazil
		? Marc Snir, Argonne National Laboratory&  University of Illinois at Urbana-Champaign, USA
		? Xian-He Sun, Illinois Institute of Technology, USA
		? Yoshio Tanaka, AIST, Japan
		? Michela Taufer, University of Delaware, USA
		? Kerry Taylor, CSIRO, Australia
		? Douglas Thain, University of Notre Dame, USA
		? Paul Watson, Newcastle University, UK
		? Jun Zhao, University of Oxford, UK

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================




From iraicu at cs.iit.edu  Tue Dec 20 18:28:08 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Tue, 20 Dec 2011 18:28:08 -0600
Subject: [Swift-devel] Call for Workshops at IEEE eScience 2012
Message-ID: <4EF12818.4000204@cs.iit.edu>

CALL FOR WORKSHOPS

8th IEEE International Conference on eScience
http://www.ci.uchicago.edu/escience2012/
October 8-12, 2012
Chicago, IL, USA

The 8th IEEE eScience conference (e-Science 2012), sponsored by the IEEE Computer Society's Technical
Committee for Scalable Computing (TCSC), will be held in Chicago Illinois from 8-12th October 2012.
The eScience 2011 conference is designed to bring together leading international and interdisciplinary
research communities, developers, and users of eScience applications and enabling IT technologies.

Multiple e-Science 2012 Workshops will be held on Monday and Tuesday, 8th and 9th October, co-
located with the main conference.

Workshops are an important part of the conference in providing opportunity for researchers to present
their work in a more focused way than the conference itself and to have discussion of particular topics
of interest to the community. We cordially invite you to submit workshop proposals on any eScience
related topic to the Workshop Chair.

To help those interested know their purpose and scope, workshop proposals should include:
	? A description of the workshop, its focus, goals, and outcome
	? A draft call for papers
	? Names and affiliations of the organizers and tentative composition of the committees
	? Expected numbers of submissions and accepted papers
	? Prior history of this workshop, if any. Please include: number of submissions, number of
accepted papers, and attendee count.

Workshop organizers are responsible for establishing a program committee, collecting and evaluating
submissions, notifying authors of acceptance or rejection in due time, ensuring a transparent and fair
selection process, organizing selected papers into sessions, and assigning session chairs. Proposals will
be selected that show clear focus and objectives in areas of emerging or developing interest guaranteed
to generate significant interest in the community.

Once accepted, the workshop should establish its own paper submission system. For each paper
selected for publication, an author must be registered for eScience 2012. Each paper must be presented
in person by at least one of the authors. It is expected that the proceedings of the eScience 2012
workshops will be published by the IEEE Computer Society Press, USA and will be made available online
through the IEEE Digital Library.

SUBMISSION PROCESS

Workshop proposals should be emailed toescience2012-workshops at fnal.gov

IMPORTANT DATES

Workshop submissions due: 23rd January 2012
Notification of workshop acceptance: 6th February 2012

While it is up to the workshop organizers to work with the authors of any papers to be published from
the workshop presenters, it should be noted that information about these will be needed by 27th
August 2012 and final camera ready papers are needed by 17th September 2012.

Workshops: 8-9 October 2012

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================




From skenny at uci.edu  Wed Dec 21 06:57:32 2011
From: skenny at uci.edu (Sarah Kenny)
Date: Wed, 21 Dec 2011 04:57:32 -0800
Subject: [Swift-devel] swift on ranger
Message-ID: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>

getting this when submitting to ranger with both the latest and our
previous version of swift (swift-r5259 cog-r3313)

Final status:  time: Wed, 21 Dec 2011 04:49:15 -0800  Finished
successfully:100
The following warnings have occurred:
1. org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
Cannot submit job: Could not submit job (qsub reported an exit code of 1).
--------------------------------------------------------------------------
Welcome to TACC's Ranger System, an NSF XD Resource
---------------------------------------------------------------------------->
Checking that you specified -V...--> Checking that you specified a time
limit...--> Checking that you specified a queue...--> Setting project...-->
Checking that you specified a parallel environment...--> Checking that you
specified a valid parallel environment name...--> Checking that the minimum
and maximum PE counts are the same...--> Checking that the number of PEs
requested is valid...------------------> Rejecting job
<------------------Your slot (or core) request is not a multiple of
16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
16.-----------------------------------------------------
Unable to run job: JSV rejected job.Exiting.

        at
org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
        at
org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
        at
org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
        at
org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
Caused by:
org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could
not submit job (qsub reported an exit code of 1).
--------------------------------------------------------------------------
Welcome to TACC's Ranger System, an NSF XD Resource
---------------------------------------------------------------------------->
Checking that you specified -V...--> Checking that you specified a time
limit...--> Checking that you specified a queue...--> Setting project...-->
Checking that you specified a parallel environment...--> Checking that you
specified a valid parallel environment name...--> Checking that the minimum
and maximum PE counts are the same...--> Checking that the number of PEs
requested is valid...------------------> Rejecting job
<------------------Your slot (or core) request is not a multiple of
16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
16.-----------------------------------------------------
Unable to run job: JSV rejected job.Exiting.

        at
org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
        at
org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
        ... 3 more

################### sites file

<config>
<pool handle="RANGER">
     <execution provider="coaster" jobManager="gt2:SGE" url="
gatekeeper.ranger.tacc.teragrid.org"/>
     <filesystem provider="gsiftp" url="gsiftp://
gridftp.ranger.tacc.teragrid.org"/>
     <profile namespace="globus" key="maxtime">86400</profile>
     <profile namespace="globus" key="maxWallTime">02:00:00</profile>
     <profile namespace="globus" key="jobsPerNode">1</profile>
     <profile namespace="globus" key="nodeGranularity">64</profile>
     <profile namespace="globus" key="maxNodes">4096</profile>
     <profile namespace="globus" key="queue">normal</profile>
     <profile namespace="karajan" key="jobThrottle">1.28</profile>
     <profile namespace="globus" key="project">TG-DBS080004N</profile>
     <profile namespace="globus" key="pe">16way</profile>
     <profile namespace="karajan" key="initialScore">10000</profile>
     <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
</pool>
</config>

same settings we've been using for a while, i'm not sure why this seems to
be popping up now, but it's rather consistent. all jobs are finishing
successfully, so it's rather confusing...any idea what i might be missing
here?

thanks
~sk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111221/f7182417/attachment.html>

From davidk at ci.uchicago.edu  Wed Dec 21 08:55:18 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Wed, 21 Dec 2011 08:55:18 -0600 (CST)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
Message-ID: <1860279107.41299.1324479318780.JavaMail.root@zimbra-mb2.anl.gov>

Sarah,

Could you please send the submit files that were generated from this run? That should help narrow it down a bit.

Thanks,
David

----- Original Message -----
> From: "Sarah Kenny" <skenny at uci.edu>
> To: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 6:57:32 AM
> Subject: [Swift-devel] swift on ranger
> getting this when submitting to ranger with both the latest and our
> previous version of swift (swift-r5259 cog-r3313)
> 
> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> successfully:100
> The following warnings have occurred:
> 1.
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> Cannot submit job: Could not submit job (qsub reported an exit code of
> 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> Could not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> ... 3 more
> 
> ################### sites file
> 
> <config>
> <pool handle="RANGER">
> <execution provider="coaster" jobManager="gt2:SGE" url="
> gatekeeper.ranger.tacc.teragrid.org "/>
> <filesystem provider="gsiftp" url="gsiftp://
> gridftp.ranger.tacc.teragrid.org "/>
> <profile namespace="globus" key="maxtime">86400</profile>
> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> <profile namespace="globus" key="jobsPerNode">1</profile>
> <profile namespace="globus" key="nodeGranularity">64</profile>
> <profile namespace="globus" key="maxNodes">4096</profile>
> <profile namespace="globus" key="queue">normal</profile>
> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> <profile namespace="globus" key="pe">16way</profile>
> <profile namespace="karajan" key="initialScore">10000</profile>
> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
> 
> same settings we've been using for a while, i'm not sure why this
> seems to be popping up now, but it's rather consistent. all jobs are
> finishing successfully, so it's rather confusing...any idea what i
> might be missing here?
> 
> thanks
> ~sk
> 
> 
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel


From wozniak at mcs.anl.gov  Wed Dec 21 09:40:22 2011
From: wozniak at mcs.anl.gov (Justin M Wozniak)
Date: Wed, 21 Dec 2011 09:40:22 -0600 (Central Standard Time)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <1860279107.41299.1324479318780.JavaMail.root@zimbra-mb2.anl.gov>
References: <1860279107.41299.1324479318780.JavaMail.root@zimbra-mb2.anl.gov>
Message-ID: <alpine.WNT.2.00.1112210939310.344@justinwozniak>


Should we provide an option to copy the submit file text into the log?

On Wed, 21 Dec 2011, David Kelly wrote:

> Sarah,
>
> Could you please send the submit files that were generated from this 
> run? That should help narrow it down a bit.
>
> Thanks,
> David
>
> ----- Original Message -----
>> From: "Sarah Kenny" <skenny at uci.edu>
>> To: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
>> Sent: Wednesday, December 21, 2011 6:57:32 AM
>> Subject: [Swift-devel] swift on ranger
>> getting this when submitting to ranger with both the latest and our
>> previous version of swift (swift-r5259 cog-r3313)
>>
>> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
>> successfully:100
>> The following warnings have occurred:
>> 1.
>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>> Cannot submit job: Could not submit job (qsub reported an exit code of
>> 1).
>> --------------------------------------------------------------------------
>> Welcome to TACC's Ranger System, an NSF XD Resource
>> ---------------------------------------------------------------------------->
>> Checking that you specified -V...--> Checking that you specified a
>> time limit...--> Checking that you specified a queue...--> Setting
>> project...--> Checking that you specified a parallel environment...-->
>> Checking that you specified a valid parallel environment name...-->
>> Checking that the minimum and maximum PE counts are the same...-->
>> Checking that the number of PEs requested is
>> valid...------------------> Rejecting job <------------------Your slot
>> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
>> <n>where <n> is a multiple of
>> 16.-----------------------------------------------------
>> Unable to run job: JSV rejected job.Exiting.
>>
>> at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
>> at
>> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
>> at
>> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
>> at
>> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
>> Caused by:
>> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
>> Could not submit job (qsub reported an exit code of 1).
>> --------------------------------------------------------------------------
>> Welcome to TACC's Ranger System, an NSF XD Resource
>> ---------------------------------------------------------------------------->
>> Checking that you specified -V...--> Checking that you specified a
>> time limit...--> Checking that you specified a queue...--> Setting
>> project...--> Checking that you specified a parallel environment...-->
>> Checking that you specified a valid parallel environment name...-->
>> Checking that the minimum and maximum PE counts are the same...-->
>> Checking that the number of PEs requested is
>> valid...------------------> Rejecting job <------------------Your slot
>> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
>> <n>where <n> is a multiple of
>> 16.-----------------------------------------------------
>> Unable to run job: JSV rejected job.Exiting.
>>
>> at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
>> at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
>> ... 3 more
>>
>> ################### sites file
>>
>> <config>
>> <pool handle="RANGER">
>> <execution provider="coaster" jobManager="gt2:SGE" url="
>> gatekeeper.ranger.tacc.teragrid.org "/>
>> <filesystem provider="gsiftp" url="gsiftp://
>> gridftp.ranger.tacc.teragrid.org "/>
>> <profile namespace="globus" key="maxtime">86400</profile>
>> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
>> <profile namespace="globus" key="jobsPerNode">1</profile>
>> <profile namespace="globus" key="nodeGranularity">64</profile>
>> <profile namespace="globus" key="maxNodes">4096</profile>
>> <profile namespace="globus" key="queue">normal</profile>
>> <profile namespace="karajan" key="jobThrottle">1.28</profile>
>> <profile namespace="globus" key="project">TG-DBS080004N</profile>
>> <profile namespace="globus" key="pe">16way</profile>
>> <profile namespace="karajan" key="initialScore">10000</profile>
>> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
>> </pool>
>> </config>
>>
>> same settings we've been using for a while, i'm not sure why this
>> seems to be popping up now, but it's rather consistent. all jobs are
>> finishing successfully, so it's rather confusing...any idea what i
>> might be missing here?
>>
>> thanks
>> ~sk
>>
>>
>>
>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>

-- 
Justin M Wozniak


From wilde at mcs.anl.gov  Wed Dec 21 09:41:43 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Wed, 21 Dec 2011 09:41:43 -0600 (CST)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <alpine.WNT.2.00.1112210939310.344@justinwozniak>
Message-ID: <1237880008.87610.1324482103234.JavaMail.root@zimbra.anl.gov>

+1

----- Original Message -----
> From: "Justin M Wozniak" <wozniak at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 9:40:22 AM
> Subject: Re: [Swift-devel] swift on ranger
> Should we provide an option to copy the submit file text into the log?
> 
> On Wed, 21 Dec 2011, David Kelly wrote:
> 
> > Sarah,
> >
> > Could you please send the submit files that were generated from this
> > run? That should help narrow it down a bit.
> >
> > Thanks,
> > David
> >
> > ----- Original Message -----
> >> From: "Sarah Kenny" <skenny at uci.edu>
> >> To: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User"
> >> <swift-user at ci.uchicago.edu>
> >> Sent: Wednesday, December 21, 2011 6:57:32 AM
> >> Subject: [Swift-devel] swift on ranger
> >> getting this when submitting to ranger with both the latest and our
> >> previous version of swift (swift-r5259 cog-r3313)
> >>
> >> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> >> successfully:100
> >> The following warnings have occurred:
> >> 1.
> >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> >> Cannot submit job: Could not submit job (qsub reported an exit code
> >> of
> >> 1).
> >> --------------------------------------------------------------------------
> >> Welcome to TACC's Ranger System, an NSF XD Resource
> >> ---------------------------------------------------------------------------->
> >> Checking that you specified -V...--> Checking that you specified a
> >> time limit...--> Checking that you specified a queue...--> Setting
> >> project...--> Checking that you specified a parallel
> >> environment...-->
> >> Checking that you specified a valid parallel environment name...-->
> >> Checking that the minimum and maximum PE counts are the same...-->
> >> Checking that the number of PEs requested is
> >> valid...------------------> Rejecting job <------------------Your
> >> slot
> >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> >> <n>where <n> is a multiple of
> >> 16.-----------------------------------------------------
> >> Unable to run job: JSV rejected job.Exiting.
> >>
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> >> at
> >> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> >> at
> >> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> >> at
> >> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> >> Caused by:
> >> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> >> Could not submit job (qsub reported an exit code of 1).
> >> --------------------------------------------------------------------------
> >> Welcome to TACC's Ranger System, an NSF XD Resource
> >> ---------------------------------------------------------------------------->
> >> Checking that you specified -V...--> Checking that you specified a
> >> time limit...--> Checking that you specified a queue...--> Setting
> >> project...--> Checking that you specified a parallel
> >> environment...-->
> >> Checking that you specified a valid parallel environment name...-->
> >> Checking that the minimum and maximum PE counts are the same...-->
> >> Checking that the number of PEs requested is
> >> valid...------------------> Rejecting job <------------------Your
> >> slot
> >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> >> <n>where <n> is a multiple of
> >> 16.-----------------------------------------------------
> >> Unable to run job: JSV rejected job.Exiting.
> >>
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> >> ... 3 more
> >>
> >> ################### sites file
> >>
> >> <config>
> >> <pool handle="RANGER">
> >> <execution provider="coaster" jobManager="gt2:SGE" url="
> >> gatekeeper.ranger.tacc.teragrid.org "/>
> >> <filesystem provider="gsiftp" url="gsiftp://
> >> gridftp.ranger.tacc.teragrid.org "/>
> >> <profile namespace="globus" key="maxtime">86400</profile>
> >> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> >> <profile namespace="globus" key="jobsPerNode">1</profile>
> >> <profile namespace="globus" key="nodeGranularity">64</profile>
> >> <profile namespace="globus" key="maxNodes">4096</profile>
> >> <profile namespace="globus" key="queue">normal</profile>
> >> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> >> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> >> <profile namespace="globus" key="pe">16way</profile>
> >> <profile namespace="karajan" key="initialScore">10000</profile>
> >> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> >> </pool>
> >> </config>
> >>
> >> same settings we've been using for a while, i'm not sure why this
> >> seems to be popping up now, but it's rather consistent. all jobs
> >> are
> >> finishing successfully, so it's rather confusing...any idea what i
> >> might be missing here?
> >>
> >> thanks
> >> ~sk
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> 
> --
> Justin M Wozniak
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From davidk at ci.uchicago.edu  Wed Dec 21 09:44:59 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Wed, 21 Dec 2011 09:44:59 -0600 (CST)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <alpine.WNT.2.00.1112210939310.344@justinwozniak>
Message-ID: <391015771.41452.1324482299772.JavaMail.root@zimbra-mb2.anl.gov>

Yep, very good idea.

----- Original Message -----
> From: "Justin M Wozniak" <wozniak at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 9:40:22 AM
> Subject: Re: [Swift-devel] swift on ranger
> Should we provide an option to copy the submit file text into the log?
> 
> On Wed, 21 Dec 2011, David Kelly wrote:
> 
> > Sarah,
> >
> > Could you please send the submit files that were generated from this
> > run? That should help narrow it down a bit.
> >
> > Thanks,
> > David
> >
> > ----- Original Message -----
> >> From: "Sarah Kenny" <skenny at uci.edu>
> >> To: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User"
> >> <swift-user at ci.uchicago.edu>
> >> Sent: Wednesday, December 21, 2011 6:57:32 AM
> >> Subject: [Swift-devel] swift on ranger
> >> getting this when submitting to ranger with both the latest and our
> >> previous version of swift (swift-r5259 cog-r3313)
> >>
> >> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> >> successfully:100
> >> The following warnings have occurred:
> >> 1.
> >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> >> Cannot submit job: Could not submit job (qsub reported an exit code
> >> of
> >> 1).
> >> --------------------------------------------------------------------------
> >> Welcome to TACC's Ranger System, an NSF XD Resource
> >> ---------------------------------------------------------------------------->
> >> Checking that you specified -V...--> Checking that you specified a
> >> time limit...--> Checking that you specified a queue...--> Setting
> >> project...--> Checking that you specified a parallel
> >> environment...-->
> >> Checking that you specified a valid parallel environment name...-->
> >> Checking that the minimum and maximum PE counts are the same...-->
> >> Checking that the number of PEs requested is
> >> valid...------------------> Rejecting job <------------------Your
> >> slot
> >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> >> <n>where <n> is a multiple of
> >> 16.-----------------------------------------------------
> >> Unable to run job: JSV rejected job.Exiting.
> >>
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> >> at
> >> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> >> at
> >> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> >> at
> >> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> >> Caused by:
> >> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> >> Could not submit job (qsub reported an exit code of 1).
> >> --------------------------------------------------------------------------
> >> Welcome to TACC's Ranger System, an NSF XD Resource
> >> ---------------------------------------------------------------------------->
> >> Checking that you specified -V...--> Checking that you specified a
> >> time limit...--> Checking that you specified a queue...--> Setting
> >> project...--> Checking that you specified a parallel
> >> environment...-->
> >> Checking that you specified a valid parallel environment name...-->
> >> Checking that the minimum and maximum PE counts are the same...-->
> >> Checking that the number of PEs requested is
> >> valid...------------------> Rejecting job <------------------Your
> >> slot
> >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> >> <n>where <n> is a multiple of
> >> 16.-----------------------------------------------------
> >> Unable to run job: JSV rejected job.Exiting.
> >>
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> >> at
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> >> ... 3 more
> >>
> >> ################### sites file
> >>
> >> <config>
> >> <pool handle="RANGER">
> >> <execution provider="coaster" jobManager="gt2:SGE" url="
> >> gatekeeper.ranger.tacc.teragrid.org "/>
> >> <filesystem provider="gsiftp" url="gsiftp://
> >> gridftp.ranger.tacc.teragrid.org "/>
> >> <profile namespace="globus" key="maxtime">86400</profile>
> >> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> >> <profile namespace="globus" key="jobsPerNode">1</profile>
> >> <profile namespace="globus" key="nodeGranularity">64</profile>
> >> <profile namespace="globus" key="maxNodes">4096</profile>
> >> <profile namespace="globus" key="queue">normal</profile>
> >> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> >> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> >> <profile namespace="globus" key="pe">16way</profile>
> >> <profile namespace="karajan" key="initialScore">10000</profile>
> >> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> >> </pool>
> >> </config>
> >>
> >> same settings we've been using for a while, i'm not sure why this
> >> seems to be popping up now, but it's rather consistent. all jobs
> >> are
> >> finishing successfully, so it's rather confusing...any idea what i
> >> might be missing here?
> >>
> >> thanks
> >> ~sk
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> 
> --
> Justin M Wozniak


From wilde at mcs.anl.gov  Wed Dec 21 09:48:16 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Wed, 21 Dec 2011 09:48:16 -0600 (CST)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <391015771.41452.1324482299772.JavaMail.root@zimbra-mb2.anl.gov>
Message-ID: <931702283.87648.1324482496044.JavaMail.root@zimbra.anl.gov>

In the incident Sarah reported, can you tell from the log if the coaster provider generated an ill-formed job request, perhaps as the script was completing? Ie, something that either exceeded the SGE limits, or possibly had e.g. a zero-node request?

- Mike

----- Original Message -----
> From: "David Kelly" <davidk at ci.uchicago.edu>
> To: "Justin M Wozniak" <wozniak at mcs.anl.gov>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 9:44:59 AM
> Subject: Re: [Swift-devel] swift on ranger
> Yep, very good idea.
> 
> ----- Original Message -----
> > From: "Justin M Wozniak" <wozniak at mcs.anl.gov>
> > To: "David Kelly" <davidk at ci.uchicago.edu>
> > Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > Sent: Wednesday, December 21, 2011 9:40:22 AM
> > Subject: Re: [Swift-devel] swift on ranger
> > Should we provide an option to copy the submit file text into the
> > log?
> >
> > On Wed, 21 Dec 2011, David Kelly wrote:
> >
> > > Sarah,
> > >
> > > Could you please send the submit files that were generated from
> > > this
> > > run? That should help narrow it down a bit.
> > >
> > > Thanks,
> > > David
> > >
> > > ----- Original Message -----
> > >> From: "Sarah Kenny" <skenny at uci.edu>
> > >> To: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User"
> > >> <swift-user at ci.uchicago.edu>
> > >> Sent: Wednesday, December 21, 2011 6:57:32 AM
> > >> Subject: [Swift-devel] swift on ranger
> > >> getting this when submitting to ranger with both the latest and
> > >> our
> > >> previous version of swift (swift-r5259 cog-r3313)
> > >>
> > >> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> > >> successfully:100
> > >> The following warnings have occurred:
> > >> 1.
> > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> > >> Cannot submit job: Could not submit job (qsub reported an exit
> > >> code
> > >> of
> > >> 1).
> > >> --------------------------------------------------------------------------
> > >> Welcome to TACC's Ranger System, an NSF XD Resource
> > >> ---------------------------------------------------------------------------->
> > >> Checking that you specified -V...--> Checking that you specified
> > >> a
> > >> time limit...--> Checking that you specified a queue...-->
> > >> Setting
> > >> project...--> Checking that you specified a parallel
> > >> environment...-->
> > >> Checking that you specified a valid parallel environment
> > >> name...-->
> > >> Checking that the minimum and maximum PE counts are the
> > >> same...-->
> > >> Checking that the number of PEs requested is
> > >> valid...------------------> Rejecting job <------------------Your
> > >> slot
> > >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> > >> <n>where <n> is a multiple of
> > >> 16.-----------------------------------------------------
> > >> Unable to run job: JSV rejected job.Exiting.
> > >>
> > >> at
> > >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> > >> at
> > >> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> > >> at
> > >> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> > >> at
> > >> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> > >> Caused by:
> > >> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> > >> Could not submit job (qsub reported an exit code of 1).
> > >> --------------------------------------------------------------------------
> > >> Welcome to TACC's Ranger System, an NSF XD Resource
> > >> ---------------------------------------------------------------------------->
> > >> Checking that you specified -V...--> Checking that you specified
> > >> a
> > >> time limit...--> Checking that you specified a queue...-->
> > >> Setting
> > >> project...--> Checking that you specified a parallel
> > >> environment...-->
> > >> Checking that you specified a valid parallel environment
> > >> name...-->
> > >> Checking that the minimum and maximum PE counts are the
> > >> same...-->
> > >> Checking that the number of PEs requested is
> > >> valid...------------------> Rejecting job <------------------Your
> > >> slot
> > >> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> > >> <n>where <n> is a multiple of
> > >> 16.-----------------------------------------------------
> > >> Unable to run job: JSV rejected job.Exiting.
> > >>
> > >> at
> > >> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> > >> at
> > >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> > >> ... 3 more
> > >>
> > >> ################### sites file
> > >>
> > >> <config>
> > >> <pool handle="RANGER">
> > >> <execution provider="coaster" jobManager="gt2:SGE" url="
> > >> gatekeeper.ranger.tacc.teragrid.org "/>
> > >> <filesystem provider="gsiftp" url="gsiftp://
> > >> gridftp.ranger.tacc.teragrid.org "/>
> > >> <profile namespace="globus" key="maxtime">86400</profile>
> > >> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> > >> <profile namespace="globus" key="jobsPerNode">1</profile>
> > >> <profile namespace="globus" key="nodeGranularity">64</profile>
> > >> <profile namespace="globus" key="maxNodes">4096</profile>
> > >> <profile namespace="globus" key="queue">normal</profile>
> > >> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> > >> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> > >> <profile namespace="globus" key="pe">16way</profile>
> > >> <profile namespace="karajan" key="initialScore">10000</profile>
> > >> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> > >> </pool>
> > >> </config>
> > >>
> > >> same settings we've been using for a while, i'm not sure why this
> > >> seems to be popping up now, but it's rather consistent. all jobs
> > >> are
> > >> finishing successfully, so it's rather confusing...any idea what
> > >> i
> > >> might be missing here?
> > >>
> > >> thanks
> > >> ~sk
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> Swift-devel mailing list
> > >> Swift-devel at ci.uchicago.edu
> > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > >
> >
> > --
> > Justin M Wozniak
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From ketancmaheshwari at gmail.com  Wed Dec 21 09:58:59 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Wed, 21 Dec 2011 09:58:59 -0600
Subject: [Swift-devel] swift on ranger
In-Reply-To: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
References: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
Message-ID: <CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>

Sarah,

I checked my sites.xml. The only difference between yours and mine being
the value of jobspernode which is 16 in my case. I have had  this value in
other multiples of 16 which has worked fine for me.


On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny <skenny at uci.edu> wrote:

> getting this when submitting to ranger with both the latest and our
> previous version of swift (swift-r5259 cog-r3313)
>
> Final status:  time: Wed, 21 Dec 2011 04:49:15 -0800  Finished
> successfully:100
> The following warnings have occurred:
> 1. org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> Cannot submit job: Could not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a time
> limit...--> Checking that you specified a queue...--> Setting project...-->
> Checking that you specified a parallel environment...--> Checking that you
> specified a valid parallel environment name...--> Checking that the minimum
> and maximum PE counts are the same...--> Checking that the number of PEs
> requested is valid...------------------> Rejecting job
> <------------------Your slot (or core) request is not a multiple of
> 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
>
>         at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
>         at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
>         at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
>         at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could
> not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a time
> limit...--> Checking that you specified a queue...--> Setting project...-->
> Checking that you specified a parallel environment...--> Checking that you
> specified a valid parallel environment name...--> Checking that the minimum
> and maximum PE counts are the same...--> Checking that the number of PEs
> requested is valid...------------------> Rejecting job
> <------------------Your slot (or core) request is not a multiple of
> 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
>
>         at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
>         at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
>         ... 3 more
>
> ################### sites file
>
> <config>
> <pool handle="RANGER">
>      <execution provider="coaster" jobManager="gt2:SGE" url="
> gatekeeper.ranger.tacc.teragrid.org"/>
>      <filesystem provider="gsiftp" url="gsiftp://
> gridftp.ranger.tacc.teragrid.org"/>
>      <profile namespace="globus" key="maxtime">86400</profile>
>      <profile namespace="globus" key="maxWallTime">02:00:00</profile>
>      <profile namespace="globus" key="jobsPerNode">1</profile>
>      <profile namespace="globus" key="nodeGranularity">64</profile>
>      <profile namespace="globus" key="maxNodes">4096</profile>
>      <profile namespace="globus" key="queue">normal</profile>
>      <profile namespace="karajan" key="jobThrottle">1.28</profile>
>      <profile namespace="globus" key="project">TG-DBS080004N</profile>
>      <profile namespace="globus" key="pe">16way</profile>
>      <profile namespace="karajan" key="initialScore">10000</profile>
>      <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
>
> same settings we've been using for a while, i'm not sure why this seems to
> be popping up now, but it's rather consistent. all jobs are finishing
> successfully, so it's rather confusing...any idea what i might be missing
> here?
>
> thanks
> ~sk
>
>
>
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
>


-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111221/6b87c704/attachment.html>

From jonmon at mcs.anl.gov  Wed Dec 21 10:04:48 2011
From: jonmon at mcs.anl.gov (Jonathan Monette)
Date: Wed, 21 Dec 2011 10:04:48 -0600
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>
References: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
	<CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>
Message-ID: <F40113C0-1DA9-4F7A-83E7-66F7D7F620A1@mcs.anl.gov>

Also, shouldn't node granularity be set to 16 on ranger an not 64?



On Dec 21, 2011, at 9:58 AM, Ketan Maheshwari <ketancmaheshwari at gmail.com> wrote:

> Sarah,
> 
> I checked my sites.xml. The only difference between yours and mine being the value of jobspernode which is 16 in my case. I have had  this value in other multiples of 16 which has worked fine for me.
> 
> 
> On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny <skenny at uci.edu> wrote:
> getting this when submitting to ranger with both the latest and our previous version of swift (swift-r5259 cog-r3313)
> 
> Final status:  time: Wed, 21 Dec 2011 04:49:15 -0800  Finished successfully:100
> The following warnings have occurred:
> 1. org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Cannot submit job: Could not submit job (qsub reported an exit code of 1). -------------------------------------------------------------------------- Welcome to TACC's Ranger System, an NSF XD Resource ----------------------------------------------------------------------------> Checking that you specified -V...--> Checking that you specified a time limit...--> Checking that you specified a queue...--> Setting project...--> Checking that you specified a parallel environment...--> Checking that you specified a valid parallel environment name...--> Checking that the minimum and maximum PE counts are the same...--> Checking that the number of PEs requested is valid...------------------> Rejecting job <------------------Your slot (or core) request is not a multiple of 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
>         at org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
>         at org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
>         at org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
>         at org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by: org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could not submit job (qsub reported an exit code of 1). -------------------------------------------------------------------------- Welcome to TACC's Ranger System, an NSF XD Resource ----------------------------------------------------------------------------> Checking that you specified -V...--> Checking that you specified a time limit...--> Checking that you specified a queue...--> Setting project...--> Checking that you specified a parallel environment...--> Checking that you specified a valid parallel environment name...--> Checking that the minimum and maximum PE counts are the same...--> Checking that the number of PEs requested is valid...------------------> Rejecting job <------------------Your slot (or core) request is not a multiple of 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
>         at org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
>         at org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
>         ... 3 more
> 
> ################### sites file
> 
> <config>
> <pool handle="RANGER">
>      <execution provider="coaster" jobManager="gt2:SGE" url="gatekeeper.ranger.tacc.teragrid.org"/>
>      <filesystem provider="gsiftp" url="gsiftp://gridftp.ranger.tacc.teragrid.org"/>
>      <profile namespace="globus" key="maxtime">86400</profile>
>      <profile namespace="globus" key="maxWallTime">02:00:00</profile>
>      <profile namespace="globus" key="jobsPerNode">1</profile>
>      <profile namespace="globus" key="nodeGranularity">64</profile>
>      <profile namespace="globus" key="maxNodes">4096</profile>
>      <profile namespace="globus" key="queue">normal</profile>
>      <profile namespace="karajan" key="jobThrottle">1.28</profile>
>      <profile namespace="globus" key="project">TG-DBS080004N</profile>
>      <profile namespace="globus" key="pe">16way</profile>
>      <profile namespace="karajan" key="initialScore">10000</profile>
>      <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
> 
> same settings we've been using for a while, i'm not sure why this seems to be popping up now, but it's rather consistent. all jobs are finishing successfully, so it's rather confusing...any idea what i might be missing here? 
> 
> thanks
> ~sk
> 
> 
>    
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> -- 
> Ketan
> 
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111221/c601f265/attachment.html>

From wilde at mcs.anl.gov  Wed Dec 21 10:20:12 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Wed, 21 Dec 2011 10:20:12 -0600 (CST)
Subject: [Swift-devel] swift on ranger
In-Reply-To: <F40113C0-1DA9-4F7A-83E7-66F7D7F620A1@mcs.anl.gov>
Message-ID: <1932609068.87907.1324484412645.JavaMail.root@zimbra.anl.gov>

Node granularity is the size increment, in nodes, of the number of nodes requested in each coaster block. So it can be anything that the user wants, as long as its valid for the local scheduler.

We recently discussed the need to improve and clarify the user guide documentation on how to specify node request parameters for the coaster provider. Im going to file this as a ticket now for 0.94.

- Mike

----- Original Message -----
> From: "Jonathan Monette" <jonmon at mcs.anl.gov>
> To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> Cc: "Sarah Kenny" <skenny at uci.edu>, "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User"
> <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 10:04:48 AM
> Subject: Re: [Swift-devel] [Swift-user] swift on ranger
> Also, shouldn't node granularity be set to 16 on ranger an not 64?
> 
> 
> 
> 
> On Dec 21, 2011, at 9:58 AM, Ketan Maheshwari <
> ketancmaheshwari at gmail.com > wrote:
> 
> 
> 
> 
> 
> Sarah,
> 
> I checked my sites.xml. The only difference between yours and mine
> being the value of jobspernode which is 16 in my case. I have had this
> value in other multiples of 16 which has worked fine for me.
> 
> 
> 
> On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny < skenny at uci.edu > wrote:
> 
> 
> getting this when submitting to ranger with both the latest and our
> previous version of swift (swift-r5259 cog-r3313)
> 
> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> successfully:100
> The following warnings have occurred:
> 1.
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> Cannot submit job: Could not submit job (qsub reported an exit code of
> 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> Could not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> ... 3 more
> 
> ################### sites file
> 
> <config>
> <pool handle="RANGER">
> <execution provider="coaster" jobManager="gt2:SGE" url="
> gatekeeper.ranger.tacc.teragrid.org "/>
> <filesystem provider="gsiftp" url="gsiftp://
> gridftp.ranger.tacc.teragrid.org "/>
> <profile namespace="globus" key="maxtime">86400</profile>
> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> <profile namespace="globus" key="jobsPerNode">1</profile>
> <profile namespace="globus" key="nodeGranularity">64</profile>
> <profile namespace="globus" key="maxNodes">4096</profile>
> <profile namespace="globus" key="queue">normal</profile>
> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> <profile namespace="globus" key="pe">16way</profile>
> <profile namespace="karajan" key="initialScore">10000</profile>
> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
> 
> same settings we've been using for a while, i'm not sure why this
> seems to be popping up now, but it's rather consistent. all jobs are
> finishing successfully, so it's rather confusing...any idea what i
> might be missing here?
> 
> thanks
> ~sk
> 
> 
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> --
> Ketan
> 
> 
> 
> 
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From skenny at uci.edu  Wed Dec 21 16:50:48 2011
From: skenny at uci.edu (Sarah Kenny)
Date: Wed, 21 Dec 2011 14:50:48 -0800
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>
References: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
	<CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>
Message-ID: <CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>

yr right ketan, if i change it to:  <profile namespace="globus"
key="jobsPerNode">16</profile> the warning message goes away. however,
there are times i don't want to run 16 jobs per node...e.g. bcs a single
job needs all the available memory so even though the node has 16
processors i can't actually use them all. so perhaps this is just a
scheduling issue with ranger/sge in that they don't want you to submit a
job that's going to leave processors idle? that seems a bit restrictive
though...

On Wed, Dec 21, 2011 at 7:58 AM, Ketan Maheshwari <
ketancmaheshwari at gmail.com> wrote:

> Sarah,
>
> I checked my sites.xml. The only difference between yours and mine being
> the value of jobspernode which is 16 in my case. I have had  this value in
> other multiples of 16 which has worked fine for me.
>
>
> On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny <skenny at uci.edu> wrote:
>
>> getting this when submitting to ranger with both the latest and our
>> previous version of swift (swift-r5259 cog-r3313)
>>
>> Final status:  time: Wed, 21 Dec 2011 04:49:15 -0800  Finished
>> successfully:100
>> The following warnings have occurred:
>> 1. org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>> Cannot submit job: Could not submit job (qsub reported an exit code of 1).
>> --------------------------------------------------------------------------
>> Welcome to TACC's Ranger System, an NSF XD Resource
>> ---------------------------------------------------------------------------->
>> Checking that you specified -V...--> Checking that you specified a time
>> limit...--> Checking that you specified a queue...--> Setting project...-->
>> Checking that you specified a parallel environment...--> Checking that you
>> specified a valid parallel environment name...--> Checking that the minimum
>> and maximum PE counts are the same...--> Checking that the number of PEs
>> requested is valid...------------------> Rejecting job
>> <------------------Your slot (or core) request is not a multiple of
>> 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
>> 16.-----------------------------------------------------
>> Unable to run job: JSV rejected job.Exiting.
>>
>>         at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
>>         at
>> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
>>         at
>> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
>>         at
>> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
>> Caused by:
>> org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could
>> not submit job (qsub reported an exit code of 1).
>> --------------------------------------------------------------------------
>> Welcome to TACC's Ranger System, an NSF XD Resource
>> ---------------------------------------------------------------------------->
>> Checking that you specified -V...--> Checking that you specified a time
>> limit...--> Checking that you specified a queue...--> Setting project...-->
>> Checking that you specified a parallel environment...--> Checking that you
>> specified a valid parallel environment name...--> Checking that the minimum
>> and maximum PE counts are the same...--> Checking that the number of PEs
>> requested is valid...------------------> Rejecting job
>> <------------------Your slot (or core) request is not a multiple of
>> 16.Syntax: -pe <pe_name> <n>where <n> is a multiple of
>> 16.-----------------------------------------------------
>> Unable to run job: JSV rejected job.Exiting.
>>
>>         at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
>>         at
>> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
>>         ... 3 more
>>
>> ################### sites file
>>
>> <config>
>> <pool handle="RANGER">
>>      <execution provider="coaster" jobManager="gt2:SGE" url="
>> gatekeeper.ranger.tacc.teragrid.org"/>
>>      <filesystem provider="gsiftp" url="gsiftp://
>> gridftp.ranger.tacc.teragrid.org"/>
>>      <profile namespace="globus" key="maxtime">86400</profile>
>>      <profile namespace="globus" key="maxWallTime">02:00:00</profile>
>>      <profile namespace="globus" key="jobsPerNode">1</profile>
>>      <profile namespace="globus" key="nodeGranularity">64</profile>
>>      <profile namespace="globus" key="maxNodes">4096</profile>
>>      <profile namespace="globus" key="queue">normal</profile>
>>      <profile namespace="karajan" key="jobThrottle">1.28</profile>
>>      <profile namespace="globus" key="project">TG-DBS080004N</profile>
>>      <profile namespace="globus" key="pe">16way</profile>
>>      <profile namespace="karajan" key="initialScore">10000</profile>
>>      <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
>> </pool>
>> </config>
>>
>> same settings we've been using for a while, i'm not sure why this seems
>> to be popping up now, but it's rather consistent. all jobs are finishing
>> successfully, so it's rather confusing...any idea what i might be missing
>> here?
>>
>> thanks
>> ~sk
>>
>>
>>
>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>
>>
>
>
> --
> Ketan
>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>



-- 
Sarah Kenny
Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
University of California Irvine, Dept. of Neurology ~ 773-818-8300
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111221/9e48e197/attachment.html>

From davidk at ci.uchicago.edu  Wed Dec 21 17:41:00 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Wed, 21 Dec 2011 17:41:00 -0600 (CST)
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>
Message-ID: <773619107.42639.1324510860877.JavaMail.root@zimbra-mb2.anl.gov>

I am able to reproduce this problem.. hoping to have a fix soon.

David

----- Original Message -----
> From: "Sarah Kenny" <skenny at uci.edu>
> To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 4:50:48 PM
> Subject: Re: [Swift-devel] [Swift-user] swift on ranger
> yr right ketan, if i change it to: <profile namespace="globus"
> key="jobsPerNode">16</profile> the warning message goes away. however,
> there are times i don't want to run 16 jobs per node...e.g. bcs a
> single job needs all the available memory so even though the node has
> 16 processors i can't actually use them all. so perhaps this is just a
> scheduling issue with ranger/sge in that they don't want you to submit
> a job that's going to leave processors idle? that seems a bit
> restrictive though...
> 
> 
> On Wed, Dec 21, 2011 at 7:58 AM, Ketan Maheshwari <
> ketancmaheshwari at gmail.com > wrote:
> 
> 
> Sarah,
> 
> I checked my sites.xml. The only difference between yours and mine
> being the value of jobspernode which is 16 in my case. I have had this
> value in other multiples of 16 which has worked fine for me.
> 
> 
> 
> 
> 
> 
> On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny < skenny at uci.edu > wrote:
> 
> 
> 
> 
> 
> getting this when submitting to ranger with both the latest and our
> previous version of swift (swift-r5259 cog-r3313)
> 
> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> successfully:100
> The following warnings have occurred:
> 1.
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> Cannot submit job: Could not submit job (qsub reported an exit code of
> 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> Could not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> ... 3 more
> 
> ################### sites file
> 
> <config>
> <pool handle="RANGER">
> <execution provider="coaster" jobManager="gt2:SGE" url="
> gatekeeper.ranger.tacc.teragrid.org "/>
> <filesystem provider="gsiftp" url="gsiftp://
> gridftp.ranger.tacc.teragrid.org "/>
> <profile namespace="globus" key="maxtime">86400</profile>
> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> <profile namespace="globus" key="jobsPerNode">1</profile>
> <profile namespace="globus" key="nodeGranularity">64</profile>
> <profile namespace="globus" key="maxNodes">4096</profile>
> <profile namespace="globus" key="queue">normal</profile>
> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> <profile namespace="globus" key="pe">16way</profile>
> <profile namespace="karajan" key="initialScore">10000</profile>
> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
> 
> same settings we've been using for a while, i'm not sure why this
> seems to be popping up now, but it's rather consistent. all jobs are
> finishing successfully, so it's rather confusing...any idea what i
> might be missing here?
> 
> thanks
> ~sk
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> --
> Ketan
> 
> 
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> 
> 
> 
> --
> Sarah Kenny
> Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
> University of California Irvine, Dept. of Neurology ~ 773-818-8300
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel


From hategan at mcs.anl.gov  Wed Dec 21 17:53:53 2011
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Wed, 21 Dec 2011 15:53:53 -0800
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>
References: <CAE50RW23ERDWOPAw69DdeuJ1bNKYNZMPucxtBaQcTs00Oufyfw@mail.gmail.com>
	<CAMUuvireO4J8yAQRL-2yVbp_yN+hHHrNMdJ1YSiHenaLAZO40Q@mail.gmail.com>
	<CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>
Message-ID: <1324511633.20131.2.camel@blabla>

On Wed, 2011-12-21 at 14:50 -0800, Sarah Kenny wrote:
> yr right ketan, if i change it to:  <profile namespace="globus"
> key="jobsPerNode">16</profile> the warning message goes away. however,
> there are times i don't want to run 16 jobs per node...e.g. bcs a
> single job needs all the available memory so even though the node has
> 16 processors i can't actually use them all. so perhaps this is just a
> scheduling issue with ranger/sge in that they don't want you to submit
> a job that's going to leave processors idle? that seems a bit
> restrictive though...

Hmmm. I think that one should be able to request 16 cores and run only
two jobs at a time on them. Have you tried pe=2way?



From davidk at ci.uchicago.edu  Thu Dec 22 01:54:25 2011
From: davidk at ci.uchicago.edu (David Kelly)
Date: Thu, 22 Dec 2011 01:54:25 -0600 (CST)
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>
Message-ID: <2130988117.43047.1324540465638.JavaMail.root@zimbra-mb2.anl.gov>

Sarah,

Can you please give this another try? I believe it should work now with your original sites.xml.

David

----- Original Message -----
> From: "Sarah Kenny" <skenny at uci.edu>
> To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 4:50:48 PM
> Subject: Re: [Swift-devel] [Swift-user] swift on ranger
> yr right ketan, if i change it to: <profile namespace="globus"
> key="jobsPerNode">16</profile> the warning message goes away. however,
> there are times i don't want to run 16 jobs per node...e.g. bcs a
> single job needs all the available memory so even though the node has
> 16 processors i can't actually use them all. so perhaps this is just a
> scheduling issue with ranger/sge in that they don't want you to submit
> a job that's going to leave processors idle? that seems a bit
> restrictive though...
> 
> 
> On Wed, Dec 21, 2011 at 7:58 AM, Ketan Maheshwari <
> ketancmaheshwari at gmail.com > wrote:
> 
> 
> Sarah,
> 
> I checked my sites.xml. The only difference between yours and mine
> being the value of jobspernode which is 16 in my case. I have had this
> value in other multiples of 16 which has worked fine for me.
> 
> 
> 
> 
> 
> 
> On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny < skenny at uci.edu > wrote:
> 
> 
> 
> 
> 
> getting this when submitting to ranger with both the latest and our
> previous version of swift (swift-r5259 cog-r3313)
> 
> Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> successfully:100
> The following warnings have occurred:
> 1.
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> Cannot submit job: Could not submit job (qsub reported an exit code of
> 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> Could not submit job (qsub reported an exit code of 1).
> --------------------------------------------------------------------------
> Welcome to TACC's Ranger System, an NSF XD Resource
> ---------------------------------------------------------------------------->
> Checking that you specified -V...--> Checking that you specified a
> time limit...--> Checking that you specified a queue...--> Setting
> project...--> Checking that you specified a parallel environment...-->
> Checking that you specified a valid parallel environment name...-->
> Checking that the minimum and maximum PE counts are the same...-->
> Checking that the number of PEs requested is
> valid...------------------> Rejecting job <------------------Your slot
> (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> <n>where <n> is a multiple of
> 16.-----------------------------------------------------
> Unable to run job: JSV rejected job.Exiting.
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> ... 3 more
> 
> ################### sites file
> 
> <config>
> <pool handle="RANGER">
> <execution provider="coaster" jobManager="gt2:SGE" url="
> gatekeeper.ranger.tacc.teragrid.org "/>
> <filesystem provider="gsiftp" url="gsiftp://
> gridftp.ranger.tacc.teragrid.org "/>
> <profile namespace="globus" key="maxtime">86400</profile>
> <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> <profile namespace="globus" key="jobsPerNode">1</profile>
> <profile namespace="globus" key="nodeGranularity">64</profile>
> <profile namespace="globus" key="maxNodes">4096</profile>
> <profile namespace="globus" key="queue">normal</profile>
> <profile namespace="karajan" key="jobThrottle">1.28</profile>
> <profile namespace="globus" key="project">TG-DBS080004N</profile>
> <profile namespace="globus" key="pe">16way</profile>
> <profile namespace="karajan" key="initialScore">10000</profile>
> <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> </pool>
> </config>
> 
> same settings we've been using for a while, i'm not sure why this
> seems to be popping up now, but it's rather consistent. all jobs are
> finishing successfully, so it's rather confusing...any idea what i
> might be missing here?
> 
> thanks
> ~sk
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> --
> Ketan
> 
> 
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> 
> 
> 
> --
> Sarah Kenny
> Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
> University of California Irvine, Dept. of Neurology ~ 773-818-8300
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel


From wilde at mcs.anl.gov  Thu Dec 22 05:28:32 2011
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 22 Dec 2011 05:28:32 -0600 (CST)
Subject: [Swift-devel] [Swift-user]    swift on ranger
In-Reply-To: <1324511633.20131.2.camel@blabla>
Message-ID: <1417017812.90783.1324553312587.JavaMail.root@zimbra.anl.gov>

The Ranger user guide:

http://www.tacc.utexas.edu/user-services/user-guides/ranger-user-guide

Has a section (below) on using < 16 cores per node. (Im not sure if "15" below is a typo; I think 16 was intended).

- Mike

Using fewer than 16 cores per node

When you want to use less than 16 MPI tasks per node, the choice of tasks per node is limited to the set of numbers {1, 2, 4, 8, 12, and 15}.

When the number of tasks you need is equal to "Number of Tasks per Node x Number of Nodes", then use the following command:

#$ -pe <TpN>way <NoN x 16>

where <TpN>is a number in the set {1, 2, 4, 8, 12, 15}.

If the Total number of Tasks that you need is less than "Number of Tasks per Node x Number of Nodes", then set the MY_NSLOTS environment variable to the Total number of Tasks <TnoT>needed. In a job script, use the following -pe option and environment variable statement:

#$ -pe <TpN>way <NoN x 16>
export MY_NSLOTS=<TnoT> # For Bourne shells

or

setenv MY_NSLOTS <TnoT> # For C shells

where <TpN>is a number in the set {1, 2, 4, 8, 12, 15}. For example, using a Bourne shell:

#$ -pe 8way 64 # Use 8 Tasks per Node, 4 Nodes requested
export MY_NSLOTS=31 # 31 tasks are launched

----- Original Message -----
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "Sarah Kenny" <skenny at uci.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 5:53:53 PM
> Subject: Re: [Swift-user] [Swift-devel] swift on ranger
> On Wed, 2011-12-21 at 14:50 -0800, Sarah Kenny wrote:
> > yr right ketan, if i change it to: <profile namespace="globus"
> > key="jobsPerNode">16</profile> the warning message goes away.
> > however,
> > there are times i don't want to run 16 jobs per node...e.g. bcs a
> > single job needs all the available memory so even though the node
> > has
> > 16 processors i can't actually use them all. so perhaps this is just
> > a
> > scheduling issue with ranger/sge in that they don't want you to
> > submit
> > a job that's going to leave processors idle? that seems a bit
> > restrictive though...
> 
> Hmmm. I think that one should be able to request 16 cores and run only
> two jobs at a time on them. Have you tried pe=2way?
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory



From skenny at uci.edu  Thu Dec 22 16:44:37 2011
From: skenny at uci.edu (Sarah Kenny)
Date: Thu, 22 Dec 2011 14:44:37 -0800
Subject: [Swift-devel] [Swift-user]  swift on ranger
In-Reply-To: <2130988117.43047.1324540465638.JavaMail.root@zimbra-mb2.anl.gov>
References: <CAE50RW37AfJHphvCDpU9vMXR4SHPbBOAHKjp4vwadqcpcSyFUg@mail.gmail.com>
	<2130988117.43047.1324540465638.JavaMail.root@zimbra-mb2.anl.gov>
Message-ID: <CAE50RW0uL8FDs5ibUMM9vuMWPYqqiQpPq0mqnft-L4FfoN8dDQ@mail.gmail.com>

yeah, latest build works...i don't get a warning when specifying 1 job per
node with 16way pe.

On Wed, Dec 21, 2011 at 11:54 PM, David Kelly <davidk at ci.uchicago.edu>wrote:

> Sarah,
>
> Can you please give this another try? I believe it should work now with
> your original sites.xml.
>
> David
>
> ----- Original Message -----
> > From: "Sarah Kenny" <skenny at uci.edu>
> > To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> > Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <
> swift-user at ci.uchicago.edu>
> > Sent: Wednesday, December 21, 2011 4:50:48 PM
> > Subject: Re: [Swift-devel] [Swift-user] swift on ranger
> > yr right ketan, if i change it to: <profile namespace="globus"
> > key="jobsPerNode">16</profile> the warning message goes away. however,
> > there are times i don't want to run 16 jobs per node...e.g. bcs a
> > single job needs all the available memory so even though the node has
> > 16 processors i can't actually use them all. so perhaps this is just a
> > scheduling issue with ranger/sge in that they don't want you to submit
> > a job that's going to leave processors idle? that seems a bit
> > restrictive though...
> >
> >
> > On Wed, Dec 21, 2011 at 7:58 AM, Ketan Maheshwari <
> > ketancmaheshwari at gmail.com > wrote:
> >
> >
> > Sarah,
> >
> > I checked my sites.xml. The only difference between yours and mine
> > being the value of jobspernode which is 16 in my case. I have had this
> > value in other multiples of 16 which has worked fine for me.
> >
> >
> >
> >
> >
> >
> > On Wed, Dec 21, 2011 at 6:57 AM, Sarah Kenny < skenny at uci.edu > wrote:
> >
> >
> >
> >
> >
> > getting this when submitting to ranger with both the latest and our
> > previous version of swift (swift-r5259 cog-r3313)
> >
> > Final status: time: Wed, 21 Dec 2011 04:49:15 -0800 Finished
> > successfully:100
> > The following warnings have occurred:
> > 1.
> > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> > Cannot submit job: Could not submit job (qsub reported an exit code of
> > 1).
> >
> --------------------------------------------------------------------------
> > Welcome to TACC's Ranger System, an NSF XD Resource
> >
> ---------------------------------------------------------------------------->
> > Checking that you specified -V...--> Checking that you specified a
> > time limit...--> Checking that you specified a queue...--> Setting
> > project...--> Checking that you specified a parallel environment...-->
> > Checking that you specified a valid parallel environment name...-->
> > Checking that the minimum and maximum PE counts are the same...-->
> > Checking that the number of PEs requested is
> > valid...------------------> Rejecting job <------------------Your slot
> > (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> > <n>where <n> is a multiple of
> > 16.-----------------------------------------------------
> > Unable to run job: JSV rejected job.Exiting.
> >
> > at
> >
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> > at
> >
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> > at
> >
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:57)
> > at
> >
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> > Caused by:
> > org.globus.cog.abstraction.impl.scheduler.common.ProcessException:
> > Could not submit job (qsub reported an exit code of 1).
> >
> --------------------------------------------------------------------------
> > Welcome to TACC's Ranger System, an NSF XD Resource
> >
> ---------------------------------------------------------------------------->
> > Checking that you specified -V...--> Checking that you specified a
> > time limit...--> Checking that you specified a queue...--> Setting
> > project...--> Checking that you specified a parallel environment...-->
> > Checking that you specified a valid parallel environment name...-->
> > Checking that the minimum and maximum PE counts are the same...-->
> > Checking that the number of PEs requested is
> > valid...------------------> Rejecting job <------------------Your slot
> > (or core) request is not a multiple of 16.Syntax: -pe <pe_name>
> > <n>where <n> is a multiple of
> > 16.-----------------------------------------------------
> > Unable to run job: JSV rejected job.Exiting.
> >
> > at
> >
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:108)
> > at
> >
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> > ... 3 more
> >
> > ################### sites file
> >
> > <config>
> > <pool handle="RANGER">
> > <execution provider="coaster" jobManager="gt2:SGE" url="
> > gatekeeper.ranger.tacc.teragrid.org "/>
> > <filesystem provider="gsiftp" url="gsiftp://
> > gridftp.ranger.tacc.teragrid.org "/>
> > <profile namespace="globus" key="maxtime">86400</profile>
> > <profile namespace="globus" key="maxWallTime">02:00:00</profile>
> > <profile namespace="globus" key="jobsPerNode">1</profile>
> > <profile namespace="globus" key="nodeGranularity">64</profile>
> > <profile namespace="globus" key="maxNodes">4096</profile>
> > <profile namespace="globus" key="queue">normal</profile>
> > <profile namespace="karajan" key="jobThrottle">1.28</profile>
> > <profile namespace="globus" key="project">TG-DBS080004N</profile>
> > <profile namespace="globus" key="pe">16way</profile>
> > <profile namespace="karajan" key="initialScore">10000</profile>
> > <workdirectory>/work/00043/tg457040/swiftwork</workdirectory>
> > </pool>
> > </config>
> >
> > same settings we've been using for a while, i'm not sure why this
> > seems to be popping up now, but it's rather consistent. all jobs are
> > finishing successfully, so it's rather confusing...any idea what i
> > might be missing here?
> >
> > thanks
> > ~sk
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> >
> >
> >
> > --
> > Ketan
> >
> >
> >
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >
> >
> >
> > --
> > Sarah Kenny
> > Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
> > University of California Irvine, Dept. of Neurology ~ 773-818-8300
> >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>



-- 
Sarah Kenny
Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III
University of California Irvine, Dept. of Neurology ~ 773-818-8300
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111222/5d5c42fa/attachment.html>

From iraicu at cs.iit.edu  Thu Dec 22 17:03:24 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Thu, 22 Dec 2011 17:03:24 -0600
Subject: [Swift-devel] CFP: The 9th Int. Conf. on Autonomic Computing (ICAC)
	2012
Message-ID: <4EF3B73C.3020204@cs.iit.edu>

CALL FOR PAPERS and WORKSHOP PROPOSALS

The 9th International Conference on Autonomic Computing (ICAC 2012)

September 17-21, 2012. San Jose, CA, USA
http://icac2012.cs.fiu.edu/
-----------------------------------------------------------------

IMPORTANT DATES

Paper and Poster Submission: March 9, 2012, 11:59pm PST
Notification: May 18, 2012
Camera-ready Due: June 8, 2012
Workshop Proposal Submission: February 10, 2012
-----------------------------------------------------------------

OVERVIEW
ICAC is the leading conference on autonomic computing techniques,
foundations, and applications. Autonomic computing refers to
methods and means for automated management of performance, fault,
security, and configuration with little involvement of users or
administrators. Systems introducing new autonomic features are
becoming increasingly prevalent, motivating research that spans
a variety of areas, from computer systems, networking, software
engineering, and data management to machine learning, control
theory, and bio-inspired computing. ICAC brings together
researchers and practitioners across these disciplines to
address multiple facets of adaptation and self-management in
computing systems and applications from different perspectives.
Autonomic computing solutions are sought for clouds, grids,
data centers, enterprise software, internet services, data
services, smart phones, embedded systems, and sensor networks.
In these environments, resources and applications must be managed
to maximize performance and minimize cost, while maintaining
predictable and reliable behavior in the face of varying
workloads, failures, and malicious threats. Papers are solicited
from all areas of autonomic computing, including (but not limited
to):

* End-to-end techniques for management of resources, workloads,
   performance, faults, power/cooling, security, and others.

* Self-managing components, such as server, storage, network
   protocols, or specific application elements, and embedded and
   mobile end systems such as smart phones.

* Decision and analysis techniques and their use, such as machine
   learning, control theory, predictive methods, probability and
   stochastic processes, queuing theory methodologies, emergent
   behavior, rule-based systems, and bio-inspired techniques.

* Monitoring systems for autonomic computing.

* Hypervisor, operating systems, hardware, or application support
   for autonomic computing.

* Novel human interfaces for monitoring and controlling autonomic
   systems.

* Management topics, such as specification and modeling of
   service-level agreements, behavior enforcement and tie-in with
   IT governance.

* Toolkits, frameworks, principles and architectures, from
   software engineering practices and experimental methodologies
   to agent-based techniques and virtualization.

* Fundamental science and theory of self-managing systems:
   understanding, controlling or exploiting system behaviors to
   enforce autonomic properties.

* Applications of autonomic computing and experiences with
   prototyped or deployed systems solving real-world problems in
   science, engineering, business and society.

Papers will be judged on originality, significance, interest,
correctness, clarity and relevance to the broader community.
Papers should report on experiences, measurements, user studies,
or other evaluations, as appropriate. Evaluations of a prototype
or large-scale deployment of systems and applications is expected.

PAPER AND POSTER SUBMISSIONS
Full papers (a maximum of 10 pages in the two-column ACM proceedings
format) and posters (2 pages) are invited on a wide variety of
topics relating to autonomic computing. Submitted papers must be
original work, and may not be under consideration for another
conference or journal. Complete formatting and submission
instructions can be found on the conference web site. Accepted
papers and posters will appear in proceedings distributed at the
conference and available electronically. Relevant top ICAC'12
papers will be invited for "fast-track" submissions to the
ACM Transactions on Autonomous and Adaptive Systems (TAAS).

WORKSHOPS, DEMONSTRATIONS AND EXHIBITION
ICAC'12 welcomes proposals for co-located workshops on topics of
interest to the autonomic computing community. Workshop proposals
should be submitted to the Workshop Chair, Fred Douglis
(f.douglis at computer.org) by February 10, 2012. Workshops are
expected to publish proceedings, and should cover areas that
complement the main program. ICAC'12 will also feature a
demonstration and exhibition session consisting of prototypes and
technology artifacts such as demonstrating autonomic software or
autonomic computing principles. Entries will be judged by a
separate committee led by the demo/exhibit chair.

INDUSTRY SESSION
One of ICAC's important roles is to bring together researchers
and practitioners from academia and industry. In its industry
session, ICAC helps fulfill this role by presenting an industry
viewpoint on technologies, products, and market needs. The
industry session also addresses current challenges, and
opportunities for academic and corporate research collaborations.
We encourage industry leaders, including entrepreneurs, product
developers, architects, managers, marketers and end users,
to submit their papers and posters reflecting such industry
perspectives as part of the regular submission process.
------------------------------------------------------------------

ORGANIZERS

GENERAL CHAIR
Dejan Milojicic, HP Labs

PROGRAM CHAIRS
Dongyan Xu, Purdue University
Vanish Talwar, HP Labs

INDUSTRY CHAIR
Xiaoyun Zhu, VMware

WORKSHOPS CHAIR
Fred Douglis, EMC

POSTERS/DEMO/EXHIBITS CHAIR
Eno Thereska, Microsoft Research

FINANCE CHAIR
Michael Kozuch, Intel

LOCAL ARRANGEMENT CHAIR
Jessica Blaine

PUBLICITY CHAIRS
Daniel Batista, University of S?o Paulo
Vartan Padaryan, ISP/Russian Academy of Sci.
Ioan Raicu, Illinois Inst. of Technology
Jianfeng Zhan, ICT/Chinese Academy of Sci.
Ming Zhao, Florida Intl. University

PROGRAM COMMITTEE
Tarek Abdelzaher, UIUC
Umesh Bellur, IIT, Bombay
Ken Birman, Cornell University
Rajkumar Buyya, Univ. of Melbourne
Rocky Chang, Hong Kong Polytechnic University
Yuan Chen, HP Labs
Alva Couch, Tufts University
Peter Dinda, Northwestern University
Fred Douglis, EMC
Renato Figueiredo, University of Florida
Mohamed Hefeeda, Qatar Computing Research Institute
Joe Hellerstein, Google
Geoff Jiang, NEC Labs
Jeff Kephart, IBM Research
Emre Kiciman, Microsoft Research
Fabio Kon, University of S?o Paulo
Michael Kozuch, Intel
Dejan Milojicic, HP Labs
Klara Nahrstedt, UIUC
Priya Narasimhan, CMU
Manish Parashar, Rutgers University
Ioan Raicu, Illinois Inst. of Technology
Omer Rana, Cardiff University
Masoud Sadjadi, Florida Intl. University
Rick Schlichting, AT&T Labs
Hartmut Schmeck, KIT
Karsten Schwan, Georgia Tech
Onn Shehory, IBM Research
Eno Thereska, Microsoft Research
Xiaoyun Zhu, VMware

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================




From iraicu at cs.iit.edu  Thu Dec 22 17:39:59 2011
From: iraicu at cs.iit.edu (Ioan Raicu)
Date: Thu, 22 Dec 2011 17:39:59 -0600
Subject: [Swift-devel] CFP: 21st International Heterogeneity in Computing
 Workshop at IEEE IPDPS 2012
Message-ID: <4EF3BFCF.4000300@cs.iit.edu>

*HCW 2012 Call for Papers
*

**

*The Twenty First International *

In conjunction with IPDPS 2012 <http://www.ipdps.org/>, May 21, 2012, 
Shanghai, China

Sponsored by the IEEE Computer Society,
through the Technical Committee on Parallel Processing (TCPP),
and by the U.S. Office of Naval Research (ONR).

* *
*

Today, most computing systems have elements of heterogeneity. 
Heterogeneity springs from the richness of environments where diversity 
and resource abundance prevail. Recognizing, capturing, and efficiently 
exploiting this diversity in an integrated and coherent manner are key 
goals of heterogeneous computing.

Heterogeneous computing systems are those with a range of diverse 
computing resources that can be on a chip, within a computer, or on a 
local or geographically distributed network. The development of 
heterogeneous multi-core chips and the pervasive use of networks by all 
segments of society mean that the number and types of heterogeneous 
computing resources are growing rapidly. This growth creates the need 
and opportunity for new research to effectively utilize these resources 
in innovative and novel ways. For example, cluster computing, grid 
computing, peer-to-peer computing, and cloud computing all involve 
elements of heterogeneity. The effective implementation of efficient 
applications in these environments, however, requires that a host of 
issues be addressed that simply do not occur in homogeneous systems.

Whereas many researchers and practitioners that use computers have a 
peripheral awareness of heterogeneity in their respective fields, few 
critically approach their fields from the heterogeneous perspective. 
This is not particularly surprising, because each field has its own 
unique challenges and imperatives that propel investigations in search 
of solutions to pressing problems. Addressing computing problems from 
the heterogeneous perspective offers at least three advantages: (i) the 
design and development of more advanced high-performance computing 
platforms, (ii) insight into new solution approaches, and (iii) exposure 
to new research opportunities and relationships among distinct research 
areas. HCW encourages the examination of both hardware and software 
systems from the perspective of heterogeneity.

With the increasing number of components in heterogeneous parallel and 
distributed systems, failure is becoming a critical factor that impacts 
application performance. High-performance computing systems, especially 
those heading towards exascale, are also becoming increasingly 
heterogeneous and hierarchical, expecting to routinely employ 
heterogeneous multicores, various accelerators (such as GPUs), deeper 
memory hierarchies, and heterogeneous and hierarchical communication 
networks. This year, HCW is specifically encouraging (but not limited 
to) submissions that explore paradigms, algorithms, and techniques for 
high performance heterogeneous computing.

*TOPICS*

------------------------------------------------------------------------
Areas or research interest include, but are not limited to, 
heterogeneity aspects of:

  * Parallel algorithms for heterogeneous and hierarchical systems,
    including manycores and hardware accelerators (FPGAs, GPUs, etc.)
  * Parallel algorithms for efficient problem solving on heterogeneous
    platforms
  * Performance models and their use in the design of parallel and
    distributed algorithms for heterogeneous platforms
  * Programming paradigms and tools
  * Fault tolerance
  * Resource allocation and scheduling
  * Computer architectures
  * Performance evaluation and management
  * High performance computing
  * Cluster, Grid and Cloud computing
  * Peer-to-peer computing
  * Ubiquitous computing
  * Application case studies
  * Task and communication scheduling
  * Task coordination and workflow

*IMPORTANT DATES*

------------------------------------------------------------------------
Paper submission: January 7, 2012
Author Notification: February 7, 2012
Camera-ready: February 21, 2012

*PAPER SUBMISSIONS*

------------------------------------------------------------------------
Prospective authors are encouraged to visit the HCW 2012 website 
(hcw.wsu.edu) for instructions on how to submit their papers.

*PUBLICATION*

------------------------------------------------------------------------
The HCW 2012 proceedings will be published through the IEEE Computer 
Society Press as part of the IPDPS CD-ROM. The authors of accepted 
papers will be also invited to submit extended versions of their work to 
a special issue of the Journal of Parallel and Distributed Computing 
(JPDC) on Heterogeneity in Parallel and Distributed Computing that is 
planned for publication in 2012.

*WORKSHOP ORGANIZATION*

------------------------------------------------------------------------
*General Chair*:
Behrooz Shirazi, Washington State University, U.S.A.

*Program Chair*:
Alexey Lastovetsky, University College Dublin, Ireland

*Steering Committee*:
H. J. Siegel, Colorado State University, U.S.A., Chair
John Antonio, University of Oklahoma, U.S.A.
Francine Berman, Rensselaer Polytechnic Institute, U.S.A.
Jack Dongarra, University of Tennessee, U.S.A.
Jerry Potter, Colorado State University, U.S.A.
Viktor K. Prasanna, University of Southern California, U.S.A.
Yves Robert, Ecole Normale Superieure de Lyon, France
Arnold Rosenberg, Colorado State University, Northeastern University, U.S.A.
Vaidy Sunderam, Emory University, U.S.A.


*Program Committee*:
Shoukat Ali, IBM, Ireland
Francisco Almeida, University of La Laguna, Spain
Rosa M. Badia, Barcelona Supercomputing Center and CSIC, Spain
Ioana Banicescu, Mississippi State University, U.S.A.
Olivier Beaumont, INRIA, France
Shuvra S. Bhattacharyya, University of Maryland, U.S.A.
George Bosilca, University of Tennessee, U.S.A.
Eddy Caron, ENS-Lyon, France
Domingo Gimenez, University of Murcia, Spain
Alexey Kalinov, Cadence Design Systems, Russia
Tahar Kechadi, University College Dublin, Ireland
Jong-Kook Kim, Korea University, South Korea
Thomas Ludwig, University of Hamburg, Germany
Tony Maciejewski, Colorado State University, U.S.A.
John P. Morrison, University College Cork, Ireland
Dana Petcu, Western University of Timisoara, Romania
Antonio Plaza, University of Extremadura, Spain
Xiao Qin, Auburn University, U.S.A.
Enrique Quintana, University of Jaume I of Castellon, Spain
Ioan Raicu, Northwestern University, U.S.A.
Alistair Rendell, Australian National University, Australia
Gudula Runger, TU Chemnitz, Germany
Vladimir Rychkov, University College Dublin, Ireland
Uwe Schwiegelshohn, University of Dortmund, Germany
Stephen L. Scott, Tennessee Tech University & Oak Ridge National 
Laboratory, U.S.A.
James T. Smith, DigitalGlobe, U.S.A.
Leonel Sousa, Technical University of Lisbon, Portugal
Stanimire Tomov, University of Tennessee, U.S.A.
Denis Trystram, IMAG, France
Carlos Varela, Rensselaer Polytechnic Institute, U.S.A.
Qin Zheng, A*Star, Singapore *

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111222/8128c2aa/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hcw_logo_small.gif
Type: image/gif
Size: 22276 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111222/8128c2aa/attachment.gif>

From ketancmaheshwari at gmail.com  Fri Dec 23 19:35:40 2011
From: ketancmaheshwari at gmail.com (Ketan Maheshwari)
Date: Fri, 23 Dec 2011 19:35:40 -0600
Subject: [Swift-devel] provider.staging.pin.swiftfiles feature
Message-ID: <CAMUuviqH02r7NE8Ru6vz4DoWgBGD1aewRaiE5EGvXMkdxGMJEQ@mail.gmail.com>

Hi,

I was looking into the provider staging trying to address the issue of
having files accessed directly at remote sites skipping provider staging (
https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=676) .

I see the option provider.staging.pin.swiftfiles of swift.properties
appears into vdl-int.k as well as worker.pl but couldn't quite got at what
is the functionality.

Could someone indicate this.

Regards,
-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20111223/29efe0e2/attachment.html>