[Swift-devel] On demand weather forecast using Swift

Fri Dec 7 14:16:00 CST 2007

Hi,

just to practicing in swift I'm porting my grid application for on demand
weather forecast  from a xml based workflow engine to Swift a workflow.

The Swift expression capability is great.

With this email here you are my (not full working) Swift code.

I have few questions:

1)      The most part of functions retrieves and array of files I
implemented using the filesys_mapper. I hope I used the right syntax, but
Swift simply does nothing. I mean, the job is not submitted to the remote
machine. I have to stop it with CTRL+C. On the logfile I have this message:

2007-12-07 20:07:49,210+0000 DEBUG Loader Recompilation suppressed.

2007-12-07 20:07:52,507+0000 INFO  unknown Using sites file: sites.xml

2007-12-07 20:07:52,508+0000 INFO  unknown Using tc.data: tc.data

2007-12-07 20:07:54,287+0000 WARN  JavaUtils Unable to find required classes
(javax.activation.DataHandler and javax.mail.internet.MimeMultipart).
Attachment support is disabled.

2007-12-07 20:07:56,560+0000 INFO  unknown Swift v0.3-dev r1507

2007-12-07 20:07:56,563+0000 INFO  unknown RunID: 20071207-2007-95unaxd2

2007-12-07 20:07:56,898+0000 INFO  AbstractDataNode Adding handle listener
"Open, 0 elements, no listeners" to "terrainDomains.$[]/0"

2007-12-07 20:07:56,927+0000 INFO  AbstractDataNode Adding handle listener
"F/terrainDomains.$[]/0" to "terrainDomains.$[]/0"

2)      All modules belonging my applications want in the ./input input
files and produce in ./output their results. I never reached this point
because the point 1 fails, how work the file mapper in this case? I mean, if
the regrid produces the regridDomains array of files mapped somewhere but
./output/REGRID_DOMAINx.gz when it is staged-in as input for the interpf
where it is copied? ./output/. ? Eventually, how can I change this behavior
in order to stage in in ./input what previously whas staged out from
./output?

Thank you in advance,

  A Swift enthusiastic - Raffaele

-----------------------

The Swift Workflow:

// Data type definition

type terrainDomainType;

type initialDataType;

type dataFileType;

type sstFileType;

type snowFileType;

type regridDomainType;

type mmInputDomainType;

type bdyOutDomainType;

type lowBdyDomainType;

type mmOutputDomainType;

// Terrain function

(terrainDomainType terrainDomains[])terrain(string lon, string lat, int
nests) {

  app { terrain lon lat nests @terrainDomains; }

}

// Pregrid function

(dataFileType dataFiles[], sstFileType sstFiles[], snowFileType snowFiles[]
)pregrid(string iDate, int hours, initialDataType initalData) {

  app { pregrid iDate hours @filename(initaialData) @dataFiles @sstFiles
@snowFiles; }

}

// Regrid function

(regridDomainType regridDomains[]) regrdi( terrainDomainType terrainDomain,
dataFileType dataFiles, sstFileType sstFiles, snowFileType snowFiles) {

  app { regrid @terrainDomain @dataFiles @sstFIles @snowFiles
@regridDomains; }

}

// Interpf function

(mmInputDomainType mmInputDomains[], bdyOutDomainType dbyOutDomains[],
lowBdyDomainType lowBdyDomains[]) interpf (regridDomainType regridDomain) {

  app { interpf @regridDomain @mmInputDomains @dbyOutDOmains @lowBdyDomains;
}

}

// Mpp (parallel MM5) function

(mmOutputDomainType mmOutputDomains[])mmp( terrainDomainType
terrainDomains[], mmInputDomainType mmInputDomains[], bdyOutDomainType
dbyOutDomains[], lowBdyDomainType lowBdyDomains[]) {

  app { mmp @terrainDomains @mmInputDomains @bdyOutDomains @lowBdyDomains
@mmOutputDomains; }

}

// Terrain output

terrainDomainType terrainDomains[] <filesys_mapper;location="./output/",
prefix="TERRAIN_DOMAIN", suffix=".gz">;

// Pregrid input data

initialDataType initialData[] <filesys_mapper;
location="gsiftp://data.uniparthenope.it//home/infonav/data/ncep/20071207/20
071207Z00", prefix="gfs">;

// Pregrid outputs

dataFileType dataFiles[] <filesys_mapper;location="./output/",
prefix="FILE", suffix=".gz">;

sstFileType sstFiles[] <filesys_mapper;location="./output/", prefix="SST",
suffix=".gz">;

snowFileType snowFiles[] <filesys_mapper;location="./output/",
prefix="SNOW", suffix=".gz">;

// Regrid outputs

regridDomainType regridDomains[] <filesys_mapper; location="./output/",
prefix="REGRID_DOMAIN", suffix=".gz">;

// Interpf outputs

mmInputDomainType mmInputDomains[] <filesys_mapper; location="./output/",
prefix="MMINPUT_DOMAIN", suffix=".gz">;

bdyOutDomainType bdyOutDomains[] <filesys_mapper; location="./output/",
prefix="BDYOUT_DOMAIN", suffix=".gz">;

lowBdyDomainType lowBdyDomains[] <filesys_mapper; location="./output/",
prefix="LOWBDY_DOMAIN", suffix=".gz">;

// MM5 output arrays

mmOutputDomainType mmOutputDomains[] <filesys_mapper; location="./output/",
prefix="MMOUTPUT_DOMAIN", suffix=".gz">;

//
----------------------------------------------------------------------------
--------

// THE WORKFLOW: on demand weather forecast

//
----------------------------------------------------------------------------
--------

// Create the terrain centered on Chicago Pier (two nest levels, max ground
resolution 27km)

terrainDomains = terrain("-87.5992", "41.8919", 2);

// Retrieve initial and boundary conditions and prepare data files for next
modules

//(dataFiles, sstFiles, snowFiles) = pregrid( "20071207Z00", 24,
initialData);

// Prepare data for each nested domain...

foreach terrainDomain,i in terrainDomains

{

  // Interpolate initial and boundary conditions over the terrain model

  //regridFiles[i] = regrid( terrainDomain, dataFiles, sstFiles, snowFiles);

  // Perform more forward interpolation and produce input data form MM5

  //(mmInputDomains[i],bdyOutDomains[i],lowBdyDomains[i]) =
interpf(regridFiles[i]);

}

// Run the model

//mmOutputDomains = mpp(terrainDomains, mmInputDomains, bdyOutDomains,
lowBdyDomains);

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20071207/1655282d/attachment.html>