[Swift-user] Problems in complex mapping situation

[Michael Wilde]

I get this:
--

RunID: 20090320-0048-id8ot1y9
Progress:
Execution failed:
         java.lang.RuntimeException: Data set initialization failed for 
result.[0][]/1. It should have been closed.
Caused by:
         Handle open: OOPSOut[] result.[0][]/1
SwiftScript trace: T1af7, Round, 0, Sim, 0, StartTemp, , TempUpdate,
Command exited with non-zero status 2

--

from the fairly lengthy script at www.ci.uchicago.edu/~wilde/oops.swift

This script was working fine, and I was trying to improve its 
performance on large datasets on the BG/P by using an ext mapper instead 
of simple_mapper to map a large 2D array of structures (OOPSOut result) 
so that it spreads across multiple directories (to avoid the GPFS 
locking issue. (I have 4K cores writing 14,000 files to one directory, 
as I see no way with simple mapper to use the array index to, for 
example, insert more directory entries in the prefix or suffix.)

So I am trying now to map the entire 2D array of structs "result" up 
front (just handling the simple case where the first dimension has only 
a single entry.)

The log is at the same URL, file oops-20090320-0048-id8ot1y9.log

In the simple first test case Im trying here (ie, map for one job), I 
think my mapper is being called with and returning this:

int$ ./OOPSOutAll.map.sh -p 1foo -d out.n -r 1 -s 1 -t "" -u ""
[0][0].pdt out.n/1foo/0/00/00/ST.TU.pdt
[0][0].rmsd out.n/1foo/0/00/00/ST.TU.rmsd
[0][0].log out.n/1foo/0/00/00/ST.TU.log
int$

I'll debug this further, but am hoping you can give some clues as to 
what the error above means. I dont get the idea of the dataset needing 
to be closed at initialization. Is intialization here mean "mapping"? 
The state notion implied by this message is confusing.

Thanks.