[Swift-devel] Re: Memory in Swift

[Jonathan Monette]

  Ok.  That seemed to fix my out of memory problem.

On 10/18/2010 05:14 PM, Justin M Wozniak wrote:
>
> Actually, to set SWIFT_HEAP_MAX for Swift you should export it in the 
> environment in which you run bin/swift .  So, in the shell you could do:
>
> SWIFT_HEAP_MAX=8192M swift ...
>
> The namespace env is there to let you set the environment for the 
> remote user job.
>
> On Mon, 18 Oct 2010, Jonathan Monette wrote:
>
>> jonmon   22946 54.6  1.2 788800 318496 pts/17  Sl+  16:42   0:39 java 
>> -Xmx256M
>>
>> I am assuming that this means the heap space is 256 Megabytes.  Here 
>> is how I set the SWIFT_HEAP_MAX variable:
>>
>> <profile key="SWIFT_HEAP_MAX" namespace="env">8192M</profile>
>>
>> Is this how you would set environment variables in the sites.xml 
>> file?  I want to make sure the variable is also set for the worker 
>> nodes.
>>
>> On 10/18/2010 04:36 PM, Justin M Wozniak wrote:
>>>
>>> What do you get if you set it to 8192M?
>>>
>>> On Mon, 18 Oct 2010, Jonathan Monette wrote:
>>>
>>>> I am on PADS.
>>>>
>>>> On 10/18/2010 01:21 PM, Justin M Wozniak wrote:
>>>>> On Mon, 18 Oct 2010, Jonathan Monette wrote:
>>>>>
>>>>>>    1) What is the maximum value for SWIFT_HEAP_MAX?  Is it the 
>>>>>> maximum memory available on the machine?  If I ask for 8Gb will 
>>>>>> the Java heap space be 8Gb?
>>>>>
>>>>> In bin/swift, that value is passed directly to the Java VM, so 
>>>>> it's implementation-specific.  What kind of machine are you on?
>>>>>
>>>>>>    2) Assuming that the above question does not fix my problem, 
>>>>>> what can be done to fix memory problems in Swift?  How should a 
>>>>>> Swift script be programmed to avoid these issues or is how Swift 
>>>>>> handles memory not optimal. I have a set of about 4100 images 
>>>>>> that will be stitched together to form a mosaic.  The size of 
>>>>>> this set is 8.1Gb.  Right now in my sites file I set 
>>>>>> SWIFT_HEAP_MAX to 4096M(4Gb) but I am still getting out of memory 
>>>>>> errors with the Java heap space.  Is Swift just not able to scale 
>>>>>> up to this size of data yet?
>>>>>
>>>>
>>>>
>>>
>>
>>
>

-- 
Jon

Computers are incredibly fast, accurate, and stupid. Human beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination.
- Albert Einstein