[Swift-devel] Tracking swift heap overflow culprit

[Mihael Hategan]

On Thu, 2014-01-30 at 19:02 -0600, Michael Wilde wrote:
> > >
> > > [yadunand at midway001 data_stress]$ jmap -histo:live 31135 | head -n 10
> > >
> > >  num     #instances         #bytes  class name
> > > ----------------------------------------------
> > >    1:       1000001       56000056  org.griphyn.vdl.mapping.DataNode
> > >    2:       1030601       32979232  java.util.HashMap$Entry
> > >    3:       2014652       32234432  java.lang.Integer
> > >    4:       1000015       24000360  org.griphyn.vdl.type.impl.FieldImpl
> > >    5:         14672        4903992  [Ljava.util.HashMap$Entry;
> > >    6:         29872        4149184  <constMethodKlass>
> > >    7:         29872        3831968  <methodKlass>

Some specific comments:

1. You need a DataNode for each swift piece of data.
2. Arrays are sparse and implemented as a map, so for each element of an
array you will have a HashMap$Entry.
3. A little too many. I've updated some code to try to avoid
instantiating Integer objects if not necessary (using Integer.valueOf
will typically use existing objects for small numbers). Unfortunately,
if you have 1,000,000 different integer values, you will end up with
1,000,000 integers. One can probably implement a version of DataNode
that is specialized for primitive values to eliminate that part.
4. Unfortunately array indices are treated as "fields" (i.e. a[1] is
treated as a.1). For structures, field instances are cached, but for
arrays, since they have different indices, not so much.
5. Every HashMap has an array of HashMap$Entry objects.
6&7. constant space stuff.

Mihael