[petsc-dev] Seeking OLCF users complaining about poor build times

[Nathan Collier]

Ok, on a whim I tried reconfiguring in home again this morning but from a
different login node. There was no load last night (on ext4) and none this
morning (on ext5), but now my configure time from /ccs/home/nate is 13m.
While not appreciably better than the /lustre time, it is suddenly much
faster than the 41m I sent previously.

Strikes me that this kind of experience is what confuses/frustrates users.

Nate




On Fri, Feb 27, 2015 at 8:20 AM, Nathan Collier <nathaniel.collier at gmail.com
> wrote:

> Barry,
>
> No problem.
>
> No packages / master and on /ccs/home/nate
>
> [nate at titan] time python no_packages.py
> ...
> real 41m38.700s
> user 1m50.075s
> sys 1m30.034s
>
> [nate at titan] time make all
> ...
> real 9m4.977s
> user 5m56.686s
> sys 28m22.310s
>
> In addition to a much slower configure time, the home areas are not
> accessible to batch submissions. So without the reconfigure script, I could
> not do a full build from home.
>
> Nate
>
>
> On Thu, Feb 26, 2015 at 11:19 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>
>>  Nathan,
>>
>>     Can you please do a configure make on your home directory system?
>>
>>      It could be since Lustre is a parallel filesystem designed for high
>> bandwidth IO from a parallel computer it is just bad for compiles etc.
>>
>>   Barry
>>
>>
>>
>> https://www.olcf.ornl.gov/support/getting-started/
>>
>> home areas
>> Home areas/directories are provided on a Network File System (NFS). The
>> user home areas are by default accessible only to the owning user.
>> Similarly the project home areas are accessible by only member of the
>> project. The areas are backed-up but available space is limited. Because
>> space is limited, each area has a quota. Users should store small source
>> code, scripts, and other similar items in the area. Users should not store
>> large job output or input in the area. Job I/O should be performed in the
>> system’s temporary work area.
>> work areas
>> Temporary work directories are provided to each user and project on
>> Lustre file systems. Similar to the home areas, by default, the user and
>> project areas are accessible to the user and project members. The areas
>> provide a large amount of storage, but are not backed-up. In general, job
>> I/O performance will be faster in the lustre areas than the NFS mounted
>> home areas. The temporary work areas are regularly purged of data that has
>> not been recently accessed, because of this all needed data should be
>> backed-up to the HPSS.
>>
>>
>> > On Feb 26, 2015, at 10:00 PM, Nathan Collier <
>> nathaniel.collier at gmail.com> wrote:
>> >
>> > > They never told you to do compiles on particular file systems or
>> anything?  For example did they ever say DON'T compile on the lustre file
>> system?
>> >
>> > I have not received any guidance about where to or not to compile. A
>> quick perusing of the user guide doesn't make that clear to me. If it isn't
>> already obvious, I am far from a Titan expert :)
>> >
>> > Do you know someone here at ORNL who I can talk to to make sure I am
>> not missing something silly?
>> >
>> > > Perhaps the 52m 'sys' time could be luster overhead
>> >
>> > It could be, I have heard a lot of complaints about this from people
>> who need to write output or read from files in their applications.
>> >
>> > Nate
>> >
>> >
>> > On Thu, Feb 26, 2015 at 10:36 PM, Satish Balay <balay at mcs.anl.gov>
>> wrote:
>> > On Thu, 26 Feb 2015, Barry Smith wrote:
>> >
>> > >
>> > > > On Feb 26, 2015, at 9:16 PM, Nathan Collier <
>> nathaniel.collier at gmail.com> wrote:
>> > > >
>> > > > Barry,
>> > > >
>> > > > > Do you know where /tmp is on the system? Presumably it is fast?
>> > > >
>> > > > I am not sure about that. There is a /tmp but I am not sure what it
>> is or if/how I can use it.
>> > >
>> > >    They never told you to do compiles on particular file systems or
>> anything?  For example did they ever say DON'T compile on the lustre file
>> system?
>> >
>> > Ah - I misunderstood earlier. You'd like to have a petsc clone in /tmp
>> > - and do the build there - and compare the difference [with the
>> > current luster build].
>> >
>> > Perhaps the 52m 'sys' time could be luster overhead - and that woud
>> > disappear with a petsc build in /tmp. [and could be much faster].
>> >
>> > Satish
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150227/21e290ae/attachment.html>