[Swift-devel] Issues to resolve for the 0.96 config mechanism

[Mihael Hategan]

I know the feeling. I get it too. I see some new stuff and I was getting
used to the old stuff, and I get this immediate anger about the new
stuff and the feeling that it's stupid and I want the old stuff back.

All I'm saying is give it some time. Try it out a bit, see how you would
do something with it, etc. It's not set in stone. We have SVN and we can
always go back.

Ultimately the questions are: can I do what I was doing before, and how
much effort does that require. Also what are the new things that I can
do and could not do before, and how much better/worse is the overall
process. I know what the previous thing could do, and you can still do
that, just not in the exact same way.

Some more inline...

On Sun, 2014-07-13 at 17:03 -0500, Michael Wilde wrote:
> On 7/13/14, 1:01 PM, Mihael Hategan wrote:
> >> - the include mechanism is new.   I think its nice and likely is very
> >> >useful, but I wonder how it will interact with or supplement the
> >> >property search path.
> > We discussed this a few days ago. We had repeated issues with magically
> > loaded files from strange locations that the users took a long time to
> > find and fix. The solution that I saw in 0.95 was even more search
> > locations, which I think was not right.
> >
> > So the philosophy in trunk is "either it stares at you or it isn't
> > there". And includes do this in a way that doesn't sacrifice
> > convenience.
> >
> I disagree with this. The properties search path in 0.95 was carefully 
> thought out, and based on a lot of experience with users, even it that 
> discussion did not take place on the list.

You can write a config file that includes all the different config files
and put it in swift/etc, if, as a user, that is what you think is best.

> 
> Unfortunately the trunk commits over-wrote the 0.95 documentation in the 
> trunk userguide, but the search path was like this:
> 
> Location of swift.properties
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Swift searches for swift.properties files in multiple locations:
> 
> 1. The etc/swift.properties file included with the Swift distribution.
> 2. $SWIFT_SITE_CONF/swift.properties - used for defining site templates.
> 3. $HOME/.swift/swift.properties
> 4. swift.properties in your current directory.
> 5. Any property file you point to with the command line argument 
> "-properties
> <file>"
> 
> Settings get read in this order. Definitions in the later files will 
> override
> any previous definitions. For example, if you have execution.retries=10 in
> $HOME/.swift/swift.properties, and execution.retries=0 in the 
> swift.properties
> in your current directory, execution.retries will be set to 0.

Yes. Well, I think this is bad. Imagine a user having to mentally go
through this sequence in order to figure out what's being set and where.
It's actually nearly impossible mentally. You have to go through the
files and remember the order.

The biggest problems we've had so far in terms of the configuration we
had were with this. A file in the search path magically loaded without
the user's knowledge. Even recently you and Yadu have experienced this,
and were wondering where some configuration value comes from that was
causing a run to break. And it took you a few hours and I had to get
involved to sort it out. And It was all documented. See
https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1281

So no, I do not think that this is a good idea.

> 
> I thought this search path notion was logical, useful, and reflected the 
> needs of users and site admins. It was a refinement of what existed in 
> Swift going back many releases.

It added extra steps to something that already had too many steps.

That said, the new stuff doesn't say you can't do that. Just that you
have to actually say it yourself that you want to do that. The reason we
had multiple search paths is because we did not support includes, so we
needed a mechanism to override only a few things. But you can simulate
it with includes and customize it in whatever way you want and it would
be obvious what's coming from where.

The point is to give the user a simple and powerful tool to do what they
need.

> 
> In 0.95 it was tied to the use of run directories (which I hope has not 
> been removed from trunk!).

No. It's still there. I like it. I only renamed runxxx.log to swift.log,
since I did not se why everything had a name that did not include the
run id except for the log. We can change this back if there is a reason.

We do probably need to fix a few issues with it. It will break with
concurrent starts of swift, and it will break on filesystems that don't
list directories in some particular order.