[R6RS] library and transformer phases

[dyb at cs.indiana.edu]

Matthew and I have agreed that there are two reasonable models for
dealing with library/transformer "phases".

  Model 1: An imported library is invoked only once for all phases, i.e.,
  run-time code, transformer code, transformer code within transformer
  code, etc., share one set of library bindings.

  Model 2: An imported library is invoked once for each phase, i.e.,
  run-time code, transformer code, transformer code within transformer
  code, etc., each have their own set of library bindings.

Model 1 is simpler while Model 2 is cleaner.

We also considered a third model in which all transformer phases share
the same set of bindings yet were distinct from run-time bindings.
This isn't any simpler than Model 1 and is less clean than Model 2,
so we discarded it.

With Model 2, we've also concluded that the programmer should be required
to specify each level at which a library should be invoked.  One
possibility is a (for --- (meta n)) syntax, where n=0 corresponds to run
time, n=1 corresponds to expand time, n=2 corresponds to expand time of
expand time, and so on.  Unfortunately, this makes eval a bit more
difficult to handle, since we'll need some sort of argument or wrapper on
the expression passed to eval to specify the meta levels at which the
libraries used by the evaluated code are invoked.  Another alternative is
to leave the invocation times implicit based on references that the
expander finds.  Although this alternative would simplify the syntax, we
discarded it because invoking a library might have side effects, and it
seems like a bad idea to leave the invocation times implicit.

With Model 2, there are also some "dark corners" where the right semantics
is not obvious.  For example, should two libraries A and B imported for
use at meta-level 2 share the same set of meta-level 3 bindings imported
from a third library C for use by their transformers?  Should a librarie
invoked at a given meta level during the expansion of one library be
reinvoked if a second library that uses it at the same level is expanded?

Model 2 is also more difficult to implement.  SRFI 72 outlines an approach
to phased expansion, which associates a level with each syntax object, but
SRFI 72 does not deal with libraries.  Extending the model to deal with
libraries would involve reconsitituting the syntax objects embedded within
the library code to increment the associated levels.  This can probably be
done with a global transformation on the expanded library source code to
"pull out" the syntax objects into an outer layer where they can be
reconsituted, so that an expanded library might look like this:

  (lambda (level situation)
    (when (eq? situation 'visit)
      (let ([t (increment-level '<syntax-object> level)]
            ...)
        <code to set up transformer bindings>))
    (when (eq? situation 'invoke)
      (let ([t (increment-level '<syntax-object> level)]
            ...)
        <code to set up variable bindings>)))

The <code to set up transformer bindings> and <code to set up variable
bindings> may have to be adjusted in other ways to set up a distinct sets
of bindings based on the level.

This is pretty hairy and will complicate the handling of libraries by the
expander.

With Model 1, none of these problems arise; the syntax, semantics, and
implementation are all simpler.  The downside of Model 1 is that run-time
code might count, perhaps unintentionally, on side effects caused by
expand-time code to the internal state of a library, thereby making it
more difficult to compile the code separately at some future point.

At this point, because of the difficulties with Model 2, Matthew and I are
inclined to go with Model 1, despite its downside.

Kent