[R6RS] library and transformer phases

[Matthew Flatt]

At Sat, 08 Jul 2006 12:48:39 -0400, dyb at cs.indiana.edu wrote:
> With Model 2, there are also some "dark corners" where the right semantics
> is not obvious.

I'm not sure where to start. They don't seem like dark corners to me,
which makes me think I need to start at the beginning. Or maybe I
misunderstand the issues.

I'll try a combination: the most direct response I can manage, followed
by an extra bit of background.

> For example, should two libraries A and B imported for
> use at meta-level 2 share the same set of meta-level 3 bindings imported
> from a third library C for use by their transformers?  

Definitely, yes.

Suppose that C defines two macros that somehow communicate together,
such as `define-record' and `record-switch' in the "You Want It When?"
paper.

Suppose that A contains something like

 (define-syntax (a-macro ..)
    .. (syntax .. (let-syntax ([.. (.. define-record ..)]) ..) ..) ..)

In other words, A has a macro that puts a reference to `define-record'
into a compile-time position relative to A.

Suppose that B exports a similar `b-macro' that similarly introduces
`record-match'.

Now suppose that D imports A and B for-expand, and it defines a macro
whose transformer implementation uses both `a-macro' and
`b-macro'. The introduced `define-record' (from C via A) should
cooperate with the introduced `record-switch' (from C via B).

[In the MzScheme implementation, in the first example of this pattern
 that I find, module C is played by `#%sc', A is `#%stxcase', B is
 `#%stxloc', `a-macro' is `syntax-case', `b-macro' is `syntax/loc',
 and D is `#%with-stx'.]

> Should a librarie
> invoked at a given meta level during the expansion of one library be
> reinvoked if a second library that uses it at the same level is expanded?

Yes, definitely, in my opinion. Library instantiations should prevent
accidental sharing of state between different module compilations, as
well as between compile time and run time.

----------------------------------------

>   Model 2: An imported library is invoked once for each phase, i.e.,
>   run-time code, transformer code, transformer code within transformer
>   code, etc., each have their own set of library bindings.

We can draw out three aspects of phase:

 * Separating bindings across phases.

 * Separating invocations across phases.

 * Separating invocations across module compilations.

The motivation for the first (distinct phase bindings) is to avoid free
variables at run and compile time --- assuming a simple connection
between the `eval-when' aspect of importing and the binding aspect of
importing.

The motivation for the second (distinct phase invocations) is to avoid
accidental sharing of state between the compilation and run time of a
given module.

The motivation for the third (distinct compilation invocations) is to
avoid accidental sharing of state across compilations of different
modules.


Maybe you can pick any subset of the bullets, depending on which
motivations you find compelling. I'm not sure, though, because
MzScheme's implementation of the first bullet relies at least on the
second one: syntax objects are phase-shifted at module-invocation
time, which depends on the phase of the invocation.


In particular, the compilation model in MzScheme is that

 1. each module is compiled individually;

 2. all (transitively) imported modules must be compiled before
    a module is compiled; and

 3. at the start of a module compilation, no modules are instantiated
    at any phase.

Thus, no module instance exists across the compilation of multiple
modules, so there's no sharing of state between the compilations
(except through the filesystem, etc.). Also, when a program is run,
the system starts with zero module instances, so there's no sharing
between compile time and run time.

[In practice, MzScheme interleaves the compilation of imported modules
 with the compilation of an importing module. But it separates module
 instantiations, so that the interleaving is not detectable (except
 through external channels, such as the filesystem or the system
 clock). This may be relevant if we end up with `eval' and a
 MzScheme-like model of separate module invocation.]


Matthew