[R6RS] Ticket #110: Remove double phase semantics

[Matthew Flatt]

At Mon, 27 Nov 2006 15:47:23 -0500, William D Clinger wrote:
> Both Matthew Flatt and Andre van Tonder have confirmed that
> the separate-binding semantics implies that macro-time values
> cannot be compared with run-time values [2,3]. 

To be clear, we mean that "it's impossible to write an expression whose
evaluation involves both run-time and compile-time values".

> It follows
> that the draft R6RS does not provide any way for a syntax-case
> macro to insert a run-time value into its output [4].

True, but confusing as stated.

In general, run time hasn't started yet, so no run-time values exist.
Macros manipulate expand-time representations of run-time expressions,
including compile-time representations of run-time constants. There is
no problem with a macro inserting the representation of a run-time
constant into the representation of a run-time expression.

>  In
> particular, it appears to be impossible to write a portable
> syntax-case macro m whose uses reliably expand into any
> specific symbol or integer constant [4].  For example, the
> following macro is apparently not portable under the weak
> guarantees provided by the draft R6RS:
> 
>     (define-syntax m (lambda (e) 13))

I agree that it's not clear whether this is valid. But the question is
not whether `13' as a compile-time expression somehow produces a
run-time 13, but whether `13' is allowed as a syntax object.

In any case, this one should certainly work:

 (define-syntax m (lambda (e) (datum->syntax e 13)))

> This is not just a theoretical problem. Several of the
> reference implementations, including most critically the
> reference implementation for generic arithmetic, define new
> types.  With the draft R6RS semantics, there does not appear
> to be any reliable way to write the reference implementation
> for arithmetic as a set of portable R6RS libraries.
>
> Symbols and arithmetic are more important to this discussion
> than records, conditions, i/o ports, hash tables, enumeration
> sets, and similar objects because symbols and numbers have
> external representations and can appear as constants in the
> macro-expanded code.

I think what you're saying is that `datum->syntax' and `syntax->datum'
must know how to deal with all things that have an external
representation, i.e., all datums.

To put it another way, things with an external representation can't be
implemented in a library, because the syntax+library system implements
libraries, so concepts with an external representation must already
exist before the library system exists.

I don't believe that allowing sharing across phases would avoid this
chicken-and-egg problem.

> Nonetheless, similar problems could arise
> even for types that have no external representation, when a
> syntax-case macro performs a comparison between values of
> such types at macro time and tries to encode the value as a
> number, string, or some other value that *does* have an
> external representation but is not guaranteed to be
> communicable from macro time to run time.

`datum->syntax' works only on datums. It should raise an exception when
given any kind of value that is not a datum. (I think this is implicit
in the conventions of R6RS, since the argument for `datum->syntax' is
called "datum", but it may make sense to be more explicit).

> Since PLT Scheme has been using the separated binding semantics
> for some time, one might ask why these problems have not been
> observed in practice.  The answer, I believe, is that PLT Scheme
> does not use the separated binding semantics consistently; it
> escapes from that semantics by writing its basic libraries in
> some other language (probably C or C++), and libraries written
> in those other languages use a shared binding semantics.

It's because the module system exists on top of of the layer that
defines external representations. 

The fact that the lower layer is in C is unfortunate, but not
important. The fact that the low layer exists outside the library
system is indeed important.

But, it's also not important that the lower layer persists across
phases. You can always separate compile time and run time in MzScheme
--- even at the level of symbols --- by restarting MzScheme.

> To fix this problem, I recommend the following.
> 
>   * Accept the recommendation of Ticket #110, and require
>     implementations of R6RS Scheme to use the shared binding
>     semantics.

This doesn't seem practical to me. A compiler like Chicken, for
example, would have to package compiled code with all state accumulated
at expand time.

>   * If there is some reason why we cannot require the shared
>     binding semantics for all libraries, and also cannot add
>     some way for libraries to declare that they must use the
>     shared binding semantics, then the R6RS should require
>     all implementations that use the separated binding semantics
>     to perform an automagical transformation of representation
>     at each phase boundary, so macros like the one shown in this
>     note will be guaranteed to work as programmers expect.

Implicitly wrap a `datum->syntax' around the result of any transformer?
That's fine with me.

>   * Requiring that automagical transformation would not be
>     enough to make portable libraries possible in general,
>     but it might become possible to write portable libraries
>     provided they don't define any record types.

I think portable libraries are possible, and I also think it's
important to restrict the range of `datum->syntax' to datums.

[Among PLT Schemers, we use the term "3-D macros" for macros that use
 `datum->syntax' on non-datums. (I think this term comes from certain
 years at Indiana --- and, for all I know, Will or Kent invented the
 term.) We don't like 3-D macros because they don't work right with
 separate compilation. That is, you get different results depending on
 whether you run directly or compile and run in a new Scheme session.
 Or, more often, the code generated by compiling with a 3-D macro
 simply cannot be marshaled to a persistent form.]

Matthew