[R6RS] modules?

[R. Kent Dybvig]

> I understand the theory, but I'm interested in seeing actual code that
> uses SC. (I think I learn a lot by seeing realistic examples in
> addition to abstract/contrived examples.) Any pointers to realistic
> code?

Most of the code I've written and seen that makes use of local modules
is proprietary.  I would be happy to show ours to you sometime, but
I can't mail it out.  There are a couple of uses of local (anonymous)
modules in the portable syntax-case implementation at

  http://www.cs.indiana.edu/chezscheme/syntax-case/psyntax.ss

but the bulk of that code was written before we added modules.  There may
be other local-module code out in the world that I haven't seen.

> > While it's true that internal modules don't work quite the same as
> > top-level modules, this is really just a reflection of the top level and
> > the concessions we make with it for interactive programming. 
> 
> Suppose that I'm willing to concede interactive programming, instead,
> and the program I want to write is
> 
> ...
>
> To make sure my modules behave like internal modules, I'll wrap my
> program in `let'. But the following doesn't work:
> 
>  (let ()
>    (module m (helper)
>      (define table ... #| big computation |# )
>      (define (helper args) ...))
>    (module n ()
>      (require m)
>      (define-syntax (macro stx) ... (helper arg) #| not in template |# ...)
>      (define x ... (marco stx) ... (helper arg) ...))
>    'done)
> 
> How should I write the `m' module so that I can use it in both macro
> implementations and run-time expressions?

You can't do it with modules alone.  The top level is certainly useful
for linking together separately compiled code and for making externally
defined libraries available for use by compile-time and run-time code.
Internal modules obviously can't be used for this purpose.  In this
respect, top-level modules differ from internal modules in exactly the
same way as top-level variable definitions differ from internal variable
definitions, as you pointed out earlier.  So please disregard my careless
comment about interactive programming.  As Oscar Waddell pointed out
to me recently, the top level really gives us a handle on the linker,
which is useful for more than just interactive programming.

In many examples like the one above, what one really wants are expand-time
helpers and data to be shared by multiple macros.  Placing them in a
separate top-level module, as you've shown and which one can do in either
of our systems, is often clumsy and leads to module name-space clutter.

Our mechanism for establishing expand-time, or "meta", variable bindings
is orthogonal to modules and allows us to have both expand-time and
run-time variable bindings in the same module or lambda/let/letrec
body, e.g.:

  (module foo (macro-helper a b)
    (meta define table
     ; pretend this is a "big computation":
      (map cons '(#\a #\b #\c) '(1 2 3)))
    (meta define lookup
      (lambda (c)
        (cond [(assq c table) => cdr] [else #f])))
    (meta define macro-helper
      (lambda (x)
        (syntax-case x ()
          [(k c)
           (with-syntax ([n (lookup (syntax-object->datum #'c))])
             #'(list '(k c) a n))])))
    (define a 'is)
    (define-syntax b
      (lambda (x) (macro-helper x))))

As with define-syntax rhs expressions, meta define rhs expressions can
evaluate references only to identifiers whose values are (already)
available in the compile-time environment, e.g., other macros and
meta variables.  They can, of course, build syntax objects containing
occurrences of any identifiers in their scope.

When we import from foo we can write macros that use the helper or insert
references to any of the exports.  This works regardless of whether foo
is top-level or internal.

  (module bar (c)
    (import foo)
    (define-syntax c
      (lambda (x) (macro-helper x))))

  (define d
    (lambda ()
      (import foo)
      (import bar)
      (list a (b #\b) (c #\c))))

  (d) -> (is ((b #\b) is 2) ((c #\c) is 3))

meta works through macro expansion, so we can write, e.g.,

  (module (a)
    (meta define-record foo (x))
    (define-syntax a
      (let ([q (make-foo #''q)])
        (lambda (x)
          (foo-x q)))))

  a -> q

where define-record is a macro that expands into a set of defines.

It is also sometimes convenient to write

  (meta begin defn ...)

or

  (meta module {exports} defn ...)

to create groups of meta bindings.

We use meta in our oop implementation, which looks something like this:

  (module oop (define-class define-interface ---)
    (meta define parse-formals
      (lambda (---) ---))
    (meta define build-generic
      (lambda (---) ---))
    (meta define-record class ; compile-time rep'n of a class
      ---)
    ---
    (define-syntax define-class
      (lambda (x)
        ---))
    (define-syntax define-interface
      (lambda (x)
        ---))
    ---)

In this case, we have no need to use the expand-time helpers and class
accessors outside the module, so we don't export them.

> How do you deal with macro-generating macros that also introduce
> regular definitions that are private to generated macro?

We do run into this, and the solution is simple: a macro-generating
macro that introduces hidden definitions should expand into an anonymous
module exporting the macro with its hidden exports.  In your example,
this means replacing the begin form in the output of define-cbr with a
module form as shown below.

  (define-syntax define-cbr
    (syntax-rules ()
      [(_ (id arg ...) body)
       (module ((id do-f))
         (define-cbr-as-cbv do-f (arg ...)
           () body)
         (define-syntax id
           (syntax-rules ()
             [(id actual (... ...))
              (do-f (lambda () actual) 
                    (... ...)
                    (lambda (v) 
                      (set! actual v))
                    (... ...))])))]))

Now if you write

  (module (f)
    (define-cbr (f a b)
      (set! a b)))

all is well, since implicit exports are propagated out as necessary.

> The bigger problem, for us, is that we'd like to restrict access to
> identifiers that do show up in macro expansions, too. In other words,
> we want the identifier used only in the context that the macro gives
> it. So there's a more general problem that I think must have a more
> general solution.

I sympathize, but I'm not sure there is a more general solution.

We thought long and hard about possible alternatives before settling on
explicitly listing hidden exports.  If there's an alternative we missed
that gives the same degree or better of expressiveness, security, and
efficiency, I'd be happy to discontinue listing hidden exports.

Kent

PS. I assume you meant to imply that "big computation" is performed only
once in your system in the following:

> (module m mzscheme
>   (define table ... #| big computation |# )
>   (define (helper args) ...)
>   (provide helper))
> (module n mzscheme
>   (require m)
>   (require-for-syntax m)
>   (define-syntax (macro stx) ... (helper arg) #| not in template |# ...)
>   (define x ... (marco stx) ... (helper arg) ...))

But isn't it actually performed once as a result of the (require m)
and once again as a result of the (require-for-syntax m)?