> Matthias Felleisen wrote:
> We spent three years and good money from a
> Large Corporation on trying to
> move the core of MrEd from Mz to
> Larceny with little to show for. --
On 6/21/07, Thomas Lord <lord at emf.net> wrote:
> That's a serious bummer. I hope you'll accept this expression of
> sympathy.
>
> I would guess that interesting lessons were learned about why
> it turned out to be harder than it seems like it should be? Why
> did it prove to be so difficult?
There were a number of reasons. MrEd is written half in Scheme, half
in C++, and the Scheme and C++ code are very closely intertwined. The
C++ code in MrEd is old and has accumulated a huge amount of cruft to
support the advanced features of DrScheme. The Scheme code in MrEd
relies heavily on the MzScheme object system, but it uses advanced
features that allow MzScheme objects to inherit from the underlying
C++ objects that are used to implement the window system.
Larceny is most definitely *not* written in C++.
Here is *one* of the C++ functions in MrEd:
--------
void wxMediaEdit::_Insert(wxSnip *isnip, long strlen, wxchar *str,
wxList *snipsl,
long start, long end, Bool scrollOk)
{
long addlen, i, sPos, s, snipStartPos;
wxSnip *gsnip, *cSnip, *beforeSnip;
wxTextSnip *snip;
wxTabSnip *tabsnip;
Bool deleted = FALSE, insertedLine = FALSE, scroll;
wxNode *node;
if (writeLocked || userLocked)
return;
if (start < 0)
return;
if (start > len)
start = len;
if (caretStyle)
if (start != end || start != startpos)
caretStyle = NULL;
if (end > -1)
if (start < end) {
#if ALLOW_X_STYLE_SELECTION
if (!delayRefresh)
needXCopy = TRUE;
#endif
if (isnip || strlen || snipsl)
BeginEditSequence();
Delete(start, end, scrollOk);
deleted = TRUE;
#if ALLOW_X_STYLE_SELECTION
if (!delayRefresh)
needXCopy = FALSE;
#endif
}
if (!isnip && !strlen && !snipsl)
return;
/* If deleted, must end edit sequence after this point... */
writeLocked = TRUE;
if (isnip || snipsl) {
int did_one;
addlen = 0;
if (snipsl) {
for (node = snipsl->First(); node; node = node->Next()) {
isnip = (wxSnip *)node->Data();
if (!isnip->count)
goto give_up;
addlen += isnip->count;
if (isnip->IsOwned())
goto give_up;
}
} else {
if (!isnip->count)
goto give_up;
addlen = isnip->count;
if (isnip->IsOwned())
goto give_up;
}
if (!CanInsert(start, addlen))
goto give_up;
OnInsert(start, addlen);
flowLocked = TRUE;
/* Make sure OnInsert didn't do something bad to the snip(s): */
if (snipsl) {
for (node = snipsl->First(); node; node = node->Next()) {
isnip = (wxSnip *)node->Data();
if (!isnip->count)
goto give_up;
if (isnip->IsOwned())
goto give_up;
}
} else {
if (!isnip->count)
goto give_up;
if (isnip->IsOwned())
goto give_up;
}
did_one = 0;
beforeSnip = NULL;
for (node = snipsl ? snipsl->First() : NULL;
!snipsl || node;
node = node ? node->Next() : node) {
if (node)
isnip = (wxSnip *)node->Data();
if ((isnip->flags & wxSNIP_NEWLINE)&&!(isnip->flags &
wxSNIP_HARD_NEWLINE))
isnip->flags -= wxSNIP_NEWLINE;
if (!len && !did_one) {
/* Special case: ignore the empty snip */
snips = lastSnip = isnip;
lineRoot = new WXGC_PTRS wxMediaLine;
isnip->line = lineRoot;
lineRoot->snip = lineRoot->lastSnip = isnip;
if (maxWidth > 0)
lineRoot->MarkCheckFlow();
} else {
if (!did_one) {
MakeSnipset(start, start);
gsnip = FindSnip(start, +2);
beforeSnip = gsnip;
} else
gsnip = beforeSnip;
if (!gsnip) {
AppendSnip(isnip);
gsnip = lastLine->lastSnip;
if (gsnip && (gsnip->flags & wxSNIP_HARD_NEWLINE)) {
wxMediaLine *lr;
lr = lineRoot;
isnip->line = lastLine->Insert(&lr, FALSE);
lineRoot = lr;
isnip->line->snip = isnip->line->lastSnip = isnip;
numValidLines++;
insertedLine = TRUE;
} else {
isnip->line = lastLine;
if (!lastLine->snip)
lastLine->snip = isnip;
lastLine->lastSnip = isnip;
if (isnip->flags & wxSNIP_HARD_NEWLINE)
insertedLine = TRUE; /* b/c added extra ghost line */
}
} else {
InsertSnip(gsnip, isnip);
if (isnip->flags & wxSNIP_HARD_NEWLINE) {
wxMediaLine *lr;
lr = lineRoot;
isnip->line = gsnip->line->Insert(&lr, TRUE);
lineRoot = lr;
insertedLine = TRUE;
numValidLines++;
if (PTREQ(gsnip->line->snip, gsnip))
isnip->line->snip = isnip;
else
isnip->line->snip = gsnip->line->snip;
isnip->line->lastSnip = isnip;
gsnip->line->snip = gsnip;
for (cSnip = isnip->line->snip;
PTRNE(cSnip, isnip);
cSnip = cSnip->next) {
cSnip->line = isnip->line;
}
gsnip->line->CalcLineLength();
gsnip->line->MarkRecalculate();
} else {
isnip->line = gsnip->line;
if (PTREQ(isnip->line->snip, gsnip))
isnip->line->snip = isnip;
}
}
if (maxWidth > 0) {
isnip->line->MarkCheckFlow();
if (isnip->line->prev
&& !(isnip->line->prev->lastSnip->flags & wxSNIP_HARD_NEWLINE))
isnip->line->prev->MarkCheckFlow();
if ((isnip->flags & wxSNIP_HARD_NEWLINE) && isnip->line->next)
isnip->line->next->MarkCheckFlow();
}
}
isnip->style = styleList->Convert(isnip->style);
isnip->SizeCacheInvalid();
isnip->line->CalcLineLength();
isnip->line->MarkRecalculate();
len += isnip->count;
did_one = 1;
SnipSetAdmin(isnip, snipAdmin);
firstLine = lineRoot->First();
lastLine = lineRoot->Last();
if (!snipsl)
break;
}
} else {
int sp, cnt;
addlen = strlen;
if (!CanInsert(start, addlen))
goto give_up;
OnInsert(start, addlen);
flowLocked = TRUE;
if (!len) {
wxStyle *style;
style = ((stickyStyles & !initialStyleNeeded)
? snips->style
: GetDefaultStyle());
snip = InsertTextSnip(start, style);
sPos = 0;
caretStyle = NULL;
lineRoot->snip = lineRoot->lastSnip = snip;
} else {
wxStyle *style;
if (start)
gsnip = FindSnip(start, -1, &sPos);
else
gsnip = NULL;
if (!gsnip || (caretStyle && (gsnip->style != caretStyle))
|| !(gsnip->flags & wxSNIP_IS_TEXT)
|| (gsnip->count + addlen > MAX_COUNT_FOR_SNIP)
|| (!stickyStyles
&& (gsnip->style != GetDefaultStyle()))) {
style = (caretStyle
? caretStyle
: (stickyStyles
? (gsnip
? gsnip->style
: snips->style) // No style: use forward
: GetDefaultStyle()));
snip = InsertTextSnip(start, style);
caretStyle = NULL;
sPos = start;
} else {
snip = (wxTextSnip *)gsnip;
if (!(snip->flags & wxSNIP_CAN_APPEND)) {
style = (stickyStyles
? snip->style
: GetDefaultStyle());
snip = InsertTextSnip(start, style);
sPos = start;
}
}
if (gsnip && (gsnip->flags & wxSNIP_HARD_NEWLINE) &&
(gsnip->next == snip)) {
/* Preceeding snip was a newline, so the new slip belongs on the next line: */
wxMediaLine *oldline = gsnip->line, *newline;
if (!oldline->next) {
wxMediaLine *lr;
lr = lineRoot;
oldline->Insert(&lr, FALSE);
lineRoot = lr;
insertedLine = TRUE;
numValidLines++;
oldline->next->lastSnip = snip;
}
newline = oldline->next;
snip->line = newline;
oldline->lastSnip = gsnip;
newline->snip = snip;
oldline->CalcLineLength();
oldline->MarkRecalculate();
}
}
s = start - sPos;
snip->flags |= wxSNIP_CAN_SPLIT;
snip->Insert(str, addlen, s);
if (snip->flags & wxSNIP_CAN_SPLIT)
snip->flags -= wxSNIP_CAN_SPLIT;
snip->line->CalcLineLength();
snip->line->MarkRecalculate();
if (maxWidth > 0) {
snip->line->MarkCheckFlow();
if (snip->line->prev
&& !(snip->line->prev->lastSnip->flags & wxSNIP_HARD_NEWLINE))
snip->line->prev->MarkCheckFlow();
}
snipStartPos = start;
str = snip->buffer;
sp = s + snip->dtext;
cnt = 0;
for (i = 0; i < addlen; i++) {
if (str[sp] == '\r')
str[sp] = '\n';
if (str[sp] == '\n' || str[sp] == '\t') {
Bool newline = (str[sp] == '\n');
MakeSnipset(i + start, i + start + 1);
snip = (wxTextSnip *)FindSnip(i + start, +1);
if (newline) {
/* Forced return - split the snip */
wxMediaLine *oldLine;
snip->flags |= wxSNIP_NEWLINE | wxSNIP_HARD_NEWLINE
| wxSNIP_INVISIBLE;
snip->flags -= (snip->flags & wxSNIP_CAN_APPEND);
insertedLine = TRUE;
if (PTRNE(snip, snip->line->lastSnip)) {
wxMediaLine *lr;
oldLine = snip->line;
lr = lineRoot;
snip->line = oldLine->Insert(&lr, TRUE);
lineRoot = lr;
numValidLines++;
snip->line->lastSnip = snip;
snip->line->snip = oldLine->snip;
/* Retarget snips moved to new line: */
for (cSnip = snip->line->snip;
PTRNE(cSnip, snip);
cSnip = cSnip->next) {
cSnip->line = snip->line;
}
oldLine->snip = snip->next;
oldLine->CalcLineLength();
oldLine->MarkRecalculate();
if (maxWidth > 0)
oldLine->MarkCheckFlow();
snip->line->CalcLineLength();
snip->line->MarkRecalculate();
if (maxWidth > 0)
snip->line->MarkCheckFlow();
} else {
/* Carriage-return inserted at the end of a auto-wrapped line.
Line lengths stay the same, but next line now starts
a paragraph. */
if (snip->line->next)
if (!snip->line->next->StartsParagraph())
snip->line->next->SetStartsParagraph(TRUE);
}
} else {
wxSnip *rsnip;
tabsnip = OnNewTabSnip();
if (tabsnip->IsOwned() || tabsnip->count) {
/* Uh-oh. */
tabsnip = new WXGC_PTRS wxTabSnip();
}
tabsnip->style = snip->style;
rsnip = SnipSetAdmin(tabsnip, snipAdmin);
if (rsnip!= tabsnip) {
/* Uh-oh. */
tabsnip = new WXGC_PTRS wxTabSnip();
tabsnip->style = snip->style;
tabsnip->SetAdmin(snipAdmin);
}
tabsnip->flags |= wxSNIP_CAN_SPLIT;
tabsnip->InsertUTF8("\t", 1, 0);
if (tabsnip->flags & wxSNIP_CAN_SPLIT)
tabsnip->flags -= wxSNIP_CAN_SPLIT;
SpliceSnip(tabsnip, snip->prev, snip->next);
tabsnip->line = snip->line;
if (PTREQ(snip->line->snip, snip))
tabsnip->line->snip = tabsnip;
if (PTREQ(snip->line->lastSnip, snip))
tabsnip->line->lastSnip = tabsnip;
DELETE_OBJ snip;
}
snip = (wxTextSnip *)FindSnip(i + start + 1, +1);
snipStartPos = i + start + 1;
str = snip->buffer;
sp = snip->dtext;
cnt = 0;
} else if (cnt > MAX_COUNT_FOR_SNIP) {
/* Divide up snip because it's too large: */
MakeSnipset(i + start, i + start);
snip = (wxTextSnip *)FindSnip(i + start, +1);
snipStartPos = i + start;
str = snip->buffer;
sp = snip->dtext + 1;
cnt = 1;
} else {
sp++;
cnt++;
}
}
firstLine = lineRoot->First();
lastLine = lineRoot->Last();
len += addlen;
}
initialStyleNeeded = 0;
revision_count++;
AdjustClickbacks(start, start, addlen, NULL);
if (!modified) {
wxUnmodifyRecord *ur;
ur = new WXGC_PTRS wxUnmodifyRecord(delayedStreak);
AddUndo(ur);
}
if (!noundomode) {
wxInsertRecord *ir;
ir = new WXGC_PTRS wxInsertRecord(start, addlen,
deleted || typingStreak || delayedStreak
|| insertForceStreak
|| !modified,
startpos, endpos);
AddUndo(ir);
}
if (delayRefresh)
delayedStreak = TRUE;
scroll = (startpos == start);
if (startpos >= start)
startpos += addlen;
if (endpos >= start)
endpos += addlen;
if (!refreshUnset) {
if (refreshStart >= start)
refreshStart += addlen;
if (refreshEnd >= start)
refreshEnd += addlen;
}
extraLine = !!(lastSnip->flags & wxSNIP_NEWLINE);
writeLocked = FALSE;
flowLocked = FALSE;
if (scroll)
caretBlinked = FALSE;
if (scroll && scrollOk) {
delayRefresh++;
ScrollToPosition(startpos);
--delayRefresh;
}
changed = TRUE;
caretStyle = NULL;
if (insertedLine) {
if (!graphicMaybeInvalid)
graphicMaybeInvalid = TRUE;
NeedRefresh(start);
} else
RefreshByLineDemand();
if (deleted)
EndEditSequence();
if (!modified)
SetModified(TRUE);
AfterInsert(start, addlen);
return;
give_up:
writeLocked = FALSE;
flowLocked = FALSE;
if (deleted)
EndEditSequence();
return;
}
----------------
Anybody still here?
So what do we do with this? There are several options:
1) Just use the code as-is and link it in as a foreign library to Larceny.
But this would require Larceny to act just like MzScheme at a very
fundamental level. (Consider that the wx event loop is tied closely to
the MzScheme threading model, and that the threading model has
intimate knowledge of the stack layout...)
This approach seemed pointless because it is simply porting the bulk
of MzScheme to a library and Larceny would be acting as a bolted on
REPL and nothing more.
2) Rewrite the code by hand.
Tried it. It is hard. Never mind the multi-page procedures, C++ code is
written at such a low level of abstraction that it is hard to
figure out how
to lift it up to the appropriate Scheme code. C++ is a truly
awful language
that encourages an imperative pointer-bashing style that is extremely
difficult to understand. In addition, the threading and evaluation model
make appearances here and there because there is no easy way in C++
to abstract these out.
3) Rewrite the code by machine.
Tried it. It is also hard. But we made a lot of progress
here. I have a
C++ -> Scheme compiler that can handle most of the MrEd window
system. Nonetheless, the low-level issues of threading and evaluation
are still a problem with this approach and I haven't figured out how to
cope with them.
We did learn quite a few things, but not what we hoped.
1) Legacy code is a problem.
2) C++ is a tar pit. (yeah, we knew this before, but it is worse than
you'd think)
3) There are few interesting problems (research-worthy problems) in
this domain, just a hell of a lot of scutwork.
This is my take on it at least.
--
~jrm
Received on Thu Jun 21 2007 - 13:42:59 UTC