30 Sep
2026
30 Sep
'26
12:26 p.m.
Ethan Dicks wrote:
On Wed, Sep 30, 2026 at 11:10 AM Paul Koning via cctalk <cctalk(a)classiccmp.org>
wrote:
Microsoft 8-bit BASIC tokenizes when you enter a new line of BASIC but
whitespace is not required between tokens. It can get confused in very
specific edge cases but mostly, it works. When executing tokenized
code, outside of quotes, the actual next-char fetch routine in low mem
(CHRGET) explicitly skips whitespace, so the token evaluator never
sees it.
The stream of tokens and untokenized args presented to the BASIC
statement evaluator sees exactly the same stream from the following
two lines...
10FORX=1TO100:PRINTX:NEXTX
and
10 FOR X = 1 TO 100 : PRINT X : NEXT X
Those spaces are stored in RAM but CHRGET skips them on each fetch.
-ethan
My recollection is that BBC Basic (on the BBC Micro) similarly ignores space
characters but not tabs or other characters that might traditionally be
regarded as whitespace. I don't recall any edge cases that cause it to be
confused but IIRC pseudovariable tokens (like TIME for example) need a space
or other separator character after them to avoid difficulties accessing actual
variables that begin with the names of pseudovariables (such as TIMER for
example).
I've tried the above two lines in BBC Basic in my BBC Micro emulator and they
both behave the same.
One slightly interesting anomaly in BBC Basic is that the REPEAT token does not
even require a colon following it. I don't know if this is by accident or by
design but the following works:
10X=1:REPEATPRINTX:X=X+1:UNTILX>100
Regards,
Peter Coghlan.
On Sep
30, 2026, at 9:30 AM, ben via cctalk <cctalk(a)classiccmp.org> wrote:
On 2026-09-29 5:57 p.m., Jim Davis via cctalk wrote:
Huh? The only language I can think of that ignores whitespace is TECO.
Just about everything else treats whitespace as a token separator. And all our daily driver languages look like
Algol or FORTRAN,
excepting Forth and APL, both written by mad scientists,
And token parsing still is from the 1950's for many languages
where blank spaces are ignored and a end of line/card or semicolon
is needed to separate id's.