Smalltalk/X Webserver

Documentation of class 'Parser':

Class: Parser

Inheritance
Description
Related information
Class protocol
Instance protocol
Private classes
Examples

Inheritance:

   Object
   |
   +--Scanner
      |
      +--Parser
         |
         +--AbstractSyntaxHighlighter
         |
         +--BreakpointAnalyzer
         |
         +--ByteCodeCompiler
         |
         +--Explainer
         |
         +--Parser::PrimitiveSpecParser

Package:: stx:libcomp

Category:: System-Compiler

Version:: rev: 1.1030 date: 2019/08/17 12:36:09; user: cg; file: Parser.st directory: libcomp; module: stx stc-classLibrary: libcomp

Author:: Claus Gittinger

Description:

Parser is used for both parsing and evaluating smalltalk expressions;
it first builds a parseTree which is then interpreted (evaluate) or
compiled. Compilation is done in the subclass ByteCodeCompiler and/or
the (planned) MachineCodeCompiler.

methods of main interest are:
    Parser evaluateExpression:...

and:
    Parser parseExpression:...
    Parser parseMethod:...

there is protocol to parse complete methods, selector specs, body only etc.

Parser is also used to find the referenced/modified inst/classvars of
a method - this is done by sending parseXXX message to a parser and asking
the parser for referencedXVars or modifiedXVars (see SystemBrowser).

You can also use parsers for all kinds of other things (ChangesBrowser for
example analyzes the expressions in the changelist ...) by looking at the
parsers tree. (Although this is somewhat dangerous, since it exports the
compilers internals ... better style is to add specialized query methods here)

One instance of Parser is created to parse one method or expression - i.e.
its not suggested to reuse parsers.


  Constant folding:

The parser has various modes for constant folding; by default, only numeric
expressions involving integers and floats are constant folded
(i.e. something like 'Float pi sin' or '1.5 + 0.3' will be reduced to a constant).

Constant folding can be turned off completely (setting FoldConstants to nil)
to ``secure folding'', which only folds constant numbers (#level1) or to #full.
In full mode, more constant expressions are folded (for example: '1.0 @ 1.0' is
reduced to a constant point), but the resulting code may not be compatible with other
smalltalk systems (consider the case, where the point is modified using #x: or #y: messages).
Therefore, this mode is a bit dangerous and disabled by default.
The current implementation, base upon a global constant-folding setting is somewhat stupid
and intermediate - a better solution would be to allow the optimization to be controlled
by a method-pragma, since it may make sense to disable optimization on a method level,
if it's known that the used constant objects are subject of modifications as described above.


  Immutable arrays:

Immutable arrays are experimental and being evaluated.
Consider the case of a method returning '#(1 2 3 4)', and that array being modified
by some other method (using #at:put:). Since the array-return is actually a return of
a reference to the compiler created array, the next invokation of the method will
return the modified array. These are hard to find bugs.
By an option, the compiler can generate immutable arrays, which don't allow modification
of its elements. For clean code, you should enable this option during development.

As mentioned above, this is experimental. If it is reported to be a useful feature,
the immutable feature will be extended to strings, point-literals etc. in a future version
of st/x.



[Instance variables:]

    classToCompileFor   <Class>             the class (or nil) we are compiling for

    selfValue           <any>               value to use as self when interpreting

    contextToEvaluateIn <Context>           the context (or nil) when interpreting

    selector            <Symbol>            the selector of the parsed method
                                            (valid after parseMethodSpecification)
    methodArgs                              internal

    methodArgNames      <Collection>        the names of the arguments
                                            (valid after parseMethodSpecification)

    methodVars                              internal

    methodVarNames      <Collection>        the names of the method locals
                                            (valid after parseMethodBodyVarSpec)

    tree                <ParseTree>         the parse tree - valid after parsing

    currentBlock                            if currently parsing for a block

    usedInstVars                            set of all accessed instances variables
                                            (valid after parsing)

    usedClassVars                           same for classVars

    usedVars                                all used variables (inst, class & globals)

    modifiedInstVars                        set of all modified instance variables

    modifiedClassVars                       same for clasVars

    localVarDefPosition <Integer>           the character offset of the local variable
                                            def. (i.e. the first '|' if any)
                                            Not yet used - prepared for automatic add of
                                            undefined variables

    evalExitBlock                           internal for interpretation

    selfNode            <Node>              cached one-and-only 'self' node
    superNode           <Node>              cached one-and-only 'super' node

    hasPrimitiveCode    <Boolean>           true, if it contains ST/X style primitive code
    hasNonOptionalPrimitiveCode
                        <Boolean>           true, if it contains ST/X style primitive code
                                            which is NOT flagged by the OPTIONAL directive.

    primitiveNr         <Integer>           the parsed ST-80 type primitive number (or nil)

    logged

    warnedUndefVars     <Set>               set of all variables which the parser has
                                            already output a warning (to avoid multiple
                                            warnings about the same variable)

[Class variables:]

    PrevClass           <Class>             class, of which properties are
                                            cached in:

    PrevInstVarNames      <Collection>      instance variablenames of cached class
    PrevClassVarNames     <Collection>      class variablenames of cached class
    PrevClassInstVarNames <Collection>      class instance variablenames of cached class

    LazyCompilation       <Boolean>         EXPERIMENTAL: lazy compilation

    ArraysAreImmutable    <Boolean>         if true, create array literals
                                            as instances of ImmutableArray,
                                            which cannot be stored into.
                                            Default is false, for compatibility.
                                            Can be turned on while developing
                                            new code to make certain that side
                                            effects are avoided.

    StringsAreImmutable   <Boolean>         same as above for string literals

    WarnST80Directives    <Boolean>         if true, give warnings about
                                            ST-80 directives (resource defs)
                                            which are ignored in st/x.
                                            defaults to false.

    FoldConstants         <Symbol>          controls how constant folding should be
                                            done.
                                            Can be one of:
                                                    nil      - no constant folding
                                                    #level1  - numeric optimizations only
                                                    #level2  - secure optimizations only
                                                    #full    - full folding

                                            level1:   arithmetic on constant numbers

                                            level2:   above PLUS array conversions with #asFloatArray,
                                                      #asDoubleArray, string concatenation

                                            full:     constant points.

Related information:

    ByteCodeCompiler
    Scanner
    ObjectFileLoader
    Workspace
    SystemBrowser

Class protocol:

Compatibility-ST80

parse: aString class: aClass: same as #parseMethod:in: for ST80 compatibility.

Compatibility-Squeak

evaluate: someString for: aReceiver logged: logged
evaluate: aStringOrStream for: aReceiver notifying: someone logged: logged

Signal constants

askForVariableTypeOfUndeclaredQuery
correctByDeclaringIdentifierAs
correctByInteractiveCorrection
correctByInteractiveRename
parseErrorSignal
parseWarningSignal
possibleCorrectionsQuery
restartCompilationSignal
undefinedSuperclassError
undefinedVariableError
undefinedVariableNotification

change & update

flushNameCache

unconditional flush name caches

usage example(s):

Parser flushNameCache

update: something with: someArgument from: changedObject

aClass has changed its definition - flush name caches if we have to

class initialization

initialize

usually set to true in your .rc file

usage example(s):

     self initialize

controlling compilation

allowArrayIndexSyntaxExtension

experimental

allowArrayIndexSyntaxExtension: aBoolean

experimental

usage example(s):

     self allowArrayIndexSyntaxExtension:true
     self allowArrayIndexSyntaxExtension:false

allowFunctionCallSyntaxForBlockEvaluation

experimental

allowFunctionCallSyntaxForBlockEvaluation: aBoolean

experimental

usage example(s):

     self allowFunctionCallSyntaxForBlockEvaluation:true
     self allowFunctionCallSyntaxForBlockEvaluation:false

allowReservedWordsAsSelectors

if true, 'self', 'super', 'thisContext', 'nil', 'true' and 'false' are allowed
as unary message selectors.

allowReservedWordsAsSelectors: aBoolean

enable/disable, if 'self', 'super', 'thisContext', 'nil', 'true' and 'false' are to be allowed
as unary message selectors.

usage example(s):

     self allowReservedWordsAsSelectors:true
     self allowReservedWordsAsSelectors:false

arraysAreImmutable

return true if arrays are immutable literals

arraysAreImmutable: aBoolean

turn on/off immutable array literals - default is false for ST-80 compatibilty.

usage example(s):

     can be added to your private.rc file:

     Compiler arraysAreImmutable:true
     Compiler arraysAreImmutable:false

compileLazy

return true if compiling lazy

compileLazy: aBoolean

turn on/off lazy compilation - return previous setting.
Actually this flag belongs into the ByteCodeCompiler subclass,
but it also controls the reporting of some errors here; therefore
its located here

usage example(s):

     Compiler compileLazy:false
     Compiler compileLazy:true

foldConstants

return a symbol describing how constants are to be folded

foldConstants: aSymbol

set the symbol describing how constants are to be folded.
It can be:
nil - no constant folding
#level1 - numeric constants only
#level2 - level1 PLUS array conversions PLUS string concatenation
#full - level2 PLUS constant points, constant rectangles (dangerous)

implicitSelfSends

return true if undefined variables with
lowercase first character are to be turned
into implicit self sends

implicitSelfSends: aBoolean

turn on/off implicit self sends

usage example(s):

     ParserFlags implicitSelfSends:true
     ParserFlags implicitSelfSends:false

lineNumberInfo

lineNumberInfo: aBoolean

stringsAreImmutable

return true if strings are immutable literals

stringsAreImmutable: aBoolean

turn on/off immutable string literals - default is false for ST-80 compatibilty.

usage example(s):

     can be added to your private.rc file:

     ParserFlags stringsAreImmutable:true
     ParserFlags stringsAreImmutable:false

warnAboutBadComments: aBoolean

controls generation of warning messages about empty comments

warnAboutVariableNameConventions

controls generation of warning messages about wrong variable names

warnAboutVariableNameConventions: aBoolean

controls generation of warning messages about wrong variable names

warnAboutWrongVariableNames

controls generation of warning messages about wrong variable names

warnAboutWrongVariableNames: aBoolean

controls generation of warning messages about wrong variable names

warnUnusedVars

controls generation of warning messages about unued method variables

warnUnusedVars: aBoolean

controls generation of warning messages about unued method variables

defaults

maxLineNumber: return the maximum lineNumber that is possibly
encoded in a methods byteCode debugging information.
Since lineNumber entries only have 1 byte available,
this is 255 for byteCode methods.

error correction

findBest: nMax selectorsFor: aString in: aClassOrNil

collect known selectors with their spelling distances to aString;
return the nMax best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class (i.e. offer
corrections from that hierarchy only).
soon OBSOLETE: to be moved to DWIM

findBest: nMax selectorsFor: aString in: aClassOrNil forCompletion: forCompletion

soon OBSOLETE and moved to DWIM.
Collect known selectors with their spelling distances to aString;
return the nMax best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class
(i.e. offer corrections from that hierarchy only).
If forCompletion isTrue, prefix sequences are preferred.
The way spelling distance is computed is a heuristic which works
well in real life (i.e. offer USEFUL suggestions)

usage example(s):

     'select:' spellAgainst:'collect'    57
     'collectWithIndex:' spellAgainst:'collect' 41

     self findBest:20 selectorsFor:'i' in:nil forCompletion:true

     self findBest:20 selectorsFor:'collect' in:Collection forCompletion:true
     self findBest:100 selectorsFor:'collect' in:Collection forCompletion:true

     self findBest:20 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true
     self findBest:100 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true

findBest: nMax selectorsFor: aString in: aClassOrNil forCompletion: forCompletion ignoreIfAnnotatedWith: annotationOrNil

usage example(s):

     'select:' spellAgainst:'collect'    57
     'collectWithIndex:' spellAgainst:'collect' 41

     self findBest:20 selectorsFor:'i' in:nil forCompletion:true

     self findBest:20 selectorsFor:'collect' in:Collection forCompletion:true
     self findBest:100 selectorsFor:'collect' in:Collection forCompletion:true

     self findBest:20 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true
     self findBest:100 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true

findBestSelectorsFor: aString

collect known selectors with their spelling distances to aString;
return the 10 best suggestions.
soon OBSOLETE: to be moved to DWIM

usage example(s):

     Parser findBestSelectorsFor:'at'
     Parser findBestSelectorsFor:'at:pu'

findBestSelectorsFor: aString in: aClassOrNil

collect known selectors with their spelling distances to aString;
return the N best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class (i.e. offer
corrections from that hierarchy only).
soon OBSOLETE: to be moved to DWIM

evaluating expressions

evaluate: aStringOrStream

return the result of evaluating an expression in aStringOrStream.
No doit-entry is added to the changeLog.

usage example(s):

     Compiler evaluate:'1 + 2'
     Compiler evaluate:'''hello world'' asSortedCollection displayString printNL'
     Compiler evaluate:'''hello world'' asSortedCollection printNL'

evaluate: aStringOrStream compile: compile

return the result of evaluating aString,
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

evaluate: aStringOrStream ifFail: failBlock

return the result of evaluating an expression in aStringOrStream.
In case of any syntax errors, return the value of failBlock.
No doit-entry is added to the changeLog.

usage example(s):

     Compiler evaluate:'1 +' ifFail:['oops']

evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor ifFail: failBlock

return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
No doIt entry is added to the change-file.
If the failBlock argument is non-nil, it is evaluated if an error occurs.

evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock

evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile

return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

evaluate: aStringOrStream logged: logged

return the result of evaluating an expression in aStringOrStream.
The argument log controls if an entry is added to the changeLog.

usage example(s):

     Compiler evaluate:'''some string''' logged:false
     Compiler evaluate:'''some string''' logged:true

evaluate: aStringOrStream notifying: requestor

return the result of evaluating aString,
errors are reported to requestor

evaluate: aStringOrStream notifying: requestor compile: compile

return the result of evaluating aString,
errors are reported to requestor.
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

evaluate: aStringOrStream notifying: requestor logged: logged

return the result of evaluating aString,
errors are reported to requestor

evaluate: aStringOrStream receiver: anObject

return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars

usage example(s):

     Compiler evaluate:'self x' receiver:(1 @ 2)

evaluate: aStringOrStream receiver: someOne logged: logged

return the result of evaluating an expression in aStringOrStream.
The argument log controls if an entry is added to the changeLog.

usage example(s):

     Compiler evaluate:'''some string''' logged:false
     Compiler evaluate:'''some string''' logged:true
     Compiler evaluate:'self class' receiver:nil logged:false
     Compiler evaluate:'self class' receiver:1 logged:false

evaluate: aStringOrStream receiver: anObject notifying: requestor

return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars (used in the inspector)

usage example(s):

     Compiler evaluate:'self x' receiver:(1 @ 2) notifying:nil

evaluate: aStringOrStream receiver: anObject notifying: requestor compile: compile

return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars (used in the inspector).
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

evaluateFrom: aStringOrStream ifFail: failBlock

return the result of evaluating an expression from aStringOrStream.
In case of any syntax errors, return the value of failBlock.
No doit-entry is added to the changeLog.

evaluateFrom: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile

return the result of evaluating the next expression from aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

general helpers

argAndVarNamesForContext: aContext

methodCommentFromSource: aString

return the method's comment.
This is done by searching for and returning the first comment
from the method's source (excluding any double-quotes).
Returns nil if there is no comment.

methodCommentsFromSource: aString

return all of the method's comments.
Returns an empty collection if there is no comment.

usage example(s):

     Parser methodCommentsFromSource:(Method compiledMethodAt:#comment) source
     Parser methodCommentsFromSource:(Object class compiledMethodAt:#infoPrinting:) source

instance creation

for: aStringOrStream in: aClass: return a new parser, reading code for aClass from aStringOrStream

parsing

blockAtLine: line in: aMethod orSource: aString numArgs: nA numVars: nV

given a lineNr in some method, return the containing BlockNode or nil.
The given lineNr must be within a block for this to work.
This is used by the debugger, to guess reverse from a lineNumber,
to the corresponding block, in order to find out the block's
variable names
(mhmh - all of this wasnt't needed, if blocks stored their characterPosition internally).
WARNING: sometimes, the block is not correctly identified, if multiple blocks
are in one line.

checkMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings

parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

usage example(s):

     self
        checkMethod:'foo
                        |local1 local2 local3|

                        local1 := local2.
                        ^ local3
                    '
        in:UndefinedObject
        ignoreErrors:true
        ignoreWarnings:true

parseExpression: aString

parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or #Error (syntactic error).
Error and warning messages are suppressed.

usage example(s):

     Parser parseExpression:''
     Parser parseExpression:'self foo'
     Parser parseExpression:'^ self foo'
     Parser parseExpression:'self:123'
     Parser parseExpression:'self:123' onError:nil

parseExpression: aString inNameSpace: aNameSpaceOrNil

parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or #Error (syntactic error).
Error and warning messages are suppressed.

parseExpression: aString inNameSpace: aNameSpaceOrNil onError: errorValue

parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or errorValue (syntactic error).
Error and warning messages are suppressed.

parseExpression: aString onError: errorValue

parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or errorValue (syntactic error).
Error and warning messages are suppressed.

parseLiteralArray: aStringOrStream

Parser literal array in given String or Stream.
Returns that array

parseMethod: aString

parse a method.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors etc.
Error and warning messages are sent to the Transcript.

parseMethod: aString in: aClass

parseMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings

parse a method in a given class.
Return a parser (if ok) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

parseMethod: aString in: aClass warnings: warnBoolean

parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The warnBoolean arguments specifies if warning
messages should be sent to the Transcript or suppressed.

This method is OBSOLETE, and left in for backward compatibility.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

parseMethodArgAndVarSpecification: aString

parse a methods selector, arg and var spec (i.e. locals);
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Error and warning messages are sent to the Transcript.
This method is OBSOLETE.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

parseMethodArgAndVarSpecification: aString in: aClass

parse a methods selector, arg and var spec in a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals.
Error and warning messages are sent to the Transcript.
This method is OBSOLETE.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

parseMethodArgAndVarSpecification: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings parseBody: parseBodyBoolean

parse a method's selector, arg and var spec in a given class;
If parseBodyBoolean is true, also parse the statements
(for primitives & resourceSpecs).
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals.
(and also: initializerExpressions)

parseMethodArgAndVarSpecificationSilent: aString

parse a methods selector, arg and var spec (i.e. locals);
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodArgAndVarSpecification:, but does NOT
display error/warning messages on the transcript.

parseMethodArgAndVarSpecificationSilent: aString in: aClass

parse a methods selector, arg and var spec in a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals.
Like #parseMethodArgAndVarSpecification:in:, but does not
display error/warning messages on the transcript.

parseMethodSilent: aString

parse a method.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors etc.
Like #parseMethod:, but warning/error messages are suppressed.

parseMethodSilent: aString in: aClass

parseMethodSpecification: aString

parse a methods selector & arg specification;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.

parseMethodSpecification: aString in: aClass

parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.

parseMethodSpecification: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings

parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
noErrors and noWarnings specify if error- and warningMessages are
to be output onto the Transcript.

parseMethodSpecificationSilent: aString

parse a methods selector & arg specification;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodSpecification:, but does not display any error/warning Messages on the transcript.

parseMethodSpecificationSilent: aString in: aClass

parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodSpecification:in:, but does not display any error/warning Messages on the transcript.

selectorInExpression: aString

parse an expression - return the selector.
Even malformed expressions
(such as missing receiver or missing arg) are parsed.
Used for the SystemBrowser's implementors/senders query-box initial text.
Returns nil if unparsable.

usage example(s):

    Parser selectorInExpression:'foo at:1 put:(5 * bar)'
    Parser selectorInExpression:'(foo at:1) at:1'
    Parser selectorInExpression:'a + 4'
    Parser selectorInExpression:'a negated'
    Parser selectorInExpression:'at:1 put:5'
    Parser selectorInExpression:'at:1 put:'
    Parser selectorInExpression:'a at:1 put:5'
    Parser selectorInExpression:'a at:1 put:'
    Parser selectorInExpression:'a := foo at:1 put:5'

withSelf: anObject parseExpression: aString notifying: someOne

parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box.

withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignore

withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings

withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil

withSelf: anObject parseExpression: aStringOrStream onError: errorValue notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil

parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or errorValue (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

private

genMakeArrayWith: elementExpressions: return a node to generate an array at runtime.
Will generate:
literal shallowCopy (if all elements are literals)
or else:
Array with:el1 ... with:elN (if N <= 8)
or else:
(Array new at:1 put:el1; ... at:N put:elN; yourself) (otherwise)
implementedInAnyClass: aSelectorStringOrSymbol
makeImmutable: anObject: helper to optionally make Array-, ByteArray- and String literals immutable.
Creates and returns an immutable copy of the object

unparsing

methodSpecificationForSelector: aSelector

given a selector such as #foo:bar:, return a string that could
serve as a methods specification source code.
To be used for code generators

usage example(s):

     Parser methodSpecificationForSelector:#foo:
     Parser methodSpecificationForSelector:#foo:bar:
     Parser methodSpecificationForSelector:#foo:bar:baz:
     Parser methodSpecificationForSelector:#+
     Parser methodSpecificationForSelector:#negated

methodSpecificationForSelector: aSelector argNames: argNames

given a selector such as #foo:bar:, return a string that could
serve as a methods specification source code.
To be used for code generators

usage example(s):

     Parser methodSpecificationForSelector:#foo:bar: argNames:#('one' 'two' 'three')
     Parser methodSpecificationForSelector:#+ argNames:#('one')
     Parser methodSpecificationForSelector:#negated

Instance protocol:

Compatibility-ST80

evaluate: aStringOrStream in: aContextOrNil to: whatIsThis

evaluate: aString in: aClassOrContext to: to notifying: aRequestor ifFail: failBlock

stupid - there seem to be differences among the various

parse: methodSource in: aClass notifying: aRequestor

parse a methods source.
Return the method's parseTree

parseSelector: aStringOrStream

parse a method's source for its selector.
Return the selector

usage example(s):

     Parser new
        parseSelector:'
parseSelector:aStringOrStream
    self initializeFor:aStringOrStream.
    self parseMethodSpec.
    ^ selector.
'

Compatibility-Squeak

parse: methodSource class: aClass: parse a methods source.
Return the method's parseTree

accessing

allowUndeclaredVariables: aBoolean
correctedSource
currentSource: return either the corrected or the requestors original source
doItTemporaries
endOfLastToken
endOfSelectorPosition: return the sourcePosition of the last character of the method's selector spec
errorFlag: return true if there where any errors (valid after parsing)

** This is an obsolete interface - do not use it (it may vanish in future versions) **
evalExitBlock: aBlock: when evaluating a return expression, this block is evaluated
getNameSpace: retrieve the nameSpace, as found in a Namespace directive
implicitSelfSends
implicitSelfSends: aBoolean: turn on/off implicit self sends
initializerExpressions: as a side effect of parsing a methodBodyVarSpec,
these are remembered and returned here.
This is to support an ST/X extension.
interactiveMode: aBoolean: support for stx-scripting service
lineNumberInfo: ** This is an obsolete interface - do not use it (it may vanish in future versions) **
lineNumberInfo: how: ** This is an obsolete interface - do not use it (it may vanish in future versions) **
moreSharedPools
moreSharedPools: aCollection
primitiveNumber: return the ST-80 style primitiveNumber or nil (valid after parsing)
primitiveResources: return the ST-80 style resource info or nil (valid after parsing).
release: (comment from inherited method)
remove all references to objects that may refer to self.
Subclasses may redefine this method but should do a 'super release'.
setNameSpace: aNameSpaceOrNameSpaceName: sent from a namespace directive, if there is no requestor.
Sets the current namespace.
setPackage: aPackageID
targetClass
targetClass: aClass
tree: return the parsetree
tree: aTree: private: set the tree - for internal use only
warnSTXHereExtensionUsed
warnSTXHereExtensionUsed: aBoolean
wasParsedForCode

code generation hooks

assignmentRewriteHookFor: anAssignmentNode: invoked whenever an assignment node has been generated;
gives subclasses a chance to rewrite (instrument) it
blockNodeRewriteHookFor: aBlockNode: invoked whenever a block node has been generated;
gives subclasses a chance to rewrite (instrument) it
messageNodeRewriteHookFor: aMessageNode: invoked whenever a message send node has been generated;
gives subclasses a chance to rewrite (instrument) it
statementListRewriteHookFor: aStatementNode: invoked whenever a statement list node has been generated;
gives subclasses a chance to rewrite (instrument) it
variableReadRewriteHookFor: aVariableNode: invoked whenever a variable node has been generated;
gives subclasses a chance to rewrite (instrument) it

coding style checks

checkBlockArgumentNameConventionsFor: aVariableName
checkBlockVariableNameConventionsFor: aVariableName
checkBracketParenthesisMistakeInIfOrWhile: aNode from: startPosition to: endPosition
checkForLowercaseVariableName: aVariableName
checkForProperUseOfArticleInVariableName: aVariableName: heuristic - and wrong !
The problem is that a simple rule like anyVowel is not sufficient.
I'd be happy to get some help on that.
checkLocalVariableNameConventionsFor: aVariableName
checkMethodArgumentNameConventionsFor: aVariableName
checkMethodVariableNameConventionsFor: aVariableName
checkPlausibilityOf: aNode from: startPosition to: endPosition: XXX: this warning is wrong, if the message is sent in a while loop
and inside a block that checks for nil.
checkReturnedValues
checkSelector: selector for: receiver inClass: cls: check whether a method with selector exists in class cls and
that the method is not obsolete.
If cls is nil, check all classes for the selector.
Return an error string on error or nil on success
checkUnusedMethodVars
isPossiblyUninitializedLocal: aNode: because I do not look into it, yet

dummy-syntax detection

markArgumentIdentifierFrom: pos1 to: pos2: intentionally left empty
markAssignedVariable: v from: pos1 to: pos2: intentionally left empty
markBadIdentifierFrom: pos1 to: pos2: intentionally left empty
markBlockArgumentIdentifierFrom: pos1 to: pos2
markBlockFrom: startPos to: endPos: intentionally left empty
markBooleanConstantFrom: pos1 to: pos2: intentionally left empty
markBracketAt: pos: intentionally left empty
markConstantFrom: pos1 to: pos2: intentionally left empty
markGlobalClassIdentifierFrom: pos1 to: pos2: intentionally left empty
markGlobalIdentifierFrom: pos1 to: pos2: intentionally left empty
markHereFrom: pos1 to: pos2: intentionally left empty
markIdentifierFrom: pos1 to: pos2: intentionally left empty
markInstVarIdentifierFrom: pos1 to: pos2: intentionally left empty
markLocalIdentifierFrom: pos1 to: pos2: intentionally left empty
markLocalVariableDeclaration: variableName from: pos1 to: pos2
markMethodArgumentIdentifierFrom: pos1 to: pos2
markMethodSelectorFrom: pos1 to: pos2: intentionally left empty
markParenthesisAt: pos: intentionally left empty
markReturnAt: pos: intentionally left empty
markSelector: sel from: pos1 to: pos2 receiverNode: aNode: intentionally left empty
markSelfFrom: pos1 to: pos2: intentionally left empty
markSuperFrom: pos1 to: pos2: intentionally left empty
markUnknownIdentifierFrom: pos1 to: pos2: intentionally left empty
markVariable: v: intentionally left empty
markVariable: v from: pos to: endPos: intentionally left empty
markVariable: v from: pos to: endPos assigned: assigned: intentionally left empty

error correction

addDoItTemporary: varName

alreadyWarnedUnimplementedSelectors

alreadyWarnedUnimplementedSelectorsPerReceiverClass

askForCorrection: aString fromList: aList

launch a selection box, which allows user to enter correction.
return newString or nil (for abort)

askForCorrection: aString fromList: aList for: originalSelector

launch a selection box, which allows user to enter correction.
return newString or nil (for abort)

askForVariableTypeWhenDeclaringUndefined: varName

checkIfAllSubclassesOf: aClass implement: aSelector

correctSelector: aSelectorString message: msg positions: posVector in: aClassOrNil for: receiverNode

notify error and correct if user wants to;
return #Error if there was no correction
or a ParseNode as returned by variable

correctSourceByDeletingFrom: start to: stop

correct (by deleting token) if user wants to;
return #Error if there was no correction;
nil if there was one.

correctVariable: varName atPosition: pos1 to: pos2

notify error and correct if user wants to;
return #Error if there was no correction
or a ParseNode as returned by variable

correctWith: correctionOperation from: pos1 to: pos2

declareUndefinedVariable: varName as: variableType

ST/X special - old classToCompileFor is obsoleted - refetch

defineAsUndeclaredVariable: aName

define varName as undeclared variable
(actually, it will be installed as a funny global named 'Undeclared:::<name>').
You can browse for methods with undeclareds by searching for global accesses
to 'Undeclared:::*' (or use the search-undeclared item in the launchers menu).

deleteDefinitionOf: varName in: defStartPosArg to: defEndPosArg

this removes the definition of varName from the declaration part i.e.
if the source was '... | foo bar baz |...'
deleting the var 'bar' changes the source to '... | foo baz |...'.
The code below tries to be somewhat smart w.r.t. spaces by
trying to remove the whiteSpace around the removed variable also.
However, this seems to not always work correctly

findBestSelectorsFor: aString

collect known selectors with their spelling distances to aString;
return the 10 best suggestions

usage example(s):

Time millisecondsToRun:[Parser new findBestSelectorsFor:'foo']

usage example(s):

Parser new findBestSelectorsFor:'findBestSel'

usage example(s):

Parser new findBestSelectorsFor:'fildBestSelectrFr'

findBestSelectorsFor: aString in: aClassOrNil

findBestVariablesFor: aString

collect known variables with their spelling distances to aString;
return the 10 best suggestions

selectorCheck: aSelectorString for: receiver position: pos1 to: pos2

selectorCheck: aSelectorString for: receiver positions: posVector

just a quick check: if a selector is totally unknown as a symbol,
has the same name as a variable, cannot be understood or is obsolete.
Simple, but catches many typos

typeOfNode: aNode

if the receiver is a constant, we know its class...

error handling

blockArgRedefined: tokenName from: pos1 to: pos2: argname reuse
disableWarningsOnCurrentMethodFor: flagName: (comment from inherited method)
ignored here
errorMessageForUndefined: variableName: Return a proper method for undefined variable named `variableName`
exitWith: something: this is the longjump out of evaluation via a return expression
identifierExpectedIn: what
isFirstWarning: something: answer true, if a warning about something has not been shown yet.
Remember, that the warning has been shown
methodArgRedefined: tokenName from: pos1 to: pos2: argname reuse
rememberWarning: something: remember, that a warning for something has been shown.
showErrorMessage: aMessage position: pos: redefined since parser can give more detailed info about
the class & selector where the error occurred.
showErrorMessageForClass: aClass: compiler parseError:'syntax error'.
undefError: aName position: pos1 to: pos2: report an undefined variable error - return true, if it should be
corrected. If not corrected, only one warning is made per undefined
variable.
warnIfPossiblyUninitializedLocal: expr
warnSTXHereExtensionUsedAt: position: only warn once
warnSTXNameSpaceUseAt: position: only warn once
warnSTXSpecialCommentAt: position to: endPosition: (comment from inherited method)
dfo
warnUnused: aNameCollection: report an unused method variable

evaluating expressions

defaultFailBlock: [:msg | self error:(msg ? 'error in eval') ]
evaluate: aStringOrStream: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream compile: compile: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream ifFail: failBlock: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream in: aContext receiver: anObject: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream in: aContext receiver: anObject ifFail: failBlock: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile checkForEndOfInput: checkForEndOfInput: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

If checkForEndOfInput is set, generate an error if EOF has not been reached
at the end of the expression.
evaluate: aStringOrStream logged: logged
evaluate: aStringOrStream notifying: someRequestor
evaluate: aStringOrStream notifying: someRequestor compile: compileBoolean
evaluate: aStringOrStream receiver: anObject: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream receiver: anObject ifFail: failBlock: return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.
evaluate: aStringOrStream receiver: receiver notifying: someRequestor compile: compileBoolean
evaluateDeclarationFor: anEnvironment

initialization

initializeFlagsFrom: aParser: initialize flags from another scanner

obsolete

correctByDeleting: correct (by deleting token) if user wants to;
return #Error if there was no correction;
nil if there was one.

parsing

block

parse a block; return a node-tree, nil or #Error

usage example(s):

blockNode lineNumber:lno.

blockBody

parse a currentBlock's block-body; return a node-tree, nil or #Error

blockExpression

parse a blockExpression; return a node-tree, nil or #Error.
Not used by ST/X's parser, but added for ST-80 compatibility.

blockStatementList

parse a blocks statementlist; return a node-tree, nil or #Error

checkForEndOfInput

check, that we have reached the end of input.
Error, if not

emptyStatement

makeSelector: rawSelector

cg: how about (to prevent creation of new symbols):

parseExpressionWithSelf: anObject notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil

parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

parseLiteralArray: aStringOrStream

Parser literal array in given String or Stream.
Returns that array

parseMethod

parse a method.
Return the parseTree or #Error.

method ::= methodSpec methodBody

parseMethod: aString in: aClass

parseMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings

parseMethodBody

parse a method's body (locals & statements).
Return a node-tree, or #Error

methodBody ::= '<' <st80Primitive> '>' #EOF
| '<' <st80Primitive> '>' <methodBodyVarSpec> <statementList> #EOF

parseMethodBodyOrEmpty

parse a method's body (locals & statements);
return a node-tree, nil or #Error.
empty (or comment only) input is accepted and returns nil.

methodBodyOrNil ::= '<' <st80Primitive> '>'
| '<' <st80Primitive> '>' <methodBodyVarSpec> <statementList>
| <empty>

parseMethodBodyVarSpec

parse a methods local variable specification, handling
possible primitive or resourceSpecs.
.
Leave spec of locals in methodLocals as a side effect.
Possibly leave initializer expressions in initExpressionsForLocals
as a side effect.
Return self.

methodBodyVarSpec ::= '|' <identifier> ... '|'
| '|' <identifier_or_initializer> ... '|'
| <empty>
identifier_or_initializer ::=
<identifier>
| <identifier> ':=' <expression> '.'

parseMethodSpec

parse a methods selector & arg specification;
Set selector and methodArgs in the receiver as a side effect.
Return the receiver or #Error.

methodSpec ::= { KEYWORD IDENTIFIER }
| binaryOperator IDENTIFIER
| IDENTIFIER

returnStatement

parse a return statement;

statement ::= '^' expression

statement

parse a statement; return a node-tree or #Error.

statement ::= '^' expression
| PRIMITIVECODE
| expression

usage example(s):

allow empty statement

statementList

parse a statementlist; return a node-tree, nil or #Error.
Statements must be separated by periods.

statementList ::= <statement>
| <statementList> . <statement>

variableTypeDeclarationFor: aVariable

experimental support for Domain variables (constraint programming support):
a variable-declaration of the form
|var (domain) ... |
declares var as a domainVariable.
Valid domains are:
min %% max - integer range domain
Bool - boolean domain
Nat - positive integer domain
Int - integer domain
#sym1 ... #sym2 - enumerated symbolic domain

warnAboutEmptyStatement

parsing-expressions

array

parse a literal array's elements

usage example(s):

self syntaxError:'unterminated array-constant; '')'' expected'

usage example(s):

always return the same object for an empty array,
         which is also immutable by definition

arrayConstant

some more special symbol consts ...

arrayIndexingExpression

parse an array index expression; this is a squeak/stx extension.
foo[x] is syntactic sugar for 'foo at:x'
and foo[x] := expr is syntactic sugar for 'foo at:x put:expr';
with commas, foo[x , y] generates foo at:x at:y.
This syntax extension must be enabled in the parserFlags as
allowArrayIndexSyntaxExtension (disabled by default)

binaryExpression

parse a binary-expression; return a node-tree, nil or #Error

binaryExpressionFor: receiverArg

parse a binary-expression; return a node-tree, nil or #Error

byteArray

parse a literal byteArray's elements

usage example(s):

literal bytearrays started with ST-80 R4 - byteArray constants are written as #[ ... ]

degeneratedKeywordExpressionForSelector

parse a keyword-expression without receiver - for the selector only.
Return the selector or nil (if it cannot be determined).
This is not used in normal parsing, but instead to extract the selector from a code fragment.
(for example, the system browsers 'implementors'-function uses this to extract a selector from
the selection)

usage example(s):

     (self new source:'hello:world:') nextToken; degeneratedKeywordExpressionForSelector
     (self new source:'hello:world') nextToken; degeneratedKeywordExpressionForSelector
     (self new source:'hello:') nextToken; degeneratedKeywordExpressionForSelector

expression

parse a cascade-expression; return a node-tree, nil or #Error.

expression ::= keywordExpression
| keywordExpression cascade

cascade ::= ';' expressionSendPart
| cascade ';' expressionSendPart

expressionSendPart ::= { KEYWORD binaryExpression }
| BINARYOPERATOR unaryExpression
| IDENTIFIER

functionCallArgList

functionCallExpression

parse a functionCall;
this is an st/x extension.
foo(x)
is syntactic sugar for
foo value:x
This syntax extension must be enabled in the parserFlags as
allowFunctionCallSyntaxForBlockEvaluation (disabled by default)

inlineObjectClassFor: slotNames

create an anonymous inline-object class for an object with slots given by names.
Reuses an existing class, if one exists.

inlineObjectFrom: pos1

an experimental ST/X feature.
InlineObject as
#{ name1: value1. ... nameN: valueN }
creates a literal object with an anon class which provides getter/setters on all
names and is preinitialized with valueI.
The initial #{ is supposed to be skipped and its position passed in as pos1.
Notice: the values must be literals too;
currently the JavaScript style (using expressions) is not supported.

keywordExpression

parse a keyword-expression; return a node-tree, nil or #Error.

keywordExpression ::= binaryexpression
| { KEYWORD-PART binaryExpression }

keywordExpressionFor: receiverArg

parse a keyword-expression; return a node-tree, nil or #Error.

keywordExpression ::= binaryexpression
| { KEYWORD-PART binaryExpression }

literalInlineObjectFor: namesAndValues

create an inline-object literal.
Reuses an existing class, if one exists.

primary

parse a primary-expression; return a node-tree, nil or #Error.
This also cares for namespace-access-paths.

primary_dolphinComputedLiteral

parse a dolphin computed literal; return a node-tree, or raise an Error.
In dolphin, these are written as: ##( expression )
and create a literal constant for the expressions value.
Right now, only a subset is supported - Strings, ByteArrays and Characters.
WARNING: this is only supported to allow file-in of dolphin code.
Since stc cannot handle this (at the moment), you should rewrite the code
if you ever plan to stc-compile it into a shared library.
The question is still: how should stc ever be able to do this, as it cannot execute
smalltalk code; it could generate code to execute it at initialization time of the
generated code, but then, it is no longer a compile-time constant
(for example, generating a compilation-Date constant is then not possible...)

primary_expression

parse a parentized expression primary; return a node-tree, or raise an Error.

primary_false

parse a false primary; return a node-tree, or raise an Error.

primary_here

parse a here primary; return a node-tree, nil or #Error.

primary_identifier

parse a primary starting with an identifier: either a variable or an assignment.
return a node-tree, or raise an Error.

primary_lazyValue

primary_nil

parse a nil primary; return a node-tree, nil or #Error.

primary_self

parse a self primary; return a node-tree, nil or #Error.

primary_simpleLiteral

parse a simple literal primary; return a node-tree, or raise an Error.

primary_squeakComputedArrayOrComputedInlineObject

parse a squeak computed array literal;
return a node-tree, or raise an Error.
In squeak, these are written as:
{ expr1 . expr2 . ... exprN }
and create a message to construct an Array containing the exprI values.
ST/X also supports immediate objects which are instances of anonymous classes,
and are written as:
{ slotName1: expr1 . slotName2: expr2 . ... slotNameN: exprN }

primary_super

parse a super primary; return a node-tree, nil or #Error.

primary_thisContext

parse a thisContext primary; return a node-tree, or raise an Error.

primary_true

parse a true primary; return a node-tree, or raise an Error.

qualifiedNameFrom: pos1

a vw3.x (and later) feature: QualifiedName is #{ id ... id }
and mapped to a global variable here.
The initial #{ is supposed to be skipped and its position passed in as pos1.

Notice, the VW implementation will generate a literal reference to a binding-cell,
which is not reified in ST/X
(does not exist in ST/X as Smalltalk object, but only inside the VM).
Thus, the generated name (a symbol) has NOT the same semantics,
and imported VW code may need some modifications to work.

qualifiedNameOrInlineObject

either a vw3.x (and later) QualifiedName #{ id ... id }
or an ST/X experimental inline object #{ name: value. ... name: value }
The initial #{ is skipped here, but the position passed down.

squeakComputedArrayExpressions

stringWithEmbeddedExpressions

there must be a closing brace

stxComputedInlineObject

parse an ST/X immediate object, which is an instance of an anonymous class,
and written as:
{ slotName1: expr1 . slotName2: expr2 . ... slotNameN: exprN }

typedArray: typeSymbol

parse a typed array's elements.
This is an ST/X extension, which is not supported by other Smalltalk implementations.
For now, the support is disabled by default;
enable with:
ParserFlags allowSTXExtendedArrayLiterals:true
or a pragma, like:
<pragma: +allowSTXExtendedArrayLiterals>

Typed literal arrays are written in scheme-style as:
#u8( element... ) - unsigned 8-bit bytes (i.e. a regular ByteArray)
#u16( element... ) - unsigned 16-bit shorts (i.e. a WordArray)
#u32( element... ) - unsigned 32-bit ints (i.e. an IntegerArray)
#u64( element... ) - unsigned 64-bit ints (i.e. a LongIntegerArray)
#s8( element... ) - signed bytes (i.e. a SignedByteArray)
#u16( element... ) - signed 16-bit shorts (i.e. a SignedWordArray)
#u32( element... ) - signed 32-bit ints (i.e. a SignedIntegerArray)
#u64( element... ) - signed 64-bit ints (i.e. a SignedLongIntegerArray)
#f16( element... ) - tiny 16-bit floats (i.e. a HalfFloatArray)
#f32( element... ) - short 32-bit floats (i.e. a FloatArray)
#f64( element... ) - 64-bit doubles (i.e. a DoubleArray)

#f( element... ) - same as f32(...)
#d( element... ) - same as f64(...)

unaryExpression

parse a unary-expression; return a node-tree, nil or #Error

unaryExpressionFor: receiverArg

parse a unary-expression; return a node-tree, nil or #Error

variable

parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.
If undefined, notify error and correct if user wants to

variableOrError

parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.

variableOrError: varName

parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.

parsing-primitives & pragmas

addAnnotationWithKey: key andArguments: arguments: was: annot := { key . arguments}.
checkForClosingAngle: verify a closing angle and skip over it
defineWellknownCTypesIn: aDictionary: a few wellknown types
generateCallToExternalFunction: fn lineNr: lineNr: the following stuff was commented (2007-07-30), to make ole: work.
Ask felix or stefan
generateReturnOfValue: aValue
generateTrapCodeForUnavailableCParser
parseAnnotationLiteral: (tokenType == #Keyword) ifTrue: [
value := '#', tokenName.
self nextToken.
^ value.
].
parseAnotationLiteral: typo in selector - left in for a while for compatibility
parseExceptionOrContextPragma: parse
<exception: #handle|raise|unwind>,
<context: #return>
context flagging pragmas.
parseExternalFunctionCallDeclaration: callType is one of c: / cdecl: / api: / apicall: ...
parseGSTExternalFunctionDeclaration: argArray: Handles GNU Smalltalk-style exteranl function declarations.
Example:
<cCall: 'cairo_close_path' returning: #void args: #(#cObject )>
parseOtherPrimitives: not really a check, but remembers endPos
parsePragma: '<' has already been parsed.
parses pragmas of the form:
<pragma: +arrayIndexSyntaxExtension>
<pragma: +STXSyntaxExtensions>
<pragma: +lazyValueExtension>
<pragma: +functionCallSyntaxForBlockEvaluation>
parsePrimitive: parse an ST-80 type primitive as '< primitive: nr >';
(return primitive number or #Error)
or a Squeak-style primitive, as '< primitive: string >';
(return primitive name or #Error)
or a V'Age-style primitive, as '< primitive: identifier >';
(return primitive name or #Error)

Also, resource specs are parsed; the result is left (as side effect) in primitiveResource.
It is used to flag methods, for faster finding of used keyboard accelerators,
and to mark resource methods (image, menu or canvas resources).

prim ::= st80Primitive | st80Pragma | stxPragma
| squeakPrimitive | vAgePrimitive | newSTXPrimitive
| externalFuncDecl
| resourceDecl

st80Primitive ::= 'primitive:' INTEGER
st80Pragma ::= 'exception:' ( 'handle | 'raise' | 'unwind' )
stxPragma ::= 'context:' 'return'

squeakPrimitive ::= 'primitive:' STRING

newSTXPrimitive ::= 'primitive'

vAgePrimitive ::= 'primitive:' IDENTIFIER
| 'sysprim:' IDENTIFIER

externalFuncDecl ::= vwExternalFuncDecl
| stvExternalFuncDecl
| squeakExternalFuncDecl
| dolphinExternalFuncDecl

vwExternalFuncDecl ::= 'c:' vwFuncDecl

stvExternalFuncDecl ::= 'api:' stvFuncDecl
| 'ole:' stvFuncDecl

squeakExternalFuncDecl ::= 'apicall:' stvFuncDecl
| 'cdecl:' stvFuncDecl

dolphinExternalFuncDecl ::= 'stdcall:' stvFuncDecl

resourceDecl ::= 'resource:' SYMBOL - leave SYMBOL in primitiveResource
| 'resource:' SYMBOL (...) - leave (SYMBOL (...)) in primitiveResource
| 'pragma:' SYMBOL - same as resource; alternative syntax
| 'pragma:' SYMBOL (...) - same as resource; alternative syntax
| 'attribute:' SYMBOL - same as resource; alternative syntax
| 'attribute:' SYMBOL (...) - same as resource; alternative syntax
parsePrimitiveOrResourceSpecOrEmpty: parse a methods primitive or resource spec
parseResourcePragma: '<' has already been parsed.
parseSTVExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr: parses ST/V function declarations of the forms
'<api: functionName argType1 .. argTypeN returnType>'
'<ole: vFunctionIndex argType1 .. argTypeN returnType>'
parseSTXOrSqueakOrDolphinExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr: parses squeak/dolphin/stx function declarations of the forms
'<stdcall: [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex argType1..argTypeN>'
'<cdecl: [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex argType1..argTypeN>'

'<cdecl: [async] [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex ( argType1..argTypeN ) module: moduleName >'
'<apicall: [async] [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex ( argType1..argTypeN ) module: moduleName >'
parseTraditionalPrimitive: parse everything after the initial '<primitive:'
parseVWTypeOrExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr: parses visualWorks type/function declarations of the form:
'<c: ...>'
primitiveNumberFromName: aPrimitiveName: for future compatibility with Squeak ...
rememberContextPragma: pragmaType value: pragmaValue
rememberContextReturnablePragma
skipForClosingAngle: skip

private

currentNameSpace

currentNameSpace: aNameSpace

currentNamespace: aNameSpace

** This is an obsolete interface - do not use it (it may vanish in future versions) **

currentPackage

currentUsedNameSpaces

findNameSpaceWith: aVariableName

private names have already been searched for.

genMakeArrayWith: elementExpressions

return a node to generate an array at runtime.
Will generate:
literal shallowCopy (if all elements are literals)
or else:
Array with:el1 ... with:elN (if N <= 8)
or else:
(Array new at:1 put:el1; ... at:N put:elN; yourself) (otherwise)

genMakeInlineObjectWith: nameExpressionDictionary

return a node to generate an inline object at runtime

inWhichClassIsClassInstVar: aString

search class-chain for the class-instance variable named aString
- return the class or nil if not found

inWhichClassIsClassVar: aString

search class-chain for the classvariable named aString
- return the class or nil if not found

isStatementAnUnconditionalReturn: aStatementNode

used on the last statement, to see if this is doing a return already
(i.e. its own return value is to be ignored.

isStatementListAnUnconditionalReturn: aStatementNode

used on a statement-list, to see if this is doing an unconditional return

isValidUnarySelector: tokenType

ParserFlags allowReservedWordsAsSelectors:true.
1234 self.

ParserFlags allowReservedWordsAsSelectors:false
1234 self.

makeImmutable: anObject

makeReferenceFor: aNode

variable references (&var)

markUnreachableCodeAsCommentIn: expr

take care (ignore) for unreachable constants which do not have a selector

noAssignmentAllowed: eMsg at: pos

eat the assign when doing highlighting only

nodeForMethodArg: varName

var used:true.

nodeForMethodVariable: varName

plausibilityCheck: aNode

do not check this for doits

selfNode

JV@2011-07-19: Changed not to share the nodes

usage example(s):

selfNode := SelfNode value:selfValue

usage example(s):

^ selfNode

queries

annotationInfo: return the annotations - if any - in a format usable for source analysis
(i.e. with start and stop positions)
valid only after parsing
annotations: return the annotations - if any - in the format stored in a method
(i.e. as an array or 2-element key-argument arrays).
valid only after parsing
classToLookForClassVars: helper - return the class to look for classVars.
If there is a context in which we evaluate, the
methods implementing class is used instead of the
class of the receiver.
classesClassInstVarNames: caching allInstVarNames for next compilation saves time ...
classesClassVarNames: caching allClassVarNames for next compilation saves time ...
classesInstVarNames: caching allInstVarNames for next compilation saves time ...
contextMustBeReturnable: misusing/misinterpreting the lineNumberInfo flag is a q&d hack; there should be an extra flag
didWarnAboutSqueakExtensions
doItSelector: the name of the method used for doit's.
The method will not be installed, but called directly,
so the name is more or less arbitrary.
hasNonOptionalPrimitiveCode: return true if there was any ST/X style primitive code (valid after parsing)
hasPrimitiveCode: return true if there was any ST/X style primitive code (valid after parsing)
isCompiling: return true if compiling code as opposed to evaluating
isDoIt
isEmptyMethod: return true (after a parse) if this is an empty (documentation) method
isSyntaxHighlighter
methodArgs: return an array with methodarg names (valid after parsing spec)
methodVars: return a collection with method variablenames (valid after parsing)
numberOfMethodArgs: return the number of methodargs (valid after parsing spec)
numberOfMethodVars: return the number of method variables (valid after parsing)
selector: return the selector (valid after parsing spec)
shouldPerformCodingStyleChecks
whichClassIncludesClassVar: aVariableName: helper: find the class in which a class variable is defined

queries-statistic

messagesPossiblySent: return a collection with possible message selectors (valid after parsing).
Includes things known or possibly used with #perform: or in specs.
The returned collection is filled by heuristics, not sharp, exact
messagesSent: return a collection with sent message selectors (valid after parsing).
Includes all sent messages (i.e. also sent super messages)
messagesSentToSelf: return a collection with message selectors sent to self only (valid after parsing)
messagesSentToSuper: return a collection with message selectors sent to super only (valid after parsing)
modifiedClassVars: return a collection with classvariable names modified by method (valid after parsing)
modifiedGlobals: return a collection with global names modified by method (valid after parsing)
modifiedInstVars: return a collection with instvariable names modified by method (valid after parsing)
modifiedPoolVars: return a collection with poolvariable names modified by method (valid after parsing)
readClassVars: return a collection with classvariable names read by method (valid after parsing)
readGlobals: return a collection with global names read by method (valid after parsing)
readInstVars: return a collection with instvariable names read by method (valid after parsing)
readPoolVars: return a collection with poolvariable names read by method (valid after parsing)
sendsSelector: selectorString
usedClassVars: return a collection with classvariable names refd by method (valid after parsing)
usedGlobals: return a collection with global names refd by method (valid after parsing)
usedInstVars: return a collection with instvariable names refd by method (valid after parsing)
usedPoolVars: return a collection with poolvariable names refd by method (valid after parsing)
usedSymbols: return a collection with used symbols (except for sent messages) (valid after parsing)
usedVars: return a collection with variable names refd by method (valid after parsing)
usesSuper: return true if the parsed method uses super (valid after parsing)

setup

arraysAreImmutable
arraysAreImmutable: aBoolean
classToCompileFor
foldConstants: aSymbolOrNil: change the constant folding level. See the classMethod for a description.
initialize: <modifier: #super> "must be called if redefined"
parseForCode: turns off certain statistics (keeping referenced variables, modified vars etc.)
Use this when parsing for compilation or evaluation
setClassToCompileFor: aClass: set the class to be used for parsing/evaluating
setContext: aContext: set the context used while evaluating
setSelf: anObject: set the value to be used for self while evaluating
stringsAreImmutable
stringsAreImmutable: aBoolean
warnAboutMissingMethodComment
warnUndeclared
warnUndeclared: aBoolean
warnUnusedVars
warnUnusedVars: aBoolean

statistic

rememberClassVarModified: name
rememberClassVarRead: name
rememberClassVarUsed: name
rememberGlobalModified: name
rememberGlobalRead: name
rememberGlobalUsed: name
rememberInstVarModified: name
rememberInstVarRead: name
rememberInstVarUsed: name
rememberLocalModified: name
rememberLocalUsed: name
rememberPoolVarModified: name
rememberPoolVarRead: name
rememberPoolVarUsed: name
rememberReturnedValue: anExpressionNode
rememberSelectorPossiblyUsed: sel
rememberSelectorUsed: sel
rememberSelectorUsed: selectorArg receiver: receiverNode: cg: thought this was a good idea;
rememberSelectorUsed: selectorArg receiver: receiverNode args: args: TODO: the heuristics below could be made more
flexible by annotating methods which do a perform somehow,
and then asking implementors of selectorArg...
rememberSelectorUsedInSelfSend: sel
rememberSelectorUsedInSuperSend: sel
rememberSymbolUsed: aSymbol
rememberVariableUsed: name

Private classes:

    AskForVariableTypeOfUndeclaredQuery
    CorrectByChangingSelector
    CorrectByDeclaringIdentifierAs
    CorrectByDeletingLocalIdentifier
    CorrectByGeneratingMissingMethod
    CorrectByGroupingMessage
    CorrectByInsertingPeriod
    CorrectByInteractiveCorrection
    CorrectByInteractiveRename
    CorrectByMakingValidHexConstant
    Correction
    ParsedAnnotation
    PossibleCorrectionsQuery
    PrimitiveSpecParser
    RestartCompilationSignal

Examples:

    Parser
        evaluate:'1+2*3'
        in:nil
        receiver:nil
        notifying:nil
        logged:false
        ifFail:nil

    Parser undefinedVariableNotification handle:[:ex |
        |badName|

        badName := ex variableName.
        ex proceedWith:(ConstantNode value:5).
    ] do:[
        ^ self compilerClass
            evaluate:'foo * 3'
            in:nil
            receiver:nil
            notifying:nil
            logged:false
            ifFail:nil
    ]

the following are experimental...

    <pragma: +arrayIndexSyntaxExtension>
    |c|

    c := Array[7].
    c[4] := 4.
    c[6] := 6.
    c[7] := 7.
    c[7] := c[4] - c[6].

    <pragma: +arrayIndexSyntaxExtension>
    |d|

    d := Dictionary new.
    d['one'] := 'eins'.
    d['two'] := 'zwei'.
    d['three'] := 'drei'.

    d['two'] , d['three']

Reparse whole image...

    Smalltalk allClassesDo:[:cls|
        cls isLoaded ifTrue:[
            Transcript show: cls name; show: '...'.
            cls methodsDo:[:mth|
                Parser parseMethod: mth source.
            ].
            Transcript showCR:'OK'.
        ]
    ]

ST/X 7.2.0.0; WebServer 1.670 at bd0aa1f87cdd.unknown:8081; Fri, 19 Apr 2024 05:21:56 GMT