eXept Software AG Logo

Smalltalk/X Webserver

Documentation of class 'Parser':

Home

Documentation
www.exept.de
Everywhere
for:
[back]

Class: Parser


Inheritance:

   Object
   |
   +--Scanner
      |
      +--Parser
         |
         +--AbstractSyntaxHighlighter
         |
         +--BreakpointAnalyzer
         |
         +--ByteCodeCompiler
         |
         +--Explainer
         |
         +--Parser::PrimitiveSpecParser

Package:
stx:libcomp
Category:
System-Compiler
Version:
rev: 1.987 date: 2018/05/25 14:08:05
user: cg
file: Parser.st directory: libcomp
module: stx stc-classLibrary: libcomp
Author:
Claus Gittinger

Description:


Parser is used for both parsing and evaluating smalltalk expressions;
it first builds a parseTree which is then interpreted (evaluate) or
compiled. Compilation is done in the subclass ByteCodeCompiler and/or
the (planned) MachineCodeCompiler.

methods of main interest are:
    Parser evaluateExpression:...

and:
    Parser parseExpression:...
    Parser parseMethod:...

there is protocol to parse complete methods, selector specs, body only etc.

Parser is also used to find the referenced/modified inst/classvars of
a method - this is done by sending parseXXX message to a parser and asking
the parser for referencedXVars or modifiedXVars (see SystemBrowser).

You can also use parsers for all kinds of other things (ChangesBrowser for
example analyzes the expressions in the changelist ...) by looking at the
parsers tree. (Although this is somewhat dangerous, since it exports the
compilers internals ... better style is to add specialized query methods here)

One instance of Parser is created to parse one method or expression - i.e.
its not suggested to reuse parsers.


  Constant folding:

The parser has various modes for constant folding; by default, only numeric
expressions involving integers and floats are constant folded
(i.e. something like 'Float pi sin' or '1.5 + 0.3' will be reduced to a constant).

Constant folding can be turned off completely (setting FoldConstants to nil)
to ``secure folding'', which only folds constant numbers (#level1) or to #full.
In full mode, more constant expressions are folded (for example: '1.0 @ 1.0' is
reduced to a constant point), but the resulting code may not be compatible with other
smalltalk systems (consider the case, where the point is modified using #x: or #y: messages).
Therefore, this mode is a bit dangerous and disabled by default.
The current implementation, base upon a global constant-folding setting is somewhat stupid
and intermediate - a better solution would be to allow the optimization to be controlled
by a method-pragma, since it may make sense to disable optimization on a method level,
if it's known that the used constant objects are subject of modifications as described above.


  Immutable arrays:

Immutable arrays are experimental and being evaluated.
Consider the case of a method returning '#(1 2 3 4)', and that array being modified
by some other method (using #at:put:). Since the array-return is actually a return of
a reference to the compiler created array, the next invokation of the method will
return the modified array. These are hard to find bugs.
By an option, the compiler can generate immutable arrays, which don't allow modification
of its elements. For clean code, you should enable this option during development.

As mentioned above, this is experimental. If it is reported to be a useful feature,
the immutable feature will be extended to strings, point-literals etc. in a future version
of st/x.



[Instance variables:]

    classToCompileFor   <Class>             the class (or nil) we are compiling for

    selfValue           <any>               value to use as self when interpreting

    contextToEvaluateIn <Context>           the context (or nil) when interpreting

    selector            <Symbol>            the selector of the parsed method
                                            (valid after parseMethodSpecification)
    methodArgs                              internal

    methodArgNames      <Collection>        the names of the arguments
                                            (valid after parseMethodSpecification)

    methodVars                              internal

    methodVarNames      <Collection>        the names of the method locals
                                            (valid after parseMethodBodyVarSpec)

    tree                <ParseTree>         the parse tree - valid after parsing

    currentBlock                            if currently parsing for a block

    usedInstVars                            set of all accessed instances variables
                                            (valid after parsing)

    usedClassVars                           same for classVars

    usedVars                                all used variables (inst, class & globals)

    modifiedInstVars                        set of all modified instance variables

    modifiedClassVars                       same for clasVars

    localVarDefPosition <Integer>           the character offset of the local variable
                                            def. (i.e. the first '|' if any)
                                            Not yet used - prepared for automatic add of
                                            undefined variables

    evalExitBlock                           internal for interpretation

    selfNode            <Node>              cached one-and-only 'self' node
    superNode           <Node>              cached one-and-only 'super' node

    hasPrimitiveCode    <Boolean>           true, if it contains ST/X style primitive code
    hasNonOptionalPrimitiveCode
                        <Boolean>           true, if it contains ST/X style primitive code
                                            which is NOT flagged by the OPTIONAL directive.

    primitiveNr         <Integer>           the parsed ST-80 type primitive number (or nil)

    logged

    warnedUndefVars     <Set>               set of all variables which the parser has
                                            already output a warning (to avoid multiple
                                            warnings about the same variable)

[Class variables:]

    PrevClass           <Class>             class, of which properties are
                                            cached in:

    PrevInstVarNames      <Collection>      instance variablenames of cached class
    PrevClassVarNames     <Collection>      class variablenames of cached class
    PrevClassInstVarNames <Collection>      class instance variablenames of cached class

    LazyCompilation       <Boolean>         EXPERIMENTAL: lazy compilation

    ArraysAreImmutable    <Boolean>         if true, create array literals
                                            as instances of ImmutableArray,
                                            which cannot be stored into.
                                            Default is false, for compatibility.
                                            Can be turned on while developing
                                            new code to make certain that side
                                            effects are avoided.

    StringsAreImmutable   <Boolean>         same as above for string literals

    WarnST80Directives    <Boolean>         if true, give warnings about
                                            ST-80 directives (resource defs)
                                            which are ignored in st/x.
                                            defaults to false.

    FoldConstants         <Symbol>          controls how constant folding should be
                                            done.
                                            Can be one of:
                                                    nil      - no constant folding
                                                    #level1  - numeric optimizations only
                                                    #level2  - secure optimizations only
                                                    #full    - full folding

                                            level1:   arithmetic on constant numbers

                                            level2:   above PLUS array conversions with #asFloatArray,
                                                      #asDoubleArray, string concatenation

                                            full:     constant points.


Related information:

    ByteCodeCompiler
    Scanner
    ObjectFileLoader
    Workspace
    SystemBrowser

Class protocol:

Compatibility-ST80
o  parse: aString class: aClass
same as #parseMethod:in: for ST80 compatibility.

Compatibility-Squeak
o  evaluate: someString for: aReceiver logged: logged

o  evaluate: aStringOrStream for: aReceiver notifying: someone logged: logged

Signal constants
o  askForVariableTypeOfUndeclaredQuery

o  correctByDeclaringIdentifierAs

o  correctByInteractiveCorrection

o  correctByInteractiveRename

o  parseErrorSignal

o  parseWarningSignal

o  possibleCorrectionsQuery

o  restartCompilationSignal

o  undefinedSuperclassError

o  undefinedVariableError

o  undefinedVariableNotification

change & update
o  flushNameCache
unconditional flush name caches
usage example(s):
Parser flushNameCache

o  update: something with: someArgument from: changedObject
aClass has changed its definition - flush name caches if we have to

class initialization
o  initialize
usually set to true in your .rc file
usage example(s):
     self initialize

controlling compilation
o  allowArrayIndexSyntaxExtension
experimental

o  allowArrayIndexSyntaxExtension: aBoolean
experimental
usage example(s):
     self allowArrayIndexSyntaxExtension:true
     self allowArrayIndexSyntaxExtension:false

o  allowFunctionCallSyntaxForBlockEvaluation
experimental

o  allowFunctionCallSyntaxForBlockEvaluation: aBoolean
experimental
usage example(s):
     self allowFunctionCallSyntaxForBlockEvaluation:true
     self allowFunctionCallSyntaxForBlockEvaluation:false

o  allowReservedWordsAsSelectors
return true, if self, super, thisContext, nil, true and false are to be allowed
as unary message selectors.

o  allowReservedWordsAsSelectors: aBoolean
enable/disable, if self, super, thisContext, nil, true and false are to be allowed
as unary message selectors.
usage example(s):
     self allowReservedWordsAsSelectors:true
     self allowReservedWordsAsSelectors:false

o  arraysAreImmutable
return true if arrays are immutable literals

o  arraysAreImmutable: aBoolean
turn on/off immutable array literals - default is false for ST-80 compatibilty.
usage example(s):
     can be added to your private.rc file:

     Compiler arraysAreImmutable:true
     Compiler arraysAreImmutable:false

o  compileLazy
return true if compiling lazy

o  compileLazy: aBoolean
turn on/off lazy compilation - return previous setting.
Actually this flag belongs into the ByteCodeCompiler subclass,
but it also controls the reporting of some errors here; therefore
its located here
usage example(s):
     Compiler compileLazy:false
     Compiler compileLazy:true

o  foldConstants
return a symbol describing how constants are to be folded

o  foldConstants: aSymbol
set the symbol describing how constants are to be folded.
It can be:
nil - no constant folding
#level1 - numeric constants only
#level2 - level1 PLUS array conversions PLUS string concatenation
#full - level2 PLUS constant points, constant rectangles (dangerous)

o  implicitSelfSends
return true if undefined variables with
lowercase first character are to be turned
into implicit self sends

o  implicitSelfSends: aBoolean
turn on/off implicit self sends
usage example(s):
     ParserFlags implicitSelfSends:true
     ParserFlags implicitSelfSends:false

o  lineNumberInfo

o  lineNumberInfo: aBoolean

o  stringsAreImmutable
return true if strings are immutable literals

o  stringsAreImmutable: aBoolean
turn on/off immutable string literals - default is false for ST-80 compatibilty.
usage example(s):
     can be added to your private.rc file:

     ParserFlags stringsAreImmutable:true
     ParserFlags stringsAreImmutable:false

o  warnAboutBadComments: aBoolean
controls generation of warning messages about empty comments

o  warnAboutVariableNameConventions
controls generation of warning messages about wrong variable names

o  warnAboutVariableNameConventions: aBoolean
controls generation of warning messages about wrong variable names

o  warnAboutWrongVariableNames
controls generation of warning messages about wrong variable names

o  warnAboutWrongVariableNames: aBoolean
controls generation of warning messages about wrong variable names

o  warnUnusedVars
controls generation of warning messages about unued method variables

o  warnUnusedVars: aBoolean
controls generation of warning messages about unued method variables

defaults
o  maxLineNumber
return the maximum lineNumber that is possibly
encoded in a methods byteCode debugging information.
Since lineNumber entries only have 1 byte available,
this is 255 for byteCode methods.

error correction
o  findBest: nMax selectorsFor: aString in: aClassOrNil
collect known selectors with their spelling distances to aString;
return the nMax best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class (i.e. offer
corrections from that hierarchy only).
soon OBSOLETE: to be moved to DWIM

o  findBest: nMax selectorsFor: aString in: aClassOrNil forCompletion: forCompletion
soon OBSOLETE and moved to DWIM.
Collect known selectors with their spelling distances to aString;
return the nMax best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class
(i.e. offer corrections from that hierarchy only).
If forCompletion isTrue, prefix sequences are preferred.
The way spelling distance is computed is a heuristic which works
well in real life (i.e. offer USEFUL suggestions)
usage example(s):
     'select:' spellAgainst:'collect'    57
     'collectWithIndex:' spellAgainst:'collect' 41

     self findBest:20 selectorsFor:'i' in:nil forCompletion:true

     self findBest:20 selectorsFor:'collect' in:Collection forCompletion:true
     self findBest:100 selectorsFor:'collect' in:Collection forCompletion:true

     self findBest:20 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true
     self findBest:100 selectorsFor:'collect:' in:SequenceableCollection forCompletion:true

o  findBestSelectorsFor: aString
collect known selectors with their spelling distances to aString;
return the 10 best suggestions.
soon OBSOLETE: to be moved to DWIM
usage example(s):
     Parser findBestSelectorsFor:'at'
     Parser findBestSelectorsFor:'at:pu'

o  findBestSelectorsFor: aString in: aClassOrNil
collect known selectors with their spelling distances to aString;
return the N best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class (i.e. offer
corrections from that hierarchy only).
soon OBSOLETE: to be moved to DWIM

evaluating expressions
o  evaluate: aStringOrStream
return the result of evaluating an expression in aStringOrStream.
No doit-entry is added to the changeLog.
usage example(s):
     Compiler evaluate:'1 + 2'
     Compiler evaluate:'''hello world'' asSortedCollection displayString printNL'
     Compiler evaluate:'''hello world'' asSortedCollection printNL'

o  evaluate: aStringOrStream compile: compile
return the result of evaluating aString,
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream ifFail: failBlock
return the result of evaluating an expression in aStringOrStream.
In case of any syntax errors, return the value of failBlock.
No doit-entry is added to the changeLog.
usage example(s):
     Compiler evaluate:'1 +' ifFail:['oops']

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
No doIt entry is added to the change-file.
If the failBlock argument is non-nil, it is evaluated if an error occurs.

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream logged: logged
return the result of evaluating an expression in aStringOrStream.
The argument log controls if an entry is added to the changeLog.
usage example(s):
     Compiler evaluate:'''some string''' logged:false
     Compiler evaluate:'''some string''' logged:true

o  evaluate: aStringOrStream notifying: requestor
return the result of evaluating aString,
errors are reported to requestor

o  evaluate: aStringOrStream notifying: requestor compile: compile
return the result of evaluating aString,
errors are reported to requestor.
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream notifying: requestor logged: logged
return the result of evaluating aString,
errors are reported to requestor

o  evaluate: aStringOrStream receiver: anObject
return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars
usage example(s):
     Compiler evaluate:'self x' receiver:(1 @ 2)

o  evaluate: aStringOrStream receiver: someOne logged: logged
return the result of evaluating an expression in aStringOrStream.
The argument log controls if an entry is added to the changeLog.
usage example(s):
     Compiler evaluate:'''some string''' logged:false
     Compiler evaluate:'''some string''' logged:true
     Compiler evaluate:'self class' receiver:nil logged:false
     Compiler evaluate:'self class' receiver:1 logged:false

o  evaluate: aStringOrStream receiver: anObject notifying: requestor
return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars (used in the inspector)
usage example(s):
     Compiler evaluate:'self x' receiver:(1 @ 2) notifying:nil

o  evaluate: aStringOrStream receiver: anObject notifying: requestor compile: compile
return the result of evaluating aString,
errors are reported to requestor. Allow access to
anObject as self and to its instVars (used in the inspector).
The compile argument specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluateFrom: aStringOrStream ifFail: failBlock
return the result of evaluating an expression from aStringOrStream.
In case of any syntax errors, return the value of failBlock.
No doit-entry is added to the changeLog.

o  evaluateFrom: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile
return the result of evaluating the next expression from aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

general helpers
o  argAndVarNamesForContext: aContext

o  methodCommentFromSource: aString
return the method's comment.
This is done by searching for and returning the first comment
from the method's source (excluding any double-quotes).
Returns nil if there is no comment.

o  methodCommentsFromSource: aString
return all of the method's comments.
Returns an empty collection if there is no comment.
usage example(s):
     Parser methodCommentsFromSource:(Method compiledMethodAt:#comment) source
     Parser methodCommentsFromSource:(Object class compiledMethodAt:#infoPrinting:) source

instance creation
o  for: aStringOrStream in: aClass
return a new parser, reading code for aClass from aStringOrStream

parsing
o  blockAtLine: line in: aMethod orSource: aString numArgs: nA numVars: nV
given a lineNr in some method, return the containing BlockNode or nil.
The given lineNr must be within a block for this to work.
This is used by the debugger, to guess reverse from a lineNumber,
to the corresponding block, in order to find out the block's
variable names
(mhmh - all of this wasnt't needed, if blocks stored their characterPosition internally).
WARNING: sometimes, the block is not correctly identified, if multiple blocks
are in one line.

o  checkMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings
parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.
usage example(s):
     self
        checkMethod:'foo
                        |local1 local2 local3|

                        local1 := local2.
                        ^ local3
                    '
        in:UndefinedObject
        ignoreErrors:true
        ignoreWarnings:true

o  parseExpression: aString
parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or #Error (syntactic error).
Error and warning messages are suppressed.
usage example(s):
     Parser parseExpression:''
     Parser parseExpression:'self foo'
     Parser parseExpression:'^ self foo'
     Parser parseExpression:'self:123'
     Parser parseExpression:'self:123' onError:nil

o  parseExpression: aString inNameSpace: aNameSpaceOrNil
parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or #Error (syntactic error).
Error and warning messages are suppressed.

o  parseExpression: aString inNameSpace: aNameSpaceOrNil onError: errorValue
parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or errorValue (syntactic error).
Error and warning messages are suppressed.

o  parseExpression: aString onError: errorValue
parse aString as an expression;
Return the parseTree (if ok), nil (for an empty string
or comment only) or errorValue (syntactic error).
Error and warning messages are suppressed.

o  parseLiteralArray: aStringOrStream
Parser literal array in given String or Stream.
Returns that array

o  parseMethod: aString
parse a method.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors etc.
Error and warning messages are sent to the Transcript.

o  parseMethod: aString in: aClass
parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
Error and warning messages are sent to the Transcript.

o  parseMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings
parse a method in a given class.
Return a parser (if ok) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

o  parseMethod: aString in: aClass warnings: warnBoolean
parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The warnBoolean arguments specifies if warning
messages should be sent to the Transcript or suppressed.

This method is OBSOLETE, and left in for backward compatibility.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  parseMethodArgAndVarSpecification: aString
parse a methods selector, arg and var spec (i.e. locals);
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Error and warning messages are sent to the Transcript.
This method is OBSOLETE.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  parseMethodArgAndVarSpecification: aString in: aClass
parse a methods selector, arg and var spec in a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals.
Error and warning messages are sent to the Transcript.
This method is OBSOLETE.

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  parseMethodArgAndVarSpecification: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings parseBody: body
parse a methods selector, arg and var spec in a given class;
If parseBody is true, also parse the statements
(for primitives & resourceSpecs).
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals

o  parseMethodArgAndVarSpecificationSilent: aString
parse a methods selector, arg and var spec (i.e. locals);
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodArgAndVarSpecification:, but does NOT
display error/warning messages on the transcript.

o  parseMethodArgAndVarSpecificationSilent: aString in: aClass
parse a methods selector, arg and var spec in a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args and locals.
Like #parseMethodArgAndVarSpecification:in:, but does not
display error/warning messages on the transcript.

o  parseMethodSilent: aString
parse a method.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors etc.
Like #parseMethod:, but warning/error messages are suppressed.

o  parseMethodSilent: aString in: aClass
parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
Like #parseMethod:in:, but warning/error messages are suppressed.

o  parseMethodSpecification: aString
parse a methods selector & arg specification;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.

o  parseMethodSpecification: aString in: aClass
parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.

o  parseMethodSpecification: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings
parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
noErrors and noWarnings specify if error- and warningMessages are
to be output onto the Transcript.

o  parseMethodSpecificationSilent: aString
parse a methods selector & arg specification;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodSpecification:, but does not display any error/warning Messages on the transcript.

o  parseMethodSpecificationSilent: aString in: aClass
parse a methods selector & arg spec for a given class;
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver etc.
Like #parseMethodSpecification:in:, but does not display any error/warning Messages on the transcript.

o  selectorInExpression: aString
parse an expression - return the selector.
Even malformed expressions
(such as missing receiver or missing arg) are parsed.
Used for the SystemBrowser's implementors/senders query-box initial text.
Returns nil if unparsable.
usage example(s):
    Parser selectorInExpression:'foo at:1 put:(5 * bar)'
    Parser selectorInExpression:'(foo at:1) at:1'
    Parser selectorInExpression:'a + 4'
    Parser selectorInExpression:'a negated'
    Parser selectorInExpression:'at:1 put:5'
    Parser selectorInExpression:'at:1 put:'
    Parser selectorInExpression:'a at:1 put:5'
    Parser selectorInExpression:'a at:1 put:'
    Parser selectorInExpression:'a := foo at:1 put:5'

o  withSelf: anObject parseExpression: aString notifying: someOne
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box.

o  withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignore
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box.

o  withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

o  withSelf: anObject parseExpression: aString notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

o  withSelf: anObject parseExpression: aStringOrStream onError: errorValue notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or errorValue (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

private
o  genMakeArrayWith: elementExpressions
return a node to generate an array at runtime.
Will generate:
Array with:el1 ... with:elN (if N <= 5)
or:
(Array new at:1 put:el1; ... at:N put:elN; yourself) (otherwise)

o  implementedInAnyClass: aSelectorStringOrSymbol

o  make: anObject immutable: whichImmutableClass
helper to optionally make Arrays, ByteArrays and Strings immutable.

o  makeImmutableArray: anArray
helper to optionally make an array immutable.

o  makeImmutableByteArray: aByteArray
helper to optionally make a byte array immutable.

o  makeImmutableString: aString
helper to optionally make a string immutable.

unparsing
o  methodSpecificationForSelector: aSelector
given a selector such as #foo:bar:, return a string that could
serve as a methods specification source code.
To be used for code generators
usage example(s):
     Parser methodSpecificationForSelector:#foo:
     Parser methodSpecificationForSelector:#foo:bar:
     Parser methodSpecificationForSelector:#foo:bar:baz:
     Parser methodSpecificationForSelector:#+
     Parser methodSpecificationForSelector:#negated

o  methodSpecificationForSelector: aSelector argNames: argNames
given a selector such as #foo:bar:, return a string that could
serve as a methods specification source code.
To be used for code generators
usage example(s):
     Parser methodSpecificationForSelector:#foo:bar: argNames:#('one' 'two' 'three')
     Parser methodSpecificationForSelector:#+ argNames:#('one')
     Parser methodSpecificationForSelector:#negated


Instance protocol:

Compatibility-ST80
o  evaluate: aStringOrStream in: aContextOrNil to: whatIsThis

o  evaluate: aString in: aClassOrContext to: to notifying: aRequestor ifFail: failBlock
stupid - there seem to be differences among the various

o  parse: methodSource in: aClass notifying: aRequestor
parse a methods source.
Return the method's parseTree

o  parseSelector: aStringOrStream
parse a method's source for its selector.
Return the selector
usage example(s):
     Parser new
        parseSelector:'
parseSelector:aStringOrStream
    self initializeFor:aStringOrStream.
    self parseMethodSpec.
    ^ selector.
'

Compatibility-Squeak
o  parse: methodSource class: aClass
parse a methods source.
Return the method's parseTree

accessing
o  allowUndeclaredVariables: aBoolean

o  correctedSource

o  currentSource
return either the corrected or the requestors original source

o  doItTemporaries

o  endOfLastToken

o  endOfSelectorPosition
return the sourcePosition of the last character of the method's selector spec

o  errorFlag
return true if there where any errors (valid after parsing)

o  evalExitBlock: aBlock
when evaluating a return expression, this block is evaluated

o  getNameSpace
retrieve the nameSpace, as found in a Namespace directive

o  implicitSelfSends

o  implicitSelfSends: aBoolean
turn on/off implicit self sends

o  interactiveMode: aBoolean
support for stx-scripting service

o  lineNumberInfo

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  lineNumberInfo: how

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  moreSharedPools

o  moreSharedPools: aCollection

o  primitiveNumber
return the ST-80 style primitiveNumber or nil (valid after parsing)

o  primitiveResources
return the ST-80 style resource info or nil (valid after parsing).

o  release
(comment from inherited method)
remove all references to objects that may refer to self.
Subclasses may redefine this method but should do a 'super release'.

o  setNameSpace: aNameSpaceOrNameSpaceName
sent from a namespace directive, if there is no requestor.
Sets the current namespace.

o  setPackage: aPackageID

o  targetClass

o  targetClass: aClass

o  tree
return the parsetree

o  tree: aTree
private: set the tree - for internal use only

o  warnSTXHereExtensionUsed

o  warnSTXHereExtensionUsed: aBoolean

o  wasParsedForCode

code generation hooks
o  assignmentRewriteHookFor: anAssignmentNode
invoked whenever an assignment node has been generated;
gives subclasses a chance to rewrite (instrument) it

o  blockNodeRewriteHookFor: aBlockNode
invoked whenever a block node has been generated;
gives subclasses a chance to rewrite (instrument) it

o  messageNodeRewriteHookFor: aMessageNode
invoked whenever a message send node has been generated;
gives subclasses a chance to rewrite (instrument) it

o  statementListRewriteHookFor: aStatementNode
invoked whenever a statement list node has been generated;
gives subclasses a chance to rewrite (instrument) it

o  variableReadRewriteHookFor: aVariableNode
invoked whenever a variable node has been generated;
gives subclasses a chance to rewrite (instrument) it

coding style checks
o  checkBlockArgumentNameConventionsFor: aVariableName

o  checkBlockVariableNameConventionsFor: aVariableName

o  checkBracketParenthesisMistakeInIfOrWhile: aNode from: startPosition to: endPosition

o  checkForLowercaseVariableName: aVariableName

o  checkForProperUseOfArticleInVariableName: aVariableName
heuristic - and wrong !
The problem is that a simple rule like anyVowel is not sufficient.
I'd be happy to get some help on that.

o  checkLocalVariableNameConventionsFor: aVariableName

o  checkMethodArgumentNameConventionsFor: aVariableName

o  checkMethodVariableNameConventionsFor: aVariableName

o  checkPlausibilityOf: aNode from: startPosition to: endPosition
this is a hack (which I don't like)

o  checkReturnedValues

o  checkSelector: selector for: receiver inClass: cls
check whether a method with selector exists in class cls and
that the method is not obsolete.
If cls is nil, check all classes for the selector.
Return an error string on error or nil on success

o  checkUnusedMethodVars

o  isPossiblyUninitializedLocal: aNode
because I do not look into it, yet

dummy-syntax detection
o  markArgumentIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markAssignedVariable: v from: pos1 to: pos2
intentionally left empty

o  markBadIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markBlockArgumentIdentifierFrom: pos1 to: pos2

o  markBlockFrom: startPos to: endPos
intentionally left empty

o  markBooleanConstantFrom: pos1 to: pos2
intentionally left empty

o  markBracketAt: pos
intentionally left empty

o  markConstantFrom: pos1 to: pos2
intentionally left empty

o  markGlobalClassIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markGlobalIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markHereFrom: pos1 to: pos2
intentionally left empty

o  markIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markInstVarIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markLocalIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markLocalVariableDeclaration: variableName from: pos1 to: pos2

o  markMethodArgumentIdentifierFrom: pos1 to: pos2

o  markMethodSelectorFrom: pos1 to: pos2
intentionally left empty

o  markParenthesisAt: pos
intentionally left empty

o  markReturnAt: pos
intentionally left empty

o  markSelector: sel from: pos1 to: pos2 receiverNode: aNode
intentionally left empty

o  markSelfFrom: pos1 to: pos2
intentionally left empty

o  markSuperFrom: pos1 to: pos2
intentionally left empty

o  markUnknownIdentifierFrom: pos1 to: pos2
intentionally left empty

o  markVariable: v
intentionally left empty

o  markVariable: v from: pos to: endPos
intentionally left empty

o  markVariable: v from: pos to: endPos assigned: assigned
intentionally left empty

error correction
o  addDoItTemporary: varName

o  alreadyWarnedUnimplementedSelectors

o  alreadyWarnedUnimplementedSelectorsPerReceiverClass

o  askForCorrection: aString fromList: aList
launch a selection box, which allows user to enter correction.
return newString or nil (for abort)

o  askForCorrection: aString fromList: aList for: originalSelector
launch a selection box, which allows user to enter correction.
return newString or nil (for abort)

o  askForVariableTypeWhenDeclaringUndefined: varName

o  checkIfAllSubclassesOf: aClass implement: aSelector

o  correctSelector: aSelectorString message: msg positions: posVector in: aClassOrNil for: receiverNode
notify error and correct if user wants to;
return #Error if there was no correction
or a ParseNode as returned by variable

o  correctSourceByDeletingFrom: start to: stop
correct (by deleting token) if user wants to;
return #Error if there was no correction;
nil if there was one.

o  correctVariable: varName atPosition: pos1 to: pos2
notify error and correct if user wants to;
return #Error if there was no correction
or a ParseNode as returned by variable

o  correctWith: correctionOperation from: pos1 to: pos2

o  declareUndefinedVariable: varName as: variableType
ST/X special - old classToCompileFor is obsoleted - refetch

o  defineAsUndeclaredVariable: aName
define varName as undeclared variable
(actually, it will be installed as a funny global named 'Undeclared:::<name>').
You can browse for methods with undeclareds by searching for global accesses
to 'Undeclared:::*' (or use the search-undeclared item in the launchers menu).

o  deleteDefinitionOf: varName in: defStartPosArg to: defEndPosArg
this removes the definition of varName from the declaration part i.e.
if the source was '... | foo bar baz |...'
deleting the var 'bar' changes the source to '... | foo baz |...'.
The code below tries to be somewhat smart w.r.t. spaces by
trying to remove the whiteSpace around the removed variable also.
However, this seems to not always work correctly

o  findBestSelectorsFor: aString
collect known selectors with their spelling distances to aString;
return the 10 best suggestions
usage example(s):
Time millisecondsToRun:[Parser new findBestSelectorsFor:'foo']
usage example(s):
Parser new findBestSelectorsFor:'findBestSel'
usage example(s):
Parser new findBestSelectorsFor:'fildBestSelectrFr'

o  findBestSelectorsFor: aString in: aClassOrNil
collect known selectors with their spelling distances to aString;
return the N best suggestions. If the argument, aClassOrNil is not nil,
the message is assumed to be sent to instances of that class (i.e. offer
corrections from that hierarchy only)

o  findBestVariablesFor: aString
collect known variables with their spelling distances to aString;
return the 10 best suggestions

o  selectorCheck: aSelectorString for: receiver position: pos1 to: pos2

o  selectorCheck: aSelectorString for: receiver positions: posVector
just a quick check: if a selector is totally unknown as a symbol,
has the same name as a variable, cannot be understood or is obsolete.
Simple, but catches many typos

o  typeOfNode: aNode
if the receiver is a constant, we know its class...

error handling
o  blockArgRedefined: tokenName from: pos1 to: pos2
argname reuse

o  disableWarningsOnCurrentMethodFor: flagName
(comment from inherited method)
ignored here

o  errorMessageForUndefined: variableName
Return a proper method for undefined variable named `variableName`

o  exitWith: something
this is the longjump out of evaluation via a return expression

o  identifierExpectedIn: what

o  isFirstWarning: something
answer true, if a warning about something has not been shown yet.
Remember, that the warning has been shown

o  methodArgRedefined: tokenName from: pos1 to: pos2
argname reuse

o  rememberWarning: something
remember, that a warning for something has been shown.

o  showErrorMessage: aMessage position: pos
redefined since parser can give more detailed info about
the class & selector where the error occurred.

o  showErrorMessageForClass: aClass
compiler parseError:'syntax error'.

o  undefError: aName position: pos1 to: pos2
report an undefined variable error - return true, if it should be
corrected. If not corrected, only one warning is made per undefined
variable.

o  warnSTXHereExtensionUsedAt: position
only warn once

o  warnSTXNameSpaceUseAt: position
only warn once

o  warnSTXSpecialCommentAt: position to: endPosition
(comment from inherited method)
dfo

o  warnUnused: aNameCollection
report an unused method variable

evaluating expressions
o  defaultFailBlock
[:msg | self error:(msg ? 'error in eval') ]

o  evaluate: aStringOrStream
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream compile: compile
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream in: aContext receiver: anObject
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream in: aContext receiver: anObject ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream in: aContext receiver: anObject notifying: requestor logged: logged ifFail: failBlock compile: compile checkForEndOfInput: checkForEndOfInput
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

If checkForEndOfInput is set, generate an error if EOF has not been reached
at the end of the expression.

o  evaluate: aStringOrStream logged: logged

o  evaluate: aStringOrStream notifying: someRequestor

o  evaluate: aStringOrStream notifying: someRequestor compile: compileBoolean

o  evaluate: aStringOrStream receiver: anObject
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream receiver: anObject ifFail: failBlock
return the result of evaluating aStringOrStream, errors are reported to requestor.
Allow access to anObject as self and to its instVars (used in the inspector).
If logged is true, an entry is added to the change-file. If the failBlock argument
is non-nil, it is evaluated if an error occurs.
Finally, compile specifies if the string should be compiled down to
bytecode or instead be interpreted from the parseTree.
The first should be done for doIts etc, where a readable walkback is
required.
The latter is better done for constants, styleSheet and resource
reading and simple sends, where the overhead of compilation is bigger
than the interpretation overhead.

o  evaluate: aStringOrStream receiver: receiver notifying: someRequestor compile: compileBoolean

o  evaluateDeclarationFor: anEnvironment

initialization
o  initializeFlagsFrom: aParser
initialize flags from another scanner

obsolete
o  correctByDeleting
correct (by deleting token) if user wants to;
return #Error if there was no correction;
nil if there was one.

parsing
o  block
parse a block; return a node-tree, nil or #Error
usage example(s):
blockNode lineNumber:lno.

o  blockBody
parse a currentBlock's block-body; return a node-tree, nil or #Error

o  blockExpression
parse a blockExpression; return a node-tree, nil or #Error.
Not used by ST/X's parser, but added for ST-80 compatibility.

o  blockStatementList
parse a blocks statementlist; return a node-tree, nil or #Error

o  checkForEndOfInput
check, that we have reached the end of input.
Error, if not

o  emptyStatement

o  makeSelector: rawSelector
cg: how about (to prevent creation of new symbols):

o  parseExpressionWithSelf: anObject notifying: someOne ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings inNameSpace: aNameSpaceOrNil
parse aString as an expression with self set to anObject;
Return the parseTree (if ok), nil (for an empty string
or comment only ) or #Error (syntactic error).

Errors and warnings are forwarded to someOne (usually some
codeView) which can highlight it and show a popup box,
iff ignoreErrors/ignoreWarnings is true respectively.

o  parseLiteralArray: aStringOrStream
Parser literal array in given String or Stream.
Returns that array

o  parseMethod
parse a method.
Return the parseTree or #Error.

method ::= methodSpec methodBody

o  parseMethod: aString in: aClass
parse a method in a given class.
Return a parser (if ok) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

o  parseMethod: aString in: aClass ignoreErrors: ignoreErrors ignoreWarnings: ignoreWarnings
parse a method in a given class.
Return a parser (if ok), nil (empty) or #Error (syntax).
The parser can be queried for selector, receiver, args, locals,
used selectors, modified instvars, referenced classvars etc.
The noErrors and noWarnings arguments specify if error and warning
messages should be sent to the Transcript or suppressed.

o  parseMethodBody
parse a methods body (locals & statements).
No more tokens may follow.
Return a node-tree, or #Error

methodBody ::= '<' st80Primitive '>' #EOF
| '<' st80Primitive '>' methodBodyVarSpec statementList #EOF


o  parseMethodBodyOrEmpty
parse a methods body (locals & statements);
return a node-tree, nil or #Error.
empty (or comment only) input is accepted and returns nil.

methodBodyOrNil ::= '<' st80Primitive '>'
| '<' st80Primitive '>' methodBodyVarSpec statementList
| <empty>

o  parseMethodBodyVarSpec
parse a methods local variable specification, handling
possible primitive or resourceSpecs.
.
Leave spec of locals in methodLocals as a side effect.
Return #Error or self.

methodBodyVarSpec ::= '|' { IDENTIFIER } '|'
| <empty>

o  parseMethodSpec
parse a methods selector & arg specification;
Set selector and methodArgs in the receiver as a side effect.
Return the receiver or #Error.

methodSpec ::= { KEYWORD IDENTIFIER }
| binaryOperator IDENTIFIER
| IDENTIFIER

o  returnStatement
parse a return statement;

statement ::= '^' expression

o  statement
parse a statement; return a node-tree or #Error.

statement ::= '^' expression
| PRIMITIVECODE
| expression

usage example(s):
allow empty statement

o  statementList
parse a statementlist; return a node-tree, nil or #Error.
Statements must be separated by periods.

statementList ::= <statement>
| <statementList> . <statement>

o  variableTypeDeclarationFor: aVariable
experimental support for Domain variables (constraint programming support):
a variable-declaration of the form
|var (domain) ... |
declares var as a domainVariable.
Valid domains are:
min %% max - integer range domain
Bool - boolean domain
Nat - positive integer domain
Int - integer domain
#sym1 ... #sym2 - enumerated symbolic domain

o  warnAboutEmptyStatement

parsing-expressions
o  array
parse a literal array's elements
usage example(s):
self syntaxError:'unterminated array-constant; '')'' expected'
usage example(s):
always return the same object for an empty array,
         which is also immutable by definition

o  arrayConstant
some more special symbol consts ...

o  arrayIndexingExpression
parse an array index expression; this is a squeak/stx extension.
foo[x] is syntactic sugar for 'foo at:x'
and foo[x] := expr is syntactic sugar for 'foo at:x put:expr';
with commas, foo[x , y] generates foo at:x at:y.
This syntax extension must be enabled in the parserFlags as
allowArrayIndexSyntaxExtension (disabled by default)

o  binaryExpression
parse a binary-expression; return a node-tree, nil or #Error

o  binaryExpressionFor: receiverArg
parse a binary-expression; return a node-tree, nil or #Error

o  byteArray
parse a literal byteArray's elements
usage example(s):
literal bytearrays started with ST-80 R4 - byteArray constants are written as #[ ... ]

o  degeneratedKeywordExpressionForSelector
parse a keyword-expression without receiver - for the selector only.
Return the selector or nil (if it cannot be determined).
This is not used in normal parsing, but instead to extract the selector from a code fragment.
(for example, the system browsers 'implementors'-function uses this to extract a selector from
the selection)
usage example(s):
     (self new source:'hello:world:') nextToken; degeneratedKeywordExpressionForSelector
     (self new source:'hello:world') nextToken; degeneratedKeywordExpressionForSelector
     (self new source:'hello:') nextToken; degeneratedKeywordExpressionForSelector

o  expression
parse a cascade-expression; return a node-tree, nil or #Error.

expression ::= keywordExpression
| keywordExpression cascade

cascade ::= ';' expressionSendPart
| cascade ';' expressionSendPart

expressionSendPart ::= { KEYWORD binaryExpression }
| BINARYOPERATOR unaryExpression
| IDENTIFIER

o  functionCallArgList

o  functionCallExpression
parse a functionCall;
this is an st/x extension.
foo(x)
is syntactic sugar for
foo value:x
This syntax extension must be enabled in the parserFlags as
allowFunctionCallSyntaxForBlockEvaluation (disabled by default)

o  inlineObjectFrom: pos1
an experimental ST/X feature.
InlineObject as #{ name1: value1. ... nameN: valueN }
creates a literal object with an anon class which provides getter/setters on all
names and is preinitialized with valueI.
The initial #{ is supposed to be skipped and its position passed in as pos1.
Notice: the values must be literals too;
currently the JavaScript style (using expressions) is not supported.

o  keywordExpression
parse a keyword-expression; return a node-tree, nil or #Error.

keywordExpression ::= binaryexpression
| { KEYWORD-PART binaryExpression }

o  keywordExpressionFor: receiverArg
parse a keyword-expression; return a node-tree, nil or #Error.

keywordExpression ::= binaryexpression
| { KEYWORD-PART binaryExpression }

o  literalInlineObjectFor: namesAndValues
fix

o  primary
parse a primary-expression; return a node-tree, nil or #Error.
This also cares for namespace-access-paths.

o  primary_dolphinComputedLiteral
parse a dolphin computed literal; return a node-tree, or raise an Error.
In dolphin, these are written as: ##( expression )
and create a literal constant for the expressions value.
Right now, only a subset is supported - Strings, ByteArrays and Characters.
WARNING: this is only supported to allow file-in of dolphin code.
Since stc cannot handle this (at the moment), you should rewrite the code
if you ever plan to stc-compile it into a shared library.
The question is still: how should stc ever be able to do this, as it cannot execute
smalltalk code; it could generate code to execute it at initialization time of the
generated code, but then, it is no longer a compile-time constant
(for example, generating a compilation-Date constant is then not possible...)

o  primary_expression
parse a parentized expression primary; return a node-tree, or raise an Error.

o  primary_false
parse a false primary; return a node-tree, or raise an Error.

o  primary_here
parse a here primary; return a node-tree, nil or #Error.

o  primary_identifier
parse a primary starting with an identifier: either a variable or an assignment.
return a node-tree, or raise an Error.

o  primary_lazyValue

o  primary_nil
parse a nil primary; return a node-tree, nil or #Error.

o  primary_self
parse a self primary; return a node-tree, nil or #Error.

o  primary_simpleLiteral
parse a simple literal primary; return a node-tree, or raise an Error.

o  primary_squeakComputedArray
parse a squeak computed array literal; return a node-tree, or raise an Error.
In squeak, these are written as: { expr1 . expr2 . ... exprN )
and create a message to construct an Array containing the exprI values

o  primary_super
parse a super primary; return a node-tree, nil or #Error.

o  primary_thisContext
parse a thisContext primary; return a node-tree, or raise an Error.

o  primary_true
parse a true primary; return a node-tree, or raise an Error.

o  qualifiedNameFrom: pos1
a vw3.x (and later) feature: QualifiedName is #{ id ... id }
and mapped to a global variable here.
The initial #{ is supposed to be skipped and its position passed in as pos1.

o  qualifiedNameOrInlineObject
either a vw3.x (and later) QualifiedName #{ id ... id }
or an ST/X experimental inline object #{ name: value. ... name: value }
The initial #{ is skipped here, but the position passed down.

o  squeakComputedArray
not reached

o  typedArray: typeSymbol
parse a typed array's elements.
This is an ST/X extension, which is not supported by other Smalltalk implementations.
For now, the support is disabled; enable with:
ParserFlags allowSTXExtendedArrayLiterals:true
Typed literal arrays are written in scheme-style as:
#u8( element... ) - unsigned 8-bit bytes (i.e. a regular ByteArray)
#u16( element... ) - unsigned 16-bit shorts (i.e. a WordArray)
#u32( element... ) - unsigned 32-bit ints (i.e. an IntegerArray)
#u64( element... ) - unsigned 64-bit ints (i.e. a LongIntegerArray)
#s8( element... ) - signed bytes (i.e. a SignedByteArray)
#u16( element... ) - signed 16-bit shorts (i.e. a SignedWordArray)
#u32( element... ) - signed 32-bit ints (i.e. a SignedIntegerArray)
#u64( element... ) - signed 64-bit ints (i.e. a SignedLongIntegerArray)
#f16( element... ) - tiny 16-bit floats (i.e. a HalfFloatArray)
#f32( element... ) - short 32-bit floats (i.e. a FloatArray)
#f64( element... ) - 64-bit doubles (i.e. a DoubleArray)

#f( element... ) - same as f32(...)
#d( element... ) - same as f64(...)

o  unaryExpression
parse a unary-expression; return a node-tree, nil or #Error

o  unaryExpressionFor: receiverArg
parse a unary-expression; return a node-tree, nil or #Error

o  variable
parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.
If undefined, notify error and correct if user wants to

o  variableOrError
parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.

o  variableOrError: varName
parse a simple (i.e. non-namespace-access path) variable;
return a node-tree, nil or #Error.
Does not advance to next token.

parsing-primitives & pragmas
o  addAnnotationWithKey: key andArguments: arguments
was: annot := { key . arguments}.

o  checkForClosingAngle
verify a closing angle and skip over it

o  defineWellknownCTypesIn: aDictionary
a few wellknown types

o  generateCallToExternalFunction: fn lineNr: lineNr
the following stuff was commented (2007-07-30), to make ole: work.
Ask felix or stefan

o  generateReturnOfValue: aValue

o  generateTrapCodeForUnavailableCParser

o  parseAnotationLiteral
(tokenType == #Keyword) ifTrue: [
value := '#', tokenName.
self nextToken.
^ value.
].

o  parseExceptionOrContextPragma
parse
<exception: #handle|raise|unwind>,
<context: #return>
context flagging pragmas.

o  parseExternalFunctionCallDeclaration
callType is one of c: / cdecl: / api: / apicall: ...

o  parseGSTExternalFunctionDeclaration: argArray
Handles GNU Smalltalk-style exteranl function declarations.
Example:
<cCall: 'cairo_close_path' returning: #void args: #(#cObject )>

o  parseOtherPrimitives
not really a check, but remembers endPos

o  parsePragma
'<' has already been parsed.
<pragma: +arrayIndexSyntaxExtension>

o  parsePrimitive
parse an ST-80 type primitive as '< primitive: nr >';
(return primitive number or #Error)
or a Squeak-style primitive, as '< primitive: string >';
(return primitive name or #Error)
or a V'Age-style primitive, as '< primitive: identifier >';
(return primitive name or #Error)

Also, resource specs are parsed; the result is left (as side effect) in primitiveResource.
It is used to flag methods, for faster finding of used keyboard accelerators,
and to mark resource methods (image, menu or canvas resources).

prim ::= st80Primitive | st80Pragma | stxPragma
| squeakPrimitive | vAgePrimitive | newSTXPrimitive
| externalFuncDecl
| resourceDecl

st80Primitive ::= 'primitive:' INTEGER
st80Pragma ::= 'exception:' ( 'handle | 'raise' | 'unwind' )
stxPragma ::= 'context:' 'return'

squeakPrimitive ::= 'primitive:' STRING

newSTXPrimitive ::= 'primitive'

vAgePrimitive ::= 'primitive:' IDENTIFIER
| 'sysprim:' IDENTIFIER

externalFuncDecl ::= vwExternalFuncDecl
| stvExternalFuncDecl
| squeakExternalFuncDecl
| dolphinExternalFuncDecl

vwExternalFuncDecl ::= 'c:' vwFuncDecl

stvExternalFuncDecl ::= 'api:' stvFuncDecl
| 'ole:' stvFuncDecl

squeakExternalFuncDecl ::= 'apicall:' stvFuncDecl
| 'cdecl:' stvFuncDecl

dolphinExternalFuncDecl ::= 'stdcall:' stvFuncDecl

resourceDecl ::= 'resource:' SYMBOL - leave SYMBOL in primitiveResource
| 'resource:' SYMBOL (...) - leave (SYMBOL (...)) in primitiveResource
| 'pragma:' SYMBOL - same as resource; alternative syntax
| 'pragma:' SYMBOL (...) - same as resource; alternative syntax
| 'attribute:' SYMBOL - same as resource; alternative syntax
| 'attribute:' SYMBOL (...) - same as resource; alternative syntax

o  parsePrimitiveOrResourceSpecOrEmpty
parse a methods primitive or resource spec

o  parseResourcePragma
'<' has already been parsed.

o  parseSTVExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr
parses ST/V function declarations of the forms
'<api: functionName argType1 .. argTypeN returnType>'
'<ole: vFunctionIndex argType1 .. argTypeN returnType>'

o  parseSTXOrSqueakOrDolphinExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr
parses squeak/dolphin/stx function declarations of the forms
'<stdcall: [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex argType1..argTypeN>'
'<cdecl: [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex argType1..argTypeN>'

'<cdecl: [async] [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex ( argType1..argTypeN ) module: moduleName >'
'<apicall: [async] [virtual|nonVirtual][const][mustFree] returnType functionNameOrIndex ( argType1..argTypeN ) module: moduleName >'

o  parseTraditionalPrimitive
parse everything after the initial '<primitive:'

o  parseVWTypeOrExternalFunctionDeclarationFrom: aStream definitionType: definitionType knownDefinitions: dictionaryOfTypesOrNil lineNr: lineNr
parses visualWorks type/function declarations of the form:
'<c: ...>'

o  primitiveNumberFromName: aPrimitiveName
for future compatibility with Squeak ...

o  rememberContextPragma: pragmaType value: pragmaValue

o  rememberContextReturnablePragma

o  skipForClosingAngle
skip

private
o  currentNameSpace

o  currentNameSpace: aNameSpace

o  currentNamespace: aNameSpace

** This is an obsolete interface - do not use it (it may vanish in future versions) **

o  currentPackage

o  currentUsedNameSpaces

o  findNameSpaceWith: aVariableName
private names have already been searched for.

o  genMakeArrayWith: elementExpressions
return a node to generate an array at runtime.
Will generate:
Array with:el1 ... with:elN (if N <= 5)
or:
(Array new at:1 put:el1; ... at:N put:elN; yourself) (otherwise)

o  inWhichClassIsClassInstVar: aString
search class-chain for the class-instance variable named aString
- return the class or nil if not found

o  inWhichClassIsClassVar: aString
search class-chain for the classvariable named aString
- return the class or nil if not found

o  isStatementAnUnconditionalReturn: aStatementNode
used on the last statement, to see if this is doing a return already
(i.e. its own return value is to be ignored.

o  isStatementListAnUnconditionalReturn: aStatementNode
used on a statement-list, to see if this is doing an unconditional return

o  isValidUnarySelector: tokenType
ParserFlags allowReservedWordsAsSelectors:true.
1234 self.

ParserFlags allowReservedWordsAsSelectors:false
1234 self.

o  makeImmutableArray: anArray

o  makeImmutableString: aString

o  makeReferenceFor: aNode
variable references (&var)

o  markUnreachableCodeAsCommentIn: expr
take care (ignore) for unreachable constants which do not have a selector

o  noAssignmentAllowed: eMsg at: pos
eat the assign when doing highlighting only

o  nodeForMethodArg: varName
var used:true.

o  nodeForMethodVariable: varName

o  plausibilityCheck: aNode
do not check this for doits

o  selfNode
JV@2011-07-19: Changed not to share the nodes
usage example(s):
selfNode := SelfNode value:selfValue
usage example(s):
^ selfNode

queries
o  annotationInfo
return the annotations - if any - in a format usable for source analysis
(i.e. with start and stop positions)
valid only after parsing

o  annotations
return the annotations - if any - in the format stored in a method
(i.e. as an array or 2-element key-argument arrays).
valid only after parsing

o  classToLookForClassVars
helper - return the class to look for classVars.
If there is a context in which we evaluate, the
methods implementing class is used instead of the
class of the receiver.

o  classesClassInstVarNames
caching allInstVarNames for next compilation saves time ...

o  classesClassVarNames
caching allClassVarNames for next compilation saves time ...

o  classesInstVarNames
caching allInstVarNames for next compilation saves time ...

o  contextMustBeReturnable
misusing/misinterpreting the lineNumberInfo flag is a q&d hack; there should be an extra flag

o  didWarnAboutSqueakExtensions

o  doItSelector
the name of the method used for doit's.
The method will not be installed, but called directly,
so the name is more or less arbitrary.

o  hasNonOptionalPrimitiveCode
return true if there was any ST/X style primitive code (valid after parsing)

o  hasPrimitiveCode
return true if there was any ST/X style primitive code (valid after parsing)

o  isCompiling
return true if compiling code as opposed to evaluating

o  isDoIt

o  isEmptyMethod
return true (after a parse) if this is an empty (documentation) method

o  isSyntaxHighlighter

o  methodArgs
return an array with methodarg names (valid after parsing spec)

o  methodVars
return a collection with method variablenames (valid after parsing)

o  numberOfMethodArgs
return the number of methodargs (valid after parsing spec)

o  numberOfMethodVars
return the number of method variables (valid after parsing)

o  selector
return the selector (valid after parsing spec)

o  shouldPerformCodingStyleChecks

o  whichClassIncludesClassVar: aVariableName
helper: find the class in which a class variable is defined

queries-statistic
o  messagesPossiblySent
return a collection with possible message selectors (valid after parsing).
Includes things known or possibly used with #perform: or in specs.
The returned collection is filled by heuristics, not sharp, exact

o  messagesSent
return a collection with sent message selectors (valid after parsing).
Includes all sent messages (i.e. also sent super messages)

o  messagesSentToSelf
return a collection with message selectors sent to self only (valid after parsing)

o  messagesSentToSuper
return a collection with message selectors sent to super only (valid after parsing)

o  modifiedClassVars
return a collection with classvariablenames modified by method (valid after parsing)

o  modifiedGlobals
return a collection with globalnames modified by method (valid after parsing)

o  modifiedInstVars
return a collection with instvariablenames modified by method (valid after parsing)

o  modifiedPoolVars
return a collection with pool variable names modified by method (valid after parsing)

o  readClassVars
return a collection with classvariablenames read by method (valid after parsing)

o  readGlobals
return a collection with global varNames read by method (valid after parsing)

o  readInstVars
return a collection with instvariablenames read by method (valid after parsing)

o  readPoolVars
return a collection with pool var names read by method (valid after parsing)

o  sendsSelector: selectorString

o  usedClassVars
return a collection with classvariablenames refd by method (valid after parsing)

o  usedGlobals
return a collection with globalnames refd by method (valid after parsing)

o  usedInstVars
return a collection with instvariablenames refd by method (valid after parsing)

o  usedPoolVars
return a collection with pool variable names refd by method (valid after parsing)

o  usedSymbols
return a collection with used symbols (except for sent messages) (valid after parsing)

o  usedVars
return a collection with variablenames refd by method (valid after parsing)

o  usesSuper
return true if the parsed method uses super (valid after parsing)

setup
o  arraysAreImmutable

o  arraysAreImmutable: aBoolean

o  classToCompileFor

o  foldConstants: aSymbolOrNil
change the constant folding level. See the classMethod for a description.

o  initialize
<modifier: #super> "must be called if redefined"

o  parseForCode
turns off certain statistics (keeping referenced variables, modified vars etc.)
Use this when parsing for compilation or evaluation

o  setClassToCompileFor: aClass
set the class to be used for parsing/evaluating

o  setContext: aContext
set the context used while evaluating

o  setSelf: anObject
set the value to be used for self while evaluating

o  stringsAreImmutable

o  stringsAreImmutable: aBoolean

o  warnAboutMissingMethodComment

o  warnUndeclared

o  warnUndeclared: aBoolean

o  warnUnusedVars

o  warnUnusedVars: aBoolean

statistic
o  rememberClassVarModified: name

o  rememberClassVarRead: name

o  rememberClassVarUsed: name

o  rememberGlobalModified: name

o  rememberGlobalRead: name

o  rememberGlobalUsed: name

o  rememberInstVarModified: name

o  rememberInstVarRead: name

o  rememberInstVarUsed: name

o  rememberLocalModified: name

o  rememberLocalUsed: name

o  rememberPoolVarModified: name

o  rememberPoolVarRead: name

o  rememberPoolVarUsed: name

o  rememberReturnedValue: anExpressionNode

o  rememberSelectorPossiblyUsed: sel

o  rememberSelectorUsed: sel

o  rememberSelectorUsed: selectorArg receiver: receiverNode
cg: thought this was a good idea;

o  rememberSelectorUsed: selectorArg receiver: receiverNode args: args
TODO: the heuristics below could be made more
flexible by annotating methods which do a perform somehow,
and then asking implementors of selectorArg...

o  rememberSelectorUsedInSelfSend: sel

o  rememberSelectorUsedInSuperSend: sel

o  rememberSymbolUsed: aSymbol

o  rememberVariableUsed: name


Private classes:

    AskForVariableTypeOfUndeclaredQuery
    CorrectByChangingSelector
    CorrectByDeclaringIdentifierAs
    CorrectByDeletingLocalIdentifier
    CorrectByGeneratingMissingMethod
    CorrectByGroupingMessage
    CorrectByInsertingPeriod
    CorrectByInteractiveCorrection
    CorrectByInteractiveRename
    CorrectByMakingValidHexConstant
    Correction
    ParsedAnnotation
    PossibleCorrectionsQuery
    PrimitiveSpecParser
    RestartCompilationSignal

Examples:


    Parser
        evaluate:'1+2*3'
        in:nil
        receiver:nil
        notifying:nil
        logged:false
        ifFail:nil
    Parser undefinedVariableNotification handle:[:ex |
        |badName|

        badName := ex variableName.
        ex proceedWith:(ConstantNode value:5).
    ] do:[
        ^ self compilerClass
            evaluate:'foo * 3'
            in:nil
            receiver:nil
            notifying:nil
            logged:false
            ifFail:nil
    ]
the following are experimental...
    <pragma: +arrayIndexSyntaxExtension>
    |c|

    c := Array[7].
    c[4] := 4.
    c[6] := 6.
    c[7] := 7.
    c[7] := c[4] - c[6].
    <pragma: +arrayIndexSyntaxExtension>
    |d|

    d := Dictionary new.
    d['one'] := 'eins'.
    d['two'] := 'zwei'.
    d['three'] := 'drei'.

    d['two'] , d['three']
Reparse whole image...
    Smalltalk allClassesDo:[:cls|
        cls isLoaded ifTrue:[
            Transcript show: cls name; show: '...'.
            cls methodsDo:[:mth|
                Parser parseMethod: mth source.
            ].
            Transcript showCR:'OK'.
        ]
    ]


ST/X 7.1.0.0; WebServer 1.663 at exept.de:8081; Sat, 17 Nov 2018 08:44:07 GMT