How to Write Primitives and Inline C Code

Contents

• Introduction
• Simple Examples
• Accessing Smalltalk Objects
  • Object Representation
  • Variables
  • Global Variables & Identifier Conflicts
  • Symbols
  • Macros & Helper Functions
    • Constants & Types
    • Query & Value Extraction Helpers
    • Object Creation Helpers
    • Garbage Collector Helpers
    • OBJ to C-Structure Casts
    • Utility Functions
• Returning a Value from a Primitive
• Local Storage in Primitives
• Stack Requirements
• Ommiting the Context Setup
• Register Variables / Method Locals
• Sending Messages from within Primitive Code
• Remembering Smalltalk Objects in External C-Code
• Allocating Object Memory in Primitives
• Allocating C-Memory in Primitives
• Allocating C-Memory from Smalltalk-Code
• Modifying Objects from within Primitive Code (Storing)
• Storing into the Receiver
• Storing into Globals and Classvariables
• Mixing Smalltalk and C-Code
• Interrupting Primitive Code
• Triggering Interrups from C-Code
• Timed Callbacks into C-Code
• Reporting Errors from C-Code
• Problems
  • Conflicting Names of Local Variables and Structures/Typedefs
  • Other Name Conflicts with Structures, Macros and Typedefs
  • Local Variables whose Name Starts with an Underscore
• Hints & Tips for Writing Primitives
• Symptoms and Strategies to Find and Fix Bugs
• Solutions for Typical Problems
  • Passing a String to a C-Function
  • Returning a String from a C-Function
  • Returning two Strings from a C-Function
  • Passing an Array of Strings to a Primitive
  • Passing a Polymorph Array with either Strings, Floats or Integers
  • Returning an Array of Strings
  • Passing a Vector of Floats
  • Processing a Square Matrix
  • Calling an Extern C-Function via a Timer

Introduction

Smalltalk/X does not support these (*); instead primitive code is entered as C-code into the methods. This allows the programmer to code everything he/she likes as C-code (beside the traditional primitives). Notice that in order to simply call any existing C function (for example, a library function from a dll), no inline C code is strictly needed: you can use the alternative FFI callout mechanism, which is very easy to use by defining a method with a C-call pragma. However, inline C code is both faster and allows for more control over what is done in the method. It also allows for arbitrary code to be written which manipulates Smalltalk or other objects without calling another function.

Primitive C-code is included right into the method code and surrounded by the special character sequences '%{' and '%}' (the quotes are not part of it).
This character sequence has been choosen as it does not conflict with existing Smalltalk programs.

Inline code can be placed wherever a Smalltalk statement may be placed.

Notice, that not all systems support the dynamic compilation to machine code (and especially: its dynamic integration) into the active system;

• On systems which do support dynamic loading, the incremental compiler will automatically call the stc compiler to create an intermediate object file and load the resulting machine code module into the system. Thus, on those systems, methods with primitive code can be edited and accepted just like any other method in the System Browser. All popular systems support dynamic loading; expecially Windows, Linux and Solaris do so.

• On systems which do not support dynamic loading of program code, you have to save the class containing the primitives into a file (fileOut), leave the system, compile it and relink the system. Systems which do not support dynamic loading may include some mobile or embedded systems, and also very old operating systems (which are actually no longer maintained anyway).

Lets start with introductionary examples, to give you some idea how primitive code looks in ST/X. The details will be described below.

Calling a simple C function:

   myPrimitive
   %{
       printf("hello world\m");
   %}
   !

   anotherPrimitive:aFlag
       aFlag ifTrue:[
   %{
	   printf("it is true\n");
   %}
       ] ifFalse:[
   %{
	   printf("no, it is not\n");
   %}
       ]
   !

   anotherPrimitive2:aFlag
   %{
       if (aFlag == true) {
	   printf("it is true\n");
       } else{
	   printf("no, it is not\n");
       }
   %}
   !

   yetAnotherPrimitive:something
       |aVariable|

       aVariable := something.
   %{
       if (aVariable == true)
	   aVariable = nil;
   %}
   . "<- notice the period - %{ ... %} is syntactically a statement
	 which must be separated by '.'-characters "

       aVariable print
   !

   add:num1 and:num2
       |sum|

   %{
       /* the code below only handles
	* SmallInteger operands ...
	*/
       if (__isSmallInteger(num1)
	&& __isSmallInteger(num2)) {
	    sum = __MKINT(sum1 + sum2);
       }
   %}.

       ^ sum
   !

   globalAccessExample
   %{
       if (@global(Debugging) == true) {
	   printf("some info message\n");
       }
   %}.
   !

   makeArrayWithStrings
   %{
       OBJ myArray;

       myArray = __ARRAY_NEW_INT(3);
       // notice the protect/unprotect. These are needed to protect myArray
       // from being garbage collected. it is required around any code which possibly
       // leads to a garbage collect.
       __PROTECT__(myArray);
       s = __mkString("string1");
       __UNPROTECT__(myArray);
       __ArrayVal(myArray)->a_element[0] = s;   // C uses 0-based indexing
       __STORE(myArray, s);     // tell GC that a reference was taken (write barriere)

       __PROTECT__(myArray);
       s = __mkString("string2");
       __UNPROTECT__(myArray);
       __ArrayVal(myArray)->a_element[1] = s;   // C uses 0-based indexing
       __STORE(myArray, s);     // tell GC that a reference was taken (write barriere)

       __PROTECT__(myArray);
       s = __mkString("string3");
       __UNPROTECT__(myArray);
       __ArrayVal(myArray)->a_element[2] = s;   // C uses 0-based indexing
       __STORE(myArray, s);     // tell GC that a reference was taken (write barriere)

       RETURN(myArray);     // not return! RETURN is a macro which handles context references
   %}.
   !

sizeof(char *) + (2 * sizeof(int32))

Notice, that beside those 12bytes, there is NO additional overhead; if you think this is much, think about malloc(), which also requires at least 4 or 8 bytes of overhead per allocated chunk (depending on the algorithm used), and does NOT provide runtime type information and automatic generational memory reclamation.

For example, the memory used by the "true" singleton object looks like:

	+-------------------------+
	|       class-pointer   --------------> True-class
	+-------------------------+
	|       size  (12)        |
	+-------------------------+
	|       additional info   +
	+-------------------------|

The size field is actually only required for objects with indexed instance variables; for other objects, the size could be fetched from its class object (which also knows how big its instances are). However, to tune the memory system, all objects include that size info, even if that is redundant. Future ST/X versions may optionally support objects without a size field.
This would save about 120k-150k bytes of object space in the initial image (there are about 35000-40000 objects initially) and would result in a noticable reduction in memory requirements if many small objects are created. Obviously, the savings are marginally for big objects. However, also a slight performance degration is to be expected (since all accesses to the size will require an additional compare & conditional branch) and a major part of object space is required for indexed objects (strings, byteArrays, methodDictionaries and symbols), which is why there are currently no plans to do this.

However, to avoid disaster if such a change may ever be made, never access the size field (or any other header field) directly.
Instead, use the access macros which are described below. Upcoming systems may be delivered with different access macros.

Lets return to an object's layout; right after the header, any named instance variables are stored.
For example, the object "(100.0 @ 200.0)" is in memory:

	+-------------------------+
	|       class-pointer   --------------> Point-class
	+-------------------------+
	|       size  (20)        |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	|       x instvar       --------------> 100.0 Float object
	+-------------------------+
	|       y instvar       --------------> 200.0 Float object
	+-------------------------+

Do not depend on the TAG bit being in a particular bit position; on some machines, this bit is stored in the sign bit, to make the smallInteger-check faster
(its done with all CPUs which have a branch on minus instruction and set the condition codes without a test instruction; for example, the 68k or mips CPUs currently have the TAG bit in the sign bit).

To stay unaffected of such details, use the check/conversion macros described below.

Although, on 32 bit machines there are actually 2 bits for use as tag bits (alignment is really at least a 4byte one), currently only one bit is used to tag small integers.
Future versions may encode other types (for example, short floats) as immediate values, to reduce memory requireements even more.
These changes will be transparent to your primitives source code, IF access macros are used everywhere.

To summarize, the object "(100 @ 200)" is in memory:

	+-------------------------+
	|       class-pointer   --------------> Point-class
	+-------------------------+
	|       size  (20)        |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	|TAG|   x instvar 100     |
	+-------------------------+
	|TAG|   y instvar 200     |
	+-------------------------+

The __Class() macro described below (and therefore the #class message sent to any object) check for the TAG bit and 'mimics' a class field pointing to SmallInteger.

There is one other special object, which is represented as an immediate object, "nil". It is represented by a NULL pointer (i.e. all zero bits). The choice of using an immediate nil (as opposed to using a singleton), was mainly to make C-code easier to write and because many machines have special instructions to deal with zeros eficiently (i.e. branch-on-zero, clear register or even having a constant zero-register).

For that reason, the UndefinedObject class is also not subclassable and does not have ``instances'' in the strict sense.

Indexed instance variables are stored after the named instance variables (or after the header if there are none).
For example, the array "#('foo' true 30 nil)" looks in memory:

	+-------------------------+
	|       class-pointer   --------------> Array-class
	+-------------------------+
	|       size  (24)        |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	|       instvar (1)     --------------> 'foo' object
	+-------------------------+
	|       instvar (2)     --------------> true object
	+-------------------------+
	|TAG|   instvar (3)  30   |
	+-------------------------+
	|       instvar (4)  NULL |
	+-------------------------+

	+-------------------------+
	|       class-pointer     |
	+-------------------------+
	|       size              |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	|       named instvar     |
	+-------------------------+
	|       instvar (1)       |
	+-------------------------+
	|       instvar (2)       |
	+-------------------------+
		   ...
	+-------------------------+
	|       instvar (N)       |
	+-------------------------+

	+-------------------------+
	|       class-pointer   --------------> String-class
	+-------------------------+
	|       size  (18)        |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	| 'h' | 'e' |  'l' |  'l' |
	+-------------------------+
	| 'o' | '\0'| '\0' | '\0' |
	+-------------------------+

Currently, the 0-byte is only added to single-byte strings. Wide strings (for example: Unicode16 and Unicode32) do not have a 0-byte at the end. This may change in the future, as it complicates the use of wide-char APIs under Windows.

Any size field reflects the number of bytes an object has - however, the memory system will always align memory in full word increments and on a full-word boundary when allocating objects.
(any padding bytes are invisible and inaccessible to Smalltalk code; for C code: the contents is undefined)

As a final example, the doubleArray "#(1.0 2.0 3.0) asDoubleArray" looks in memory:

	+-------------------------+
	|       class-pointer   --------------> FloatArray-class
	+-------------------------+
	|       size  (36)        |
	+-------------------------+
	|       additional info   |
	+-------------------------|
	|     optional padding    |
	+-------------------------|
	|                         |
	|     1.0 as a C double   |
	|                         |
	+-------------------------+
	|                         |
	|     2.0 as a C double   |
	|                         |
	+-------------------------+
	|                         |
	|     3.0 as a C double   |
	|                         |
	+-------------------------+

true, false or nil

to access the corresponding Smalltalk object.
'nil' is actually a macro for a constant (usually 0); while 'true' and 'false' are object pointers.
Nil may not be dereferenced (i.e. do not apply the _qXXX macros which are described below).

@global(xyz)

if xyz is a global variable (other than above).
(this is a bad example, globals should start with an upper case letter, as in @global(Xyz) or @global(ByteArray))

@global(class:xyz)

if xyz is a class variable in class.
(this is a bad example, classvars and class names should start with an upper case letter, as in ``@global(MyClass:Xyz)'' or ``@global(Object:AbortSignal)'')
Notice, that the classes name is encoded in the name - this allows for any classvar to be accessed from primitive code - even if that variable is defined in multiple (super-) classes.

__INST(xyz)

if 'xyz' is an instance variable.

__CINST(xyz)

if 'xyz' is a class instance variable.
(do not confuse ``class variables'' with ``class instance variables''.)

xyz

if 'xyz' is a method variable, method argument, block variable or block argument AND 'xyz' is NOT a C keyword.

__xyz

if 'xyz' is a method variable, method argument, block variable or block argument AND 'xyz' is a C keyword.
(i.e. all variables named after C-keywords, such as "int" or "char")

    ...
    #include <stc.h>            // get the st/x definitions

    #undef true                 // get rid of st/x definitions
    #undef false                // which conflict with C definitions
    ...

    #include <your-file.h>      // get C definitions (possibly redefining true, false and others)

    #define true __STX_true
    #define false __STX_false   // get back st/x versions
    ...

Symbols

The downside of MKSYMBOL is that it is called at run time and involves some execution time overhead (not too much, as the table is hashed, and therefore fast; but still there is a function call and hash key computation involved.

Constant symbols, can be created at compilation/system startup time, and require no execution time (actually a single memory fetch, which executes in the nanosecond order).

To specify a compile-time symbol, stc supports selector translation. A construct of the form:

	@symbol(...)

Typically, the overhead of MKSYMBOL is in the 20-30ns order, compared to 3-5ns for @symbol.

As an example, to return one of the symbols ["#+" "#foo" "#bar" "#bar:with:with:"] from a C primitive, write:

    myPrimitive
    %{
	...
	switch (something) {
	    case a:
		RETURN ( @symbol(+) );
	    ...
	    case x:
		RETURN ( @symbol(foo) );
	    case y:
		RETURN ( @symbol(bar) );
	    case z:
		RETURN ( @symbol(bar:with:with:) );
	    default:
		RETURN (nil);
	}
    %}

PLEASE: DO use this feature, since it allows future changes made to the translation mechanism without making existing primitive codes source incompatible.

Late note:
Starting with rel 1.4 of ST/X, it is no longer optional but required that you use the above syntax, since the compiler has to generate special declarations in systems which use dynamic or shared libraries, for symbols to be unified. Since the compiler does not parse and analyze primitive code (except for @symbol and @global), it had otherwise no chance of knowing which symbols are required.
Although undetected by stc, it is considered an error if you use symbols directly - so better forget all of the above.

Types

Another highly useful definition is "INT", which is defined as an integer type with a size large enough to hold a casted "OBJ". It is also the type used for smallInteger values.
On most systems, this is a 32bit integer; however, on alpha, x86_64 and other 64bit cpus, it is defined as a 64bit integer.

Be very careful to declare integer variables as "INT" in your primitive code. Not "int" or "long" in your - it may otherwise fail to run correctly if your code is to be ported to another machine. Notice, that "long" is NOT always defined to be of pointer size; thanks to wise microsoft decisions, it is especially not true for 64bit windows!

Since C originally had no boolean type, all check macros (such as "__isSmallInteger()") return 1 (one) as true and 0 (zero) as false. Those are marked with a type of "bool" in the section below (although they really return "int")

Macros & Helper Functions

INT

an integer type of appropriate type to hold smallInteger values and tagged pointers.
On 32bit machines, this is defined as "int" or "long" (i.e. a 32bit integer type).
On the alpha (or other 64bit) CPU, this is defined as "long long" (i.e. a 64bit integer type). Never ever use "int", "long" or "int32" or similar machine types inside your primitive code, and do not expect that an "int" or "long" is able to be of the same size as a pointer (instead, always use "INT").

UINT

like INT but unsigned

_MIN_INT

the minimum value a smallInteger can have.
This returns the same value as "SmallInteger minVal".
In general, that is the smallest signed integer which can be represented with one bit less than the wordsize of the CPU.
On 32bit machines, this is defined as "0x40000000".
On a 64bit CPU, this is defined as "0x4000000000000000L"

_MAX_INT

the maximum value a smallInteger can have.
This returns the same value as "SmallInteger maxVal".
In general, that is the largest signed integer which can be represented with one bit less than the wordsize of the CPU.
On 32bit machines, this is defined as "0x3FFFFFFF".
On a 64bit CPU, this is defined as "0x3FFFFFFFFFFFFFFFL"

OHDR_SIZE

the size (in bytes) of an object header.
Useful when objects are allocated via the __NEW* functions below.

OBJ __Class(OBJ x)

returns the class of 'x'. Handles immediate objects (i.e. is "safe")
This is typically used in a primitives entry sequence, to check for the arguments being of the expected types.
(i.e. "if (__Class(arg) == @global(SomeClassHere)) ...")

OBJ __qClass(OBJ object_x)

same, but 'object_x' must be non-Nil, non-SmallInteger (i.e. reference to a "real", non-immediate object).
Use only if you are certain about this (it is slightly faster than _Class macro, but leads to a segmentation violation, if applied to non pointer objects, because it unconditionally indirects through the object header's class slot).
If you do not understand the above, use the __Class macro.

int __Size(OBJ x)

the size of the object (including header) in bytes (0 for nil or integers).
Do not confuse this with the value returned by Smalltalk's #size message. This macro returns the memory size in bytes - not the logical size.

Be careful with strings: the 0-byte counts here.
(use __stringSize() to get the net size without header and 0-byte).

Be also careful with arrays: you have to subtract the size of the header and divide by "sizeof(OBJ)" to get the number of elements.
(better use the __arraySize() macro, which does this for you)

int __qSize(OBJ object_x)

like above, but 'object_x' may not be nil or a SmallInteger. Use only if you are certain. (this is slighly faster than the _Size() macro, but leads to a segmentation violation, if applied to non pointer objects)

(x == nil), (x == true), (x == false)

to check for nil, true and false

bool __isObject(OBJ x)

1 if 'x' is not a SmallInteger; 0 (zero) otherwise

bool __isNonNilObject(OBJ x) (macro)

1 if 'x' is neither a SmallInteger nor nil; 0 (zero) otherwise
(i.e. a 'real' object for which the _q-macros are allowed. You should only dereference a given OBJ (-pointer) in C, if it passed the __isNonNilObject test before - otherwise, segmentation violations may occur).

bool __isSmallInteger(OBJ x)

1 if 'x' is a SmallInteger

bool __bothSmallInteger(OBJ x, OBJ y)

1 if both 'x' and 'y' are SmallIntegers
(this saves some typing for the lazy guy, and a branch for the CPU)

bool __isLargeInteger(OBJ x)

1 if 'x' is a LargeInteger

bool __isInteger(OBJ x)

1 if 'x' is either a LargeInteger or a SmallInteger

bool __isArray(OBJ x)

1 if 'x' is an Array
(i.e. elements can be accessed with __ArrayInstPtr(x)->a_element[index0])

bool __isByteArray(OBJ x)

1 if 'x' is a ByteArray (but not an instance of a subclass of it)
(i.e. elements can be accessed with __ByteArrayInstPtr(x)->a_element[index0])
Attention: Please consider using __isByteArrayLike or __isBytes.

bool __isFloat(OBJ x)

1 if 'x' is a Float (double precision float), but not an instance of a subclass.
(i.e. C-double value can be extracted with the __floatVal macro)
Attention: Please consider using __isFloatLike.

bool __isShortFloat(OBJ x)

1 if 'x' is a ShortFloat (single precision float)
(i.e. C-float value can be extracted with the __shortFloatVal macro)

bool __isFraction(OBJ x)

1 if 'x' is a Fraction (but not an instance of a subclass of it)

bool __isString(OBJ x)

1 if 'x' is a String (but not an instance of a subclass of it)
(i.e. a C-unsigned char pointer can be extracted with __stringVal(x))

bool __isUnicode16String(OBJ x)

1 if 'x' is a Unicode16String
(i.e. a C-unsigned int16 pointer can be extracted with __unicode16StringVal(x))

bool __isUnicode32String(OBJ x)

1 if 'x' is a Unicode32String
(i.e. a C-unsigned int32 pointer can be extracted with __unicode32StringVal(x))

bool __isSymbol(OBJ x)

1 if 'x' is a Symbol
Since symbols are derived from string, the __stringVal macro can also be applied to symbols.
Attention: Please consider using __isSymbolLike.

bool __isCharacter(OBJ x)

1 if 'x' is a Character

bool __isPoint(OBJ x)

1 if 'x' is a Point

bool __isBlock(OBJ x)

1 if 'x' is a Block
Attention: Please use __isBlockLike

bool __isExternalBytes(OBJ x)

1 if 'x' is an ExternalBytes instance
Attention: Please use __isExternalBytesLike

bool __isExternalAddress(OBJ x)

1 if 'x' is an ExternalAddress instance

bool __isExternalFunction(OBJ x)

1 if 'x' is an ExternalFunction instance

bool __isByteArrayLike(OBJ x)

1 if 'x' is either a ByteArray or an ImmutableByteArray

bool __isBytes(OBJ x)

1 if 'x' is ByteArray-like (i.e. ByteArray or subclass)

bool __isWords(OBJ x)

1 if 'x' is WordArray-like (i.e. WordArray or TwoByteArray or subclass)

bool __isBlockLike(OBJ x)

1 if 'x' is an instance of Block or of a subclass

bool __isSymbolLike(OBJ x)

1 if 'x' is an instance of Symbol or of a subclass

bool __isMethodLike(OBJ x)

1 if 'x' is an instance of Method or of a subclass

bool __isFloatLike(OBJ x)

1 if 'x' is an instance of Float or of a subclass

bool __isBehaviorLike(OBJ x)

1 if 'x' is an instance of Behavior or of a subclass

bool __isExternalBytesLike(OBJ x)

1 if 'x' is an instance of ExternalBytes or of a subclass thereof

INT __intVal(OBJ x)

the 31 bit (x86_64, alpha: 63 bit) signed integer value; 'x' must be a SmallInteger. No check is done whether the passed argument is really a smallInteger; you will get a wrong numeric value (actually something based on the object's address) if applied to non smallIntegers.
This is the reverse of __MKSMALLINT().
Sorry: the name was badly choosen; it should be called ``__smallIntVal()''.
(but its too late to change this now, as many users have already written C-primitive code using those macros...)

unsigned INT __longIntVal(OBJ x)

returns the 32 bit (x86_64, alpha: 64 bit) unsigned integer value; 'x' must be a SmallInteger or a 4-byte (8-byte) LargeInteger.
This does check for the argument being of correct type, and returns 0 if x is neither a small or largeInteger, or if the largeIntegers value does not fit into 32/64 bit.
Since 0 (zero) is a valid value, you have to check x agains beeing equal to __MKSMALLINT(0) before and care for that case.
This is the reverse of __MKUINT().
Sorry: the name was badly choosen; it should be called ``__uintVal()'', but the old name is kept for backward compatibility.

INT __signedLongIntVal(OBJ x)

returns the 32 bit (x86_64, alpha: 64 bit) signed integer value; 'x' must be a SmallInteger or a 4-byte (8-byte) LargeInteger.
This does check for the argument being of correct type, and returns 0 if x is neither a small or largeInteger, or if the largeIntegers signed value does not fit into 32/64 bits.
Since 0 (zero) is a valid value, you have to check x agains beeing equal to __MKSMALLINT(0) before and care for that case.
This is the reverse of __MKINT().
Sorry: the name was badly choosen; it should be called ``__intVal()''.

int __unsignedLong64IntVal(OBJ x, uint64 *pInt) (function)

returns the 64 bit unsigned integer value; 'x' must be a Small- or LargeInteger with an appropriate value.
If the class and value of the object are correct, the value is stored as a 64bit unsigned integer through the given pointer and the function returns 1.
Otherwise, a 0 is returned.
This is the reverse of __MKUINT64().

int __signedLong64IntVal(OBJ x, int64 *pInt) (function)

returns the 64 bit unsigned integer value; 'x' must be a Small- or LargeInteger with an appropriate value.
If the class and value of the object are correct, the value is stored as a 64bit unsigned integer through the given pointer and the function returns 1.
Otherwise, a 0 is returned.
This is the reverse of __MKINT64() / __MKLARGEINT64().

double __floatVal(OBJ x)

the double value; 'x' must be a Float. Does not check - you must check for 'x' being an instance of Float before invoking this (using "__isFloat()" or "__isFloatLike()").
This is the reverse of __MKFLOAT().

float __shortFloatVal(OBJ x)

the float value; 'x' must be a ShortFloat. Does not check the argument for being an instance of ShortFloat (use "__isShortFloat()").
This is the reverse of __MKSFLOAT().

char * __stringVal(OBJ x)

the character-pointer, pointing to the 0-terminated characters of the String object. 'x' must be an instance of String or a subclass of it. Does not check (use "__isString()" or "__isStringLike()").
This is the reverse of __MKSTRING().

unsigned short * __unicode16StringVal(OBJ x) (macro)

the 2-byte-character-pointer; 'x' must be a Unicode16String. Does not check. Notice that (currently) multibyte string objects are NOT 0-terminated. If required, you have to copy the bytes into a local buffer and 0-terminate them manually, before passing them to C. This may change in a future version.
This is the reverse of __MKUSTRING().

char * __symbolVal(OBJ x)

the character-pointer, pointing to the 0-terminated characters of the Symbol object. 'x' must be a Symbol. Does not check.
This is the reverse of __MKSYMBOL() or @symbol() (for constant symbols).

OBJ __characterVal(OBJ x) (macro)

the asciicode/codePoint as ST-SmallInteger; 'x' must be a Character. Does not check for x to be a valid character object.
This is the reverse of __MKCHARACTER() or __MKUCHARACTER().
Be careful: the result is still an OBJ - not a C-int. You still have to extract the asciiValue/codePoint using the __intVal() macro.
To check an argument for being a character and getting its C equivalent, use:

char ch;

if (__isCharacter(arg)) {
    ch = __intVal( __characterVal(arg) );
    if (ch <= 0xFF) {
	... iso8859-1 ...
    } else {
	... unicode ...
    }
}

Notice that character encoding is always unicode; however, for single byte characters, iso8859 encoding is used which is a subset of unicode.

void * __externalBytesAddress(x)

the address of an externalBytes underlying memory; 'x' must be an instance of ExternalBytes or a subclass thereof. Does not check.
This is the reverse of __MKEXTERNALBYTES().

OBJ __externalBytesSize(x)

returns the number of bytes in an externalBytes memory block, or nil if that size is not known. Notice, that this nil-check is still required, and the intVal must be taken in order to get the size as C-int. Does not check.

void * __externalAddressVal(x)

the C-pointer of an externalAddress; 'x' must be an ExternalAddress instance. Does not check.
This is the reverse of __MKEXTERNALADDRESS().

void * __externalFunctionVal(x)

the C-pointer of an externalFunction; 'x' must be an ExternalFunction instance. Does not check.
This is the reverse of __MKEXTERNALFUNCTION().

OBJ __point_X(OBJ p)

the x-instance as ST-Object; 'x' must be a Point. Does not check.

OBJ __point_Y(OBJ p)

the y-instance as ST-Object; 'x' must be a Point. Does not check.

int __stringSize(OBJ s)

the size of the string-object; 's' must be a String. Does not check.
In contrast to __Size(), this returns the strings net size (i.e. without header and 0-byte).

int __arraySize(OBJ a)

the number of elements in the array-object; 'a' must be an Array. Does not check.

int __byteArraySize(OBJ a)

the number of elements in the byteArray-object; 'a' must be a ByteArray. Does not check.

OBJ __AT_(OBJ receiver, int index) (function) [1]

sends #at: to the receiver. For certain classes (String, Array) no real send is performed; instead the value is extracted directly.

OBJ __AT_PUT_(OBJ receiver, int index, OBJ value) (function) [1]

sends #at:put: to the receiver. For certain classes (String, Array) no real send is performed; instead the value is stored directly.

OBJ __SIZE(OBJ receiver) (function) [1]

sends #size to the receiver. For certain classes (String, Array) no real send is performed; instead the size is determined directly.

Object Creation Helpers

bool __ISVALIDINTEGER(INT ival)

checks if 'ival' is in the valid range for a smallInteger. (i.e. in [_MIN_INT .. _MAX_INT])

OBJ __MKSMALLINT(INT ival)

makes a SmallInteger object with value 'ival'. The argument must be within the valid SmallInteger range which is typically 31 bit (alpha: 63 bit).
For security and portability, compare 'ival' against _MIN_INT and _MAX_INT (using __ISVALIDINTEGER) before using this macro.
Attention: will be obsoleted by __mkSmallInteger()

OBJ __MKFLOAT(double dval) (function) [1]

makes a Float object with value 'dval'.
Notice that the Smalltalk class "Float" corresponds to the C-double type, whereas the Smalltalk "ShortFloat" holds a "float" value. (this was done for compatibility with VisualAge- and Dolphin Smalltalk systems)

OBJ __MKSFLOAT(float fval) (function) [1]

makes a ShortFloat object with value 'fval'
Notice that the Smalltalk class "Float" corresponds to the C-double type, whereas the Smalltalk "ShortFloat" holds a "float" value. (this was done for compatibility with VisualAge- and Dolphin Smalltalk systems)

OBJ __MKFRACT_I(int num, int denom) (function) [1]

makes a Fraction object with integral numerator and integral denominator
The arguments must be in the valid smallInteger range (which is not checked in this function).

OBJ __MKEMPTYSTRING(int len) (function) [1]

makes an empty String object for 'len' 1-byte-characters with undefined contents. Actually, the allocation includes space for an extra zero byte. So do not count that in the passed len argument.
The returned object is an instance of String, which can only hold 8-bit characters.

OBJ __MKEMPTYUSTRING(int len) (function) [1]

makes an empty Unicode16String object for 'len' 2-byte-characters with undefined contents.
The returned object is an instance of Unicode16String, which can hold 16-bit characters. The allocation does currently NOT include space for a zero character at the end.
This function was introduced with ST/X release 5.2.1.

OBJ __MKSTRING(char *sval) (function) [1]

makes a String object with value 'sval'.

Warning: 'sval' is not allowed to point into a Smalltalk object, because the string allocation itself could lead to a garbage collect to happen, which could make any passed-in object-pointer useless. To create a new string from a passed-in Smalltalk string, use the following procedure:

1. protect the st-object from which you want to copy (__PROTECT__)
2. allocate an empty string (__MKEMPTYSTRING)
3. unprotect the st-object (__UNPROTECT__)
4. copy the bytes (bcopy)

of course, the PROTECT/UNPROTECT is only needed if the reference is not via a local/argument of the current context.

OBJ __MKSTRING_L(char *sval, int len) (function) [1]

as above, if you know the length (is somewhat faster, since a call to strlen() is avoided)
Warning: 'sval' is not allowed to point into a Smalltalk object (see above).

OBJ __MKSTRING_ST(OBJ stStringObject) (function) [1]

use this, top copy an existing Smalltalk string.

OBJ __MKSTRING_ST_L(OBJ stStringObject, int len) (function) [1]

like above, if you know the length or only want to copy the first len characters.

OBJ __MKSYMBOL(char *sval, 0) (function) [1]

makes a Symbol object with name 'sval'
For constant symbols, always use the @symbol() macro, which creates the symbol at compilation time and does not take any time at execution time. In contrast, MKSYMBOL creates the symbol at execution time and requires a search in the hashtable of existing symbols.

OBJ __MKCHARACTER(unsigned char c) (macro)

makes a Character object with asciiValue 'c'
'c' must be in the range [0 .. 255].
No actual object creation is done, as all 256 single-byte character objects are created and remembered at startup time.
The character is assumed to have unicode coding (of which iso8859-1 and ascii are single-byte subsets).

OBJ __MKUCHARACTER(unsigned short uc) (function) [1]

makes a Character object with unicode-codePoint 'uc'
'uc' must be in the range [0 .. 0xFFFF].
This may or may not actually create a new object, depending on the passed codePoint: the first 1024 characters are shared and reused (immediate objects), and for those, this function simply returns a pointer to one of those preallocated character objects. For higher codepoints, an object allocation is performed and thus, garbage collection may happen.
The character is assumed to have unicode coding.
This function was introduced with ST/X release 5.2.1.

OBJ __MKLARGEINT(INT lval) (function) [1]

makes a LargeInteger object with value 'lval'
Be careful to only create largeIntegers if 32 bits (alpha: 64 bits) of value are really needed; use __MKSMALLINT() (if you are certain) or __MKINT() (if you don't know). Otherwise, this will return an unnormalized LargeInteger (i.e. a LargeInteger with a value in the SmallInteger range).
Most code in the numeric classes is not prepared to handle unnormalized largeIntegers; therefore strange things may happen.

OBJ __MKULARGEINT(unsigned INT uval) (function) [1]

makes a LargeInteger object with value 'uval'
Same care as above.

OBJ __MKINT(INT val) (function) [1]

makes a Small or LargeInteger object with value 'val'
This checks the value of the passed number and returns a correctly typed object. If in doubt, always use this or __MKUINT below to pass integers from C to Smalltalk.

OBJ __MKUINT(unsigned INT uval) (function) [1]

makes a Small or LargeInteger object with value 'uval'

OBJ __MKINT64(int64 *iP) (function) [1]

makes a LargeInteger object with up to 64 bits of value.
The argument is a pointer to an int64 structure (see stc.h).
This checks the passed arguments value and returns a correctly typed object (i.e. SmallInteger, 4-byte LargeInteger or 8-byte LargeInteger as required).

OBJ __MKUINT64(unsigned int64 *iP) (function) [1]

makes a LargeInteger object with up to 64 bits of value
The argument is a pointer to an unsigned int64 structure (see stc.h).
This checks the passed arguments value and returns a correctly typed object (i.e. SmallInteger, 4-byte LargeInteger or 8-byte LargeInteger as required).

OBJ __MKLARGEINT64(int sign, int low32bits, int hi32Bits) (function) [1]

same as __MKINT64, but expects sign, low and hi bits as separate arguments. (this is sometimes useful, when a datum is either computed or read from an external source)
This checks the passed arguments values and returns a correctly typed object (i.e. SmallInteger, 4-byte LargeInteger or 8-byte LargeInteger as required).

OBJ __MKEXTERNALBYTES(void *ptr) (function) [1]

makes an ExternalBytes object from an arbitrary pointer 'ptr'
This is useful to pass a block of memory which was allocated by C-code up to Smalltalk. It can be accessed there via protocol found in ExternalBytes.
This does not allocate the bytes-block; however, a Smalltalk object which points to the passed memory is allocated.
Notice:
The created ExternalBytes object does not know the size of the memory block, and therefore does no bound checking when individual bytes are accessed. This makes your code less robust, of course.
Therefore, it is recommended to ALWAYS use the function below if the size of the memory block is known.

OBJ __MKEXTERNALBYTES_N(void *ptr, int nBytes) (function) [1]

like above, passing the number of (byte-) elements.
If possible, use this to create externalBytes objects, since knowing the size of the data block enables for bounds checking operations when elements are accessed from Smalltalk code - this makes your code more robust.

OBJ __MKEXTERNALADDRESS(void *ptr) (function) [1]

makes an ExternalAddress object from an arbitrary pointer 'ptr'

OBJ __MKEXTERNALFUNCTION(voidFunc *ptr) (function) [1]

makes an ExternalFunction object from an arbitrary function pointer 'ptr'

OBJ __MKPOINT_INT(INT x, INT y) (function) [1]

makes a Point object given two integer coordinates; x and y must be within the valid smallInteger range.

OBJ __MKPOINT_DOUBLE(double x, double y) (function) [1]

makes a Point object given two double coordinates

OBJ __ARRAY_NEW_INT(int n) (function) [1]

makes an Array with 'n' slots (nilled) elements.

OBJ __ARRAY_NEW_INT_WITHALL(int n, OBJ val) (function) [1]

makes an Array with 'n' slots, all referring to 'val'.
The 'val' argument must be a valid OBJ.

OBJ __ARRAY_WITH1(OBJ element) (function) [1]

makes an Array with 1 element. The argument must be a valid OBJ.

OBJ __ARRAY_WITH2(OBJ e1, e2) (function) [1]

makes an Array with 2 elements. The arguments must be valid OBJs.

OBJ __ARRAY_WITH[3-5](OBJ e1, e2, ...) (function) [1]

same with up to 5 elements.

OBJ __STRING_NEW_INT(int n) (function) [1]

makes an String with 'n' elements; the string is initialized with spaces.

OBJ __BYTEARRAY_UNINITIALIZED_NEW_INT(int n) (function) [1]

makes a ByteArray with 'n' elements; the contents is undefined.

OBJ __BYTEARRAY_NEW_INT(int n) (function) [1]

makes a ByteArray with 'n' elements; the elements are set to zero.

OBJ __BASICNEW(OBJ cls) (function) [1]

low level instance creation. This does NOT actually send #basicNew to the class - only the memory is allocated and the instance variables nilled. (i.e. a classes private basicNew method would not be invoked by this).

OBJ __BASICNEW_INT(OBJ cls, int n) (function) [1]

like above, with indexed instance variables.

OBJ __NEW(OBJ cls) (function) [1]

instance creation. Sends #new to cls.

OBJ __NEW_INT(OBJ cls, int n) (function) [1]

instance creation with indexed instance variables. Sends #new: to cls.

OBJ __stArrayFromCIntArray(int *pInts, int nItems) (function) [1]

makes an Array filled with integer values from a C-array of ints. Individual elements of the returned array may be smallIntegers or largeIntegers, depending on their value.

OBJ __stArrayFromCUIntArray(unsigned int *pInts, int nItems) (function) [1]

makes an Array filled with integer values from a C-array of unsigned ints. Individual elements of the returned array may be smallIntegers or largeIntegers, depending on their value.

OBJ __stArrayFromCShortArray(short *pInts, int nItems) (function) [1]

makes an Array filled with integer values from a C-array of short ints. All elements of the returned array will be smallIntegers.

OBJ __stArrayFromCUShortArray(unsigned short *pInts, int nItems) (function) [1]

makes an Array filled with integer values from a C-array of unsigned short ints. All elements of the returned array will be smallIntegers.

OBJ __stArrayFromCFloatArray(float *pInts, int nItems) (function) [1]

makes an Array filled with floats from a C-array of floats.

OBJ __stArrayFromCDoubleArray(double *pInts, int nItems) (function) [1]

makes an Array filled with floats from a C-array of doubles.

OBJ __stFloatArrayFromCFloatArray(pfloat *pInts, int nItems) (function) [1]

makes a FloatArray filled with values from a C-array of floats.

OBJ __stDoubleArrayFromCDoubleArray(double *pInts, int nItems) (function) [1]

makes an DoubleArray filled with values from a C-array of doubles.

[1] these macros/functions (may) allocate object memory and therefore may trigger a garbage collect - read more on this below.

garbage collector interface

__PROTECT__(var)

tell the VM, that var contains a reference to a valid Smalltalk object which should not be collected. This is required if C-code holds a reference to a Smalltalk object and there is a chance for a garbage collect operation to occur.
This macro must be used in a stack-like fashion together with the UNPROTECT macro. For every PROTECT, there MUST be a corresponding UNPROTECT; otherwise, some VM internal table will overflow. Notice, that this internal table is relatively small (some 30 entries or so), and no mechanism whatsowever exists in the VM to clean up this table. Thus, PROTECT/UNPROTECT are strictly not allowed iff there is any chance of a message send (i.e. a call into smalltalk code), an interrupt or a context switch is to happen inside. This is because then, the table could overflow because a user may end up in a debugger and/or exception handler and the UNPROTECT may never be executed.

Only use PROTECT/UNPROTECT around calls to a memory allocation VM routine; never around message sends or when interrupts are enabled
See more below in ``Allocating object memory in primitives''.

__UNPROTECT__(var)

get any updated pointer back from the VM into var, and release the VM's remembering as installed with PROTECT. This macro MUST be used in a stack-like fashion together with the PROTECT macro. For every PROTECT, there MUST be a corresponding UNPROTECT; otherwise, some VM internal table will overflow.
Also notice, that you have to use the UNPROTECT macro after EVERY object allocation, to refetch any other possibly changed object reference.
See more below in ``Allocating object memory in primitives''.

OBJ __STORE(OBJ dst, OBJ val)

to tell the garbage collector, that a store of val was made into dst; the STORE macro is not needed for local variables (method locals) or if you are certain that the stored object is either nil or a SmallInteger. The macro checks for those, so it does not hurt (much) to place this macro after every non-context store.
Please read further information on why a store macro is required and how it is used.

OBJ __GSTORE(OBJ val)

to tell the garbage collector, that a store of val was made into a global (or classVariable);
This is equivalent to __STORE(Smalltalk, val).

You have to take care of this, either by using the PROTECT/UNPROTECT macros, or by placing the variables into the current context (i.e. effectively keeping those references in a Smalltalk method local instead of a C variable which is unknown to the grabage collector).
All pointers into an object are to be considered invalid after a garbage collect, even if the underlying base pointer was PROTECTED/UNPROTECTED.
Therefore, never create any C pointers into an object; instead, always use indexed accesses, and protect the base object.

Currently (and maybe forever), pointers in ST/X are DIRECT object pointers, meaning that the values of these variables point directly to the underlying object's structure (some of which can be found in the stc.h include file).

Since I cannot guarantee that this statement remains true in the future (indirection makes things like the garbage collector or the become:-operation much much simpler), you should always use access macros such as "__InstPtr(o)->field" instead of "o->field".
These access macros are also defined in stc.h:

OBJ to C-structure casts

__objPtr(o)

same as o, to hide indirection if ever implemented, always use "__objPtr(o)->field" instead of "o->field" !

__InstPtr(o)

to access instvars by index [0 .. instSize-1]

Direct access to instance variables it is NOT recommended, since you have to modify your primitives whenever ther instance variable layout of the accessed object changes.

For primitive code in methods, use the "__INST(name)" macro to access named instance variables.

__PointInstPtr(o)

to access x and y of a point (actually, simply a cast to the points structure)

For example, to access the x component of a point instance, use:

    ...
    if (__isPoint(stVar)) {
	OBJ xComponent;

	xComponent = __PointInstPtr(stVar)->p_x;  /* see "stc.h" for p_x */
	if (__isSmallInteger(xComponent)) {
	    INT xValue;

	    xValue = __intVal(xComponent);
	    ...
	}
    }
    ...

of course, never forget to check the types with the __isXXX macros - just to make certain.

__ArrayInstPtr(o)

to access elements of an array.

For example, to access the 10th element of a Smalltalk array, use:

    ...
    if (__isArray(stVar)) {
	OBJ el_10;

	/* notice: c-indexing starts at 0 */
	el_10 = __ArrayInstPtr(stVar)->a_element[9];
	...
    }
    ...

another example (summing the elements of an array):

    ...
    if (__isArray(stVar)) {
	int nElements;
	INT sum;
	int index;
	OBJ element;

	nElements = __arraySize(stVar);
	sum = 0;
	for (index=0; index < nElements; index++) {
	    element = __ArrayInstPtr(stVar)->a_element[index];
	    if (! __isSmallInteger(element)) {
		RETURN (nil);
	    }
	    sum += __intVal(element);
	}
	RETURN ( __MKSMALLINT(sum));   /* use "__MKINT()" if there is a chance
					* that sum does not fit into 31 bits
					*/
    }
    ...

notice, stc is not too bad: the code generated by compiling the corresponding ST code is not much slower.

and so on ...
There are macros for all objects which are known to the runtime system
i.e.

• __ByteArrayInstPtr(),
• __StringInstPtr(),
• __SymbolInstPtr(),
• __CharacterInstPtr()

If there will ever be a switch to indirect pointers, only those macros have to be changed instead of all primitive code.

late news:

Originally, all those macros were named _XXX(). However, some compilers got confused by a definition of both a macro named "_XXX" and a global variable with the same name.

In the current ST/X release, this was true for the _isBlock macro, where a corresponding '_isBlock' c-variable exists (for the isBlock symbol).

The macro has been renamed to __isBlock to avoid this problem This renaming will be done for all and every other macro.
It has already been done for most macros, but some remain to use a single underscore. Be prepared for some minor (name-) changes in that area.
For the intermediate time, please excuse the confusion due to some macros starting with two underscores, others with only one.

With release 2.10.5.4, almost all macros have been renamed for a double-underscore name. The old names are available as synonym for easier migration, but will vanish in a future version.

int id = __STX_AddTimeout(func, deltaT, arg)

installs a timeout, which arranges for the C-function func to be called after deltaT milliseconds. arg is passed to the function as argument (use a pointer to a structure, if more than one argument is required).
The function will be called by the scheduler (i.e. at highest priority), but will NOT interrupt the garbage collector (i.e. it can be used for soft realtime only).

The return value is an integer, which identifies the timeout.

The passed timeout-function is called only once - for cyclic calling, another call to __STX_AddTimeout should be made in the called function.

__STX_RemoveTimeout(int id)

removes a timeout as previously installed with __STX_AddTimeout. The argument must be the timeouts id, as previously returned.

__STX_SignalSemaphore(aSemaphore)

signals a semaphore. Before using this function, carefully read the section on how references to Smalltalk objects are kept & remembered in C-code.

Returning a value from a primitive

examples:

    myMethodReturningOne
    %{
	/* return a SmallInteger */
	RETURN ( __MKSMALLINT(1) );
    %}
    !

    anotherMethodReturningOne
	|retVal|

    %{
	/* return a SmallInteger */
	retVal = __MKSMALLINT(1);
    %}.
	^ retVal
    !

    mySpecialTrigMethod:arg
    %{
	if (__isFloat(arg)) {
	    /* compute a Float */

	    double fVal = __floatVal(arg);
	    RETURN ( __MKFLOAT(sin(exp(fVal) * 1.2345)) );
	}
    %}
    .
	self primitiveFailed
    !

    myMethodReturningSymbol
    %{
	/* return a Symbol */
	RETURN ( @symbol(fooBar) );
	}
    %}
    !

    myMethodReturningString
    %{
	/* return a String */
	RETURN ( __MKSTRING("hello world") );
	}
    %}
    !

    anotherMethodReturningString
	|retVal|

    %{
	char buffer[100];

	strcpy(buffer, "hello ");
	strcat(buffer, getLogin());
	/* return a String */
	retVal = __MKSTRING(buffer);
    %}.
	^ retVal
    !

    aMethodReturningArrayWithStrings
    %{
	OBJ arr, s;

	/*
	 * you need a lot of PROTECT/UNPROTECT,
	 * if references are not reachable via the context...
	 */
	arr = __ARRAY_NEW_INT(2);
	__PROTECT__(arr);
	s = __MKSTRING("hello");    // may GC and make reference in arr obsolete
	__UNPROTECT__(arr);         // fetch possibly updated ref
	__ArrayInstPtr(arr)->a_element[0] = s;
	__STORE(arr, s);            // tells memory manager that a reference exists

	__PROTECT__(arr);
	s = __MKSTRING("hello");    // may GC and make reference in arr obsolete
	__UNPROTECT__(arr);         // fetch possibly updated ref
	__ArrayInstPtr(arr)->a_element[1] = s;
	__STORE(arr, s);            // tells memory manager that a reference exists
	RETURN(arr);
    %}
    !

    anotherMethodReturningArrayWithStrings
	|arr s|
    %{
	/*
	 * you dont need PROTECT/UNPROTECT,
	 * if references ARE reachable via the context...
	 */
	arr = __ARRAY_NEW_INT(2);
	s = __MKSTRING("hello");
	__ArrayInstPtr(arr)->a_element[0] = s;
	__STORE(arr, s);

	s = __MKSTRING("hello");
	__ArrayInstPtr(arr)->a_element[1] = s;
	__STORE(arr, s);
    %}.
	^ arr
    !

    myDestructiveUpperCaseToLowerCase:aString
    %{
	char *cp;
	char c;

	/* check if argument is a string */
	if (__isString(aString)) {
	    /*
	     * get the C-character pointer to the characters
	     * this is allowed here, since no danger for a garbage
	     * collect exists here.
	     */
	    cp = __stringVal(aString);

	    /* walk over string till end (0-byte) is reached */
	    while ((c = *cp)) {
		if (isUpper(c)) {
		    *cp = toLower(c);
		}
		cp++;
	    }
	    RETURN ( aString );
	}
    %}
    .
	self primitiveFailed
    !

The easiest way of handling this situation is by declaring these locals as method locals (in contrast to c-variables). Method locals are located in the context, which is investigated (and updated) by the garbage collector:

instead of:

      aMethodName
      %{
	  OBJ aLocal;

	   ...
	  do something with aLocal
	   ...
      %}

      aMethodName
	  |aLocal|
      %{
	    ...
	  do something with aLocal
	    ...
      %}

Of course, this protection is only needed if your primitive code calls other methods and/or allocates storage - otherwise there is no danger since the garbage collector will only run when new objects are allocated (which is always possible when calling other methods).
Warning: the previous statement is not true, iff you declare your primitive code as being interruptable (which is described below). If you do this, an interrupt may occur at any time, leading to a thread-switch to some other Smalltalk process, which may of course trigger a garbage collect.

For coding examples, see the primitives in "libbasic/SmallInt.st" or "libbasic/Float.st". Also, even though a bit more complex, looking into "libview/XWorkstation.st" also gives a lot of insight.

Stack management

For this, the STC compiler generates code which checks for the amount of required stack on method- or block entry, and grows the stack if required.

Of course, called C-functions do not do so, therefore great care must be taken when calling c library functions which use big stack frames. Especially C-functions, which use alloca, or declare big automatic arrays or generate a very deep call-nesting.

ST/X internally keeps 2 classes of stacks:

• the so called C-Stack
  this is an automatic growing stack, which is virtually unlimited in size. This is the machine stack, which was allocated by the operating system for the "main()" function. Only (exactly) one such C-stack is available. The C-stack is non-scattered. The operating system typically maintains this stack and grows it as required (by detecting memory references below the current stack pointer and allocating more stack pages as requried). This stack may bounce into the heap memory area (which grows upward).
  Its actuall size limit is typically some 8Mb, 16Mb or even more.

• per thread (Smalltalk process) stacks
  these are maintained by the Smalltalk runtime system and are semi automatically grown (and shrunk) as required by the thread. Each Smalltalk process (thread) has its own threadStack. ThreadStacks are allocated in fragments (typically multiples of the systems page size) and may be allocated as non-consecutive pages. (so called spaghetti-stack)
  In contrast to the C-stack, the growing is done in software (in every method's and block's entry code sequence).

Primitive code which calls out to a C-function will have this default amount of stack available for its called C-function, which may or may not be enough. In some cases, it may be required to arrange for a bigger thread-stack-frame to be reserved, or to do a temporary switch to the (unlimited) C-stack.

Stack requirements

To tell the stc compiler, that a primitive needs more stack, a stack declaration should be added - this is done by inserting a comment of the form:

    /* STACK:n */

    /* UNLIMITEDSTACK */

Detecting the first declaration, stc will make certain that n bytes of stack are available for the method containing that primitive. You may have to guess on what a good value for n is. Taking a bigger value may be less performant, but give you more security.

The second declaration will produce code to switch to the unlimited c-stack for the execution of the method containing that primitive. This stack is grown by the operating system and unlimited (not really, but the limit is typically some 8 or 16 megabytes).

There is one big catch in using UNLIMITEDSTACK: - it is not reentrant. Methods running on the unlimited stack may NOT send other Smalltalk messages. The reason is that all messages might eventually lead to a process switch into another thread and the c-stack cannot hold frames for multiple processes in a non first-in/last-out order.

To summarize things, your primitive might now look like:

    myMethod

    %{ /* STACK:3500 */

	char aBigBuffer[3000];
	...
	...
    %}

A rule of thumb is the size of local data arrays (i.e. 3000 bytes of aBigBuffer) plus some 500 bytes for the context and other housekeeping locals. (500 is actually too high for most situations; but better estimate too high than too low.)

A new version with more protection against stack violations is being prepared and soon available. This will at least protect agains violations near the top of the stack - however, there is (currently and in the near future) no insurance against violations due to big alloca chunks.

You are always on the bright side of life, if an UNLIMITEDSTACK declaration is added - however, these methods MUST be leaf methods (i.e. they may not send any further messages).

Late note:
The newest ST/X releases can be started with the VM commandLine argument "-MprotectStack". This makes the memory page below a threads stack non-accessible (by using mprotect), and helps to find stack problems.
However, not all operating systems provide this functionality, and also it does not help against alloca operations, which access memory below that memory page.

Ommiting the context setup

This is done by adding a comment of the form "/* NOCONTEXT */" to the first line of the primitive code.

BIG WARNING:
Since those methods do not have a context, there is no way for the garbage collector to update object references. Thus neither self, nor locals, nor arguments are valid after a garbage collect. You must take great care (using PROTECT/UNPROTECT) to not loose any pointers in case of a garbage collect.
NOCONTEXT primitives should only be written by experienced users or for methods which do NOT allocate memory and do NOT send other Smalltalk messages (i.e. which are completely save from ever entering the garbage collector).

For the curious:

the macros __PROTECT__(ptr) and __UNPROTECT__(ptr) can be used to tell the garbage collector about variable values to be updated - see some primitives in libbasic for examples.
Thus, with careful use of these __PROTECT__ macros, it is possible to define NOCONTEXT methods even if there is a possibility of garbage collection.
These macros are used as:

    {
	OBJ myRef;
	...
	__PROTECT__(myRef);
	...
	do something which may lead to a GC
	this invalidates myRef
	...
	__UNPROTECT__(myRef);
	...
	myRef valid again
    }

Not all C-compilers complain when the address is taken of a register variable; some silently make the variable a non register one.
For portability of your code, please use the NOREGISTER pragma even if your compiler is a tolerant one iff your primitive takes the address of the variable.

Multiple of these kludge comment pragmas may be in one comment as in:

	  %{ /* STACK:4096  NOREGISTER */

Low performance but space saving code

    OBJ sel, receiver, arg1, ... argi;

    val = __SSEND<i>(receiver, sel, 0, arg1, arg2, ... argi);

This performs a message send which uses a single static inline cache slot for all sends - therefore it does not require any additional memory for caching.

Although, the performance of this is relatively poor (if compared to a fully cached send), this is the most space effective code sequence - use this, if the message send is seldom performed ('SSEND' stands for 'Space-saving Send').

Expect a performance of roughly 500k to 2M sends per second depending on CPU and memory speed (750ns per send on a P5/200) - for highly polymorphic message sends, the performance may degrade to lower numbers.

The stc compiler generates this type of message sends if the "+optSpace" compilation option is enabled.

Medium performance

    OBJ sel, receiver, arg1, ... argi;
    OBJ val;
    static struct inlineCache dummy = _DUMMYILC<i>;

    val = _SEND<i>(receiver, sel, nil, &dummy, arg1, arg2, ... argi)

This results in a medium-speed message send - the methodLookup is cached in the global method cache, but no inline cache is used.

Expect a performance of roughly 700k to 3M sends per second. (550ns per send on a P5/200)

High performance

If your message send is often performed, you may like to use the inline caching facility, which keeps the target of the last send and speeds up future sends - effectively making the next send an indirect function call.
For very highspeed sends, use:

	static struct inlineCache ilc = _ILC<i>;

	val = (*ilc.ilc_func)(receiver, sel COMMA_SENDER, nil, &ilc, args);

Expect a performance of roughly 2M to 15M sends per second. (105ns per send on a P5/200)

The above code also shows good performance for polymorphic message sends. However, this requires additional memory for caching.

For highly polymorphic sends (as when enumerating all objects), the performance may degrade to below the speed of the medium-speed send above, since the polymorph cache handling may create more overhead than savings in rare cases.

The stc compiler generates this code sequence, unless the +optSpace option is enabled. The JIT-compiler always generates this code sequence.

Examples:

      myMethod:argument
	  |local|
      %{
	  static struct inlineCache dummy = _DUMMYILC0;

	  local =_SEND0(argument, @symbol(redraw) COMMA_SENDER, nil, &dummy);
      %}
      !

      myMethod:argument
	  |local|

	  local := argument redraw
      !

Notice:

Since the SEND-code produced by the compiler is usually as good as handwritten code - you should avoid writing primitives which send messages, if possible.
(actually the compilers add additional hints to the cache management which further speeds up sends to self, constants and classes).

Primitives should be written for things which cannot be written in Smalltalk (for example: interface to databases, interface to graphics etc) or which are very time-consuming and can be considerable tuned in c (for example: copying arrays, String search, Image rotation etc.)

A good strategy is to do all nescessary Smalltalk stuff before entering the primitive code, and not sending anything from inside. (See examples in XWorkstation.st, Array.st and especially the image manipulation methods in DepthXImage.st etc.)

Also, keep in mind, that the interface will be changed if I get an idea of a faster send, and YOU have to update the code in this case; whereas all high level Smalltalk code will not be affected by these changes.

Remembering Smalltalk objects in external C-code

All of the above is done, iff the garbage collector knows about external memory locations which hold references to some Smalltalk object.

__ADD_REFCELL( OBJ *cellRef )

announces to the garbage collector, that cell contains a reference to some Smalltalk object, and should be investigated/updated when garbage is collected.
The cell will NOT survive a snapshot-save & restart - i.e. it has to be reinstalled when an image is restarted.

__REMOVE_REFCELL( OBJ *cellRef)

to deinstall a cell. After that, the value in the cell has to be considered to be invalid.

    static OBJ semaRef = nil;

    void
    doTrigger(dummyArg)
    {
	__STX_SignalSemaphore(semaRef);
	semaRef = nil;
	__REMOVE_REFCELL( &semaRef );
    }

    void
    triggerSemaAfter(theSema, delta)
	OBJ theSema;
	int delta;
    {

	/*
	 * remember the semaphore
	 */
	semaRef = theSema;
	__ADD_REFCELL( &semaRef );

	/*
	 * install a timeout
	 */

	__STX_AddTimeout(doTrigger, delta, 0);
    }

    startShow
	|mySema|

	mySema := Semaphore new.
    %{
	triggerSemaAfter(mySema, 1000);
    %}.
	mySema wait

In most cases, it is possible to do the allocation outside of the primitive code as in:

    someMethod
	|localBytes|

	localBytes := ByteArray new:10000.
	%{
	    ...
	    do something with localBytes in the primitive
	    ...
	%}

If you really have to allocate in a primitive, here is how its done using a call to the "__STX___new()" function as:

	newObject = __STX___new(size, SENDER);

This returns space for an object with (size - OHDR_SIZE) bytes, where OHDR_SIZE is the space (overhead) required for an object header. The object header includes the size field, the class, and some flags needed by the garbage collector.

Since an object header is always required, the call is better written as:

	newObject = __STX___new(bytesWanted + OHDR_SIZE, SENDER);

Notice, that this function may return nil in case the memory manager has problems allocating the memory. This may happen only in one situation: if the object memory is full, and the operating system is not willing to satisfy a request for more memory. (I.e. if the memory requirements hit any virtual memory size limits of the operating system).

Since this does not happen in normal situations, forgetting the non-nil test is a common mistake - which will get unnoticed for quite some time but may later lead to a segmentation violation, when the system is about to run out of space.
ST/X can deal with low memory situations gracefully, as it keeps some emergency memory aside, which is used to handle an out-of-memory exception. A handler for this (Smalltalk-) exception will try to free memory (sending lowSpaceCleanup) and proceed. If you do not check for a nil return, you will break this graceful emergency scheme. Therefore: you MUST check the value returned and handle the nil case (usually by raising an exception).

Initialized vs. Uninitialized Memory

Also, proper setup of the contents must be done shortly after the object is created: even if you think that no accesses are possible to the uninitialized object, the garbage collector may want to peek into it to find references to other objects.

example: allocate a ByteArray.

    {
	...
	/*
	 * notice, there may be a garbage collect here ...
	 * thus invalidating all local pointers, which are neither
	 * in the context, nor have been PROTECTED
	 *
	 * don't forget the OHDR in the given size ...
	 */
	newObject = __STX___new(100 + OHDR_SIZE, SENDER);
	if (newObject != nil) {
	    /*
	     * MUST set the class
	     */
	    __objPtr(newObject)->o_class = ByteArray;
	    /*
	     * except for ByteArrays and Strings, MUST nil-out instvars
	     * but care for the first OHDR_SIZE header bytes - don't clear
	     * those.
	     */
	    bzero((char *)__objPtr(newObject) + OHDR_SIZE, 100);
	}
	RETURN ( newObject );
    }

    {
	...
	newPoint = __STX___new(sizeof(OBJ)*2 + OHDR_SIZE, SENDER);
	if (newPoint != nil) {
	    /*
	     * MUST set the class
	     */
	    __objPtr(newPoint)->o_class = Point;
	    /*
	     * must set the fields
	     */
	    __InstPtr(newPoint)->i_instvars[0] = _MKSMALLINT(1);  /* p x:1 */
	    __InstPtr(newPoint)->i_instvars[1] = _MKSMALLINT(0);  /* p y:0 */
	}
	RETURN ( newPoint );
    }

The situation becomes a bit more difficult, if previously allocated objects are to be stored into a newly created object. Since every object allocation may lead to a GC, we have to make certain that these other pointers are not lost.
To help here, use the above mentioned PROTECT/UNPROTECT macros. The PROTECT macro tells the VM, that some value is still referenced from primitive code (possibly not reachable via regular object references) and that the referenced object should be protected in case of a garbage collect.
This protection stays active until the UNPROTECT macro releases the protection.
These macros have to be used in a stack-like fashion; every PROTECT must be followed by a corresponding UNPROTECT. If you forget any UNPROTECT, some VM internal table will sooner or later overflow, and bad things happen to it.
(for the curious:
PROTECT/UNPROTECT are actually stack push/pop operations for a VM protection stack. The pointers found on this stack are taken to be references to living objects by the garbage collector and updated as appropriate. If you forget to UNPROTECT, this stack may overflow. Also, you will get back the updated pointer value via UNPROTECT. Any other object reference (in a C-variable) will be osolete and may point to an invalid location.

The following example shows what has to be done if the stored values are allocated in the primitive.

    primitivePoint
    %{
	OBJ newPoint;
	OBJ newX;
	OBJ newY;

	/*
	 * create a float for some X value:
	 */
	newX = __MKFLOAT(1.2345);

	/*
	 * in case allocation failed
	 */
	if (newX == nil) {
	    RETURN (nil);
	}

	/*
	 * create a float for the y value.
	 * WARNING: newX may be invalid after that, if not protected,
	 * because the allocation of newY may lead into a garbage collect
	 */
	__PROTECT__(newX);
	newY = __MKFLOAT(3.14159);
	__UNPROTECT__(newX);
	/*
	 * in case allocation failed
	 */
	if (newY == nil) {
	    RETURN (nil);
	}

	/*
	 * create a point for the x and y values.
	 * WARNING: newX and newY may be invalid after that, if
	 * we do not protect them, because the allocation of the
	 * new point may lead into a garbage collect ...
	 */
	__PROTECT__(newX);
	__PROTECT__(newY);
	newPoint = __STX___new(sizeof(OBJ)*2 + OHDR_SIZE, SENDER);
	__UNPROTECT__(newY);  // fetch possibly changed reference
	__UNPROTECT__(newX);  // fetch possibly changed reference
	if (newPoint == nil) {
	    RETURN (nil);
	}

	/*
	 * MUST set the class
	 */
	__objPtr(newPoint)->o_class = Point;
	/*
	 * must set the fields
	 */
	__InstPtr(newPoint)->i_instvars[0] = newX;
	__InstPtr(newPoint)->i_instvars[1] = newY;
	/*
	 * see below
	 */
	__STORE(newPoint, newX);
	__STORE(newPoint, newY);

	RETURN ( newPoint );
    %}

    primitivePoint
	|newX newY newPoint ok|

    %{
	ok = false;

	/*
	 * create a float for some X value:
	 */
	newX = __MKFLOAT(1.2345);
	/*
	 * always check for allocation failure ...
	 */
	if (newX != nil) {
	    /*
	     * create a float for the y value.
	     */
	    newY = __MKFLOAT(3.14159);
	    if (newY != nil) {
		/*
		 * create a point for the x and y values.
		 */
		newPoint = __STX___new(sizeof(OBJ)*2 + OHDR_SIZE, SENDER);
		if (newPoint != nil) {
		    /*
		     * MUST set the class
		     */
		    __objPtr(newPoint)->o_class = Point;

		    /*
		     * must set the fields
		     */
		    __InstPtr(newPoint)->i_instvars[0] = newX;
		    __InstPtr(newPoint)->i_instvars[1] = newY;

		    /*
		     * see below
		     */
		    __STORE(newPoint, newX);
		    __STORE(newPoint, newY);

		    /*
		     * good
		     */
		    ok = true;
		}
	    }
	}
    %}.
	ok ifTrue:[
	    ^ newPoint
	].

	"/ mhmh, something went wrong ...

	ObjectMemory allocationFailureSignal raise

For experts:
there are also macros for very hi-speed allocation (without a function call). These macros will directly manipulate the storage managers free-pointers, thus allowing an object to be allocated with just a few machine instructions. These macros are in the order of 10-20 times faster than ordinary malloc/free calls if no garbage collection is involved (*3).
The macros are:

__qNew(newObject, int size)

tries to do a quick (inline) new; if no memory is available, calls the general __STX___new() for a garbage collect.

__canDoQuickNew(int size)

returns 1 if a quick new is possible WITHOUT a garbage collect. (i.e. returns zero if a quick new operation would do a GC)

__qCheckedNew(newObject, int size)

does the quick new; is only allowed after a canDoQuickNew() which returned true.

__qMKFLOAT(newObject, double dval)

like __MKFLOAT, but tries to allocate inline; if no memory is available, the general __MKFLOAT is invoked which performs a garbage collect.

You should start to write your primitive using #new on the Smalltalk level or (at least) the normal __STX___new() function and later decide if its worth to tune the allocation (I'd say: seldom) .

Notice, that stc does generate high performance code for some classes' new operations. For example, it may decide to generate inline allocation code for a float, point or array creation.
This stc generated code may be faster than what you write - especially, if you are not too familiar with all those internals.

Allocating C memory in primitives

    dangerousMethod
    %{
	char *cp;

	cp = malloc(100);
	SEND(...);
	...
	free(cp);
    %}

Due to the message send in this method (the SEND call), it is possible, that the free() call is never executed. This happens if either a signal raise (with a handler and unwind) or a block return to some upper method, or a process termination occurs in or below the method reached via the send.

Of course, this will not lead to a crash, but instead to more and more memory being allocated over time. This memory will never be freed since the garbage collector does not know anything about this C memory.
These errors are especially hard to find, since Smalltalk does not know about it and they do not show any fatal conditions for quite a while (eventually, allocation failures will occur, though)

There are three possible solutions to fix the above, of which 2 suffer from other problems (they are described for didactic reasons):

• Bad Solution 1: keep a handle and cleanup when called the next time
  

      fixedMethod
      %{
  	static char *saveHandle;
  	char *cp;
  
  	if (saveHandle) {
  	    free(saveHandle);
  	}
  	cp = saveHandle = malloc(100);
  	if (cp) {
  	    ...
  	    SEND(...);
  	    ...
  	    free(cp); saveHandle = (char *)0;
  	}
      %}
  

  this will free the memory when the method is called the next time and the previous free was not executed for some reason. This is not the very best solution, but at least, unfreed memory will not accumulate.
  Be aware, that above method is not reentrant - this means, that it will lead to trouble, if another process executes the same method (for example, if another process becomes runnable while the SEND is executed).
  If this happens, the other invocation will free the memory and this in turn will lead to trouble, when the first invocation is continued.

• Bad Solution 2: keep a handle in a global C variable and provide a cleanup method
  

      !MyClass class primitiveVariables!
  
      %{
  	static char *saveHandle;
      %}
      ! !
  
      ...
  
      cleanupMethod
      %{
  	if (saveHandle) {
  	    free(saveHandle);
  	    saveHandle = (char *)0;
  	}
      %}
      !
  
      fixedMethod
      %{
  	char *cp;
  
  	if (saveHandle) {
  	    free(saveHandle);
  	}
  	cp = saveHandle = malloc(100);
  	if (cp) {
  	    ...
  	    SEND(...);
  	    ...
  	    free(cp); saveHandle = (char *)0;
  	}
      %}
  

  Also not the very best solution, since it requires that the cleanup code is called. However, the system already provides a hook for low space cleanup actions (the #lowSpaceCleanup class method), which is invoked before running out of memory. This method could do the cleanup in the above example.
  Also, this suffers from the same reentrancy problem as above code

• Good Solution: use an instance of ExternalBytes which is garbage collected, and the memory is freed by the finalizer.
  

      fixedMethod
  	|memory|
  
  	memory := ExternalBytes unprotectedNew:100.
      %{
  	char *cp;
  
  	cp = (char *)(__ExternalBytesInstPtr(memory)->e_address);
  	...
  	SEND(...);
  	...
      %}.
  	memory free
  

  Here, the memory as held by the externalBytes object will be freed by the garbage collector in case the #free send is not performed.

We highly recommend this solution, since it also solves the reentrancy problem. (If the memory block is passed to a C-function which remembers that memory reference (e.g. it should not be freed by the smalltalk collector but by explicit freeing), use "ExternalBytes protectedNew:"
Please read more on ExternalByte below.

Allocating C memory from Smalltalk code

In addition, you can either manually free the underlying C memory or let it be freed by the garbage collectors finalization procedure.

For example, to allocate a block of 100 bytes, use:

    |bytes|

    ...
    bytes := ExternalBytes new:100.
    ...

    ...
    {
	char *cp;

	cp = (char *)(__externalBytesAddress(bytes));
	cp[i] = ...
    }
    ...

    ...
    bytes at:1 put:16r13.
    ...
    foo := bytes at:10.
    ...

In the above example, the externalBytes memory is never automatically freed. You have to manually free the memory later with:

    ...
    bytes free.
    ...

However, this may not be a good choice in your concrete application; therefore, ExternalBytes offers a mechanism for automatic freeing. This is implemented with the finalization mechanisms described elsewhere. To get some memory which is freed automatically, use:

    ...
    bytes := ExternalBytes unprotectedNew:100.
    ...

    ...
    char *cp;
    OBJ bytes;

    cp = (char *) malloc(100);
    bytes = __MKEXTERNALBYTES(cp);
    ...
    RETURN (bytes);

    ...
    char *cp;
    OBJ bytes;

    cp = (char *) __externalBytesAddress(bytes);
    free(cp);
    ...
    __ExternalBytesInstPtr(bytes)->e_address = (OBJ)0;
    ...

It is better to free it from Smalltalk with:

    ...
    bytes free.
    ...

Therefore, every store of a non-SmallInteger, non-Nil object into another object must be checked for these special situations. To do so, after every store, a macro of the form:

	dest->i_inst[3] = value;     /* the store */
	__STORE(dest, value);        /* the writeBarrier-macro */

Omitting the STORE macro may leave you with a running program for a while or longer but lead to a bad crash at a later time. The crash will also usually hit some other object and it will be very hard to find the actual cause. The kind of crash is hard to predict, as arbitrary objects may become inconsistent or get invalid contents. For example, if a bytearray or string is affected, it may even silently get changed elements without ever getting a warning message from the compiler.

Sometimes it may even go totally unnoticed up to the time when storage allocation patterns or types of stored objects change.

Therefore: Never forget this macro for pointer stores!

It can ONLY be omitted if you are absolutely certain that the stored value is either nil nor a SmallInteger. Also for all non-pointer-arrays (i.e. ByteArray. FloatArray, DoubleArray and String) this store-check is not needed for the elements.

If you get one of the following error messages from the garbage collector:

    GC [error]: tried to mark free inst ... in ...
    GC [error]: tried to mark strange object: ...
    MEM [error]: bad class: ... in object at ...
    MEM [error]: class of ... is nil in scavenge
    MEM [error]: class of inst[...] ... is nil in scavenge

From our past experience, almost all (>80%) of all trouble came from user written primitives which had one or more STORE checks missing (the rest being missing PROTECT/UNPROTECT macros).

Late note:
The newest ST/X releases can be started with the VM commandLine argument "-MprotectSurv". This makes unsused portions of the object memory inaccessible (by using mprotect), and may help to find those problems, since missing STORE macros usualy lead to object references not being updated correctly and therefore tends to result in pointers into the alternative semispace of the copying garbage collector.
However, this is not a replacement for a careful code review, as not all operating systems provide this functionality and it also this does not guarantee to detect all such bugs.

Storing into another Object's Instvars (not recommended)

Doing so is a very very bad practice, because another object's instance slots can only be accessed by index. Thus it makes your code invalid, if the accessed object's instance layout changes.

If you ever have to return structured data or fill a passed-in object from a primitive, we recommend passing in or returning an array, and moving the slots into the target object via setter calls on the smalltalk level afterwards. Take a look at the event-returning code in XWorkstation to get the idea.

This may not be the "superfast" solution, but it is save and maintainable.

If you really have to care for every nanosecond, pass in the instvar offsets of the target object to make your primitive code independent from the target object's layout (either as argument or via preinitialized class variables).

Be reminded that this is more of a theoretical advice - even the author of the ST/X system never felt a need to do this.

Storing into the receiver

	__INST(xyz) = value;
	__STORE(self, value);

    myMethod
	myInstance := someThing.
    %{
	/* other stuff */
    %}
    .
	myInstance := somethingElse

	__STORE(Smalltalk, XYZ);

For the curious: these macros are found in "stc.h"; all of them can be used in expressions and evaluate to the stored value.

Mixing Smalltalk and C-code

As an artifical example, here is a Smalltalk loop over a collection, collecting its (integer) elements into an externalBytes object: (you may need this, to pass an integer vector to some external C function later)

    externalBytesVectorFor:someCollection
	|sz index element vector|

	sz := someCollection size.
	vector := ExternalBytes new:(sz * ExternalBytes sizeofInt).

	index := 1.
	[index <= sz] whileTrue:[
	    element := someCollection at:index.
    %{
	    INT iVal;
	    int *pI;

	    iVal = __longIntVal(element);
	    pI = (int *)(__ExternalBytesInstPtr(vector)->e_address);
	    pI[_intVal(index) - 1] = iVal;
    %}.
	    index := index + 1
	].
	^ vector

For most primitives, this behavior is correct and simplifies the writing of primitive code, because you don't have to care for effects due to multiple processes, reentrancy, context switching etc.
Also, typical C-libraries (Xlib, Stdio etc.) are not thread safe or reentrant and therefore, library functions should normally not be interrupted.

However, primitive code in which a blocking I/O or a (long) wait operation occurs, should be interruptable and tell the interrupt system to do immediate interrupt processing and handle the interrupt even while executing primitive code. Otherwise there would be no chance to get out of a blocking read (for example, when reading from a socket/pipe to which noone is writing).

To tell ST/X that interrupts should be handled immediately, two macros are provided in the "stc.h include file. These should be placed around interruptable code, as in:

    ...
    __BEGIN_INTERRUPTABLE__
    n = read(fd, buffer, count);
    __END_INTERRUPTABLE__
    ...

Of course, also "CTRL-C" processing is done using signals - therefore even pressing "CTRL-C" has no effect and there is no chance to get into the debugger.
Beside the obvious read() and write() many other C library functions possibly block for longer time - sometimes unexpected as with gethostname().

The handling of interrupts is not done the same on all systems (not even within the Unix world). Especially when system calls are interrupted, things may be very different (BSD vs. SYS5 signals; i.e. restarted call vs. EINTR return).
Your code should take care of this, by checking for an errno of EINTR and retrying the operation as appropriate. Therefore, the above was an oversimplification; real code should look like:

    ...
    __BEGIN_INTERRUPTABLE__
    do {
	n = read(fd, buffer, count);
    } while ((n < 0) && (errno == EINTR));
    __END_INTERRUPTABLE__
    ...

As a side effect, enabling immediate interrupt processing enables the VM to reschedule to another process. If your interruptable primitive code runs at a lower priority, higher priority processes can suspend it and become active. This allows long computations (even in c-code) to be performed in the background.

On the other hand, if an interrupt occurs and is handled immediately, you have to be very careful in coding your primitive. Anything can happen there, especially recursive entry into this method, a garbage collect, context unwinds, long returns or even process termination have to be considered.

Be VERY VERY careful when enabling interrupts in a primitive method which calls c-library functions; most system's libraries are NOT prepared for reentrant calls. This may result in memory leaks, locks, flags which are not set correct, stack overflows ...

For this very reason, there are only a few places in the system (blocking select and I/O system calls) where interrupts are currently enabled.

As an example, the following does some long computation in C, which is interruptable. Therefore, it can be performed in the background without disturbing foreground processes:

    longComputation
    %{
	int run, i, j;

	__BEGIN_INTERRUPTABLE__

	for (run=0; run<20; run++) {
	  for (i=0; i<50; i++) {
	    for (j=0; j<1000000; j++) {
	    }
	  }
	  printf("looping\n");
	}

	__END_INTERRUPTABLE__
    %}

    "
     [XXX new longComputation] forkAt:3
    "

    interruptableDestructiveUpperCaseToLowerCase:aString
    %{
	int i, len;
	char c, *cp;

	/* check if argument is a string */
	if (__isString(aString)) {
	    /*
	     * fetch size once
	     */
	    len = __stringSize(aString);
	    /*
	     * walk over string till len characters have been processed
	     */
	    for (i = 0; i < len; i++) {
		/*
		 * allow interrupts only here; let the loop body run uninterrupted
		 */
		__BEGIN_INTERRUPTABLE__
		__END_INTERRUPTABLE__
		/*
		 * check if the string has magically changed its identity or size
		 */
		if (! __isString(aString)
		 || (len != __stringSize(aString)) break;
		/*
		 * refetch the actual pointer - the old one may be obsolete
		 */
		cp = __stringVal(aString)->s_element;
		if (isUpper(c = cp[i])) {
		    cp[i] = toLower(c);
		}
	    }
	    RETURN ( aString );
	}
    %}
    .
	self primitiveFailed
    !

• __newGCGeneration__
  incremented for any newSpace garbage collect (i.e. scavenge)

• __oldGCGeneration__
  incremented for any oldSpace garbage collect (i.e. full compress)

• __becomeGeneration__
  incremented with any #become or #becomeNil

• __pointerGeneration__
  incremented with any of the above

    interruptableDestructiveUpperCaseToLowerCase:aString
    %{
	int i, len;
	char c, *cp;
	extern int __pointerGeneration__;
	int currentGeneration = __pointerGeneration__;

	/* check if argument is a string */
	if (__isString(aString)) {
	    /*
	     * fetch size once
	     */
	    len = __stringSize(aString);
	    cp = __stringVal(aString)->s_element[0];
	    /*
	     * walk over string till len characters have been processed
	     */
	    for (i = 0; i < len; i++) {
		__BEGIN_INTERRUPTABLE__
		__END_INTERRUPTABLE__
		/*
		 * check if the string has magically changed its identity or size
		 */
		if (__pointerGeneration__ != currentGeneration) {
		    if (! __isString(aString)
		     || (len != __stringSize(aString)) break;
		    /*
		     * refetch the actual pointer - the old one is obsolete
		     */
		    cp = __stringVal(aString)->s_element[i];
		    currentGeneration = __pointerGeneration__;
		}
		if (isUpper(c = *cp)) {
		    *cp = toLower(c);
		}
		cp++;
	    }
	    RETURN ( aString );
	}
    %}
    .
	self primitiveFailed
    !

    __stxRegisterCustomInterrupt__();

    __stxHandleInterrupt__();

In the Smalltalk world, this will send a #customInterrupt message to the CustomInterruptHandler.

Using custom interrupts is especially useful for callBacks and other C functions which want to interrupt Smalltalk processing and get immediate response.

Timed callBacks into C code

    int id = __STX_AddTimeout(func, millis, arg)

    id = Processor
	    addTimeoutFunctionCall:anExternalFunction
	    for:nil
	    afterMilliseconds:millis
	    with:arg

The timeout is installed as a oneShot - i.e. to get some cyclic calling, you have to reinstall another timeout from within the called function.

Any pending timeout can be removed with:

    __STX_RemoveTimeout(id)

    Processor removeTimeoutWithID:id

An example & demonstration can be found in the "doc/coding/cCallBack directory.

Reporting errors from C code

The above should work in most situations;
however, there are cases where the error is reported asynchronously, and there is noone to return an errorCode to.

For example, take the error mechanism in the Xlib:
here, in case of an error, a handler (c-)function is called from withing Xlib, which gets additional error information passed as arguments.

In ST/X, these asynchronous errors must be handled as errorInterrupts, since there is no particular method available, to which the handler could return this error information.
To do so, the (C-)handler function should call one of the VM functions

    __errorInterruptWithID__(id)
or:
    __errorInterruptWithIDAndParameter__(id, param)

For example, the XErrorHandler uses the symbol #DisplayError as errorID and passes the display connections ID as (integer-) parameter.

Calling one of the above functions leads to an error interrupt being registered, which eventually invokes the errorInterrupt:with: method (in Object). The actual interrupt processing may be done asynchronously (for example, if the VM is currently in the garbage collector, or a noninterruptable primitive).
This method uses the passed id to find a specific handler and forward the error to that handler; again by sending #errorInterrupt:with: to it.
The specific handler must have been registered previously via

ObjectMemory registerErrorInterruptHandler:aHandler forID:anId

If the above sounds confusing, lets see a concrete example:
assume that you want to use some C library, which calls an error function asynchronously, and this error should be translated into a private (Smalltalk-) signal.
First, create the signal in your class's initialize method and provide access to it:

!MyClass class methodsFor:'initialization'!

initialize
    ...
    MySignal := Object errorSignal newSignalMayProceed:true.
    MySignal nameClass:self message:#mySignal.
    ...

...

!MyClass class mathodsFor:'Signal constants'!

mySignal
    ^ MySignal
!
....

!MyClass class methodsFor:'error handling'!

errorInterrupt:id with:someParameter
    mySignal raise
!
....

!MyClass class primitiveFunctions!

subsystemError() {
	__errorInterruptWithIDAndParameter__(@symbol(MySubSystem), nil);
}

...
ObjectMemory registerErrorInterruptHandler:self forID:#MySubSytem

Further examples:
see how DeviceWorkstation and XWorkstation handles errors occuring in the graphic interface.

Problems

    %{
	struct abc {
	    int field1;
	    char field2;
	};
    %}

    method
	|local1 field2|
	...

Workaround: rename the local variables.

other name conflicts with structures, macros and typedefs

This will be changed soon - at least for names in "stc.h". Compiling code with such conflicts will usualy lead to errors in the C-compilation phase. Since stc does not parse (and understand) the structure of primitive code, it will not notice this conflict.

local variables whose name starts with an underscore

This will be changed to use a different renaming scheme either for local variables or symbols (or both).

For now, be when a locals name starts with an underscore; we suggest you use 'normal' variable names for now.

Solutions for typical problems

Most problems involved with primitives deal with memory management; especially, when data structures as returned or filled by some C function have to be converted into Smalltalk objects.

• passing a string to a C function

  The following passes a string to the printf function:

      myPrintf:someString
  	"print someString, with printf"
  
      %{
  	/*
  	 * first, check if its a string that was passed - just to make sure
  	 */
  	if (__isString(someString)) {
  	    char *format = __stringVal(someString);
  
  	    printf(format);
  	    RETURN ( self );
  	}
  	/* fall through in case of a non-string... */
      %}.
  	self primitiveFailed
  

• returning a string from a C function

  The following returns the string as returned by the getlogin() function:

      getLoginName
  	"return a string with the user's name"
  
      %{
  	char *name;
  
  	name = (char *)getlogin();
  	if (name) {
  	    RETURN ( __MKSTRING(name) );
  	} else {
  	    RETURN ( nil );
  	}
      %}
  

• returning two strings from a C function

  Since Smalltalk does not directly support returning multiple values, we have to wrap multiple values into one return object. Here, we place them into an array of 2 elements (the later releases of ST/X provide private classes - so a better solution would be to define a private class for holding those two values and return an instance of it instead of an array).
  The example is interesting, since it has to take care that the objects which where allocated first will be handled correctly in case of a garbage collection (which may happen during allocation of the last object, for example).
  Here, we take the easy road, and simply put allocated objects into local variables of the method (remember, these context variables are always seen and handled correctly by the garbage collector), and let Smalltalk code construct the return array:

      getUserAndGroupName
  	"return an array of two strings, filled with the user's name
  	 and the group name"
  
  	|user group|
      %{
  	char *userName, *groupName;
  	int userId, groupID;
  
  	userID = (int) getuid();
  	groupId = (int) getgid();
  	userName = (char *) getpwuid(userID)
  	groupName = (char *) getgrgid(userID)
  
  	user = __MKSTRING(userName);
  	group = __MKSTRING(groupName);
      %}.
  	^ Array with:user with:group
  
  	"
  	 alternative:
  
  	^ (IdentityDictionary new)
  		at:#userName put:user;
  		at:#groupName put:group;
  		yourself
  	"
  

• passing an array of strings to a primitive

  The following sends each line from a collection of strings via printf to the standard output:
  (of course, this is for the demonstration only; you better use Smalltalk's printOn:)

      printAll:aCollectionOfStrings
  	"send all strings to the standard output"
  
  	|arrayOfStrings|
  
  	"/ only handle arrays in the primitive
  	"/ convert, if required
  
  	arrayOfStrings := aCollectionOfStrings asArray.
      %{
  	char *name;
  
  	/*
  	 * check - we never know ...
  	 * (in case of allocation failure,
  	 *  or if interrupted and set to something else in the debugger ...)
  	 */
  	if (__isArray(arrayOfStrings)) {
  	    int nElements = __arraySize(arrayOfStrings);
  
  	    for (i=0; i < nElements; i++) {  /* notice: C-index starts with 0 */
  		OBJ element;
  
  		element = __ArrayInstPtr(arrayOfStrings)->a_element[i];
  		/*
  		 * the bad guy could have placed something else into
  		 * the array; check it.
  		 */
  		if (__isString(element)) {
  		    printf("%s\n", __stringVal(element));
  		} else {
  		    /*
  		     * mhmh - what should we do here ?
  		     */
  		    printf("*** non-string ***\n");
  		}
  	    }
  	    RETURN ( self );
  	}
      %}.
  	self primitiveFailed
  

• passing a polymorph array with either strings, floats or integers

  Like above, but allow non-strings entries.

  	...
  	...
  	element = __ArrayInstPtr(arrayOfStrings)->a_element[i];
  	if (__isString(element)) {
  	    printf("%s\n", __stringVal(element));
  	} else {
  	    if (__isSmallInteger(element)) {
  		printf("%d\n", __intVal(element));
  	    } else {
  		if (__isFloat(element)) {
  		    printf("%f\n", __floatVal(element));
  		} else {
  		    /*
  		     * mhmh - what should we do here ?
  		     */
  		    printf("*** bad element ***\n");
  		}
  	    }
  	}
  	...
  	...
  

• returning an array of strings

  this is more complicated, since we have to be careful about memory allocation and garbage collector effects (reread the previous chapters, if things are unclear):
  The following example is especially interesting, since we do not know in advance, how big the array will be.
  Also, the called X function internally uses alloca() and therefore may require a huge stack frame for its operation. Thus, we have to tell the stc compiler to temporarily switch from the bounded thread stack to the unbounded C stack.
  You normally do not know this in advance; consider this a first-class candidate when running into memory problems (i.e. SIGBUS or SIGSEGV or ST/X crashes). Or, always use UNLIMITEDSTACK to make sure - the overhead is only a few microseconds.

      getAvailableFontsFromXServer
  	"return anArray filled with font names"
  
      %{  /* UNLIMITEDSTACK */
  
  	int nnames = 1000;
  	int available = nnames + 1;
  	char **fonts;
  	OBJ arr, str;
  	int i;
  
  	for (;;) {
  	    /*
  	     * ask for nnames font-names
  	     */
  	    fonts = XListFonts(myDpy, __stringVal(pattern), nnames, &available);
  	    if ((fonts == NULL) || (available < nnames)) break;
  	    /*
  	     * mhmh - there are more. Double the size and ask again
  	     */
  	    XFreeFontNames(fonts);
  	    nnames = available * 2;
  	}
  	/*
  	 * no names returned
  	 */
  	if (fonts == NULL) {
  	    RETURN ( nil );
  	}
  
  	/*
  	 * ok; got 'available' names, pointed to by 'fonts'
  	 *
  	 * first, create a Smalltalk array:
  	 */
  	arr = __ARRAY_NEW_INT(available);
  	if (! arr) {
  	    /*
  	     * allocation failed
  	     */
  	    RETURN (nil);
  	}
  	/*
  	 * then, enter each string into it.
  	 * Be careful, creating a string may lead to a garbage collect;
  	 * therefore, we PROTECT/UNPROTECT the array reference.
  	 * (could have used a method local variable for the same effect)
  	 */
  	for (i=0; i < available; i++) {
  	    __PROTECT__(arr);
  	    str = __MKSTRING(fonts[i]);
  	    __UNPROTECT__(arr);
  	    __ArrayInstPtr(arr)->a_element[i] = str;
  	    /*
  	     * do not forget the write-barrier;
  	     * arr may be in a different memory space than the newly
  	     * created string ...
  	     */
  	    __STORE(arr, str);
  	}
  	RETURN (arr);
      %}
  

  (the above code is actually found in the XWorkstation class. BTW: This class and GLXWorkstation are good sources of many more primitive code examples)

• passing a vecor of floats

  some of you may want to use ST/X's inline C features to speedup number crunching. The following shows a very high performance version, using an instance of FloatArray. In contrast to regular arrays, these to not keep their elements as pointer references, but instead directly hold the numeric values. (it does not really make sense, to write a primitive operating on a regular array containing floats; such a primitive will show comparable performance to the corresponding compiled Smalltalk code, since most time is spent in the garbage collector.)
  Therefore, it is possible to operate on these elements without any need to allocate temporary storage, or care for any shared references.
  The following code is about as fast as one can get; if your system is attached to a vector processing unit, you may even pass the whole vector down to that unit for processing ...
  To make the example simple, lets simply negate all elements in the array:

      negateFloatArray:aFloatArray
  	"in place negate all floats in the array"
  
      %{
  	if (__Class(aFloatArray) == @global(FloatArray)) {
  	    int nElements = __floatArraySize(aFloatArray);
  	    float *fPtr = __FloatArrayInstPtr(aFloatArray)->f_element;
  
  	    while (nElements) {
  		*fPtr = -(*fPtr);
  	    }
  	    RETURN ( aFloatArray );
  	}
      %}.
  	self primitiveFailed
  

  since no allocation is done (and no other sends are performed), we can further speed this up, by ommiting the context setup.
  Also, we might add code to handle doubleArrays as well:

      negateFloatOrDoubleArray:aFloatOrDoubleArrayArray
  	"in place negate all floats in the array"
  
      %{  /* NOCONTEXT */
  	OBJ cls;
  
  	cls = __Class(aFloatOrDoubleArrayArray);
  	if (cls == @global(FloatArray)) {
  	    int nElements = __floatArraySize(aFloatOrDoubleArrayArray);
  	    float *fPtr = __FloatArrayInstPtr(aFloatOrDoubleArrayArray)->f_element;
  
  	    while (nElements) {
  		*fPtr = -(*fPtr);
  	    }
  	    RETURN ( aFloatOrDoubleArrayArray );
  	}
  	if (cls == @global(DoubleArray)) {
  	    int nElements = __doubleArraySize(aFloatOrDoubleArrayArray);
  	    double *dPtr = __DoubleArrayInstPtr(aFloatOrDoubleArrayArray)->d_element;
  
  	    while (nElements) {
  		*dPtr = -(*dPtr);
  	    }
  	    RETURN ( aFloatOrDoubleArrayArray );
  	}
      %}.
  	self primitiveFailed
  

  the above leads us directly to the next example:

• processing a square matrix

  lets assume, we have already created two subclasses of FloatArray, called HomogeneousPoint3D and TransformationMatrix. Lets further assume, that HomogeneousPoint3D always creates instances containing exactly 4 elements (for x, y, z and W components) while TransformationMatrix always creates instances containing exactly 16 elements (for a 4 x 4 transformation matrix).

  Then take your Foley, vanDam, Feiner & Hughes from your library, and start to code:

      multiplyVector:point with:matrix into:result
  	"in-place multiply; we do not return any newly created objects,
  	 instead, we destructively place things into the result vector.
  	 This gives us more control over object allocation, and allows
  	 result vectors to be reused.
  	 Well, well, well; at some point, we have to make a compromise ..."
  
      %{  /* NOCONTEXT */
  	if ((__Class(point)== @global(HomogeneousPoint3D))
  	 && (__Class(result)== @global(HomogeneousPoint3D))
  	 && (__Class(matrix)== @global(TransformationMatrix))) {
  	    float *p = __FloatArrayInstPtr(point)->f_element;
  	    float *r = __FloatArrayInstPtr(result)->f_element;
  	    float *m = __FloatArrayInstPtr(matrix)->f_element;
  
  	    r[0] = p[0] * m[0][0] + p[1] * m[1][0] + p[2] * m[2][0] + p[3] * m[3][0];
  	    r[1] = p[0] * m[0][1] + p[1] * m[1][1] + p[2] * m[2][1] + p[3] * m[3][1];
  	    r[2] = p[0] * m[0][2] + p[1] * m[1][2] + p[2] * m[2][2] + p[3] * m[3][2];
  	    r[3] = p[0] * m[0][3] + p[1] * m[1][3] + p[2] * m[2][3] + p[3] * m[3][3];
  	    RETURN ( self );
  	}
      %}.
  	self primitiveFailed
  

• Enable additional pointer checks in the VM
  Whenever new primititives have been added, enable the additional pointer checking code in the VM, by running ST/X with the "-Mcheck" argument. This instructs the VM to perform additional checking of pointers for being valid in the garbage collect and memory allocation functions and may help to detect bugs earlier.
  Let the program run in this mode for a while (a few weeks).

  Notice:
  The overhead created by this checking is usually not noticed (typically below 1% in memory intensive applications), therefore, it may be a good idea to always run the system in this mode.

  Also notice, that these checks cannot cover all possible errors - especially illegal pointers into objects (as opposed to pointers to non-object memory) are not detected and will certainly lead to crashes.

• Never forget to PROTECT/UNPROTECT references
  Whenever a garbage collection happens, objects may be moved (from new- to survivor or to old-space. Also when oldSpace is compressed). For all reachable objects (i.e. reachable via the context chain or globals), references are updated. However, you may end up in getting obsolete and invalid pointers if either:
  • an object is not reachable and it has not been protected
  • a reference was not known to the GC and still points to the old memory
  • a reference in a local variable has not been updated after a GC
  This means that you have to make sure that all objects are known to th GC, either by holding a reference in the context, or by adding a reference to it to the protect-list.

  It also means, that after any operation which possibly resulted in a GC, you have to refetch (UNPROTECT) any pointers. Be especially careful with local variables which refer to Smalltalk objects.

• Never forget the STORE-macro
  This has been and still is one of the most often encountered bug in primitives, and usually leads to a VM-crash sooner or later.

  Whenever an OBJ-reference is stored into some object, you MUST place a __STORE(dst, ptr) macro after the store operation.
  If this is ommited, the VM may loose track of ptr and treat it as free garbage with the next scavenge operation (Garbage collect).
  The result will be that ptr gets overwritten later (whenever new objects are created) and desaster happens.
  This is especially dangerous, because the crash occurs delayed - often much later.

  A typical symptom of such a bug are VM messages, complaining about bad sizes in object headers, invalid pointers or class references.
  We have made the VM very robust towards such bugs, and often the system survives even those invalid object references (typically, these are nilled when encountered).
  However, even if the VM and object system survives, your objects will get instance variable slots silently nilled in such situations, which will certainly not be what your program expects... (;-)

• Always check argument types
  Even if you are certain that some primitive is always being passed a certain type (i.e. a SmallInteger, a String etc), please add code to check for this and do a primitiveFailure if not.

  Especially when debugging, there are often situations when incorrect arguments are passed to a primitive, and funny things can happen if you interpret these wrong (all the swamp of C programming may surround you ;-).
  For proper checking, use a construct like:

      myPrimitive:arg
  	...
      %{
  	/*
  	 * check for correct type - even if you think that can
  	 * never happen ...
  	 */
  	if (__isSmallInteger(arg)) {
  	    ...
  	    /*
  	     * do something with the integers value
  	     */
  	    ... __intVal(arg) ...
  	    ...
  	    RETURN ( ... );
  	}
      %}.
  	self primitiveFailed
  

  failing to do this check may not be fatal in the above integer case (simply taking a wrong value), but may lead to hard to find bugs if you take the argument as a string, symbol or other pointer value.

• Never forget the RETURN-macro The RETURN macro is quite diferent from a regular return: it also checks for a pending interrupt (for single stepping or thread switching) and takes special actions if a reference was taken of the current context.

• when compiling, stc gives funny error messages about block nesting, unclosed comments etc.

  action:

  • check parenthesis, bracket and brace-nesting;

  • check nesting of #ifdef .. #endif directives;

  • check unclosed comments, single- and double-quote strings

  • check for missing semicolons

  • check if you wrote a Smalltalk comment ("...") within the c-code

• stc compiles ok, but c compiler gives funny error messages; and the line numbers as reported may not be those of primitive code

  action:

  • like above: check parenthesis, bracket and brace-nesting, comments and unterminated strings in previous primitive code; if missing or too many, the whole program structure may get out of sync and errors are reported where no errors are.

• stc compiles ok, but c compiler errs with undefined names

  action:

  • check if you used a structure-field name as an argument and/or local

  • check parenthesis, bracket and brace-nesting;

  • check unclosed comments, single- and double-quote strings

  • check for missing semicolons

  • check if you wrote a Smalltalk comment ("...") within the c-code

• Accepting in the browser, the primitive compiles but you get messages about '__con' being undefined from the linker.

  This happens on Linux, if you installed the a.out version on an ELF system. The loader is not able to load an ELF object module and will fail with the above behavior.

  On all systems, this happens if the include files are not correctly installed and/or the path variables pointing to the include directory are not set correctly.

  action:

  • on Linux: check if you installed the correct ST/X version.

  • open the Launcher's compiler-settings dialog and check if the stx-include directory is found in the include setting.

  • check if the include files are all installed correctly - especially the files named "stc.h and "stxIntern.h" are required.

• all runs fine, but system runs into segv when executing one of the _isXXX macros (can only happen on some systems)

  action:

  • look for typing errors; there is a special danger in the _isXXX macros, since some are also known as Smalltalk-symbols. This error occurs, when you write '_isString' instead of '__isString' on systems, which do not differentiate between text- and data symbols. (i.e. in this case the C compiler creates a function-call to the data-symbol _isString - which is the C variable of the #isString symbol)

• the program runs fine for a while, then the VM outputs messages about bad classes, invalid method-arrays etc.

  action:

  • check garbage collector effects (are there any ST-objects stored into C-data ?)

  • check for uninitialized instance variables of objects created in primitives

  • check for boundary violations when accessing ST objects (array elements, strings) in primitive code

  • check if you stored/returned a C integer or float, ommitting the MKSMALLINT or MKFLOAT macro).
    Look for c-compiler warnings.

  • check if you stored/returned a C pointer instead of a proper st-pointer.
    Look for c-compiler warnings.

  • check for missing __STORE macros when assigning to an objects instance variable or to an ST-array element.
    Run ST/X with the -MprotectSurv command line argument.
    

  • check for stack overflows. Try if the error vanishes if you give your primitive a huge stack.
    Run ST/X with the -MprotectStack command line argument.
    

  • Run ST/X with the -Mcheck command line argument for more pointer checks in the VM.
    

  • re-read description in this document

• my primitive works, but crashes when called for instances of a subclass (where the class is of variable size and the subclass has additional instvars)

  the memory layout of subclass instances is not the same as for instances of the class where your primitive is defined. Therefore access to the variable part (the indexed instvars) is wrong (the indexed instvars are stored right behind the named instvars).

  action:

  • in your primitive, either check the receiver for being an instance of the class, or take care of the changed instSize.
    See some primitives in Array and String, of which some are prepared to handle subclass instances, others simply fall back into a super send.

• I pass a string to a c function which remembers this string. After some time, the system crashes when C uses this string

  you probably passed the value of the string using the _stringVal() macro. This macro returns a (char *) to the characters of the string object.
  At the time you pass this to the c function, everything runs fine and looks ok. However, if the C function remembers that pointer, the next GC may move the string around, leaving the (remembered) c pointer pointing to nowhere (actually, it will point to some random object).
  The next access from c will access a totally random object and may lead to a bus error or segmentation violation.

  action:

  • never pass pointers of or pointers based on ST objects to C functions which remember these.
    You MUST get the value out of Smalltalk memory (i.e. by malloc'ing some memory, copying the value and passing that pointer to your c function).

    An alternative mechanism for pinning down objects at a fixed address already exist in the VM (and are internally used), but not yet released for public use and especially: not documented.

• primitives by number (*):
  Actually, this is not really true; ST/X also supports a mechanism for custom C functions to be registered by number, and have them invoked by
  "ExternalFunction callCustomFunction:nr". This mechanism allows (some) integration of external functions even if no stc compiler is available.

• how about C++ (**):
  This is also true for C++, since Smalltalk's objects are much more dynamic than those of C++, and therefore require more information to be accessible at execution time. For example, new classes may be added/removed or changed or objects may change their class. Therefore, these conversions are also required if Smalltalk code is combined with C++. I.e. it is not possible, to provide a simple 1-to-1 mapping of Smalltalk objects to C++ objects.

• allocation speed (*3): Even with garbage collection, mosts Smalltalk's memory systems are much faster than the malloc()/free() combination, if the objects are allocated temporarily and reclaimed within a short time period. (Measured P90/SGI Indy: 1-2 us per object vs. 9us for malloc/free - these times are from the 90's - these days, allocation speed is in the 100ns order)
  Things are of course different if static or non-malloced data is used in C/C++, or if the objects have to be reclaimed from oldSpace (which is more expensive to reclaim).

Copyright © 1995 Claus Gittinger, all rights reserved