eXept Software AG Logo

Smalltalk/X Webserver

Documentation of class 'ObjectFileLoader':
Home

Documentation
www.exept.de
Everywhere
for:
[back]

Class: ObjectFileLoader

Inheritance:

   Object
   |
   +--ObjectFileLoader
Package:
stx:libcomp
Category:
System-Compiler
Version:
rev: 1.397 date: 2019/04/10 07:36:21
user: cg
file: ObjectFileLoader.st directory: libcomp
module: stx stc-classLibrary: libcomp
Author:
Claus Gittinger

Description:


This class knowns how to dynamically load in external object-modules.

WARNING:
  As to date, this is still experimental and WITHOUT ANY WARRANTY.
  It is still being developed and the code may need some cleanup and more
  robustness.

There are basically two totally different mechanisms to do this:
    a) if there exists some dynamic-link facility such as:
       GNU-DL, dlopen (sparc, SYS5.4), rld_open (NeXT),
       or LoadLibrary (Win32), this is used

    b) if no such facility exists, the normal linker is used to
       link the module to text/data address as previously malloced,
       and the object file is loaded into that space.

Not all mechanisms work fully satisfying.
For example, the sun dl*-functions do an exit on link-errors
(which is certainly not what we want here :-(;
the NeXT mechanism does not allow for selective unloading
(only all or the last loaded module).

The only really useful packages are the GNU-dl package,
the SGI/Unixware/sys5.4/linux/osx libdl packages and the win32 dll's.
The GNU-dl package is only available for a.out file formats
(which is more or less obsolete);

For the above reasons, dynamic object loading is only
officially released for SYS5.4-based, linux, osx and win32 systems.

When stable, the functionality contained herein will be moved into the VM.
(It is needed there, to allow reloading of objectfiles upon
 image restart; i.e. before any class is reinitialilized).

Right now, reloading of object files is done by Smalltalk code,
which fails, if code is to be loaded which is required before the
load takes place, or which is used by the loader itself.
(i.e. for now, many primitives from libbasic cannot be dynamically
loaded at image restart time).
This will fix itself, once we do object file loading in the VM.

Class protocol:

accessing
o  copyLibrariesWhenLoading
under windows, a library file (dll) which has been loaded cannot be overwritten.
This makes it troublesome, to recompile a new lib while a smalltalk is being executed.
When CopyLibrariesWhenLoading is true, ddl files are copied to a tempdir before loading,
so that the original can still be recompiled.
Makes loading a bit slower (due to the copying),
so it is off by default and must be turned on explicitly during development.

usage example(s): 

     self copyLibrariesWhenLoading
     self copyLibrariesWhenLoading:true

o  copyLibrariesWhenLoading: aBoolean
under windows, a library file (dll) which has been loaded cannot be overwritten.
This makes it troublesome, to recompile a new lib while a smalltalk is being executed.
When CopyLibrariesWhenLoading is true, ddl files are copied to a tempdir before loading,
so that the original can still be recompiled.
Makes loading a bit slower (due to copying and less sharing/caching of already loaded dlls),
so it is of by default and must be turned on explicitly during development.

usage example(s): 

     self copyLibrariesWhenLoading:true

change & update
o  update: something with: aParameter from: changedObject
sent, when image is saved or restarted

defaults
o  hasValidBinaryExtension: aFilename
return true, if aFilename looks ok for binary loading.
This checks only the extension - not its contents.
(So an attempt to load the file may fail due to format errors or unresolved symbols)
This is very machine specific.

usage example(s): 

     ObjectFileLoader hasValidBinaryExtension:'foo.st'
     ObjectFileLoader hasValidBinaryExtension:'foo.o'
     ObjectFileLoader hasValidBinaryExtension:'foo.so'
     ObjectFileLoader hasValidBinaryExtension:'foo.dll'
     ObjectFileLoader hasValidBinaryExtension:'foo.DLL'
     ObjectFileLoader hasValidBinaryExtension:'foo.obj'
     ObjectFileLoader hasValidBinaryExtension:'foo.sl'

o  linkArgs

o  linkArgs: anArgString
MSC:
ParserFlags linkCommand:'link32'.
ParserFlags linkArgs:' /NOPACK /NOLOGO /DEBUG /MACHINE:I386 /DLL /OUT:%1.dll /DEF:%1.def'

Borland:
ParserFlags linkCommand:'ilink32'.
ParserFlags linkArgs:'-ap -E0 -Tpd -s -c -L\Programme\Borland\CBuilder3\lib -L..\libbc librun.lib import32.lib'
'
'/NOPACK /NOLOGO /DEBUG /MACHINE:I386 /DLL /OUT:%1.dll /DEF:%1.def'

Unix:
ParserFlags linkCommand:'ld'
ParserFlags linkArgs:'-shared'

o  linkCommand

o  linkCommand: aCommand

o  linkSharedArgs

o  linkSharedArgs: anArgString
Linux on AMD64 in 32bit mode:
LinkSharedArgs := '-shared -m elf_i386'

o  loadableBinaryObjectFormat
return a symbol describing the expected binary format
for object files to be loadable.
This is very machine specific.

usage example(s): 

     ObjectFileLoader loadableBinaryObjectFormat

o  nm: file
this should return a string to list the namelist of file.
The output of the command should be one line for each symbol,
formatted as:
addr segment name
This is parsed and read by #namesMatching:segment:in:.

If your default nm command cannot produce this, write a little
shell or awk script, to generate this and return a corresponding command
below.
This entry is obsolete (no longer req'd) and will vanish.

o  objectFileSuffix
return the fileName extension used for objects,
as generated by myself (output of cc).
This is very machine specific.

o  sharedLibrarySuffix
return the fileName suffix used for dynamic loadable objects,
(as generated by myself - output of linker/makefiles).
This is very os/machine specific.

usage example(s): 

     ObjectFileLoader sharedLibrarySuffix

o  sharedLibrarySuffixes
return a collection of possible fileName suffixes used for dynamic loadable objects.
If multiple suffixes are possible, the first one should be the suffix generated by myself,
when I create loadable modules (methods with primitive code).
This is very os/machine specific.

usage example(s): 

     ObjectFileLoader sharedLibrarySuffixes

o  useBorlandC

o  validBinaryExtensions
return a collection of valid filename extension for binary files.
This is very machine specific.

usage example(s): 

     ObjectFileLoader validBinaryExtensions

dynamic object loading
o  loadCPlusPlusObjectFile: aFileName
load a c++ object file (.o-file) into the image.
This method is not maintained (name mangling and static
initialization is so different among systems ...)

o  loadClass: aClassName fromObjectFile: aFileName
load a compiled class (.o-file) into the image.
Returns the class or nil.

usage example(s): 

     ObjectFileLoader loadClass:'Tetris'      fromObjectFile:'../clients/Tetris/Tetris.o'
     ObjectFileLoader loadClass:'TetrisBlock' fromObjectFile:'../clients/Tetris/TBlock.o'
     ObjectFileLoader loadClass:'Foo'         fromObjectFile:'classList.o'

o  loadLibrary: aLibraryFileName
load a library; search in some standard places and using
standard suffixes (i.e. suffix may be omitted from LibraryFileName).
Returns a handle or raise an error.
Use this to attach C-libraries.

usage example(s): 

     ObjectFileLoader loadLibrary:'libc.so.6'
     ObjectFileLoader loadLibrary:'libjpeg'
     ObjectFileLoader loadLibrary:'odbc32'

o  loadMethodObjectFile: aFileName
load an object file (.o-file) for a single method into the image;
This does a slightly different initialization.
Return an object handle; nil on error

o  loadObjectFile: aFileName
load an object file (.o-file) into the image;
the class name is not needed (multiple definitions may be in the file).
This may be either a smalltalk object or a C-object file.
The object file's init function (if any) is called, and the module
is unloaded if it returns failure (use lowLevel load, to load a file
without automatic initialization).
Return nil on error, an objectFile handle if ok.

o  loadObjectFile: pathNameOrFilename invokeInitializeMethods: invokeInitializeMethods
load an object file (.dll or .so-file) into the image;
the class name is not needed (multiple definitions may be in the file).
This may be either a smalltalk object or a C-object file.
The object file's init function (if any) is called, and the module
is unloaded if it returns failure (use lowLevel load, to load a file
without automatic initialization).
Returns nil on error, or the objectFile's handle if ok.

o  unloadAllObsoleteObjectFiles
unload all dynamically loaded object files for which the classes/method
has been already garbage collected.

usage example(s): 

     ObjectFileLoader unloadAllObsoleteObjectFiles

o  unloadObjectFile: aFileName
unload an object file (.o-file) from the image.
DANGER ALERT: currently, you have to make sure that no references to
objects of this module exist - in future versions, the system will keep
track of these. For now, use at your own risk.
(especially critical are blocks-functions).

o  unloadObjectFileAndRemoveClasses: aFileName
unload an object file (.o-file) from the image and remove all
corresponding classes from the system.
DANGER ALERT: currently, you have to make sure that no references to
objects of this module exist - in future versions, the system will keep
track of these. For now, use at your own risk.
(especially critical are blocks-functions).

dynamic object queries
o  findFunction: functionName suffix: suffix in: handle
look for the init function and returns its address

o  findInitDefinitionFunction: functionName in: handle
look for the initDefinition function and return its address or nil

o  findInitFunction: functionName in: handle
look for the init function and return its address or nil

o  getFunction: aStringOrSymbol from: handle
return the address of a function from a dynamically loaded object file.
Handle must be the one returned previously from loadDynamicObject.
Return the address of the function, or nil on any error.

o  getListOfUndefinedSymbolsFrom: aHandle
return a collection of undefined symbols in a dynamically loaded object file.
Handle must be the one returned previously from loadDynamicObject.
Not all systems allow an object with undefined symbols to be
loaded (actually, only dld does).

o  getSymbol: aString function: isFunction from: aHandle
return the address of a symbol/function from a dynamically loaded object file.
Handle must be the one returned previously from loadDynamicObject.
Return the address of the symbol, or nil on any error.

o  hasUndefinedSymbolsIn: handle
return true, if a module has undefined symbols in it.
This is only possible if the system supports loading
modules with undefined things in it - most do not

o  initFunctionBasenameForFile: pathNameOrFilename
return the expected initFunction's name, given a fileName

usage example(s): 

     ObjectFileLoader initFunctionBasenameForFile:'libstx_libbasic.so'  (unix)
     ObjectFileLoader initFunctionBasenameForFile:'libstx_libbasic.dll' (msdos)
     ObjectFileLoader initFunctionBasenameForFile:'demo.so'
     ObjectFileLoader initFunctionBasenameForFile:'demo.o'
     ObjectFileLoader initFunctionBasenameForFile:'demo.obj'

o  isCPlusPlusObject: handle
return true, if the loaded object is a c++ object module.
This is not yet fully implemented/supported.

o  isObjectiveCObject: handle
return true, if the loaded object is an objective-C object module.
This is not yet implemented/supported

o  isSmalltalkObject: handle
return true, if the loaded object is a smalltalk object module

o  listUndefinedSymbolsIn: handle
list undefined objects in a module on the transcript

o  namesMatching: aPattern segment: segmentPattern in: aPathName
search for entries which match a pattern.
This is obsolete & rubbish - it will vanish soon

o  releaseSymbolTable
this is needed on NeXT to forget loaded names. If this wasnt done,
the same class could not be loaded in again due to multiple defines.
On other architectures, this is not needed and therefore a noop.

image save/restart
o  invalidateAndRememberAllObjectFiles
invalidate code refs into all dynamically loaded object files.
Required before writing a snapshot image.

o  reloadAllRememberedObjectFiles
reload all object modules as were loaded when the image was saved.
Some care has to be taken, if files are missing or otherwise corrupted.

o  rememberAllObjectFiles
remember all loaded objectModules in the
PreviouslyLoadedObjects classVariable.
Called when an image is restarted to reload all modules which
were loaded in the previous life

o  revalidateAllObjectFiles
revalidate code refs into all dynamically loaded object files.
Required after writing a snapshot image.

o  unloadAllObjectFiles
unload all dynamically loaded object files from the image.
see DANGER ALERT in ObjectFileLoader>>unloadObjectFile:

usage example(s): 

     ObjectFileLoader unloadAllObjectFiles

o  unloadAndRememberAllObjectFiles
remember all modules and unload them

initialization
o  initialize
Verbose := true.

usage example(s): 

     LinkArgs := LinkCommand := nil.
     ObjectFileLoader initialize

o  lastError

o  libPath
see comment in #libPath:

o  libPath: aSharedLibraryPath
set the pathnames of directories, where other shared objects
are searched, in case a loaded module requires another module shared
library object to be loaded.
Currently, this is only required to be set for AIX systems;
ELF based linux and solaris systems specify the libPath in the
LD_LIBRARY_PATH environment variable.

usage example(s): 

     ObjectFileLoader libPath:'.:/usr/lib:/usr/local/lib'

o  linkErrorMessage

o  searchedLibraries
see comment in #searchedLibraries:

o  searchedLibraries: aCollectionOfArchivePathNames
set the pathnames of archives which are to be searched
when unresolved references occur while loading in an object
file. On systems which support shared libraries (all SYS5.4 based
systems), this is usually not required. Instead, modules which are to
be filed in (.so files) are to be prelinked with the appropriate
shared libraries. The dynamic loader then cares about loading those
modules (keeping track of which modules are already loaded).
Only systems in which the dynamic load is done 'manually' by st/x
(i.e. currently only linux and a.out-sunos) need to set this.

o  verbose
return true, if debug traces are turned on

o  verbose: aBoolean
turn on/off debug traces

usage example(s): 

ObjectFileLoader verbose:true

linking objects
o  createLoadableObjectFor: baseFilenameString
given an oFile, arrange for it to be loadable.
On ELF systems, this means that it has to be linked with the
-shared option into a .so file;
DLD based loaders (linux a.out) can directly load a .o file;
Other systems may require more ...

lowlevel object loading
o  initializeLoader
initialize dynamic loader if required

o  loadDynamicObject: pathNameOrFilename
load an object-file (load/map into my address space).
Return a non-nil handle if ok, nil otherwise.
No bindings or automatic initializations are done
- only a pure (low-level) load is performed.
For class-files or C-objects to be loaded with this method,
it is your responsibility to fetch any init-functions and
call them as appropriate.
This function is not supported on all architectures.

usage example(s): 

sys5.4:
     |handle|
     handle := ObjectFileLoader loadDynamicObject:'../stc/mod1.so'.
     ObjectFileLoader getFunction:'module1' from:handle

o  loadModulesFromListOfUndefined: list
try to figure out what has to be loaded to resolve symbols from list.
return a list of handles of loaded objects

o  primLoadDynamicObject: pathName into: anInfoBuffer
load an object-file (map into my address space).
Return an OS-handle (whatever that is) - where some space
(a 3-element array) has to be passed in for this.
The first two entries are used in a machine dependent way,
and callers may not depend on what is found there
(instead, only pass around handles transparently).
This function is not supported on all architectures.

o  primUnloadDynamicObject: aHandle
unload an object-file (unmap from my address space).
This is a low-level entry, which does not care if there are
still any code references (from blocks or methods) to this
module. Calling it for still living classes will definitely
lead to some fatal conditions to occur later.

o  unloadDynamicObject: handle
close an object-file (unmap from my address space)
and remove the entry from the remembered object file set.
This is a low-level entry, which does not care if there are
still any code references (from blocks or methods) to this
module. Calling it for still living classes will definitely
lead to some fatal conditions to occur later.

queries
o  binaryClassFilenameForPackage: aPackageId inDirectory: packageDirOrNil
Is there a shared library (.dll or .so) ?

o  canLoadObjectFiles
return true, if dynamic loading is possible.
Currently, only ELF based systems, AIX and linux a.out can do this.

usage example(s): 

     ObjectFileLoader canLoadObjectFiles

o  handleForDynamicObject: pathNameOrFilename
answer the handle of pathName (or nil if it has not been loaded)

o  handleFromID: id
given an id, return the pathName, or nil for static modules

usage example(s): 

     ObjectFileLoader handleFromID:1

o  loadedObjectFiles
return a collection containing the names of all
dynamically loaded objects.

usage example(s): 

     ObjectFileLoader loadedObjectFiles

o  loadedObjectHandles
return a collection of all handles

usage example(s): 

     ObjectFileLoader loadedObjectHandles

o  loadedObjectHandlesDo: aBlock
enumerate all handles

usage example(s): 

     ObjectFileLoader loadedObjectHandlesDo:[:h | h pathName printNL]

o  moduleNamed: aNameString
return the module named aNameString

usage example(s): 

     ObjectFileLoader moduleNamed:'ctypes'

o  pathNameFromID: id
given an id, return the pathName, or nil for static modules

usage example(s): 

     ObjectFileLoader pathNameFromID:1

o  primCanLoadObjectFiles
return true, if loading is possible using a standard mechanism

usage example(s): 

     ObjectFileLoader primCanLoadObjectFiles

signal access
o  objectFileLoadErrorNotification

st object file handling
o  callInitFunctionAt: address specialInit: special forceOld: forceOld interruptable: interruptable argument: argument identifyAs: handle returnsObject: returnsObject
call a function at address - a very dangerous operation.
This is needed to call the classes init-function after loading in a
class-object file or to call the CTORS when c++ modules are loaded.
ForceOld (if true) will have the memory manager
allocate things in oldSpace instead of newSpace.
If special is true, this is a smalltalk moduleInit (i.e. pass the vm-entries table);
otherwise, its called without arguments.
If it's a smalltalk-moduleInit, it may be either for a classPackage (which should
install all of its classes) or for a single incrementally compiled methdod
(it should then NOT install it, but return the method object instead).

DANGER: Internal & highly specialized. Don't use in your programs.
This interface may change without notice.

o  classPresentCheck: aClassOrClassName
callBack from class registration code in VM:
make certain, that aClassOrClassName is loaded too ...
(req'd if a subclass of an autoloaded class has been loaded).
This is now invoked both for superClasses AND preRequisite classnames (for extensions)

o  deinitializeClassesFromModule: handle
send #deinitialize and an #aboutToUnload notification
to all classes of a module.

o  invalidateModule: handle
invalidate all of the classes code objects ...

o  loadStatusFor: className
ask VM if class-hierarchy has been completely loaded, and return the status.
If className is non-nil, check the load status of this class only,
otherwise check the status of all known classes.

Answer an array with the status symbol and optionally the name of
a class with a bad status.

o  moduleInit: phase forceOld: forceOld interruptable: interruptable
initialization phases after registration.
DANGER: Pure magic; internal only -> don't use in your programs.

o  performModuleInitAt: initAddr for: className identifyAs: handle
Initialize a loaded smalltalk module.

o  performModuleInitAt: initAddr invokeInitializeMethods: invokeInitializeMethods for: className identifyAs: handle
Initialize a loaded smalltalk module.

o  primModuleInit: phase forceOld: forceOld interruptable: interruptable
initialization phases after registration.
DANGER: Pure magic; internal only -> don't use in your programs.

o  revalidateModule: handle
revalidate all of the classes code objects ...

o  saveCallInitFunctionAt: address in: aFilenameOrNil specialInit: special forceOld: forceOld interruptable: interruptable argument: argument identifyAs: handle returnsObject: returnsObject

o  unregisterModule: handle
unregister classes in the VM.
This invalidates all of the classes code objects ...

under test - loading dlls
o  loadObjectFiles: aSeqOfFiles
load a sequence of DLLs... first load all dlls with #moduleInit:0
then call #moduleInit:1, #moduleInit:2 and #moduleInit:3 for all loaded DLLs.

*** UNDER TEST ...
used by ExpeccoStartup under windows.

not supported is C,...

Private classes:

    ObjectFileLoadError
    ObjectFileLoadErrorNotification
    RegistrationFailedErrorNotification
    SuperClassMissingErrorNotification

ST/X 7.2.0.0; WebServer 1.670 at bd0aa1f87cdd.unknown:8081; Wed, 24 Apr 2024 12:27:26 GMT