Smalltalk/X Webserver

Documentation of class 'TableData':

Class: TableData

Inheritance
Description
Related information
Class protocol
- instance creation
Instance protocol

Inheritance:

   Object
   |
   +--TableData

Package:: stx:libbasic2

Category:: Collections-Sequenceable-Table

Version:: rev: 1.4 date: 2019/05/30 09:36:31; user: cg; file: TableData.st directory: libbasic2; module: stx stc-classLibrary: libbasic2

Author:: Claus Gittinger

Description:

Unfinished
 Ongoing work to support some algorithms on table data (such as CSV files).
 
 Snippets to read and process tabular (CSV) data
 Snippets to generate a plot.

 The category DWD (Deutscher Wetter Dienst / German weather service)
 contains specific code to deal with historic weather data.
 

 [instance variables:]
     rowData                 the actual data
     columnNames             name per column 
     columnTypes             type per column (default is String)
     tableName               name - only used for labeling graphs
     
 [class variables:]

Related information:

Class protocol:

instance creation

fromFile: filename: self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'
rows: rowData columnNames: names
rows: rowData columnNames: names tableName: tableName

Instance protocol:

accessing

getColumn: index

return a column (by index) as a vector

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        getColumn:1.

getColumnNamed: name

return a column (by name) as a vector

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        getColumnNamed:'TXK'.

getColumns: indexCollection

return multiple columns (by index vector) as a vector of columns

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        getColumns:#(1 2 3).

getColumnsNamed: names

return multiple columns (by name vector) as a vector of columns

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        getColumnsNamed:#('MESS_DATUM' 'TXK' 'TNK').

tableName: aString

set the table's name

tableNamePrefix: aString

prepend a prefix to the table's name

tableNameSuffix: aString

append a suffix to the table's name

analysis

addBincoSlidingMean3ForColumnNamed: colName

add a column with the sliding binco mean (1/4 + 1/2 + 1/4).
As binco:3 is quite common,
that is a tuned version of addBincoSlidingMean:3 forColumnNamed:colName

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerYearOfColumnNamed:'TXK')
                addBincoSlidingMean3ForColumnNamed:'Mean_Per_Year_TXK')
                    tableName:'Feldberg/Schwarzwald';
                    plot:{ #x -> 'MESS_DATUM' . #y -> { 'Mean_Per_Year_TXK' . 'Sliding_Binco_Mean_of_Mean_Per_Year_TXK' .  }}.

addBincoSlidingMean: n forColumnNamed: colName

add a column with the sliding binco mean (1/2^n + ... + 1/4 + 1/2 + 1/4 + ... + 1/2^n)

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerYearOfColumnNamed:'TXK')
                addBincoSlidingMean:5 forColumnNamed:'Mean_Per_Year_TXK')
                    tableName:'Feldberg/Schwarzwald';
                    plot:{ #x -> 'MESS_DATUM' . 
                           #y -> { 'Mean_Per_Year_TXK' . 'Sliding_Binco_Mean_of_Mean_Per_Year_TXK' .  }}.

addSlidingMean: n forColumnNamed: colName

add a column with the sliding mean.
The sliding mean looks meaner, but may introduce lag (phase shift),
which binco avoids.

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerYearOfColumnNamed:'TXK')
                addSlidingMean:11 forColumnNamed:'Mean_Per_Year_TXK')
                    tableName:'Feldberg/Schwarzwald';
                    plot:{ #x -> 'MESS_DATUM' . #y -> { 'Mean_Per_Year_TXK' . 'Sliding_Mean_of_Mean_Per_Year_TXK' .  }}.

extractRowsWhere: filterBlock

return a new table containing only rows for which filterBlock evaluates to true

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractRowsWhere:[:row | (row at:1) startsWith:'1945'])
                    plot
        
     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractRowsWhere:[:row | ((row at:1) from:5 to:6) = '01'])
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:8)
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:9)
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10)
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10)
                    tableNameSuffix:'-Echterdingen';
                    plot

analysis - DWD

extractMonth: monthIndex

return a new table containing only rows for that month.
This is specific to DWD data

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot
        
     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    tableNameSuffix:'-Echterdingen';
                    plot

extractMonth: monthIndex columnName: monthColumnName

return a new table containing only rows for that month.
This is specific to DWD data

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot
        
     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    tableNameSuffix:'-Echterdingen';
                    plot

extractMonth: monthIndex columnName: monthColumnName format: dateFormat

return a new table containing only rows for that month.
This is specific to DWD data

usage example(s):

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot
        
     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
                    tableNameSuffix:'-Echterdingen';
                    plot

     (((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
            meanPerMonthOfColumnNamed:'TXK')
                extractMonth:3 columnName:'MESS_DATUM' format:'%4y%2m')
                    tableNameSuffix:'-Echterdingen';
                    plot

     |table|
     table := (self new 
            readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt').
     1 to:12 do:[:month |        
         ((table
                meanPerMonthOfColumnNamed:'TXK')
                    extractMonth:month columnName:'MESS_DATUM' format:'%4y%2m')
                        tableNameSuffix:'-Echterdingen';
                        plot
     ]

extractMonth: month day: day

return a new table containing only rows for that day in month

usage example(s):

     ((self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
        extractMonth:1 day:1)
            tableName:'1st Januar, Feldberg';
            plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' . 'TNK' }}

extractMonth: month day: day columnName: monthColumnName format: dateFormat

return a new table containing only rows for that day in month

usage example(s):

     ((self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
        extractMonth:1 day:1 columnName:'MESS_DATUM' format:'%4y%2m%2d')
            plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' . 'TNK' }}

meanPerMonthOfColumnNamed: colName

return a new table containing the arithmetic mean per month of a column

usage example(s):

     ((self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerMonthOfColumnNamed:'TXK')
                plot

meanPerYearOfColumnNamed: colName

return a new table containing the arithmetic mean per year of a column

usage example(s):

     ((self new 
        read:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
            meanPerYearOfColumnNamed:'TXK')
                plot

helpers

indexOfColumnNamed: name

find a column index by name

usage example(s):

     self new 
        read:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        indexOfColumnNamed:'MESS_DATUM'.

instance creation

rows: rowDataArg columnNames: columnNamesArg
rows: rowDataArg columnNames: columnNamesArg tableName: tableNameArg

plotting

plot: |data tmax|

data := self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'.
tmax := data withColumnsNamed:#('MESS_DATUM' 'TXK' 'TNK').
tmax plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' 'TNK' }} .
plot: optionalSpec: FileBrowserV2 openOn:tmpFile.
plot: optionalSpec file: fileName: 'plot(',xCol,', '
plotFile: fileName: |data tmax|

data := self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'.
tmax := data withColumnsNamed:#('MESS_DATUM' 'TXK').
tmax tableName:'Test'.
tmax plot.
writeCSVToStream: aStream: save myself as CSV onto a stream

processing

removeColumn: index

destructively remove a column

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        removeColumn:1;
        inspect.

removeColumnNamed: name

destructively remove a column

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        removeColumnNamed:'STATIONS_ID';
        removeColumnNamed:'QN_3';
        removeColumnNamed:'FX';
        removeColumnNamed:'FM';
        removeColumnNamed:'RSK';
        removeColumnNamed:'RSKF';
        removeColumnNamed:'SDK';
        removeColumnNamed:'SHK_TAG';
        removeColumnNamed:'NM';
        removeColumnNamed:'VPM';
        removeColumnNamed:'PM';
        removeColumnNamed:'TMK';
        removeColumnNamed:'UPM';
        removeColumnNamed:'eor';
        inspect.

withColumns: indexCollection

return a new TableData instance, containing only the given columns

usage example(s):

     (self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
        withColumns:#(1 2 3).

withColumnsNamed: nameCollection

return a new TableData instance, containing only the given columns

usage example(s):

     (self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
        withColumnsNamed:#('MESS_DATUM' 'TXK').

withoutColumn: index

return a new TableData instance, without the given columns

usage example(s):

     self new 
        readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
        withoutColumn:1.

withoutColumns: indexCollection

return a new TableData instance, without the given columns

usage example(s):

     (self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
        withoutColumns:#(1 2 3).

reading

readCSV: filename: self new
readCSV:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
inspect.
readCSV: filename separator: separatorCharacter: self new
readCSV:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
inspect.
readDemoCSV

ST/X 7.2.0.0; WebServer 1.670 at bd0aa1f87cdd.unknown:8081; Thu, 25 Apr 2024 16:49:47 GMT