|
Class: TableData
Object
|
+--TableData
- Package:
- stx:libbasic2
- Category:
- Collections-Sequenceable-Table
- Version:
- rev:
1.4
date: 2019/05/30 09:36:31
- user: cg
- file: TableData.st directory: libbasic2
- module: stx stc-classLibrary: libbasic2
- Author:
- Claus Gittinger
Unfinished
Ongoing work to support some algorithms on table data (such as CSV files).
Snippets to read and process tabular (CSV) data
Snippets to generate a plot.
The category DWD (Deutscher Wetter Dienst / German weather service)
contains specific code to deal with historic weather data.
[instance variables:]
rowData the actual data
columnNames name per column
columnTypes type per column (default is String)
tableName name - only used for labeling graphs
[class variables:]
instance creation
-
fromFile: filename
-
self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'
-
rows: rowData columnNames: names
-
-
rows: rowData columnNames: names tableName: tableName
-
accessing
-
getColumn: index
-
return a column (by index) as a vector
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
getColumn:1.
|
-
getColumnNamed: name
-
return a column (by name) as a vector
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
getColumnNamed:'TXK'.
|
-
getColumns: indexCollection
-
return multiple columns (by index vector) as a vector of columns
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
getColumns:#(1 2 3).
|
-
getColumnsNamed: names
-
return multiple columns (by name vector) as a vector of columns
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
getColumnsNamed:#('MESS_DATUM' 'TXK' 'TNK').
|
-
tableName: aString
-
set the table's name
-
tableNamePrefix: aString
-
prepend a prefix to the table's name
-
tableNameSuffix: aString
-
append a suffix to the table's name
analysis
-
addBincoSlidingMean3ForColumnNamed: colName
-
add a column with the sliding binco mean (1/4 + 1/2 + 1/4).
As binco:3 is quite common,
that is a tuned version of addBincoSlidingMean:3 forColumnNamed:colName
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerYearOfColumnNamed:'TXK')
addBincoSlidingMean3ForColumnNamed:'Mean_Per_Year_TXK')
tableName:'Feldberg/Schwarzwald';
plot:{ #x -> 'MESS_DATUM' . #y -> { 'Mean_Per_Year_TXK' . 'Sliding_Binco_Mean_of_Mean_Per_Year_TXK' . }}.
|
-
addBincoSlidingMean: n forColumnNamed: colName
-
add a column with the sliding binco mean (1/2^n + ... + 1/4 + 1/2 + 1/4 + ... + 1/2^n)
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerYearOfColumnNamed:'TXK')
addBincoSlidingMean:5 forColumnNamed:'Mean_Per_Year_TXK')
tableName:'Feldberg/Schwarzwald';
plot:{ #x -> 'MESS_DATUM' .
#y -> { 'Mean_Per_Year_TXK' . 'Sliding_Binco_Mean_of_Mean_Per_Year_TXK' . }}.
|
-
addSlidingMean: n forColumnNamed: colName
-
add a column with the sliding mean.
The sliding mean looks meaner, but may introduce lag (phase shift),
which binco avoids.
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerYearOfColumnNamed:'TXK')
addSlidingMean:11 forColumnNamed:'Mean_Per_Year_TXK')
tableName:'Feldberg/Schwarzwald';
plot:{ #x -> 'MESS_DATUM' . #y -> { 'Mean_Per_Year_TXK' . 'Sliding_Mean_of_Mean_Per_Year_TXK' . }}.
|
-
extractRowsWhere: filterBlock
-
return a new table containing only rows for which filterBlock evaluates to true
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractRowsWhere:[:row | (row at:1) startsWith:'1945'])
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractRowsWhere:[:row | ((row at:1) from:5 to:6) = '01'])
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:8)
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:9)
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10)
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10)
tableNameSuffix:'-Echterdingen';
plot
|
analysis - DWD
-
extractMonth: monthIndex
-
return a new table containing only rows for that month.
This is specific to DWD data
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
tableNameSuffix:'-Echterdingen';
plot
|
-
extractMonth: monthIndex columnName: monthColumnName
-
return a new table containing only rows for that month.
This is specific to DWD data
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
tableNameSuffix:'-Echterdingen';
plot
|
-
extractMonth: monthIndex columnName: monthColumnName format: dateFormat
-
return a new table containing only rows for that month.
This is specific to DWD data
usage example(s):
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:1 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:2 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:8 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:9 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:10 columnName:'MESS_DATUM' format:'%4y%2m')
tableNameSuffix:'-Echterdingen';
plot
(((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt')
meanPerMonthOfColumnNamed:'TXK')
extractMonth:3 columnName:'MESS_DATUM' format:'%4y%2m')
tableNameSuffix:'-Echterdingen';
plot
|table|
table := (self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19530101_20171231_04931.txt').
1 to:12 do:[:month |
((table
meanPerMonthOfColumnNamed:'TXK')
extractMonth:month columnName:'MESS_DATUM' format:'%4y%2m')
tableNameSuffix:'-Echterdingen';
plot
]
|
-
extractMonth: month day: day
-
return a new table containing only rows for that day in month
usage example(s):
((self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
extractMonth:1 day:1)
tableName:'1st Januar, Feldberg';
plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' . 'TNK' }}
|
-
extractMonth: month day: day columnName: monthColumnName format: dateFormat
-
return a new table containing only rows for that day in month
usage example(s):
((self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
extractMonth:1 day:1 columnName:'MESS_DATUM' format:'%4y%2m%2d')
plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' . 'TNK' }}
|
-
meanPerMonthOfColumnNamed: colName
-
return a new table containing the arithmetic mean per month of a column
usage example(s):
((self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerMonthOfColumnNamed:'TXK')
plot
|
-
meanPerYearOfColumnNamed: colName
-
return a new table containing the arithmetic mean per year of a column
usage example(s):
((self new
read:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
meanPerYearOfColumnNamed:'TXK')
plot
|
helpers
-
indexOfColumnNamed: name
-
find a column index by name
usage example(s):
self new
read:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
indexOfColumnNamed:'MESS_DATUM'.
|
instance creation
-
rows: rowDataArg columnNames: columnNamesArg
-
-
rows: rowDataArg columnNames: columnNamesArg tableName: tableNameArg
-
plotting
-
plot
-
|data tmax|
data := self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'.
tmax := data withColumnsNamed:#('MESS_DATUM' 'TXK' 'TNK').
tmax plot:{ #x -> 'MESS_DATUM' . #y -> { 'TXK' 'TNK' }} .
-
plot: optionalSpec
-
FileBrowserV2 openOn:tmpFile.
-
plot: optionalSpec file: fileName
-
'plot(',xCol,', '
-
plotFile: fileName
-
|data tmax|
data := self fromFile:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt'.
tmax := data withColumnsNamed:#('MESS_DATUM' 'TXK').
tmax tableName:'Test'.
tmax plot.
-
writeCSVToStream: aStream
-
save myself as CSV onto a stream
processing
-
removeColumn: index
-
destructively remove a column
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
removeColumn:1;
inspect.
|
-
removeColumnNamed: name
-
destructively remove a column
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
removeColumnNamed:'STATIONS_ID';
removeColumnNamed:'QN_3';
removeColumnNamed:'FX';
removeColumnNamed:'FM';
removeColumnNamed:'RSK';
removeColumnNamed:'RSKF';
removeColumnNamed:'SDK';
removeColumnNamed:'SHK_TAG';
removeColumnNamed:'NM';
removeColumnNamed:'VPM';
removeColumnNamed:'PM';
removeColumnNamed:'TMK';
removeColumnNamed:'UPM';
removeColumnNamed:'eor';
inspect.
|
-
withColumns: indexCollection
-
return a new TableData instance, containing only the given columns
usage example(s):
(self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
withColumns:#(1 2 3).
|
-
withColumnsNamed: nameCollection
-
return a new TableData instance, containing only the given columns
usage example(s):
(self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
withColumnsNamed:#('MESS_DATUM' 'TXK').
|
-
withoutColumn: index
-
return a new TableData instance, without the given columns
usage example(s):
self new
readCSV:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
withoutColumn:1.
|
-
withoutColumns: indexCollection
-
return a new TableData instance, without the given columns
usage example(s):
(self fromFile:'~/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt')
withoutColumns:#(1 2 3).
|
reading
-
readCSV: filename
-
self new
readCSV:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
inspect.
-
readCSV: filename separator: separatorCharacter
-
self new
readCSV:'/Users/cg/Documents/klima/data/DWD/daily/kl/historical/produkt_klima_tag_19450101_20171231_01346.txt';
inspect.
-
readDemoCSV
-
|