Aggregation/Scaling performance

Source: vignettes/agg_scale_performance.Rmd

Aggregations and scaling with hierarchyUtils should perform as fast as basic data.table code. For basic use cases there should only be slightly more overhead due to the assertions and added flexibility included in hierarchyUtils.

Aggregation timing comparison

This vignette makes aggregation timing comparisons between hierarchyUtils and data.table for example input data when aggregating across an interval variable like ‘age’ or a categorical variable like ‘sex’. The example input data increases in size as more draws are added.

Example input data 

#> Key: <location, year_start, sex, age_start, value1, value2, draw>
#>        location year_start    sex age_start value1 value2  draw
#>           <num>      <num> <char>     <num>  <num>  <num> <int>
#>     1:        1       1950 female         0      1      1     1
#>     2:        1       1950 female         1      1      1     1
#>     3:        1       1950 female         2      1      1     1
#>     4:        1       1950 female         3      1      1     1
#>     5:        1       1950 female         4      1      1     1
#>    ---                                                         
#> 13628:        1       2020   male        91      1      1     1
#> 13629:        1       2020   male        92      1      1     1
#> 13630:        1       2020   male        93      1      1     1
#> 13631:        1       2020   male        94      1      1     1
#> 13632:        1       2020   male        95      1      1     1

Timing comparison

col_stem col_type n_draws method n_input_rows user.self sys.self elapsed
age interval 1 data.table 13,632 0.079 0.000 0.040
age interval 1 hierarchyUtils 13,632 1.757 0.010 1.705
age interval 10 data.table 136,320 0.245 0.000 0.153
age interval 10 hierarchyUtils 136,320 2.525 0.047 2.006
age interval 100 data.table 1,363,200 0.756 0.000 0.391
age interval 100 hierarchyUtils 1,363,200 8.184 0.156 6.168
sex categorical 1 data.table 13,632 0.013 0.000 0.007
sex categorical 1 hierarchyUtils 13,632 0.101 0.000 0.058
sex categorical 10 data.table 136,320 0.074 0.000 0.037
sex categorical 10 hierarchyUtils 136,320 0.557 0.000 0.402
sex categorical 100 data.table 1,363,200 0.669 0.020 0.365
sex categorical 100 hierarchyUtils 1,363,200 5.135 0.007 4.004