Each system is characterized by operating system type, CPU type, mkin version, saemix version and R version. A compiler was available, so if no analytical solution was available, compiled ODE models are used.

Every fit is only performed once, so the accuracy of the benchmarks is limited.

For the initial mmkin fits, we use all available cores.

n_cores <- parallel::detectCores()

Test data

Please refer to the vignette dimethenamid_2018 for an explanation of the following preprocessing.

dmta_ds <- lapply(1:7, function(i) {
  ds_i <- dimethenamid_2018$ds[[i]]$data
  ds_i[ds_i$name == "DMTAP", "name"] <-  "DMTA"
  ds_i$time <- ds_i$time * dimethenamid_2018$f_time_norm[i]
  ds_i
})
names(dmta_ds) <- sapply(dimethenamid_2018$ds, function(ds) ds$title)
dmta_ds[["Elliot"]] <- rbind(dmta_ds[["Elliot 1"]], dmta_ds[["Elliot 2"]])
dmta_ds[["Elliot 1"]] <- NULL
dmta_ds[["Elliot 2"]] <- NULL

Test cases

Parent only

parent_mods <- c("SFO", "DFOP", "SFORB", "HS")
parent_sep_const <- mmkin(parent_mods, dmta_ds, quiet = TRUE, cores = n_cores)
parent_sep_tc <- update(parent_sep_const, error_model = "tc")

t1 <- system.time(sfo_const <- saem(parent_sep_const["SFO", ]))[["elapsed"]]
t2 <- system.time(dfop_const <- saem(parent_sep_const["DFOP", ]))[["elapsed"]]
t3 <- system.time(sforb_const <- saem(parent_sep_const["SFORB", ]))[["elapsed"]]
t4 <- system.time(hs_const <- saem(parent_sep_const["HS", ]))[["elapsed"]]
t5 <- system.time(sfo_tc <- saem(parent_sep_tc["SFO", ]))[["elapsed"]]
t6 <- system.time(dfop_tc <- saem(parent_sep_tc["DFOP", ]))[["elapsed"]]
t7 <- system.time(sforb_tc <- saem(parent_sep_tc["SFORB", ]))[["elapsed"]]
t8 <- system.time(hs_tc <- saem(parent_sep_tc["HS", ]))[["elapsed"]]
anova(
  sfo_const, dfop_const, sforb_const, hs_const,
  sfo_tc, dfop_tc, sforb_tc, hs_tc) |> kable(, digits = 1)
npar AIC BIC Lik
sfo_const 5 796.3 795.3 -393.2
sfo_tc 6 798.3 797.1 -393.2
dfop_const 9 709.4 707.5 -345.7
sforb_const 9 710.0 708.1 -346.0
hs_const 9 713.7 711.8 -347.8
dfop_tc 10 669.8 667.7 -324.9
sforb_tc 10 662.8 660.7 -321.4
hs_tc 10 667.3 665.2 -323.6

The above model comparison suggests to use the SFORB model with two-component error. For comparison, we keep the DFOP model with two-component error, as it competes with SFORB for biphasic curves.

illparms(dfop_tc)
## [1] "sd(log_k2)"
illparms(sforb_tc)
## [1] "sd(log_k_DMTA_bound_free)"

For these two models, random effects for the transformed parameters k2 and k_DMTA_bound_free could not be quantified.

One metabolite

We remove parameters that were found to be ill-defined in the parent only fits.

one_met_mods <- list(
  DFOP_SFO = mkinmod(
    DMTA = mkinsub("DFOP", "M23"),
    M23 = mkinsub("SFO")),
  SFORB_SFO = mkinmod(
    DMTA = mkinsub("SFORB", "M23"),
    M23 = mkinsub("SFO")))

one_met_sep_const <- mmkin(one_met_mods, dmta_ds, error_model = "const",
  cores = n_cores, quiet = TRUE)
one_met_sep_tc <- mmkin(one_met_mods, dmta_ds, error_model = "tc",
  cores = n_cores, quiet = TRUE)

t9 <- system.time(dfop_sfo_tc <- saem(one_met_sep_tc["DFOP_SFO", ],
    no_random_effect = "log_k2"))[["elapsed"]]
t10 <- system.time(sforb_sfo_tc <- saem(one_met_sep_tc["SFORB_SFO", ],
    no_random_effect = "log_k_DMTA_bound_free"))[["elapsed"]]

Three metabolites

For the case of three metabolites, we only keep the SFORB model in order to limit the time for compiling this vignette, and as fitting in parallel may disturb the benchmark. Again, we do not include random effects that were ill-defined in previous fits of subsets of the degradation model.

illparms(sforb_sfo_tc)
three_met_mods <- list(
  SFORB_SFO3_plus = mkinmod(
    DMTA = mkinsub("SFORB", c("M23", "M27", "M31")),
    M23 = mkinsub("SFO"),
    M27 = mkinsub("SFO"),
    M31 = mkinsub("SFO", "M27", sink = FALSE)))

three_met_sep_tc <- mmkin(three_met_mods, dmta_ds, error_model = "tc",
  cores = n_cores, quiet = TRUE)

t11 <- system.time(sforb_sfo3_plus_const <- saem(three_met_sep_tc["SFORB_SFO3_plus", ],
    no_random_effect = "log_k_DMTA_bound_free"))[["elapsed"]]

Results

Benchmarks for all available error models are shown. They are intended for improving mkin, not for comparing CPUs or operating systems. All trademarks belong to their respective owners.

Parent only

Constant variance for SFO, DFOP, SFORB and HS.

CPU OS mkin saemix t1 t2 t3 t4
Ryzen 7 1700 Linux 1.2.0 3.2 2.140 4.626 4.328 4.998
Ryzen 7 1700 Linux 1.2.2 3.2 2.427 4.550 4.217 4.851
Ryzen 9 7950X Linux 1.2.1 3.2 1.352 2.813 2.401 2.074
Ryzen 9 7950X Linux 1.2.2 3.2 1.328 2.738 2.336 2.023
Ryzen 9 7950X Linux 1.2.3 3.2 1.118 2.036 2.010 2.088
Ryzen 9 7950X Linux 1.2.3 3.2 1.419 2.374 1.926 2.398
Ryzen 9 7950X Linux 1.2.4 3.2 0.972 2.550 1.987 2.055
Ryzen 9 7950X Linux 1.2.5 3.2 0.994 2.546 1.999 2.070

Two-component error fits for SFO, DFOP, SFORB and HS.

CPU OS mkin saemix t5 t6 t7 t8
Ryzen 7 1700 Linux 1.2.0 3.2 5.678 7.441 8.000 7.980
Ryzen 7 1700 Linux 1.2.2 3.2 5.352 7.201 8.174 8.401
Ryzen 9 7950X Linux 1.2.1 3.2 2.388 3.033 3.532 3.310
Ryzen 9 7950X Linux 1.2.2 3.2 2.341 2.968 3.465 3.341
Ryzen 9 7950X Linux 1.2.3 3.2 2.159 3.584 3.307 3.460
Ryzen 9 7950X Linux 1.2.3 3.2 2.348 3.134 3.253 3.530
Ryzen 9 7950X Linux 1.2.4 3.2 2.127 3.587 3.433 3.595
Ryzen 9 7950X Linux 1.2.5 3.2 2.152 3.316 3.309 3.337

One metabolite

Two-component error for DFOP-SFO and SFORB-SFO.

CPU OS mkin saemix t9 t10
Ryzen 7 1700 Linux 1.2.0 3.2 24.465 800.266
Ryzen 7 1700 Linux 1.2.2 3.2 25.193 798.580
Ryzen 9 7950X Linux 1.2.1 3.2 11.247 285.216
Ryzen 9 7950X Linux 1.2.2 3.2 11.242 284.258
Ryzen 9 7950X Linux 1.2.3 3.2 11.796 216.012
Ryzen 9 7950X Linux 1.2.3 3.2 12.841 292.688
Ryzen 9 7950X Linux 1.2.4 3.2 12.160 265.934
Ryzen 9 7950X Linux 1.2.5 3.2 11.249 290.918

Three metabolites

Two-component error for SFORB-SFO3-plus

CPU OS mkin saemix t11
Ryzen 7 1700 Linux 1.2.0 3.2 1289.198
Ryzen 7 1700 Linux 1.2.2 3.2 1312.445
Ryzen 9 7950X Linux 1.2.1 3.2 489.939
Ryzen 9 7950X Linux 1.2.2 3.2 482.970
Ryzen 9 7950X Linux 1.2.3 3.2 392.364
Ryzen 9 7950X Linux 1.2.3 3.2 483.027
Ryzen 9 7950X Linux 1.2.4 3.2 456.252
Ryzen 9 7950X Linux 1.2.5 3.2 479.637