Model Comparison Fit

Overview

This app shows some basic fitting of data to 2 simple infection models. This shows the concept of model/hypothesis testing. Read about the model in the “Model” tab. Then do the tasks described in the “What to do” tab.

The Model

Model Overview

This app fits 2 different models to virus load data from humans.

Model 1

This model consists of 4 compartments. The following entities are modeled:

  • U - uninfected cells
  • I - infected cells
  • V - (free) virus
  • X - T-cells

The following processes/flows are included in the model:

  • Virus infects cells at rate b.
  • Infected cells produce new virus at rate p, are killed by T-cells at rate k and die due to other causes at rate dI.
  • Free virus is removed at rate dV or goes on to infect further uninfected cells. It can also infect new cells at rate b, with unit conversion factor g.
  • T-cells are activated proportional to virus at rate a and undergo exponential growth at rate r.

Model 2

This model also consists of 4 compartments. The following entities are modeled:

  • U - uninfected cells
  • I - infected cells
  • V - (free) virus
  • X - B-cells/antibodies

The following processes/flows are included in the model:

  • Virus infects cells at rate b.
  • Infected cells produce new virus at rate p and die at rate dI.
  • Free virus is removed by antibodies at rate k, and by other mechanisms at rate dV. It can also infect new cells at rate b, with unit conversion factor g.
  • B-cells/antibodies grow exponentially proportional to virus at rate a and decay at rate dX.

Model Diagram

The diagram illustrates both compartmental models, with colors to indicate mechanisms that are part of either model 1 or model 2.

Flow diagram for the 2 models.

Model Equations

Implementing the models as continuous-time, deterministic systems leads to the following set of ordinary differential equations for model 1:

\[\dot U = - bUV\] \[\dot I = bUV - d_I I - kIX\] \[\dot V = pI - d_V V - gbUV\] \[\dot X = a V + r X\]

Model 2 is described by these equations: \[\dot U = - bUV\] \[\dot I = bUV - d_I I \] \[\dot V = pI - d_V V - k V X - gbUV\] \[\dot X = a V X - d_X X\]

Comments

Note that both models are rather simple and - arguably - somewhat unrealistic. For instance the T-cells grow exponentially without any limit. This model can be justified if one only tries to fit data from an acute infection - as done here for influenza - where the infection is over (i.e. virus drops below level of detection) before T-cells reach their peak. For both models, we ignored birth/death of uninfected cells, which is possibly ok for an acute infection (but needs to be determined based on known biology for a given system).

We also assume for model 2 that the dynamics of B-cells and antibodies track each other close enough that we can lump them together into a single equation.

In general, models need to be carefully tailored to the question and setting. This is true for models used in fitting even more than for models that are used without trying to perform data fitting.

Data source

The data being used in this app comes from (Hayden et al. 1996). Specifically, the data being fit is the ‘no intervention’ viral load data shown in Figure 2 of the paper.

Fitting Approach

The basic fitting approach is the one described in the Basic model fitting app, i.e. we minimize the sum of squares of the log differences between model predictions and virus load data, taking censored values into account.

Model comparison

There are different ways to evaluate how well a model fits to data, and to compare between different models. This app shows the approach of using Akaike’s “An Information Criterion” (AIC), or more precisely, we’ll use the one corrected for small sample size, AICc . If we fit by minimizing the sum of squares (SSR), as we do here, the formula for the AICC is: \[ AICc = N \log(\frac{SSR}{N})+2(K+1)+\frac{2(K+1)(K+2)}{N-K} \] where N is the number of data points, SSR is the sum of squares residual at the final fit, and K is the number of parameters being fit. A lower value means a better model. One nice feature of the AIC is that one can compare as many models as one wants without having issues with p-values and correcting for multiple comparison, and the models do not need to be nested (i.e. each smaller model being contained inside the larger models). That said, AIC has its drawbacks. Nowadays, if one has enough data available, the best approach is to evaluate model performance by a method like cross-validation (Hastie, Tibshirani, and Friedman 2011).

For evaluation of models with different AIC, there is fortunately no arbitrary, “magic” value (like p=0.05). A rule of thumb is that if models differ by AIC of more than 2, the one with the smaller one is considered statistically better supported (don’t use the word ‘significant’ since that is usually associated with a p-value<0.05, which we don’t have here). I tend to be more conservative and want AIC differences to be at least >10 before I’m willing to favor a given model. Also, I think that visual inspection of the fits is useful. If one model has a lower AIC, but the fit doesn’t look that convincing biologically (e.g. very steep increases or decreases in some quantity), I’d be careful drawing very strong conclusions.

Note that the absolute value of the AIC is unimportant and varies from dataset to dataset. Only relative differences matter. And it goes without saying that we can only compare models that are fit to exactly the same data.

What to do

The tasks below are described in a way that assumes everything is in units of days (rate parameters, therefore, have units of inverse days). If any quantity is not given in those units, you need to convert it first (e.g. if it says a week, you need to convert it to 7 days).

Task 1:

Task 2:

Task 3:

Further Information

References

Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. 2011. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics). Springer.
Hayden, F G, J J Treanor, R F Betts, M Lobo, J D Esinhart, and E K Hussey. 1996. “Safety and Efficacy of the Neuraminidase Inhibitor Gg167 in Experimental Human Influenza.” JAMA 275: 295–99.