In more complex Markov models in health economic evaluation, transition probabilities between states can vary with time. These models are called non-homogeneous or time-inhomogeneous Markov models. A further distinction can be made depending on whether:
However in some special cases non-homogeneous Markov models can be used in situation 2, when the state with transition probabilities depending on individual-time is the starting state and it is not possible to go back to that state after having left it. In this situation individual-time is equivalent to model-time. This is the case in the following example.
If you are not familiar with heemod
, first consult the introduction vignette vignette("introduction", package = "heemod")
.
This example is an implementation of the assessment of a new total hip replacement (THR) technology described in chapter 3.5 of Decision Modelling for Health Economic Evaluation. A more detailed report is available at this location, event though this reports goes a bit further in the analysis. For the sake of simplicity we will not reproduce exactly the analysis from the book. See vignette vignette("reproduction", package = "heemod")
for an exact reproduction of the analysis.
This model has 5 states:
omrPTHR
);omrRTHR
);Two transition probabilities are time-varying in this model:
Other-cause death probabilities (mortality rate mr
) for the United Kingdom is taken from WHO databases using the get_who_mr()
function. The variable sex
, taking values 0
and 1
, must be recoded in "FMLE"
and "MLE"
before being passed to this function.
standardRR
increases with time with the following formula (a Weibull distribution):\[ P_{revision} = 1 - \exp(\lambda \times ((t-1)^\gamma-t^\gamma)) \]
Where \(t\) is the time since revision, \(\gamma = 1.45367786\) and:
\[ \lambda = exp(cons + ageC \times age + maleC \times sex) \]
Where age
and sex
(female = 0, male = 1) are individual characteristics, cons
= -5.49094, ageC
= -0.0367 and maleC
= 0.768536.
standardRR
is modified by the relative risk rrNP1
= 0.260677.\[ P_{revision} = 1 - \exp(\lambda \times rrNP1 \times ((t-1)^\gamma-t^\gamma)) \]
rrr
) probability is set to be constant at 0.04 per year.The key element to specify time-varying elements in heemod
is through the use of a package-defined variable, markov_model
. This variable takes increasing values with each cycles, starting from 1. For example the age of individuals at any moment can be defined as Initial age + markov_cycle
.
In order to build this more complex Markov model, parameters need to be defined through define_parameters()
. The equations decribed in the previous section can be written easily, here for a female population (sex
= 0) starting at 60 years old (age_init
= 60).
Note that state values that are going to be discounted need to be defined through define_parameters()
.
param <- define_parameters(
age_init = 60,
sex = 0,
# age increases with cycles
age = age_init + markov_cycle,
# operative mortality rates
omrPTHR = .02,
omrRTHR = .02,
# re-revision mortality rate
rrr = .04,
# parameters for calculating primary revision rate
cons = -5.49094,
ageC = -.0367,
maleC = .768536,
lambda = exp(cons + ageC * age_init + maleC * sex),
gamma = 1.45367786,
rrNP1 = .260677,
# revision probability of primary procedure
standardRR = 1 - exp(lambda * ((markov_cycle - 1) ^ gamma -
markov_cycle ^ gamma)),
np1RR = 1 - exp(lambda * rrNP1 * ((markov_cycle - 1) ^ gamma -
markov_cycle ^ gamma)),
# age-related mortality rate
sex_cat = ifelse(sex == 0, "FMLE", "MLE"),
mr = get_who_mr(age, sex_cat, country = "GBR"),
# state values
u_SuccessP = .85,
u_RevisionTHR = .30,
u_SuccessR = .75,
c_RevisionTHR = 5294
)
param
## 20 unevaluated parameters.
##
## age_init = 60
## sex = 0
## age = age_init + markov_cycle
## omrPTHR = 0.02
## omrRTHR = 0.02
## rrr = 0.04
## cons = -5.49094
## ageC = -0.0367
## maleC = 0.768536
## lambda = exp(cons + ageC * age_init + maleC * sex)
## gamma = 1.45367786
## rrNP1 = 0.260677
## standardRR = 1 - exp(lambda * ((markov_cycle - 1)^gamma - markov_cycle^gamma))
## np1RR = 1 - exp(lambda * rrNP1 * ((markov_cycle - 1)^gamma - markov_cycle^gamma))
## sex_cat = ifelse(sex == 0, "FMLE", "MLE")
## mr = get_who_mr(age, sex_cat, country = "GBR")
## u_SuccessP = 0.85
## u_RevisionTHR = 0.3
## u_SuccessR = 0.75
## c_RevisionTHR = 5294
Now that parameters are defined, the probability transitions can be easily written:
mat_standard <- define_matrix(
state_names = c(
"PrimaryTHR",
"SuccessP",
"RevisionTHR",
"SuccessR",
"Death"
),
0, C, 0, 0, omrPTHR,
0, C, standardRR, 0, mr,
0, 0, 0, C, omrRTHR+mr,
0, 0, rrr, C, mr,
0, 0, 0, 0, 1
)
mat_standard
## An unevaluated matrix, 5 states.
##
## PrimaryTHR SuccessP RevisionTHR SuccessR Death
## PrimaryTHR C omrPTHR
## SuccessP C standardRR mr
## RevisionTHR C omrRTHR + mr
## SuccessR rrr C mr
## Death 1
mat_np1 <- define_matrix(
state_names = c(
"PrimaryTHR",
"SuccessP",
"RevisionTHR",
"SuccessR",
"Death"
),
0, C, 0, 0, omrPTHR,
0, C, np1RR, 0, mr,
0, 0, 0, C, omrRTHR+mr,
0, 0, rrr, C, mr,
0, 0, 0, 0, 1
)
mat_np1
## An unevaluated matrix, 5 states.
##
## PrimaryTHR SuccessP RevisionTHR SuccessR Death
## PrimaryTHR C omrPTHR
## SuccessP C np1RR mr
## RevisionTHR C omrRTHR + mr
## SuccessR rrr C mr
## Death 1
While it is possible to plot the matrix thanks to the diagram
package, the results may not always be easy to read.
plot(mat_standard)
Utilities and costs are then associated to states. In this model costs are discounted at a rate of 6% and utilities at a rate of 1.5%.
Now that parameters, transition matrix and states are defined we can define the models for the control group and the NP1 treatment.
mod_standard <- define_model(
transition_matrix = mat_standard,
PrimaryTHR = define_state(
utility = 0,
cost = 394
),
SuccessP = define_state(
utility = discount(u_SuccessP, .015),
cost = 0
),
RevisionTHR = define_state(
utility = discount(u_RevisionTHR, .015),
cost = discount(c_RevisionTHR, .06)
),
SuccessR = define_state(
utility = discount(u_SuccessR, .015),
cost = 0
),
Death = define_state(
utility = 0,
cost = 0
)
)
mod_standard
## An unevaluated Markov model:
##
## 5 states,
## 2 state values
mod_np1 <- define_model(
transition_matrix = mat_np1,
PrimaryTHR = define_state(
utility = 0,
cost = 579
),
SuccessP = define_state(
utility = discount(u_SuccessP, .015),
cost = 0
),
RevisionTHR = define_state(
utility = discount(u_RevisionTHR, .015),
cost = discount(c_RevisionTHR, .06)
),
SuccessR = define_state(
utility = discount(u_SuccessR, .015),
cost = 0
),
Death = define_state(
utility = 0,
cost = 0
)
)
Both models can now be run for 60 years. By default models are computed for 1000 person starting in PrimaryTHR
. To replicate results from the original study we assume transitions occur at the end of each cycles. This is necessary because THR cost defined in state PrimaryTHR
need to be accounted for all 1000 individuals.
res_mod <- run_models(
standard = mod_standard,
np1 = mod_np1,
parameters = param,
cycles = 60,
cost = cost,
effect = utility,
method = "end"
)
## Fetching mortality data from WHO server.
## Using latest year: 2015
A comparison of both models can be done with summary()
. The incremental cost and effect are displayed in columns .cost
and .effect
.
summary(res_mod)
## 2 Markov models run for 60 cycles.
##
## Initial states:
##
## N
## PrimaryTHR 1000
## SuccessP 0
## RevisionTHR 0
## SuccessR 0
## Death 0
##
## Counting method: 'end'.
##
## utility cost
## standard 16118.73 525457.1
## np1 16170.51 613783.6
##
## Efficiency frontier:
##
## standard np1
##
## Model difference:
##
## Cost Effect ICER
## np1 88.32658 0.05177584 1705.942
The new treatment costs £1687 more per QALY gained.
It should be noted that this result differs from the original study. This difference is explained by higher population-level all-causes mortality rates in the original study than in the WHO database (used here). See vignette vignette("reproduction", package = "heemod")
for an exact reproduction of the analysis.
We can plot the counts per state for the standard model:
plot(res_mod, model = "standard", type = "counts")
And the np1 model:
plot(res_mod, model = "np1", type = "counts")