Multiple readers and Multiple Modality (MRMC) case

Issei Tsunoda

2019-05-04

Modality Comparison Issue ==============

Radiologists are sometimes consider which methods are more useful to detect lesions in radiographs from MRI, CT, PET,…

This package gives radiologists the solution of this Modality comparison issue by Bayesian approaches.

Work Flow ============================

Work Flow of R codes

Prepare data

#An example dataset for the case of Multiple readers and Multiple Modalities.
dat <- BayesianFROC::dataList.Chakra.Web

Fitting

# Fitting for your data with respect to the hierarchical Bayesian model.
fit<-BayesianFROC::fit_Bayesian_FROC(dat)

Draw Curves

 #Draw curves for the 1st modality and 2nd reader
 DrawCurves(

   #This is estimates 
        fit,
   
   # Modatity ID whose curves are drawn.
      modalityID =1,
   
   # Reader ID whose curves are drawn.
      readerID   =2)

Functions for MRMC

  • Make a data of MRMC —————————- create_dataset() or convertFromJafroc()
  • Fitting ( comparison of modalities) ———– fit_Bayesian_FROC()
  • Draw the FROC curve —————————– DrawCurves_MRMC_pairwise()

Data for MRMC =========================

Example.

Two readers and two modalities and three kind of confidence levels.

Confidence Level Modality ID Reader ID Number of Hits Number of False alarms
3 = definitely present 1 1 \(H_{3,1,1}\) \(F_{3,1,1}\)
2 = equivocal 1 1 \(H_{2,1,1}\) \(F_{2,1,1}\)
1 = questionable 1 1 \(H_{1,1,1}\) \(F_{1,1,1}\)
3 = definitely present 1 2 \(H_{3,1,2}\) \(F_{3,1,2}\)
2 = equivocal 1 2 \(H_{2,1,2}\) \(F_{2,1,2}\)
1 = questionable 1 2 \(H_{1,1,2}\) \(F_{1,1,2}\)
3 = definitely present 2 1 \(H_{3,2,1}\) \(F_{3,2,1}\)
2 = equivocal 2 1 \(H_{2,2,1}\) \(F_{2,2,1}\)
1 = questionable 2 1 \(H_{1,2,1}\) \(F_{1,2,1}\)
3 = definitely present 2 2 \(H_{3,2,2}\) \(F_{3,2,2}\)
2 = equivocal 2 2 \(H_{2,2,2}\) \(F_{2,2,2}\)
1 = questionable 2 2 \(H_{1,2,2}\) \(F_{1,2,2}\)

This package has example data, for example, the following object in this package is an MRMC dataset:

BayesianFROC::dataList.Chakra.Web

The dictionary for data

R console Definitions
m Modality ID,
q Reader ID,
c Confidence level,
h Number of hits for each reader, confidence level and modality.
f Number of false alarms for each reader, confidence level and modality.
NL Number of Lesions.
NI Number of Images.
M Number of modalities.
Q Number of Readers.

Note the confidence level should be the above order, i.e., 5,4,3,2,1,5,4,3,… and not 1,2,3,4,5,1,2,3,… . If you make by your hand, please be careful !!

This data can be shown more intuitive manner by the following code: Note that what we need to run the function, the above data is only needed to run other procedures. The following code merely show the data.

BayesianFROC:::viewdata(BayesianFROC::dataList.Chakra.Web)

Making your own data

Compatibility for Jafroc:

Note that this dataset is transformed from an Rjafroc dataset made by Chakraborty.

By the function BayesianFROC:convertFromJafroc(), users can transform the Jafroc datasets to this package dataset.

Helpassistance for user’s making data:

Please use BayesianFROC::create_dataset() to make a dataset.

If you have some Jafroc data, then please use BayesianFROC::convertFromJafroc().

In this package, the author use the sheet names and column names in programs, so user have to obey the strict rules. For details, please use help("convertFromJafroc").

Fit =============================

To fit the model, it is same as the case of single reader and single modality as follows:

# I do not why, but Machine cannot find some function in Rcpp. So I have to load the package Rcpp.
library(Rcpp)

# Prepare dataset
dat <- BayesianFROC::dataList.Chakra.Web

#Fitting
fit <-BayesianFROC::fit_Bayesian_FROC(dat)

It is different to the single reader and single modality case user should run the another function DrawCurves_MRMC_pairwise() to draw the curves such as FROC, AFROC, or cumulative hits and cumulative false alarms. Since drawing the curve is very heavy procedures, so the author separate this drawing process from fitting procedures.

Hierarchical Bayesian Model to Compare modalities.

We use \(H_{c,m,r}\) instead of \(H_c\), with \(H_{c,m,r}\) indicating the number of hits by the \(r^{\text{th}}\) reader using \(m^{\text{th}}\) modality with his or her confidence level \(c\), and we will use \(F_{c,m,r}\) similarly. Other quantities are also extended in this manner by adding subscripts \(m,r\), indicating that calculations are performed, respectively, for each \(r^{\text{th}}\) reader and \(m^{\text{th}}\) modality.

In the function BayesianFROC::fit_Bayesian_FROC() of this package for MRMC data, we implement the following statistical models which will be used to explain how the data \((H_{c,m,r},F_{c,m,r})\) arise for number of lesions \(N_L\).

\[H_{c,m,r} \sim \text{Binomial}( p_{c,m,r}, N_L ), \] \[F_{c,m,r} \sim \text{Poisson} ( ( \lambda _{c} - \lambda _{c+1})N_L ), \] \[\lambda _{c} = - \log \Phi (z_{c }), \] \[p_{c,m,r} := \Phi (\frac{z_{c +1}-\mu_{m,r}}{\sigma_{m,r}})-\Phi (\frac{z_{c}-\mu_{m,r}}{\sigma_{m,r}}), \]
\[A_{m,r} := \Phi (\frac{\mu_{m,r}/\sigma_{m,r}}{\sqrt{(1/\sigma_{m,r})^2+1}}), \] \[A_{m,r} \sim \text{Normal} (A_{m},\sigma_{r}^2), \] \[dz_c := z_{c+1}-z_{c}, \] \[dz_c, \sigma_{m,r} \sim \text{Uniform}(0,\infty), \] \[z_{c,m,r} \sim \text{Uniform}( -\infty,100000), \] \[A_{m} \sim \text{Uniform}(0,1). \]

Our model has parameters \(z_{1}, dz_1,dz_2,\cdots, dz_{C}\), \(A_{m}\), \(\sigma_{r}\), \(\mu_{m,r}\), and \(\sigma_{m,r}\).

Drawing the curves ====================


# Prepare a dataset
dat <- BayesianFROC::dataList.Chakra.Web


# Fitting
fit <- BayesianFROC::fit_Bayesian_FROC(dat)





 #Draw curves for the 1st modality and 2nd reader
 DrawCurves(
   
   #This is estimates for dat
        fit,
   
   # Modatity ID whose curves are drawn.
      modalityID =1,
   
   # Reader ID whose curves are drawn.
      readerID   =2)

Note that you need to input your dataset dat and estimates in fit to draw the curves. Of course you should specify reader and modality whose curves you want to write.

To identify reader and modality ID you can use vector, for example, modalityID = c(1,3) for the first modality and the second modalities. readerID = c(1,2) for the first and the second readers.

Questions and Supports

If user has any questions, please tell me.

tsunoda.issei1111

by

gmail.com