redundant memorandum

Discrete and Continuous rating is essentially same.

Radiograph

Radiograph

Recall that the format of FROC data is the following table:

Confidence Level No. of Hits No. of False alarms
5 = definitely present \(H_{5}\) \(F_{5}\)
4 = probably present \(H_{4}\) \(F_{4}\)
3 = equivocal \(H_{3}\) \(F_{3}\)
2 = probably absent \(H_{2}\) \(F_{2}\)
1 = questionable \(H_{1}\) \(F_{1}\)

Here, we use the confidence level as predictor and if we use some bio-marker instead of the above discrete rating and if bio marker is not discrete, then it corresponds continuous rating. Here, bio marker means, e.g., TSH/EULIA, FT4/EULIA, FT3/EULIA, T4/EULIA, T3, Baso, Eosino, Neutro, Stab, Seg, Lympho, Mono, MCV, MCHC, TP, A/G, TTT, ZTT, AST(GOT), ALT(GPT), gamma-GT, LD, LAP, CK, L/H, Na, K, CL, Ca, IP, Mg, Fe, CRP, RF, ASO, HbA1c(NGSP), RAST, IgE, …, etc.

But, sequential rating reduce to the above table by dividing region such that bio-marker fall into some partition. Thus we may assume that the FROC data has the above format and it is independent whether rating is discrete or continuous.

Rating (instead of Confidence Level) No. of Hits No. of False alarms
0.8< biomarker < 1.0 \(H_{5}\) \(F_{5}\)
0.6< biomarker < 0.8 \(H_{4}\) \(F_{4}\)
0.4< biomarker < 0.6 \(H_{3}\) \(F_{3}\)
0.2< biomarker < 0.4 \(H_{2}\) \(F_{2}\)
0.0< biomarker < 0.2 \(H_{1}\) \(F_{1}\)

Good biomaker has coerce data such that the hits and false alarms has the following monotonicity conditions (not nesessay to satisfy presicely);

\[ H_1 < H_2 < H_3 < H_4 < H_5 \\ F_1 > F_2 > F_3 > F_4 > F_5 \\ \]

If data does not have these monotonicity, then the model fitting will be not better.

Another criterion whether bio marker is useful is that if the estimated AUC is far from 1/2 then the bio-marker is more important. Note that if bio-marker is less than 1/2, then by taking inverse order, we will obtain the bio-marker which is greater than 1/2. Thus bio-marker is significantly important if and only if the absolute value of (AUC - 1/2) is more greater.

So, in FROC context, the rating is not required the confidence level, but for simplicity and confidence level is always significant, thus we use in the general theory.

So, the word “rating” is more general than “confidence level”.

Note that confidence level cannot be said good predictor, since each reader has their trend to take what confidence level he or she use under the FROC trial. So, some reader use confidence level only the lowest and highest one, and such usage of confidence rating mislead our model or estimates. So, to avoid such bias caused reader, we sometimes recommend to use bio marker instead confidence level for the rating.

Anyway the most important thing is that rating is a more general notion than the confidence level.

If there are several bio markers, then we need to reduce them into one dimensional characteristic such as linear combinations of them and this reduced characteristic, we can examine their effect to the observer performance, but …

Future research direction

In the ROC context, comparison issues are solved by Analysis of Variance (ANOVA), and thus, we have to consider the alternative ANOVA in Bayesian sense. I have to develop this methodology for FROC context.

Since our models provide only comparison of pairwise. I think it is not so hard, since nowadays many tools about Bayesian is given, so we immediately get such alternative methods.

I am tired to hit the key board, my hand ache is …

I’m not sure whether my explanation is successful …