Analysis of the after images experiment data using the Bayesian color model

Jure Demšar, Erik Štrumbelj and Grega Repovš

2019-12-13

Introduction

In the afterimages task participants were asked to fix their gaze on a fixation point in the middle of the computer screen. Stimulus – a colored rectangle – was then shown above the fixation point. After 20 seconds the rectangle disappeared and a color palette was shown on the right-hand side of the screen. Participants were asked to keep their gaze on the fixation point while using the mouse to select the color that best matches the color of the afterimage that appeared above the fixation point. Then a colored rectangle of the selected color and same size as before was shown below the fixation point. Finally, participants confirmed their selection. For each trial the color of the stimulus rectangle, the response in RGB and the response time were recorded. The goal of this study was to determine which of the two color coding mechanisms (trichromatic or opponent-process), better explains the color of the afterimages. We used six differently colored rectangles: red, green, blue, cyan, magenta, yellow.

We start our analysis by loading the experiment and stimuli data. The experiment data include subject index, reaction time, response in RGB format, stimuli name (e.g blue) and stimuli values in RGB and HSV. The stimuli data set includes only the information about stimuli (names, RGB and HSV values).

# libs
library(bayes4psy)
library(cowplot)
library(dplyr)
library(ggplot2)

# load data
data_all <- after_images

# load stimuli
stimuli <- after_images_stimuli

Once we load required libraries and thata we can start fitting the Bayesian color models. Below is a detailed example of fitting the Bayesian color model for red color stimuli. Note here that, due to vignette limitations, all fits are built using only two chains, using more chains in parallel is usually more efficient.

# prepare data
data_red <- data_all %>% filter(stimuli == "red")
data_red <- data.frame(r=data_red$r,
                       g=data_red$g,
                       b=data_red$b)

# fit
fit_red <- b_color(colors=data_red, chains=2)

# inspect
plot_trace(fit_red)

print(fit_red)
## Inference for Stan model: color.
## 2 chains, each with iter=2000; warmup=1000; thin=1; 
## post-warmup draws per chain=1000, total post-warmup draws=2000.
## 
##             mean se_mean   sd     2.5%      25%      50%      75%    97.5% n_eff   Rhat
## mu_r       60.41    0.06 4.07    52.53    57.55    60.43    63.17    68.35  5092   1.00
## sigma_r    58.30    0.04 2.93    52.90    56.21    58.12    60.21    64.39  4327   1.00
## mu_g      164.16    0.04 3.06   158.20   162.11   164.17   166.23   170.34  5655   1.00
## sigma_g    44.46    0.03 2.05    40.65    43.04    44.38    45.77    48.71  4700   1.00
## mu_b      235.87    0.05 3.00   229.76   234.00   235.95   237.87   241.58  3231   1.00
## sigma_b    45.27    0.04 2.22    41.30    43.72    45.13    46.72    49.83  3505   1.00
## mu_h        0.45    3.14 3.14    -2.75    -2.69     0.46     3.59     3.64     1 113.37
## kappa_h     5.22    0.01 0.48     4.34     4.90     5.20     5.53     6.23  4044   1.00
## mu_s        0.78    0.00 0.01     0.76     0.78     0.78     0.79     0.81  4507   1.00
## sigma_s     0.21    0.00 0.01     0.19     0.20     0.21     0.21     0.23  4617   1.00
## mu_v        0.95    0.00 0.01     0.94     0.95     0.95     0.96     0.97  3059   1.00
## sigma_v     0.10    0.00 0.01     0.09     0.10     0.10     0.11     0.12  5658   1.00
## lp__    -1940.94    0.09 2.45 -1946.47 -1942.43 -1940.73 -1939.13 -1936.96   791   1.00
## 
## Samples were drawn using NUTS(diag_e) at Fri Dec 13 21:43:15 2019.
## For each parameter, n_eff is a crude measure of effective sample size,
## and Rhat is the potential scale reduction factor on split chains (at 
## convergence, Rhat=1).
plot_hsv(fit_red)

The function plot_hsv was developed specially for the color model. Input data points are visualized with circles, mean of the fit is visualized with a solid line and the 95% HDI of the underlying distribution is visualized as a colored band. If we are satisfied with the fit we repeat the whole process five more times for the remaining five colors of stimuli. For brevity only code for fitting the model is executed here, the inspections of fits are ommited.

# green
data_green <- data_all %>% filter(stimuli == "green")
data_green <- data.frame(r=data_green$r,
                         g=data_green$g,
                         b=data_green$b)
fit_green <- b_color(colors=data_green, chains=2)

# blue
data_blue <- data_all %>% filter(stimuli == "blue")
data_blue <- data.frame(r=data_blue$r,
                        g=data_blue$g,
                        b=data_blue$b)
fit_blue <- b_color(colors=data_blue, chains=2)

# yellow
data_yellow <- data_all %>% filter(stimuli == "yellow")
data_yellow <- data.frame(r=data_yellow$r,
                          g=data_yellow$g,
                          b=data_yellow$b)
fit_yellow <- b_color(colors=data_yellow, chains=2)

# cyan
data_cyan <- data_all %>% filter(stimuli == "cyan")
data_cyan <- data.frame(r=data_cyan$r,
                        g=data_cyan$g,
                        b=data_cyan$b)
fit_cyan <- b_color(colors=data_cyan, chains=2)

# magenta
data_magenta <- data_all %>% filter(stimuli == "magenta")
data_magenta <- data.frame(r=data_magenta$r,
                           g=data_magenta$g,
                           b=data_magenta$b)
fit_magenta <- b_color(colors=data_magenta, chains=2)

We start the analysis by loading data about the colors predicted by the trichromatic or the opponent-process theory. We can then use the plot_distributions_hsv function of the Bayesian color model to produce a visualization of the accuracy of both color coding mechanisms predictions for each stimuli independently. Each graph visualizes the fitted distribution, displayed stimuli and responses predicted by the trichromatic and opponent-process coding. This additional information can be added to the visualization via annotation points and lines.

# load theory predictions
trichromatic <- after_images_trichromatic
opponent_process <- after_images_opponent_process

# red
stimulus <- "red"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)
    
points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)
    
plot_red <- plot_distributions_hsv(fit_red, points=points,
                                   lines=lines, hsv=TRUE)

plot_red <- plot_red + ggtitle("Red") +
  theme(plot.title = element_text(hjust = 0.5))

# green
stimulus <- "green"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)

points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)

plot_green <- plot_distributions_hsv(fit_green, points=points,
                                     lines=lines, hsv=TRUE)
plot_green <- plot_green + ggtitle("Green") +
  theme(plot.title = element_text(hjust = 0.5))

# blue
stimulus <- "blue"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)

points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)

plot_blue <- plot_distributions_hsv(fit_blue, points=points,
                                    lines=lines, hsv=TRUE)
plot_blue <- plot_blue + ggtitle("Blue") +
  theme(plot.title = element_text(hjust = 0.5))

# yellow
stimulus <- "yellow"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)

points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)

plot_yellow <- plot_distributions_hsv(fit_yellow, points=points,
                                      lines=lines, hsv=TRUE)
plot_yellow <- plot_yellow + ggtitle("Yellow") +
  theme(plot.title = element_text(hjust = 0.5))


# cyan
stimulus <- "cyan"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)

points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)

plot_cyan <- plot_distributions_hsv(fit_cyan, points=points,
                                    lines=lines, hsv=TRUE)
plot_cyan <- plot_cyan + ggtitle("Cyan") +
  theme(plot.title = element_text(hjust = 0.5))


# magenta
stimulus <- "magenta"
lines <- list()
lines[[1]] <- c(trichromatic[trichromatic$stimuli == stimulus, ]$h,
                trichromatic[trichromatic$stimuli == stimulus, ]$s,
                trichromatic[trichromatic$stimuli == stimulus, ]$v)
lines[[2]] <- c(opponent_process[opponent_process$stimuli == stimulus, ]$h,
                opponent_process[opponent_process$stimuli == stimulus, ]$s,
                opponent_process[opponent_process$stimuli == stimulus, ]$v)

points <- list()
points[[1]] <- c(stimuli[stimuli$stimuli == stimulus, ]$h_s,
                 stimuli[stimuli$stimuli == stimulus, ]$s_s,
                 stimuli[stimuli$stimuli == stimulus, ]$v_s)

plot_magenta <- plot_distributions_hsv(fit_magenta, points=points,
                                       lines=lines, hsv=TRUE)
plot_magenta <- plot_magenta + ggtitle("Magenta") +
  theme(plot.title = element_text(hjust = 0.5))

We can then use the cowplot library to combine the plots into a single figure.

plot_grid(plot_red, plot_green, plot_blue,
          plot_yellow, plot_cyan, plot_magenta,
          ncol=3, nrow=2, scale=0.9)

This last figure features the comparison of predictions between the thrichromatic and the oponent-process color coding. The long solid line visualizes the trichromatic color coding prediction while the dashed line visualizes the opponent-process color coding. Short solid line represents the mean hue of the fit and the the colored band the 95% HDI of the distribution underlying the fit. The small colored circle visualizes the color of the presented stimuli. In the case of blue and yellow stimuli the dashed line is not visible because both color codings predict the same outcome. The prediction based on the thrichromatic color coding seems more accurate as its prediction is always inside the 95% of the most probable subject’s responses and is always closer to the mean predicted hue than the opponent-process prediction. The opponent-process prediction is outside of the 95% of the most probable subject’s responses in cases of red and green stimuli.