---
title: "Rraven: Connecting R and Raven Sound Analysis Software"
pagetitle: Rraven, connecting R and Raven Sound Analysis Software
author:
- Marcelo Araya-Salas, PhD
date: "`r Sys.Date()`"
output:
rmarkdown::html_document:
self_contained: yes
toc: true
toc_depth: 3
toc_float:
collapsed: false
smooth_scroll: true
vignette: >
%\VignetteIndexEntry{Rraven: Connecting R and Raven Sound Analysis Software}
%\usepackage[utf8]{inputenc}
%\VignetteEncoding{UTF-8}
%\VignetteDepends{Rraven}
%\VignetteDepends{viridis}
%\VignetteEngine{knitr::rmarkdown}
editor_options:
chunk_output_type: console
params:
EVAL: !r identical(Sys.getenv("NOT_CRAN"), "true")
---
```{r, echo = FALSE, message = FALSE}
# remove all objects
rm(list = ls())
# unload all non-based packages
out <- sapply(paste('package:', names(sessionInfo()$otherPkgs), sep = ""), function(x) try(detach(x, unload = FALSE, character.only = TRUE), silent = T))
#load packages
library(warbleR)
library(Rraven)
library(knitr)
library(kableExtra)
options(knitr.table.format = "html")
opts_chunk$set(comment = "")
opts_knit$set(root.dir = tempdir())
options(width = 150, max.print = 100)
#website to fix gifs
#https://ezgif.com/optimize
```
The `Rraven` package is designed to facilitate the exchange of data between R and [Raven sound analysis software](https://www.ravensoundsoftware.com/) ([Cornell Lab of Ornithology](https://www.birds.cornell.edu/home)). [Raven](https://www.ravensoundsoftware.com/) provides very powerful tools for the analysis of (animal) sounds. R can simplify the automatization of complex routines of analyses. Furthermore, R packages as [warbleR](https://cran.r-project.org/package=warbleR), [seewave](https://cran.r-project.org/package=seewave) and [monitoR](https://cran.r-project.org/package=monitoR) (among others) provide additional methods of analysis, working as a perfect complement for those found in [Raven](https://www.ravensoundsoftware.com/). Hence, bridging these applications can largely expand the bioacoustician's toolkit.
Currently, most analyses in [Raven](https://www.ravensoundsoftware.com/) cannot be run in the background from a command terminal. Thus, most `Rraven` functions are design to simplify the exchange of data between the two programs, and in some cases, export files to [Raven](https://www.ravensoundsoftware.com/) for further analysis. This vignette provides detailed examples for each function in `Rraven`, including both the R code as well as the additional steps in [Raven](https://www.ravensoundsoftware.com/) required to fully accomplished the analyses. Raven Pro must be installed to be able to run some of the code. Note that the animations explaining these additional [Raven](https://www.ravensoundsoftware.com/) steps are shown in more detail in the [github](https://github.com/maRce10/Rraven) version of this vignette, which can be downloaded as follows (saves the file "Rraven.github.html" in your current working directory):
```{r eval = FALSE}
download.file(
url = "https://raw.githubusercontent.com/maRce10/Rraven/master/gifs/Rraven.hitgub.html",
destfile = "Rraven.github.html")
```
The downloaded file can be opened by any internet browser.
Before getting into the functions, the packages must be installed and loaded. I recommend using the latest developmental version, which is found in [github](https://github.com/maRce10/Rraven). To do so, you need the R package [remotes](https://cran.r-project.org/package=remotes) (which of course should be installed!). Some [warbleR](https://cran.r-project.org/package=warbleR) functions and example data sets will be used, so [warbleR](https://cran.r-project.org/package=warbleR) should be installed as well:
```{r, eval = FALSE}
remotes::install_github("maRce10/warbleR")
remotes::install_github("maRce10/Rraven")
#from CRAN would be
#install.packages("warbleR")
#load packages
library(warbleR)
library(Rraven)
```
Let's also use a temporary folder as the working directory in which to save all sound files and data files:
```{r eval= F, echo=T}
setwd(tempdir())
#load example data
data(list = c("Phae.long1", "Phae.long2", "Phae.long3", "Phae.long4", "lbh_selec_table", "selection_files"))
#save sound files in temporary directory
writeWave(Phae.long1, "Phae.long1.wav", extensible = FALSE)
writeWave(Phae.long2, "Phae.long2.wav", extensible = FALSE)
writeWave(Phae.long3, "Phae.long3.wav", extensible = FALSE)
writeWave(Phae.long4, "Phae.long4.wav", extensible = FALSE)
#save Raven selection tables in the temporary directory
out <- lapply(1:4, function(x)
writeLines(selection_files[[x]], con = names(selection_files)[x]))
#this is the temporary directory location (of course different each time is run)
getwd()
```
```{r eval= T, echo=F}
#load example data
data(list = c("Phae.long1", "Phae.long2", "Phae.long3", "Phae.long4", "lbh_selec_table", "selection_files"))
#save sound files in temporary directory
writeWave(Phae.long1, file.path(tempdir(), "Phae.long1.wav"), extensible = FALSE) #save sound files
writeWave(Phae.long2, file.path(tempdir(), "Phae.long2.wav"), extensible = FALSE)
writeWave(Phae.long3, file.path(tempdir(), "Phae.long3.wav"), extensible = FALSE)
writeWave(Phae.long4, file.path(tempdir(), "Phae.long4.wav"), extensible = FALSE)
#save Raven selection tables in temporary directory
out <- lapply(1:4, function(x)
writeLines(selection_files[[x]], con = file.path(tempdir(), names(selection_files)[x])))
#providing the name of the column with the sound file names
# rvn.dat <- imp_raven(sound.file.col = "Begin.File", all.data = FALSE)
#this is the temporary directory location (of course different each time is run)
# getwd()
```
---
## Importing data from Raven
### *imp_raven*
This function imports [Raven](https://www.ravensoundsoftware.com/) selection tables. Multiple files can be imported at once. [Raven](https://www.ravensoundsoftware.com/) selection tables including data from multiple recordings can also be imported. It returns a single data frame with the information contained in the selection files. We already have 2 [Raven](https://www.ravensoundsoftware.com/) selection tables in the working directory:
```{r, eval=T, echo=T}
list.files(path = tempdir(), pattern = "\\.txt$")
```
This code shows how to import all the data contained in those files into R:
```{r, eval=FALSE}
#providing the name of the column with the sound file names
rvn.dat <- imp_raven(all.data = TRUE, path = tempdir())
head(rvn.dat)
```
```{r, eval=TRUE, echo=F, message=F}
#providing the name of the column with the sound file names
rvn.dat <- imp_raven(all.data = TRUE, path = tempdir())
kbl <- kable(head(rvn.dat), align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 11)
scroll_box(kbl, width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
Note that the 'waveform' view data has been removed. It can also be imported as follows (but note that the example selection tables have no waveform data):
```{r, eval=FALSE}
rvn.dat <- imp_raven(all.data = TRUE, waveform = TRUE,
path = tempdir())
```
[Raven](https://www.ravensoundsoftware.com/) selections can also be imported in a 'selection.table' format so it can be directly input into [warbleR](https://cran.r-project.org/package=warbleR) functions. To do this you need to set `warbler.format = TRUE`:
```{r, eval=FALSE}
#providing the name of the column with the sound file names
rvn.dat <- imp_raven(sound.file.col = "End.File",
warbler.format = TRUE, path = tempdir())
head(rvn.dat)
```
```{r, eval=TRUE, echo=FALSE}
#providing the name of the column with the sound file names
rvn.dat <- imp_raven(sound.file.col = "End.File",
warbler.format = TRUE, path = tempdir())
kbl <- kable(head(rvn.dat), align = "c", row.names = F, escape = FALSE)
kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = TRUE, font_size = 12)
# scroll_box(kbl, width = "808",
# box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
The data frame contains the following columns: sound.files, channel, selec, start, end, and selec.file. You can also import the frequency range parameters in the 'selection.table' by setting 'freq.cols' tp `TRUE`. The data frame returned by "imp_raven" (when in the 'warbleR' format) can be input into several [warbleR](https://cran.r-project.org/package=warbleR) functions for further analysis. For instance, the following code runs additional parameter measurements on the imported selections:
```{r, eval=FALSE, echo=TRUE}
# convert to class selection.table
rvn.dat.st <- selection_table(rvn.dat, path = tempdir())
sp <- spectro_analysis(X = rvn.dat, bp = "frange", wl = 150,
pb = FALSE, ovlp = 90, path = tempdir())
head(sp)
```
```{r, eval=TRUE, echo=FALSE}
# convert to class selection.table
rvn.dat.st <- selection_table(rvn.dat)
sp <- spectro_analysis(X = rvn.dat, bp = "frange", wl = 150, pb = FALSE, ovlp = 90, path = tempdir())
kbl <- kable(head(sp), align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 11)
scroll_box(kbl, width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
And this code creates song catalogs:
```{r, eval = FALSE}
# create a color palette
trc <- function(n) terrain.colors(n = n, alpha = 0.3)
# plot catalog
catalog(X = rvn.dat.st[1:9, ], flim = c(1, 10), nrow = 3, ncol = 3,
same.time.scale = TRUE, spec.mar = 1, box = FALSE,
ovlp = 90, parallel = 1, mar = 0.01, wl = 200,
pal = reverse.heat.colors, width = 20,
labels = c("sound.files", "selec"), legend = 1,
tag.pal = list(trc), group.tag = "sound.files", path = tempdir())
```

---
This is just to mention a few analysis that can be implemented in [warbleR](https://cran.r-project.org/package=warbleR).
`Rraven` also contains the function `imp_syrinx` to import selections from Syrinx sound analysis software (although this program is not been maintained any longer).
### *extract_ts*
The function extracts parameters encoded as time series in [Raven](https://www.ravensoundsoftware.com/) selection tables. The resulting data frame can be directly input into functions for time series analysis of acoustic signals as in the [warbleR](https://cran.r-project.org/package=warbleR) function `freq_DTW`. The function needs an R data frame, so the data should have been previously imported using `imp_raven`. This example uses the `selection_file.ts` example data that comes with `Rraven`:
```{r, eval=FALSE}
#remove previous raven data files
unlink(list.files(pattern = "\\.txt$", path = tempdir()))
#save Raven selection table in the temporary directory
writeLines(selection_files[[5]], con = file.path(tempdir(),
names(selection_files)[5]))
rvn.dat <- imp_raven(all.data = TRUE, path = tempdir())
# Peak freq contour dif length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)")
head(fcts)
```
```{r, eval=T, echo=FALSE}
#remove previous raven data files
unlink(list.files(pattern = "\\.txt$", path = tempdir()))
#save Raven selection table in the temporary directory
writeLines(selection_files[[5]], con = file.path(tempdir(), names(selection_files)[5]))
#save Raven selection table in the temporary directory
rvn.dat <- imp_raven(all.data = TRUE, path = tempdir())
# Peak freq contour dif length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)")
kbl <- kable(head(fcts), align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 11)
scroll_box(kbl, width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
Note that these sequences are not all of equal length (one has NAs at the end).
`extract_ts` can also interpolate values so all time series have the same length:
```{r, eval=F, echo=T}
# Peak freq contour equal length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)", equal.length = TRUE)
#look at the last rows wit no NAs
head(fcts)
```
```{r, eval=T, echo = F}
# Peak freq contour equal length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)",
equal.length = TRUE)
kbl <- kable(head(fcts), align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 11)
scroll_box(kbl, width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
And the length of the series can also be specified:
```{r, eval=FALSE}
# Peak freq contour equal length 10 measurements
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)",
equal.length = T, length.out = 10)
head(fcts)
```
```{r, eval=TRUE, echo=FALSE}
# Peak freq contour equal length 10 measurements
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)",
equal.length = T, length.out = 10)
kbl <- kable(head(fcts), align = "c", row.names = F, escape = FALSE)
kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 14)
# scroll_box(kbl, width = "900px",
# box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
The time series data frame can be directly input into the `freq_DTW` [warbleR](https://cran.r-project.org/package=warbleR) function to calculate [Dynamic Time Warping](https://en.wikipedia.org/wiki/Dynamic_time_warping) distances:
```{r, eval=F, echo=T}
freq_DTW(ts.df = fcts, path = tempdir())
```
```{r, eval=T, echo=F}
kbl <- kable(freq_DTW(ts.df = fcts, path = tempdir()), align = "c", row.names = T, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = T, font_size = 12)
# row_spec(0, angle = 0)
scroll_box(kbl, height = "500px", width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
---
### *relabel_colms*
This is a simple function to relabel columns so they match the selection table format used in [warbleR](https://cran.r-project.org/package=warbleR):
```{r, eval = F, echo = T}
#to simplify the example select a subset of the columns
st1 <- rvn.dat[ ,1:7]
#check original column names
st1
```
```{r, eval = T, echo = F}
#to simplify the example select a subset of the columns
st1 <- rvn.dat[ ,1:7]
#check original column names
kbl <- kable(st1, align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 14)
```
```{r, eval = F, echo = T}
# Relabel the basic columns required by warbleR
relabel_colms(st1)
```
```{r, eval = T, echo = F}
rc <- relabel_colms(st1)
#check original column names
kbl <- kable(rc, align = "c", row.names = F, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 14)
```
Additional columns can also be relabeled:
```{r, eval = F, echo = T}
# 2 additional column
relabel_colms(st1, extra.cols.name = "View",
extra.cols.new.name = "Raven view")
```
```{r, eval = T, echo = F}
# plus 2 additional column
rc <- relabel_colms(st1, extra.cols.name = "View",
"Raven view")
kbl <- kable(rc, align = "c", row.names = F, escape = FALSE)
kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE, font_size = 14)
```
---
### *imp_corr_mat*
The function imports the output of a batch correlation routine in [Raven](https://www.ravensoundsoftware.com/). Both the correlation and lag matrices contained in the output '.txt' file are read and both waveform and spectrogram (cross-correlation) correlations can be imported.
This example shows how to input the sound files into [Raven](https://www.ravensoundsoftware.com/) and how to bring the results back to R. First, the selections need to be cut as single sound files for the [Raven](https://www.ravensoundsoftware.com/) correlator to be able to read it. We can do this using the `cut_sels` function from [warbleR](https://cran.r-project.org/package=warbleR):
```{r, eval=F, echo=T}
#create new folder to put cuts
dir.create(file.path(tempdir(), "cuts"))
# add a rowname column to be able to match cuts and selections
lbh_selec_table$rownames <- sprintf("%02d",1:nrow(lbh_selec_table))
# cut files
cut_sels(X = lbh_selec_table, mar = 0.05, path = tempdir(), dest.path =
file.path(tempdir(), "cuts"),
labels = c("rownames", "sound.files", "selec"), pb = FALSE)
#list cuts
list.files(path = file.path(tempdir(), "cuts"))
```
Every selection is in its own sound file (labeled as `paste("rownames", "sound.files", "selec")`). Now open [Raven](https://www.ravensoundsoftware.com/) and run the batch correlator on the 'cuts' folder as follows:

And then import the output file into R:
```{r, eval=F, echo=T}
# Import output (change the name of the file if you used a different one)
xcorr.rav <- imp_corr_mat(file = "BatchCorrOutput.txt", path = tempdir())
```
```{r, eval=T, echo=F}
#save Raven selection table in the temporary directory
writeLines(selection_files[[6]], con = file.path(tempdir(), names(selection_files)[6]))
# Import output (change the name of the file if you used a different one)
xcorr.rav <- imp_corr_mat(file = "BatchCorrOutput.txt", path = tempdir())
```
The function returns a list containing the correlation matrix:
```{r, eval=F}
xcorr.rav$correlation
```
```{r, eval=T, echo=F}
kbl <- kable(xcorr.rav$correlation, align = "c", row.names = T, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = T, font_size = 12)
# row_spec(0, angle = 0)
scroll_box(kbl, height = "500px", width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
and the time lag matrix:
```{r, eval=F}
xcorr.rav$`lag (s)`
```
```{r, eval=T, echo=F}
kbl <- kable(xcorr.rav$`lag (s)`, align = "c", row.names = T, escape = FALSE)
kbl <- kable_styling(kbl, bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = T, font_size = 12)
scroll_box(kbl, height = "500px", width = "808px",
box_css = "border: 1px solid #ddd; padding: 5px; ", extra_css = NULL)
```
This output is ready for stats. For instance, the following code runs a mantel test between cross-correlation (converted to distances) and [warbleR](https://cran.r-project.org/package=warbleR) spectral parameter pairwise dissimilarities:
```{r}
#convert cross-corr to distance
xcorr.rvn <- 1- xcorr.rav$correlation
#sort matrix to match selection table
xcorr.rvn <- xcorr.rvn[order(rownames(xcorr.rvn)), order(colnames(xcorr.rvn))]
#convert it to distance matrix
xcorr.rvn <- as.dist(xcorr.rvn)
# measure acoustic parameters
sp.wrblR <- spectro_analysis(lbh_selec_table, bp = c(1, 11), wl = 150,
pb = FALSE, path = tempdir())
#convert them to distance matrix
dist.sp.wrblR <- dist(sp.wrblR)
vegan::mantel(xcorr.rvn, dist.sp.wrblR)
```
There is actually a good match between the two methods!
---
## Exporting R data to Raven
### *exp_raven*
*exp_raven* saves a selection table in '.txt' format that can be directly opened in Raven. No objects are returned into the R environment. The following code exports a data table from a single sound file:
```{r, eval=FALSE, echo=T}
# Select data for a single sound file
st1 <- lbh_selec_table[lbh_selec_table$sound.files == "Phae.long1.wav", ]
# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE, path = tempdir())
```
If the path to the sound file is provided, the functions exports a 'sound selection table' which can be directly open by [Raven](https://www.ravensoundsoftware.com/) (and which will also open the associated sound file):
```{r, eval=FALSE, echo=T}
# Select data for a single sound file
st1 <- lbh_selec_table[lbh_selec_table$sound.files == "Phae.long1.wav",]
# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE,
sound.file.path = tempdir(), path = tempdir())
```

This is useful to add new selections or even new measurements:

If several sound files are available, users can either export them as a single selection file or as multiple selection files (one for each sound file). This example creates a multiple sound file selection:
```{r, eval=FALSE, echo=T}
exp_raven(X = lbh_selec_table, file.name = "Phaethornis multiple sound files",
sound.file.path = tempdir(), single.file = TRUE, path = tempdir())
```
These type of tables can be opened as a multiple file display in [Raven](https://www.ravensoundsoftware.com/).
---
## Running Raven from R