Using the multiverse in RMarkdown: Introduction to multiverse code chunks

library(knitr)
library(dplyr)
library(tidyr)
library(purrr)
library(multiverse)

Introduction

In this document we will describe the usage of the multiverse code block. The multiverse code block is an alternative to the R code block which will parse any code run in it directly in the multiverse, and reduces the need for auxiliary functions such as (inside). In addition, the default analysis in the multiverse will be executed in the environment in which the multiverse has been declared, which would usually be the global environment. In that case the results of the default analysis can be inspected directly in an R code block, similar to what an user would do if they were running a single universe analysis.

We demonstrate how the two methods are equivalent, and result in the same multiverse.

In this example, we will use the data which compares different modalities for physical visualizations. See (frequentist-multiverse-analysis) for more details.

data("userlogs")

As with any multiverse analysis, we first need to define the multiverse object. This step is the same for both methods. We will define two separate objects for illustrating the two different ways of adding code to the multiverse for analysis. In the multiverse analysis in this vignette, we will perform a log transformation on the duration variable of the dataset using both the inside function as well as the multiverse code block

M_inside = multiverse()
M_block = multiverse()

Inside() and Multiverse analysis in R scripts

The idea of the inside function is to allow us to write code to be executed within the multiverse and not directly in R, thus allowing us to make use of a flexible syntax for declaring the different possible analysis combinations within the multiverse. The inside function takes two arguments:

the name of the multiverse object
the expression (code) to be passed into the multiverse enclosed within a pair of curly braces, {

The expression passed into the multiverse is not executed directly, allowing us to parse and expand the expression provided by the user has declared using our branch and parameters, into their corresponding analysis combinations.

inside(M_inside, {
    data_transform <- branch(data_transform,
        "log-transformed" ~ log,
        "untransformed" ~ identity
      )
    
    duration <- do.call(data_transform, list(userlogs$duration))
})

Multiverse in RMarkdown

Although the inside function is the only way to add code to the multiverse in an RScript, RMarkdown notebooks allow us the opportunity to use different language engines (not just limited to R). This flexibility also provides an opportunity to write multiverse code directly into a code block, instead of using auxillary functions.

The language associated with a code block is provided by the first argument: ```{r} Here the first argument is r hence the code in the associated block will be executed in R. To convert a code block to execute in multiverse, change the first argument to multiverse. Thus the code block would be: ```{multiverse}

To execute a code block in multiverse, the user needs to provide two additional arguments: label and inside.

The label is a unique identifier for the code block, and each code block in the same document must have a different label. However, the label argument does not need to be explicitly specified (as is the case with an R code block). Therefore, ```{multiverse, label=default-m-1 ...}, ```{multiverse, default-m-1 ...} and ```{multiverse default-m-1 ...} are all equivalent.

The inside argument takes in only multiverse objects, and is used to indicate the multiverse object which will the code inside the code block will be associated with. Thus declaring a multiverse code block would be: ```{multiverse, default-m-1, inside = M}

We provide the ability to declare multiverse code block as an AddIn in RStudio. Users can click on AddIns toolbar menu in RStudio (see the image below). This would create a multiverse code block at the location of the cursor in the document.

Alternately, users can insert a multiverse We also allow users to create a keyboard shortcut to declare a multiverse code block inside a RMarkdown document. This can be done through the following steps:

Tools > AddIns > Browse AddIns… > Keyboard Shortcuts
Next, in the filter input field, type multiverse. You will see one result with “Insert multiverse code chunk” as the name.
Click on the Shortcut field and press Cmd+Option+M (on Mac OS) or Ctrl+Shift+Alt+M (on Windows).
Click “Apply” and exit the dialog box

If you are experiencing issues creating a keyboard shortcut for code blocks, or if you are experiencing a situation where the shortcut needs to be re-created everytime you open a new RStudio session, please refer to the Debugging Keyboard Shortcuts section at the end of this document.

Multiverse code block

The declaration of the chunk below is (this gets hidden when we knit the document): ```{multiverse default-m-1, inside = M_block}

```{multiverse default-m-1, inside = M_block}
data_transform <- branch(data_transform,
    "log-transformed" ~ log,
    "untransformed" ~ identity
  )

duration <- do.call(data_transform, list(userlogs$duration))
```

The result of this code block will be identical to the result using inside(). We first compare whether the multiverse table is generated properly for both the multiverse objects.

expand(M_inside)

## # A tibble: 2 × 6
##   .universe data_transform  .parameter_assignment .code        .results .errors
##       <int> <chr>           <list>                <list>       <list>   <list> 
## 1         1 log-transformed <named list [1]>      <named list> <env>    <lgl>  
## 2         2 untransformed   <named list [1]>      <named list> <env>    <lgl>

expand(M_block)

## # A tibble: 2 × 6
##   .universe data_transform  .parameter_assignment .code        .results .errors
##       <int> <chr>           <list>                <list>       <list>   <list> 
## 1         1 log-transformed <named list [1]>      <named list> <env>    <lgl>  
## 2         2 untransformed   <named list [1]>      <named list> <env>    <lgl>

As you can see above, both the methods yield the same multiverse table. Next, we inspect the .code column of the multiverse object. This column contains the code used to generate each analysis combination in the multiverse. The only differences here arise from how the expressions are stored. The multiverse code block creates a named list, whereas the inside function creates a unnamed list for each row of this column.

Below is the output from the first multiverse object (M_inside), which uses the inside function:

## [[1]]
## [[1]]$`1`
## {
##     data_transform <- log
##     duration <- do.call(data_transform, list(userlogs$duration))
## }
## 
## 
## [[2]]
## [[2]]$`1`
## {
##     data_transform <- identity
##     duration <- do.call(data_transform, list(userlogs$duration))
## }

This is the output from the second multiverse object (M_block), which uses the multiverse code block:

## [[1]]
## [[1]]$`default-m-1`
## {
##     data_transform <- log
##     duration <- do.call(data_transform, list(userlogs$duration))
## }
## 
## 
## [[2]]
## [[2]]$`default-m-1`
## {
##     data_transform <- identity
##     duration <- do.call(data_transform, list(userlogs$duration))
## }

Notes on using multiverse code blocks

In RStudio using RMarkdown

During interactive use with RStudio, multiverse code blocks will work very similarly to a R code block. It will only execute the default analysis (i.e. the analysis path obtained by combining the first option of each parameter) in the global environment. Any output that is generated will have the formatting that output of R code blocks do.

Knitting to HTML using knitr

When a document is knit, currently all analysis paths are executed but the document only shows the analysis path corresponding to the default analysis. We are currently developing interactive features which will allow the author to implement interactivity into the rendered HTML document; this will also allow readers to interact with the analysis and investigate the robustness of the implemented analysis themselves.

Knitting to HTML using knitr

Currently we do not support knitting of RMarkdown documents with multiverse code blocks to PDF documents.

Debugging Keyboard Shortcuts

On OS X Systems

One issue that I have encountered in the past, is that I had to create the keyboard shortcut everytime I opened a new RStudio session. As discussed here, RStudio keybindings are saves as JSON files in the directory ~/.R/rstudio/keybindings/, but this directory was missing for me.

Check if the directory exists for you: 1. Open Terminal 2. Enter: cd ~/.config/rstudio/keybindings

If it returns an error such as cd: no such file or directory, this means that you do not have the directory. The fix is to create this directory which can be done through the following steps:

Open Terminal
Enter mkdir ~/.config/rstudio/keybindings/

If it returns “Permission denied” you might have to run the command as the superuser in the Terminal: sudo mkdir ~/.config/rstudio/keybindings/ This will request you to enter your login password.

Abhraneel Sarma

2024-10-06

Introduction

Inside() and Multiverse analysis in R scripts

Multiverse in RMarkdown

Multiverse code block

Notes on using multiverse code blocks

In RStudio using RMarkdown

Knitting to HTML using knitr

Knitting to HTML using knitr

Debugging Keyboard Shortcuts

On OS X Systems