Augmenting data analysis with interactive web graphics

class: center, middle, inverse, title-slide

# Augmenting data analysis with interactive web graphics
### Carson Sievert
### Slides: <a href="https://talks.cpsievert.me" class="uri">https://talks.cpsievert.me</a> <br /> <br /> <i class="fa  fa-twitter "></i> <a href='https://twitter.com/cpsievert'><span class="citation">@cpsievert</span></a> <br /> <i class="fa  fa-github "></i> <a href='https://github.com/cpsievert'><span class="citation">@cpsievert</span></a> <br /> <i class="fa  fa-envelope "></i> <a href="mailto:cpsievert1@gmail.com">cpsievert1@gmail.com</a> <br /> <i class="fa  fa-globe "></i> <a href="https://cpsievert.me/" class="uri">https://cpsievert.me/</a> <br /> <br /> Slides released under <a href='https://github.com/cpsievert/talks/blob/gh-pages/LICENSE'>Creative Commons</a>

---

class: principles

## About me

* PhD in statistics at Iowa State with Heike Hofmann & Di Cook (Dec 2016) 
  
  * Thesis: [Interfacing R with web technologies](http://lib.dr.iastate.edu/etd/15422/)
  
* CEO of [Sievert Consulting](https://consulting.cpsievert.me/) LLC (Jan 2017)

* Clients: plotly, NOAA, Sandia Labs, O'Reilly
  
* I ❤️ interactive data visualization
  
  * Maintain/author R 📦s: plotly, LDAvis, animint

---
background-image: url(../20171207/workflow.svg)
background-size: contain
class: inverse

# Data science workflow

---
background-image: url(../20171207/workflow1.svg)
background-size: contain
class: inverse

# Expository vis

---
background-image: url(../20171207/workflow2.svg)
background-size: contain
class: inverse

# Exploratory vis

## Problem: analysts juggle many languages (R, JavaScript, python)

---
background-image: url(../20171207/workflow2.svg)
background-size: 400px
background-position: 90% 8%
class: inverse, middle, center

## Interactive graphics *can* augment exploratory analysis, but are only *practical* when we can iterate quickly

---
class: principles

## Interactive graphics augment exploration<sup>1</sup>

* Identify structure that otherwise goes missing ([Tukey 1972](http://stat-graphics.org/movies/prim9.html)).
    
* Search for information quickly without fully specified questions<sup>2</sup> ([Unwin & Hofmann, 2000](https://www.researchgate.net/publication/2425912_GUI_and_Command-line_-_Conflict_or_Synergy))
    * Multiple linked views are the optimal framework for posing queries about data ([Buja, Cook, & Swayne 1996](https://www.jstor.org/stable/1390754))
    
* Diagnose and understand models ([Wickham, Cook, & Hofmann 2015](http://onlinelibrary.wiley.com/doi/10.1002/sam.11271/abstract)).

.footnote[
---

[1]: Statisticians were building (very advanced!) int graphics systems decades ago --   <http://stat-graphics.org/movies/>

[2]: Worried about inference? See visual ([Majumder et al 2013](http://amstat.tandfonline.com/doi/abs/10.1080/01621459.2013.808157?journalCode=uasa20#.Wl01_ZM-dTY)) and post-selection ([Berk et al 2013](https://projecteuclid.org/euclid.aos/1369836961)) inference frameworks.
]

---
class: principles

## Interactive graphics augment exploration

* Identify structure that otherwise goes missing ([Tukey 1972](http://stat-graphics.org/movies/prim9.html)).
    
* <div class="highlight">Search for information quickly</div> without fully specified questions ([Unwin & Hofmann, 2000](https://www.researchgate.net/publication/2425912_GUI_and_Command-line_-_Conflict_or_Synergy))
    * <div class="highlight">[Multiple linked views] are the optimal framework for posing queries about data</div> ([Buja, Cook, & Swayne 1996](https://www.jstor.org/stable/1390754))
    
* Diagnose and understand models ([Wickham, Cook, & Hofmann 2015](http://onlinelibrary.wiley.com/doi/10.1002/sam.11271/abstract)).

---
background-image: url(https://i.imgur.com/Id2nVFG.gif)
background-size: contain

---
class: middle, inverse
background-image: url(https://i.imgur.com/Id2nVFG.gif)
background-size: contain

# Generally useful for comparing within/across panels!

---
class: center, middle

# Should an interactive graphics system be purely graphical (from a user perspective)?

.footnote[
### Over a system where users have to write code?
]

---
class: inverse, center, middle

### From Hofmann & Unwin, [*GUI and Command-line - Conflict or Synergy?*](https://www.researchgate.net/publication/2425912_GUI_and_Command-line_-_Conflict_or_Synergy)

# "Statistical software should be expressive, powerful, fast and reliable, but it should also be intuitive, easy to use, flexible, forgiving and consistent"

---
class: inverse, center, middle

# Statistical software should be <font color='red'>expressive, powerful, fast and reliable</font>, but it should also be <font color='#119dff'>intuitive, easy to use, flexible, forgiving and consistent</font>

.footnote[

### <font color='red'>Programming languages are good at this!</font>

### <font color='#119dff'>Graphical interfaces are good at this!</font>

]

---
background-image: url(https://i.imgur.com/c7NJRa2.gif)
background-size: contain
class: inverse

# The ideal 'power' system has best of both worlds

---
background-image: url(../gifs/swamp.gif)
background-size: contain
class: inverse, center, bottom

## It is all too easy for statistical thinking to be **swamped by programming tasks.** -- Brian D. Ripley

---
### An example with Texas housing prices

```r
library(ggplot2)
library(DT)
datatable(txhousing)
```

---
## Annual variation in price by city

```r
ggplot(txhousing, aes(month, median, group = year)) + geom_line() +
  facet_wrap(~city, ncol = 2)
```

![](index_files/figure-html/unnamed-chunk-3-1.png)

---
### *Quickly* compare within and across cities

```r
*tx <- SharedData$new(txhousing, ~year)
p <- ggplot(tx, aes(month, median, group = year)) + geom_line() +
  facet_wrap(~city, ncol = 2)
highlight(ggplotly(p), dynamic = TRUE, selectize = TRUE)
```

---
### Share (default) selections

```r
highlight(ggplotly(p), defaultValues = 2006)
```

---
class: middle, center, inverse

# Beyond trellis (i.e. facet) displays

---
### Query missing values by city

---
class: bottom, left 
background-image: url(../20171207/pipeline.svg)
background-size: contain

## The 'data pipeline'

---
background-image: url(../20171207/server-client.svg)
background-size: contain
class: middle, right

# Where does the pipeline reside?  &nbsp;

---
background-image: url(../20171207/server-client-dim.svg)
background-size: contain
class: middle, right

# No server required! &nbsp;

---
background-image: url(../20171207/server-client-dim.svg)
background-size: contain
class: middle, right

# Easier to share, scale, & maintain &nbsp;

---
### Works with 'aggregated' traces

---
class: middle, principles

## The implementation

```r
d <- SharedData$new(mpg)
dots <- plot_ly(d, color = ~class, x = ~displ, y = ~cyl)
*boxs <- plot_ly(d, color = ~class, x = ~class, y = ~cty) %>% add_boxplot()
*bars <- plot_ly(d, x = ~class, color = ~class)

subplot(dots, boxs) %>%
  subplot(bars, nrows = 2) %>%
  layout(barmode = "overlay") %>%
  highlight("plotly_selected")
```

plotly.js dynamically recomputes summary stats as a function of selection

---
class: principles

## Interactive graphics augment exploration!

* Identify structure that otherwise goes missing ([Tukey 1972](http://stat-graphics.org/movies/prim9.html)).

* Search for information quickly without fully specified questions ([Unwin & Hofmann, 2000](https://www.researchgate.net/publication/2425912_GUI_and_Command-line_-_Conflict_or_Synergy))
    * Multiple linked views are the optimal framework for posing queries about data ([Buja, Cook, & Swayne 1996](https://www.jstor.org/stable/1390754))
    
    
* <div class="highlight">Diagnose and understand models</div> ([Wickham, Cook, & Hofmann 2015](http://onlinelibrary.wiley.com/doi/10.1002/sam.11271/abstract)).

---
### See relationships evolve over time ([made via ggplot2](https://plotly-book.cpsievert.me/linking-animated-views.html))

---
background-image: url(../20170803/tour.gif)
background-size: contain
class: middle

### Interactively [plot models in data space](../20180118/tour.html) ([code](https://github.com/ropensci/plotly/blob/master/demo/animation-tour-USArrests.R))

---
class: inverse, middle, center

# Works both _without_ and _with_ shiny!

---
background-image: url(https://i.imgur.com/MGbR5AI.gif)
background-size: contain
class: principles

<div align="right">
<a href='https://github.com/cpsievert/bcviz'>https://github.com/cpsievert/bcviz</a>
</div>

---
background-image: url(https://i.imgur.com/T7GSpv9.gif)
background-size: contain
class: principles

<https://github.com/cpsievert/zikar>

---
background-image: url(https://i.imgur.com/csIUJX0.gif)
background-size: contain
class: principles

<https://github.com/cpsievert/apps>

---
## Summary

### Interactive graphics *can* augment exploratory analysis, but are only *practical* when we can iterate quickly

### Quickly pose queries about data and make comparisons with __plotly__ + __crosstalk__

---
class: middle, center, principles

## Thanks! Questions?

Slides: <https://talks.cpsievert.me>

Learn more: <https://plotly-book.cpsievert.me>

#### Contact

<i class="fa  fa-twitter "></i> <a href='https://twitter.com/cpsievert'>@cpsievert</a> <br />
<i class="fa  fa-github "></i> <a href='https://github.com/cpsievert'>@cpsievert</a> <br />
<i class="fa  fa-envelope "></i> <cpsievert1@gmail.com> <br />
<i class="fa  fa-globe "></i> <https://cpsievert.me/>