index.Rmd

---
title: "tidy data and tidy tools"
subtitle: "Overview of tools of effective data analysis"
author: "Eric Leung"
date: "2020-05-17"
output:
  xaringan::moon_reader:
    self_contained: no
    css: xaringan-themer.css
    lib_dir: libs
    nature:
      highlightStyle: github
      highlightLines: true
      countIncrementalSlides: true
---

```{r setup, include=FALSE, warning=FALSE}
options(htmltools.dir.version = FALSE)
knitr::opts_chunk$set(
  message = FALSE,
  echo = TRUE
)

# Load libraries
library(xaringanthemer)

# Set up styling
style_mono_accent(
  base_color = "#242943",
  header_font_google = google_font("Josefin Sans"),
  text_font_google   = google_font("Montserrat", "300", "300i"),
  code_font_google   = google_font("Fira Mono")
)
```

# Overview

- Tidy data

- Core and useful tidyverse functions

- Useful packages


---

# Tidy data overview

![Tidy data](https://d33wubrfki0l68.cloudfront.net/6f1ddb544fc5c69a2478e444ab8112fb0eea23f8/91adc/images/tidy-1.png)

*Source*: https://r4ds.had.co.nz/tidy-data.html

---

# Tidy data are easier to work with inside the tidyverse

> "Tidy datasets are all alike, but every messy dataset is messy in its own
way."
> 
> –– Hadley Wickham

---

# Objective

Take built-in phyloseq object and convert it to a tidy data set. Then wrangle
the data using tidy principles and tidyverse packages<sup>*</sup>.

.footnote[[*] The "tidyverse" packages are a suite of packages that adhere to
tidy data principles]


---

# Load packages

```{r}
library(phyloseq)

# install.packages("remotes")
# remotes::install_github("mikemc/speedyseq")
library(speedyseq)  # Optional
```


---

# phyloseq data example

```{r load_pkgs_data}
# Load data example
data("GlobalPatterns")
GlobalPatterns
```

--

Let's convert this into a tidy data set.

---

# `psmelt()` converts to tidy data

```{r melt_phyloseq}
psm_gp <- psmelt(GlobalPatterns)  # Create tidy data
```

--

```{r load_convert_tibble, warning=FALSE}
library(tibble)
psm_gp <- tibble(psm_gp)  # Make nicer data frame
```

--

```{r explore_melted_df}
head(psm_gp, 5)  # View first five rows
```


---

# Quick note on the tidyverse

- The {tidyverse} package is a suite of packages

- Home website https://www.tidyverse.org/

- Running `library(tidyverse)` will load the main tidyverse packages

- The following code loads the appropriate packages within the tidyverse when
  needed

---

# Let's clean the `Descriptions` column

```{r dplyr_basics, warning=FALSE}
library(dplyr)

psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  head(5)
```


---

# Just keep stool samples

```{r}
psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  filter(SampleType == "Feces") %>%
  head(5)
```

--

The descriptions have lots of information in them that we might want to
visualize later on with ggplot2.

--

Difficult to plot when the data is locked in text form.

**Note**: some of the following code may not be "best" design decision


---

# Load core data wrangling libraries

```{r load_tidyr, warning=FALSE}
library(tidyr)
```

--

```{r}
# Column must be character, `Description` is currently a factor
psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  filter(SampleType == "Feces") %>%
  mutate(Description = as.character(Description)) %>%  #<<
  separate_rows(Description, sep = ", ") %>%           #<<
  head(5)
```


---

# Let's move values into columns

```{r}
# Create `value` column to keep value
psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  filter(SampleType == "Feces") %>%
  mutate(Description = as.character(Description)) %>%
  separate_rows(Description, sep = ", ") %>%
  mutate(value = 1) %>%                  #<<
  pivot_wider(names_from = Description,  #<<
              values_from = value) %>%   #<<
  head(5)
```


---

# We can reverse our pivot

```{r}
psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  filter(SampleType == "Feces") %>%
  mutate(Description = as.character(Description)) %>%
  separate_rows(Description, sep = ", ") %>%
  mutate(value = 1) %>%
  pivot_wider(names_from = Description,
              values_from = value) %>%
  pivot_longer(cols = !c(OTU, Sample, SampleType),  #<<
               names_to = "Description",            #<<
               values_to = "Value") %>%             #<<
  head(5)
```


---

# Future steps

- With the code here, you could further process the data to filter on specific
  features.

- For example, you could focus on stool samples from a specific twins or a whole
  body study.

- The {stringr} package within the tidyverse will prove useful for checking the
  contents of strings

---

# Functions reviewed

- **Note**: the following functions are in the form `<package>::<function>`

- **`dplyr::select()`** = select, subset, and remove columns from data frame

- **`dplyr::filter()`** = remove or keep specific rows based on information in
  data frame

- **`dplyr::mutate()`** = convert or transform column values

- **`tidyr::separate_rows()`** = take column values and separate them into
  multiple rows

- **`tidyr:pivot_wider()`** and **`tidyr::pivot_longer()`** = transform data
  frame from long to wide data


---

# Useful packages

- janitor = miscellaneous cleaning

- unheadr = weird nested headers

- tidycells = multiple tables spaced out in Excel


---

# janitor for cleaning columns and creating frequency tables

```{r show_janitor, warning=FALSE}
library(janitor)

psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  filter(SampleType == "Feces") %>%
  mutate(Description = as.character(Description)) %>%
  separate_rows(Description, sep = ", ") %>%
  clean_names() %>%  #<<
  head(5)
```


---

# janitor for creating frequency tables

```{r show_freq_tabs}
psm_gp %>%
  select(OTU, Sample, SampleType, Description) %>%
  tabyl(SampleType) %>%  #<<
  adorn_totals()         #<<
```


---

# unheadr for nested headers

**Before**

| scientific\_name        | common\_name                 | red\_list\_status | mass\_kg |
| :---------------------- | :--------------------------- | :---------------- | -------: |
| Asia                    | NA                           | NA                |       NA |
| CERCOPITHECIDAE         | NA                           | NA                |       NA |
| Trachypithecus obscurus | Dusky Langur                 | NT                |     7.13 |
| Presbytis sumatra       | Black Sumatran Langur        | EN                |     6.00 |


---

# unheadr for nested headers

**After**

| scientific\_name        | common\_name                 | red\_list\_status | mass\_kg | family          |
| :---------------------- | :--------------------------- | :---------------- | -------: | :-------------- |
| Trachypithecus obscurus | Dusky Langur                 | NT                |     7.13 | CERCOPITHECIDAE |
| Presbytis sumatra       | Black Sumatran Langur        | EN                |     6.00 | CERCOPITHECIDAE |


---

# tidycells for odd cell arrangements

![](https://r-rudra.github.io/tidycells/articles/ext/marks.png)


---

# Resources

- R for Data Science https://r4ds.had.co.nz
- Ted's list of underrated tidyverse functions 
  https://hugo-portfolio-example.netlify.app/projects/tidyverse_functions/
- {janitor} R package https://github.com/sfirke/janitor
- {unheadr} R package https://github.com/luisDVA/unheadr
- {tidycells} https://r-rudra.github.io/tidycells/
- Blog post on cleaning messy data 
  https://rfortherestofus.com/2019/12/how-to-clean-messy-data-in-r/