Elements of Data Science
SDS 322E

H. Sherry Zhang
Department of Statistics and Data Sciences
University of Texas at Austin

Get the repository for today:
library(usethis)
create_from_github("SDS322E-26FALL/0401-viz3", fork = FALSE)

Learning objectives

Expand your ggplot2 skills:

• Syntax:
  • New geometries for distributions of a continuous variable:
    • basic: geom_boxplot(), geom_density()
    • advance: geom_jitter(), geom_violin(), ggbeeswarm::geom_quasirandom()
    • geom_bar(), geom_col(), geom_histogram()
  • The bins and binwidth argument in geom_histogram()
• Identify and generate next steps when results are unexpected, recognizing that exploratory data analysis is an iterative process (the movie example)

Plots the distribution of a continuous variable

Example: the monthly temperature of the JFK airport in New York City

library(nycflights13)
(jfk_df <- weather |> 
    filter(origin == "JFK") |>
    mutate(month = as.factor(month)) |> 
    select(origin, year, month, day, temp))

# A tibble: 8,706 × 5
   origin  year month   day  temp
   <chr>  <int> <fct> <int> <dbl>
 1 JFK     2013 1         1  39.0
 2 JFK     2013 1         1  39.0
 3 JFK     2013 1         1  39.9
 4 JFK     2013 1         1  39.9
 5 JFK     2013 1         1  39.0
 6 JFK     2013 1         1  37.9
 7 JFK     2013 1         1  39.0
 8 JFK     2013 1         1  39.9
 9 JFK     2013 1         1  39.9
10 JFK     2013 1         1  41  
# ℹ 8,696 more rows

How about geom_point()?

jfk_df |> 
  ggplot(aes(x = month, y = temp)) + 
  geom_point()

• 😢 the points are squashed together, we can’t see the distribution
• 😆 but it does allow us to see there is one particular point with the lowest temperature in May

How about geom_jitter()?

As an alternative to geom_point(), geom_jitter() adds a small amount of random noise to the position of each point, which helps to spread out the points and make them more visible.

jfk_df |> 
  ggplot(aes(x = month, y = temp, group = month)) + 
  geom_jitter(width = 0.2)

• 😭 the points are loosen up, but we can’t see the distribution

How about geom_boxplot()?

jfk_df |> 
  ggplot(aes(x = month, y = temp, group = month)) + 
  geom_boxplot()

• 😢 now we can see the five number summary - better than all the points lining up in one line

How about geom_violin()?

jfk_df |> 
  ggplot(aes(x = month, y = temp, group = month)) +
  geom_violin()

• 🙋 now we can see the distribution and the long whisker in May signals the interesting low temperature in May. Sure…

How about geom_quasirandom()?

jfk_df |> 
  ggplot(aes(x = month, y = temp, group = month)) + 
  ggbeeswarm::geom_quasirandom(size = 0.5)

• 😀 the long whisker in May signals the interesting low temperature in May
• 😄 Now we can see the distribution: most of the days in March is around 40F, but In November, the temperature is bi-modal: part of it clusters are around 40F and another around 50F.

Why are distributions important? (1/4)

I have simulated three sets of observations (100 observations each), dt, and plot them using geom_boxplot().

Does the boxplot tell you anything about the distribution of the data?

dt |>  
  ggplot(aes(x = x, y = value, group = x)) + 
  geom_boxplot()

Is it so?

Why are distributions important? (2/4)

dt |> 
  ggplot(aes(x = x, y = value, group = x)) + 
  geom_violin()

Why are distributions important? (3/4)

dt |> 
  ggplot(aes(x = x, y = value, group = x)) + 
  ggbeeswarm::geom_quasirandom(width = 0.3)

Why are distributions important? (4/4)

dt |> 
  ggplot(aes(x = value, group = x, 
             fill = as.factor(x))) + 
  geom_density() + 
  facet_wrap(vars(x), ncol = 1)

You don’t need to know the following for this class but in case you’re interested in how the data is generated.

set.seed(1234)
dt <- tibble(id = 1: 100, 
             x1 = 1: 100, 
             x2 = rnorm(100, 50, 30), 
             x3 = c(rnorm(50, 25, 10), 
                    rnorm(50, 70, 10))) |> 
  pivot_longer(names_to = "x",
               values_to = "value", 
               cols = -id) |>
  mutate(x = parse_number(x)) |> 
  filter(between(value, 0, 99))

Remark 1: What happen if I don’t have facets?

dt |> 
  ggplot(aes(x = value, group = x, 
             fill = as.factor(x))) + 
  geom_density() 

This doesn’t look very nice - we can’t see the distributions at the back.

What should we do?

dt |> 
  ggplot(aes(x = value, group = x, 
             fill = as.factor(x))) + 
  geom_density(alpha = 0.5) 

A common way to alleviate this issue is to add some transparency to the fill color.

In this case, a faceted plot is better because it shows the three groups clearer.

Remark 2: does adding color make it better?

jfk_df |> 
  ggplot(aes(x = month, y = temp, group = month, color = month)) + 
  ggbeeswarm::geom_quasirandom(size = 0.5)

• 😿 the color doesn’t add any more information because month is already on the x-axis

What’s the issue with the plot on the left?

set.seed(123)
tibble(x = rnorm(10000), 
       y = rnorm(10000)) |> 
  ggplot(aes(x = x , y = y)) + 
  geom_point()

set.seed(123)
tibble(x = rnorm(10000), 
       y = rnorm(10000)) |> 
  ggplot(aes(x = x , y = y)) + 
  geom_point(size = 0.1)

When plotting too many points, we may consider reduce the point size to avoid overplotting.

More on distributions

library(ggplot2movies)
movies

# A tibble: 58,788 × 24
   title     year length budget rating votes    r1    r2    r3    r4    r5    r6
   <chr>    <int>  <int>  <int>  <dbl> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
 1 $         1971    121     NA    6.4   348   4.5   4.5   4.5   4.5  14.5  24.5
 2 $1000 a…  1939     71     NA    6      20   0    14.5   4.5  24.5  14.5  14.5
 3 $21 a D…  1941      7     NA    8.2     5   0     0     0     0     0    24.5
 4 $40,000   1996     70     NA    8.2     6  14.5   0     0     0     0     0  
 5 $50,000…  1975     71     NA    3.4    17  24.5   4.5   0    14.5  14.5   4.5
 6 $pent     2000     91     NA    4.3    45   4.5   4.5   4.5  14.5  14.5  14.5
 7 $windle   2002     93     NA    5.3   200   4.5   0     4.5   4.5  24.5  24.5
 8 '15'      2002     25     NA    6.7    24   4.5   4.5   4.5   4.5   4.5  14.5
 9 '38       1987     97     NA    6.6    18   4.5   4.5   4.5   0     0     0  
10 '49-'17   1917     61     NA    6      51   4.5   0     4.5   4.5   4.5  44.5
# ℹ 58,778 more rows
# ℹ 12 more variables: r7 <dbl>, r8 <dbl>, r9 <dbl>, r10 <dbl>, mpaa <chr>,
#   Action <int>, Animation <int>, Comedy <int>, Drama <int>,
#   Documentary <int>, Romance <int>, Short <int>

We want to have an idea about the distribution of the movie lengths.

On top of a density plot, we could also do a histogram.

More on distributions

movies |> 
  ggplot(aes(x = length)) + 
  geom_histogram()

This is bad - what should we do next?

movies |> filter(length > 2000) |> select(title:votes)

# A tibble: 2 × 6
  title            year length budget rating votes
  <chr>           <int>  <int>  <int>  <dbl> <int>
1 Cure for Insom…  1987   5220     NA    3.8    59
2 Longest Most M…  1970   2880     NA    6.4    15

movies |> 
  filter(length < 2000) |> 
  ggplot(aes(x = length)) + 
  geom_histogram()

This seems equally bad - what next?

More on distributions

movies |> 
  filter(length < 300) |>
  ggplot(aes(x = length)) +
  geom_histogram()

movies |> 
  filter(length < 200) |>
  ggplot(aes(x = length)) +
  geom_histogram()

This is something - What can you see?

There is a peak around 10 mins and another peak around 90 mins.

This dataset include short films about 0-20 mins long and standard films that are on average 90-100 mins.

More on distributions

stat_bin() using bins = 30. Pick better value with binwidth.

It uses 30 bins by default and ask you to consider whether this is appropriate.

movies |> 
  filter(length < 200) |>
  ggplot(aes(x = length)) +
  geom_histogram(bins = 5)

This choice is bad because you can’t see the two peaks anymore.

movies |> 
  filter(length < 200) |>
  ggplot(aes(x = length)) +
  geom_histogram(bins = 50)

This choice is arguably more or less the same as the default choice.

More on distributions

We could also change with the binwidth argument:

movies |> 
  filter(length < 200) |>
  ggplot(aes(x = length)) +
  geom_histogram(binwidth = 10)

Still more or less the same

movies |> 
  filter(length < 200) |>
  ggplot(aes(x = length)) +
  geom_histogram(binwidth = 1)

Wahoo - we see something very different!

These spikes are real and correspond to movies like to round their length to the nearest multiple of 5 or 10 minutes.