Elements of Data Science
SDS 322E

H. Sherry Zhang
Department of Statistics and Data Sciences
University of Texas at Austin

Get the repository for today:
library(usethis)
create_from_github("SDS322E-26FALL/0402-viz4", fork = FALSE)

Assessment heads-up

When completing your assessment, remember to use line breaks on your code. For example,

gapminder |>
  ggplot(aes(x = year, y = lifeExp)) +
  geom_line(aes(group = country)) +
  geom_point(data = gapminder_2007, color = "red", size = 3)

is better than

gapminder |>
  ggplot(aes(x = year, y = lifeExp)) + geom_line(aes(group = country)) + geom_point(data = gapminder_2007, color = "red", size = 3)

Make sure when you knit a pdf, the code is not cut off. You will lose marks if we can’t see your code.

There are only 8 questions in your homework 1. Gradescope may ask you to crop for Question 9, but we don’t have a Question 9. Just ignore it.

Learning objectives

Syntax:

New geometry: geom_bar() and geom_col()
The position argument in geom_bar(): fill and dodge
compare a data.frame vs. a tibble
use the forcats package to manipulate factor variables
- as_factor(), fct_reorder(), and fct_recode()
change legend title in the plot with labs()

Which one is better: pie chart or bar chart?

Pie chart is almost never a good idea, because it is hard to compare the size of the slices e.g. A vs. C

General Social Survey data

It is a small subset of the questions from the 2016 General Social Survey, or GSS. The GSS is a long-running survey of American adults that asks about a range of topics of interest to social scientists.

# the package associated with the textbook 
# Data Visualization A practical introduction by Kieran Healy
library(socviz) 
gss_sm

# A tibble: 2,867 × 32
   year    id ballot   age childs sibs  degree race  sex   region income16 relig
  <dbl> <dbl> <labe> <dbl>  <dbl> <lab> <fct>  <fct> <fct> <fct>  <fct>    <fct>
1  2016     1 1         47      3 2     Bache… White Male  New E… $170000… None 
2  2016     2 2         61      0 3     High … White Male  New E… $50000 … None 
3  2016     3 3         72      2 3     Bache… White Male  New E… $75000 … Cath…
4  2016     4 1         43      4 3     High … White Fema… New E… $170000… Cath…
# ℹ 2,863 more rows
# ℹ 20 more variables: marital <fct>, padeg <fct>, madeg <fct>, partyid <fct>,
#   polviews <fct>, happy <fct>, partners <fct>, grass <fct>, zodiac <fct>,
#   pres12 <labelled>, wtssall <dbl>, income_rc <fct>, agegrp <fct>,
#   ageq <fct>, siblings <fct>, kids <fct>, religion <fct>, bigregion <fct>,
#   partners_rc <fct>, obama <dbl>

Display count

ggplot(data = gss_sm, mapping = aes(x = bigregion)) +
  geom_bar()

Your time (1/2)

create_from_github("SDS322E-26FALL/0402-viz4", fork = FALSE)

We have created the plot that shows the count and proportion of bigregion:

ggplot(data = gss_sm, mapping = aes(x = bigregion)) +
    geom_bar()

There is also a geometry called geom_col() that does something similar. Read the example in the documentation with ?geom_col() and create the plot above using geom_col(). Which geom would you prefer?

Solution

geom_col() requires you to pre-compute the count or proportion:

gss_cnt <- gss_sm |> count(bigregion) |> mutate(prop = n / sum(n))
gss_cnt

# A tibble: 4 × 3
  bigregion     n  prop
  <fct>     <int> <dbl>
1 Northeast   488 0.170
2 Midwest     695 0.242
3 South      1052 0.367
4 West        632 0.220

ggplot(gss_cnt, aes(x = bigregion, y = n)) + geom_col()

When there are two categorical variables

Show the count of (big)region and religion through the fill aesthetic

gss_sm |> 
  ggplot(aes(x = bigregion, fill = religion)) + 
  geom_bar()

Good or bad?

Again the “pie chart issue” - it is difficult to compare middle category, e.g. Catholic.

When there are two categorical variables

We can use position = "fill" to expand the bars to [0, 1]

gss_sm |> 
  ggplot(aes(x = bigregion, fill = religion)) + 
  geom_bar(position = "fill")

When there are two categorical variables

We can use position = "dodge" to make the bars within each group side-by-side:

gss_sm |> 
  ggplot(aes(x = bigregion, fill = religion)) + 
  geom_bar(position = "dodge")

In this plot, it is easy to compare for each (big)region, the count of each religion category.

Would you say it is easy to compare the same religion across different (big)regions?

When there are two categorical variables

To observe the change of count, our eyes need to trace the bars across regions.

gss_sm |> 
  ggplot(aes(x = bigregion, fill = religion)) + 
  geom_bar(position = "dodge")

A better display would be to have each religion as a group and color by region

gss_sm |> 
  ggplot(aes(x = religion, fill = bigregion)) + 
  geom_bar(position = "dodge")

When you arrange the aesthetics differently, it tells different stories.

When there are two categorical variables

This point is probably easier to justify in facets

To observe differences of religion within each (big)region group

gss_sm |> 
  ggplot(aes(x = religion)) + 
  geom_bar() + 
  facet_wrap(vars(bigregion), ncol = 1)

To observe differences of region within each religion group

gss_sm |>
  ggplot(aes(x = bigregion)) + 
  geom_bar() + 
  facet_wrap(vars(religion), ncol = 2)

Data frame vs. tibble

Data frame:

mtcars |> head(5)

                   mpg cyl disp  hp drat    wt  qsec vs am gear carb
Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2

Tibble:

as_tibble(mtcars)

# A tibble: 32 × 11
    mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
  <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1  21       6   160   110  3.9   2.62  16.5     0     1     4     4
2  21       6   160   110  3.9   2.88  17.0     0     1     4     4
3  22.8     4   108    93  3.85  2.32  18.6     1     1     4     1
4  21.4     6   258   110  3.08  3.22  19.4     1     0     3     1
5  18.7     8   360   175  3.15  3.44  17.0     0     0     3     2
# ℹ 27 more rows

A few things that are inconvenient with a data frame:

A data frame will print all the rows, while a tibble only prints the first 10 rows. You will need to scroll all the way up to see the column names in a data frame.
A tibble has a few prints that make it easier to know your data, e.g. 1) data dimension: 32 x 11, 2) variable type: <dbl>

Convert a data frame to a tibble

For some historical reasons, a data frame allows you to specify a rowname:

mtcars |> head(3)

               mpg cyl disp  hp drat    wt  qsec vs am gear carb
Mazda RX4     21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
Mazda RX4 Wag 21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
Datsun 710    22.8   4  108  93 3.85 2.320 18.61  1  1    4    1

There is a convenient function to make it as a variable in a tibble:

rownames_to_column(mtcars, var = "model") |> head(3)

          model  mpg cyl disp  hp drat    wt  qsec vs am gear carb
1     Mazda RX4 21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
2 Mazda RX4 Wag 21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
3    Datsun 710 22.8   4  108  93 3.85 2.320 18.61  1  1    4    1

Then convert it to a tibble:

rownames_to_column(mtcars, var = "model") |> as_tibble() |> head(3)

# A tibble: 3 × 12
  model          mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
  <chr>        <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 Mazda RX4     21       6   160   110  3.9   2.62  16.5     0     1     4     4
2 Mazda RX4 W…  21       6   160   110  3.9   2.88  17.0     0     1     4     4
3 Datsun 710    22.8     4   108    93  3.85  2.32  18.6     1     1     4     1

Anything wrong with the color in this plot?

mtcars: mpg vs. disp colored by cyl

ggplot(mtcars, aes(x = mpg, y = disp)) + 
  geom_point(aes(color = cyl))

The variable cyl only has three values (4, 6, 8) but it is mapped to a continuous scale.

Factor basics

my_fct <- c("apple", "banana", "orange", "apple")
# create a factor from a vector
as.factor(my_fct)

[1] apple  banana orange apple 
Levels: apple banana orange

We can do technically do the same with numbers (integers):

my_fct2 <- c(4, 6, 8, 4)
as.factor(my_fct2)

[1] 4 6 8 4
Levels: 4 6 8

Change a variable to a factor in a tibble

Can you spot the difference before and after?

as_tibble(mtcars)

# A tibble: 32 × 11
     mpg   cyl  disp    hp  drat    wt  qsec    vs
   <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
 1  21       6  160    110  3.9   2.62  16.5     0
 2  21       6  160    110  3.9   2.88  17.0     0
 3  22.8     4  108     93  3.85  2.32  18.6     1
 4  21.4     6  258    110  3.08  3.22  19.4     1
 5  18.7     8  360    175  3.15  3.44  17.0     0
 6  18.1     6  225    105  2.76  3.46  20.2     1
 7  14.3     8  360    245  3.21  3.57  15.8     0
 8  24.4     4  147.    62  3.69  3.19  20       1
 9  22.8     4  141.    95  3.92  3.15  22.9     1
10  19.2     6  168.   123  3.92  3.44  18.3     1
# ℹ 22 more rows
# ℹ 3 more variables: am <dbl>, gear <dbl>,
#   carb <dbl>

as_tibble(mtcars) |> 
  mutate(cyl = as.factor(cyl))

# A tibble: 32 × 11
     mpg cyl    disp    hp  drat    wt  qsec    vs
   <dbl> <fct> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
 1  21   6      160    110  3.9   2.62  16.5     0
 2  21   6      160    110  3.9   2.88  17.0     0
 3  22.8 4      108     93  3.85  2.32  18.6     1
 4  21.4 6      258    110  3.08  3.22  19.4     1
 5  18.7 8      360    175  3.15  3.44  17.0     0
 6  18.1 6      225    105  2.76  3.46  20.2     1
 7  14.3 8      360    245  3.21  3.57  15.8     0
 8  24.4 4      147.    62  3.69  3.19  20       1
 9  22.8 4      141.    95  3.92  3.15  22.9     1
10  19.2 6      168.   123  3.92  3.44  18.3     1
# ℹ 22 more rows
# ℹ 3 more variables: am <dbl>, gear <dbl>,
#   carb <dbl>

<dbl> means double - the variable is a continuous variable <fct> means factor - the variable is discrete/ a factor variable

Factor example 1: change a variable to a factor

The most pedantic way

mtcars2 <- mtcars |> 
  mutate(cyl = as.factor(cyl))

ggplot(mtcars2, 
       aes(x = mpg, y = disp, color = cyl)) + 
  geom_point(size = 5)

Often, people just do

ggplot(mtcars, aes(x = mpg, y = disp)) + 
  geom_point(aes(color = as.factor(cyl)), size = 5)

This is legit because ggplot2 allows you to input an “expression” of the variable (not just the variable itself).

Change the legend name

The legend title is now ugly - we can change it with labs():

ggplot(mtcars, aes(x = mpg, y = disp)) + 
  geom_point(aes(color = as.factor(cyl)), size = 5) + 
  labs(color = "cylinder")

You can also change the legend title using scale_color_brewer():

ggplot(mtcars, aes(x = mpg, y = disp)) + 
  geom_point(aes(color = as.factor(cyl)), size = 5) + 
  scale_color_brewer(palette = "Dark2", 
                     name = "cylinder")

Factor example 2: reorder factor levels

mtcars_tbl <- rownames_to_column(
  mtcars, var = "model") |> 
  as_tibble() |> 
  mutate(cyl = as.factor(cyl))
mtcars_tbl

# A tibble: 32 × 12
   model   mpg cyl    disp    hp  drat    wt  qsec
   <chr> <dbl> <fct> <dbl> <dbl> <dbl> <dbl> <dbl>
 1 Mazd…  21   6      160    110  3.9   2.62  16.5
 2 Mazd…  21   6      160    110  3.9   2.88  17.0
 3 Dats…  22.8 4      108     93  3.85  2.32  18.6
 4 Horn…  21.4 6      258    110  3.08  3.22  19.4
 5 Horn…  18.7 8      360    175  3.15  3.44  17.0
 6 Vali…  18.1 6      225    105  2.76  3.46  20.2
 7 Dust…  14.3 8      360    245  3.21  3.57  15.8
 8 Merc…  24.4 4      147.    62  3.69  3.19  20  
 9 Merc…  22.8 4      141.    95  3.92  3.15  22.9
10 Merc…  19.2 6      168.   123  3.92  3.44  18.3
# ℹ 22 more rows
# ℹ 4 more variables: vs <dbl>, am <dbl>,
#   gear <dbl>, carb <dbl>

mtcars_tbl |> 
  ggplot(aes(x = disp, y = model, fill = cyl)) + 
  geom_col() + 
  scale_fill_brewer(palette = "Dark2")

We would like the variable model to be ordered according to disp.

Factor example 2: reorder factor levels

df <- tibble(group = c("A", "B", "C"),
             values = c(3, 1, 2))
df

# A tibble: 3 × 2
  group values
  <chr>  <dbl>
1 A          3
2 B          1
3 C          2

Two main arguments:

.f: the factor variable you want to reorder
x: the variable you want to order by

res <- df |> 
  mutate(group = fct_reorder(group, values))
res

# A tibble: 3 × 2
  group values
  <fct>  <dbl>
1 A          3
2 B          1
3 C          2

Now we see <fct> instead of <chr> for group.

By default .desc = FALSE

not in descending order, in ascending order: B - C - A (from small to large)

res$group

[1] A B C
Levels: B C A

Factor example 2: reorder factor levels

df2 <- tibble(
  group = c(rep("A", 3), rep("B", 3), rep("C", 3)),
  values = c(c(3, 4, 5), c(1, 6, 7), c(2, 5, 8))
  )
df2

# A tibble: 9 × 2
  group values
  <chr>  <dbl>
1 A          3
2 A          4
3 A          5
4 B          1
5 B          6
6 B          7
7 C          2
8 C          5
9 C          8

Argument .fun = median means we order by the median of values for each group.

The medians are A: 4, B: 6, C: 5

The order (from small to large) is A - C - B.

res2 <- df2 |> 
  mutate(group = fct_reorder(group, values))

res2$group

[1] A A A B B B C C C
Levels: A C B

Previously we only have one value for each group, so the order is the same as the value itself.

Factor example 2: reorder factor levels

df2 <- tibble(
  group = c(rep("A", 3), rep("B", 3), rep("C", 3)),
  values = c(c(3, 4, 5), c(1, 6, 7), c(2, 5, 8))
  )
df2

# A tibble: 9 × 2
  group values
  <chr>  <dbl>
1 A          3
2 A          4
3 A          5
4 B          1
5 B          6
6 B          7
7 C          2
8 C          5
9 C          8

We can change this .fun argument to make it order by min, max, mean, or others:

e.g. Let’s order by the minimum of the each group

The minimum are A: 3, B: 1, C: 2.

The order (from small to large) is B - C - A.

res3 <- df2 |> 
  mutate(group = fct_reorder(group, values, .fun = min))
res3$group

[1] A A A B B B C C C
Levels: B C A

Factor example 2: reorder factor levels

Use fct_reorder() to reorder a factor by another variable

mtcars_tbl <- rownames_to_column(
  mtcars, var = "model") |> 
  as_tibble() 

mtcars_tbl2 <- mtcars_tbl |> 
  mutate(cyl = as.factor(cyl)) |> 
  mutate(model = fct_reorder(model, disp))

mtcars_tbl2 |> 
  ggplot(aes(x = disp, y = model, fill = cyl)) + 
  geom_col() + 
  scale_fill_brewer(palette = "Dark2")

mtcars_tbl |> head(3)

# A tibble: 3 × 12
  model    mpg cyl    disp    hp  drat    wt  qsec
  <chr>  <dbl> <fct> <dbl> <dbl> <dbl> <dbl> <dbl>
1 Mazda…  21   6       160   110  3.9   2.62  16.5
2 Mazda…  21   6       160   110  3.9   2.88  17.0
3 Datsu…  22.8 4       108    93  3.85  2.32  18.6
# ℹ 4 more variables: vs <dbl>, am <dbl>,
#   gear <dbl>, carb <dbl>

There is also a fct_reorder2() that allows you to order a factor by two variables

Your time (2/2)

Start from this code, how would you change it into the plot on the right?

mtcars |> 
  rownames_to_column(var = "model") |> 
  ggplot(aes(x = disp, y = model, fill = cyl)) + 
  geom_col()

Your time (bonus)

Take the gapminder data and focus on all the European countries. Plot the lifeExp for each country and order the country by their max lifExp. Reproduce this plot:

x-axis: country
y-axis: lifeExp
geometry: points
We are only using the data in Europe
- How do you subset the data to only European ones?
We order the country by the maximum life expectancy

You can play around with order by mean, median, min, etc if you have extra time.

Solution

gapminder |> 
  filter(continent == "Europe") |>
  ggplot(aes(x = lifeExp, y = fct_reorder(country, lifeExp, max))) +
  geom_point() + 
  labs(y = "Country")

Appendix

A deeper dive into geom_bar() and geom_col()

`ggplot2` component: `stat`

In geom_bar(), stat = "count" is used, so the geometry first calculate the count of x or y from the data before plotting.

geom_bar

function (mapping = NULL, data = NULL, stat = "count", position = "stack", 
    ..., just = 0.5, lineend = "butt", linejoin = "mitre", na.rm = FALSE, 
    show.legend = NA, inherit.aes = TRUE) 
{
    layer(mapping = mapping, data = data, geom = "bar", stat = stat, 
        position = position, show.legend = show.legend, inherit.aes = inherit.aes, 
        params = list2(na.rm = na.rm, just = just, lineend = lineend, 
            linejoin = linejoin, ...))
}
<bytecode: 0x113624358>
<environment: 0x1136218a0>

In geom_point() and geom_bar(), stat = "identity" is used, so there is no statistical transformation

Poke into how ggplot2 works internally

dt <- tibble(x = c(1, 1, 2, 2), y = c(1, 10, 3, 20), 
             group = c(1, 2, 1, 2))
dt

# A tibble: 4 × 3
      x     y group
  <dbl> <dbl> <dbl>
1     1     1     1
2     1    10     2
3     2     3     1
4     2    20     2

ggplot(data = dt, 
       aes(x = x, y = y, group = group)) +
  geom_line() + 
  geom_point()

Based on your specification, ggplot2 will generate the tibble below internally:

# A tibble: 4 × 4
      x     y group PANEL
  <dbl> <dbl> <int> <fct>
1     1     1     1 1    
2     1    10     2 1    
3     2     3     1 1    
4     2    20     2 1

Poke into how ggplot2 works internally

The internally data object for geom_bar() with a default stat = "count" is more complex:

# A tibble: 4 × 7
  count  prop x      width flipped_aes PANEL group
  <dbl> <dbl> <mppd> <dbl> <lgl>       <fct> <int>
1   488     1 1        0.9 FALSE       1         1
2   695     1 2        0.9 FALSE       1         2
3  1052     1 3        0.9 FALSE       1         3
4   632     1 4        0.9 FALSE       1         4

The column count and prop are the statistics calculated from the data.

It will then plot x on the x-axis, the generated variable count on the y-axis, each bar is a group, there is only one panel (no facet), and bar is its own entity (group differs for each).

You can check the computed variables for each geom in the documentation

after_stat(count) - number of points in bin.
after_stat(prop) - groupwise proportion

What if we want to use a different variable calculated after stat?

You need to use after_stat() to tell ggplot2 prop is not an original variable in the dataset, but something accessible only “after stat”.

ggplot(data = gss_sm, mapping = aes(x = bigregion)) +
    geom_bar(aes(y = after_stat(prop)))

This is bad - what has happened?

What if we want to use a different variable calculated after stat?

The proportion is calculated for each individual group, hence 1 for all prop:

# A tibble: 4 × 7
  count  prop x      width flipped_aes PANEL group
  <dbl> <dbl> <mppd> <dbl> <lgl>       <fct> <int>
1   488     1 1        0.9 FALSE       1         1
2   695     1 2        0.9 FALSE       1         2
3  1052     1 3        0.9 FALSE       1         3
4   632     1 4        0.9 FALSE       1         4

We can force the group to be 1:

This is the internal data now:

# A tibble: 4 × 7
  count  prop x      width flipped_aes group PANEL
  <dbl> <dbl> <mppd> <dbl> <lgl>       <int> <fct>
1   488 0.170 1        0.9 FALSE           1 1    
2   695 0.242 2        0.9 FALSE           1 1    
3  1052 0.367 3        0.9 FALSE           1 1    
4   632 0.220 4        0.9 FALSE           1 1

ggplot(gss_sm, aes(x = bigregion)) +
  geom_bar(aes(y = after_stat(prop), group = 1))

Elements of Data ScienceSDS 322E

Assessment heads-up

Learning objectives

Which one is better: pie chart or bar chart?

General Social Survey data

Display count

Your time (1/2)

Solution

When there are two categorical variables

When there are two categorical variables

When there are two categorical variables

When there are two categorical variables

When there are two categorical variables

Data frame vs. tibble

Convert a data frame to a tibble

Anything wrong with the color in this plot?

Factor basics

Change a variable to a factor in a tibble

Factor example 1: change a variable to a factor

Change the legend name

Factor example 2: reorder factor levels

Factor example 2: reorder factor levels

Factor example 2: reorder factor levels

Factor example 2: reorder factor levels

Factor example 2: reorder factor levels

Factor example 2: reorder factor levels

Your time (2/2)

Your time (bonus)

Solution

Appendix

ggplot2 component: stat

Poke into how ggplot2 works internally

Poke into how ggplot2 works internally

What if we want to use a different variable calculated after stat?

What if we want to use a different variable calculated after stat?

Elements of Data Science
SDS 322E

`ggplot2` component: `stat`