Elements of Data Science
SDS 322E

H. Sherry Zhang
Department of Statistics and Data Sciences
University of Texas at Austin

Get the repository for today:
library(usethis)
create_from_github("SDS322E-26FALL/0202-dplyr", fork = FALSE)

Learning objective

We will learn the five most basic dplyr verbs to wrangle data, including:

  • filter(): filter rows by a predicate
  • mutate(): create or modify variables
  • group_by(): group data by one or more variables
  • summarize(): summarize data by groups
  • arrange(): sort data by one or more variables

These are the fundamentals for building up more complex data wrangling…


By the end of the class, you are expected to write code to answer question like this:

Task: We want to get the proportion of flights with departure delay larger than 3 hours.

Artwork by @allison_horst

Predicate functions: ==, !=

  • a == b checks whether a is equal to b
  • a != b checks whether a is NOT equal to b
1 == 1 
[1] TRUE
1 == 2
[1] FALSE
1 != 2
[1] TRUE

In the flights data:

# by a single value
flights |> filter(month == 1)
# A tibble: 27,004 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      517            515         2      830
2  2013     1     1      533            529         4      850
3  2013     1     1      542            540         2      923
4  2013     1     1      544            545        -1     1004
5  2013     1     1      554            600        -6      812
# ℹ 26,999 more rows
# ℹ 12 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

Predicate functions: %in%

For multiple values, you need%in%:

  • a %in% c(x1, x2, ...) checks
    • whether a is one of the values in the vector c(x1, x2, ...)
  • !a %in% c(x1, x2, ...) checks
    • whether a is NOT one of the values in the vector c(x1, x2, ...)
3 %in% c(1, 2) 
[1] FALSE
!(3 %in% c(1, 2))
[1] TRUE

In the flights data:

# by multiple value
flights |> filter(month %in% c(1, 3))
# A tibble: 55,838 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      517            515         2      830
2  2013     1     1      533            529         4      850
3  2013     1     1      542            540         2      923
4  2013     1     1      544            545        -1     1004
5  2013     1     1      554            600        -6      812
# ℹ 55,833 more rows
# ℹ 12 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

Predicate functions: &, |

You can also combine multiple predicate functions together with & (and) and | (or).

(1 == 1) & (1 == 2) # TRUE and FALSE -> FALSE
[1] FALSE
(1 == 1) | (1 == 2) # TRUE or FALSE -> TRUE
[1] TRUE

In the flights data:

flights |> filter(month == 1 & dep_delay > 120)
# A tibble: 593 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      848           1835       853     1001
2  2013     1     1      957            733       144     1056
3  2013     1     1     1114            900       134     1447
4  2013     1     1     1540           1338       122     2020
5  2013     1     1     1815           1325       290     2120
# ℹ 588 more rows
# ℹ 12 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

You can also write it as:

flights |> filter(month == 1, dep_delay > 120)
# A tibble: 593 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      848           1835       853     1001
2  2013     1     1      957            733       144     1056
3  2013     1     1     1114            900       134     1447
4  2013     1     1     1540           1338       122     2020
5  2013     1     1     1815           1325       290     2120
# ℹ 588 more rows
# ℹ 12 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

dplyr syntax

DATA |> filter(PREDICATE) 
# basic
flights |> filter(dep_delay > 120)

# by a single value
flights |> filter(month == 1)

# by multiple value
flights |> filter(month %in% c(1, 3))

# by negation
flights |> filter(month != 1)
flights |> filter(!month %in% c(1, 3))

# by multiple condition
flights |> filter(month == 1 & dep_delay > 120)
flights |> filter(month == 1, dep_delay > 120)
flights |> filter(month == 1 | dep_delay > 120)

What goes wrong here?

flights |> filter(month = 1)
flights |> filter(month == 1)
# A tibble: 27,004 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      517            515         2      830
2  2013     1     1      533            529         4      850
3  2013     1     1      542            540         2      923
4  2013     1     1      544            545        -1     1004
5  2013     1     1      554            600        -6      812
# ℹ 26,999 more rows
# ℹ 12 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

Your time (1/5)

usethis::create_from_github("SDS322E-26FALL/0203-dplyr")

Apply the predicate functions to answer the following questions:

  1. Find the flights that depart from JFK
  2. Find the flights that depart from JFK in January
  3. Find the flights that depart from EWR and LGA and not in January

You may start from:

flights |> filter(...)

Solution

  1. Find the flights that depart from JFK
flights |> filter(origin == "JFK")
  1. Find the flights that depart from JFK and LGA in January
flights |> filter(origin %in% c("JFK", "LGA"), month == 1)
# or
flights |> filter(origin %in% c("JFK", "LGA") & month == 1)
  1. Find the flights that not depart from EWR and LGA and not in January
flights |> filter(!origin %in% c("EWR", "LGA"), month != 1)

Artwork by @allison_horst

mutate() syntax

DATA |> mutate(VARIABLE = EXPRESSION)

If VARIABLE already exists, it modifies the column; otherwise, it will create a new one.

flights |> mutate(gain = dep_delay - arr_delay)
# A tibble: 336,776 × 20
   year month   day dep_time sched_dep_time dep_delay arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>
1  2013     1     1      517            515         2      830
2  2013     1     1      533            529         4      850
3  2013     1     1      542            540         2      923
4  2013     1     1      544            545        -1     1004
5  2013     1     1      554            600        -6      812
# ℹ 336,771 more rows
# ℹ 13 more variables: sched_arr_time <int>, arr_delay <dbl>,
#   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>,
#   dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>, gain <dbl>

You can put multiple mutations together, or separately

These two are identical:

flights |> 
  mutate(gain = dep_delay - arr_delay, 
         speed = distance / (air_time / 60))
# A tibble: 336,776 × 21
   year month   day dep_time sched_dep_time
  <int> <int> <int>    <int>          <int>
1  2013     1     1      517            515
2  2013     1     1      533            529
3  2013     1     1      542            540
4  2013     1     1      544            545
5  2013     1     1      554            600
# ℹ 336,771 more rows
# ℹ 16 more variables: dep_delay <dbl>,
#   arr_time <int>, sched_arr_time <int>,
#   arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>,
#   air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>, gain <dbl>, …
flights |> 
  mutate(gain = dep_delay - arr_delay) |> 
  mutate(speed = distance / (air_time / 60))
# A tibble: 336,776 × 21
   year month   day dep_time sched_dep_time
  <int> <int> <int>    <int>          <int>
1  2013     1     1      517            515
2  2013     1     1      533            529
3  2013     1     1      542            540
4  2013     1     1      544            545
5  2013     1     1      554            600
# ℹ 336,771 more rows
# ℹ 16 more variables: dep_delay <dbl>,
#   arr_time <int>, sched_arr_time <int>,
#   arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>,
#   air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>, gain <dbl>, …

We can also compare them properly :)

v1 <- flights |> 
  mutate(gain = dep_delay - arr_delay, 
         speed = distance / (air_time / 60))

v2 <- flights |> 
  mutate(gain = dep_delay - arr_delay) |> 
  mutate(speed = distance / (air_time / 60))

identical(v1, v2)
[1] TRUE

Your time (2/5)

Read the documentation of mutate() and modify the following code to put the gain column:

flights |> 
  mutate(gain = dep_delay - arr_delay)
  • as the first column,
  • after the day variable, and
  • keep only the columns used to create the gain column

Solution

flights |> mutate(gain = dep_delay - arr_delay, .before = 1) 
flights |> mutate(gain = dep_delay - arr_delay, .after = day)
flights |> mutate(gain = dep_delay - arr_delay, .keep = "used")

Why these arguments have “.” in front of everything?

  • Function developer needs to accommodate the fact that users will do all sorts of things with their functions, e.g. you may name a column after for the number of mins after the scheduled time
  • If after is also an argument name inside mutate(), it will then be confusing whether you are referring to the column, after, or the argument.
  • Prefixing the argument name with . avoids this problem – it’s a good practice.

Illustration by Andrew Heiss

Illustration by Andrew Heiss

Illustration by Andrew Heiss

With the flights data

DATA |> group_by(VARIABLEs) |> summarize(EXPRESSION)

Example:

flights |> group_by(month)
# A tibble: 336,776 × 19
# Groups:   month [12]
   year month   day dep_time sched_dep_time
  <int> <int> <int>    <int>          <int>
1  2013     1     1      517            515
2  2013     1     1      533            529
3  2013     1     1      542            540
4  2013     1     1      544            545
5  2013     1     1      554            600
# ℹ 336,771 more rows
# ℹ 14 more variables: dep_delay <dbl>,
#   arr_time <int>, sched_arr_time <int>,
#   arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>,
#   air_time <dbl>, distance <dbl>, hour <dbl>,
#   minute <dbl>, time_hour <dttm>

The header tells you that the data is grouped by month and there are 12 groups (12 months).

I hope you remember

Why we get all the NAs here?

flights |>
  group_by(month) |>
  summarize(avg_dep_delay = mean(dep_delay))
# A tibble: 12 × 2
  month avg_dep_delay
  <int>         <dbl>
1     1            NA
2     2            NA
3     3            NA
4     4            NA
5     5            NA
# ℹ 7 more rows
flights |>
  group_by(month) |>
  summarize(avg_dep_delay = 
              mean(dep_delay, na.rm = TRUE))
# A tibble: 12 × 2
  month avg_dep_delay
  <int>         <dbl>
1     1          10.0
2     2          10.8
3     3          13.2
4     4          13.9
5     5          13.0
# ℹ 7 more rows

summarize(), summarise(), and summary()

  • summarize()/ summarise() are dplyr verbs to summarize groups in a way you specified
  • summary() is a base R function to provide the 5 number summary (min, quantile, median, mean, max) for each variable in a data frame
summary(flights$dep_delay)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.     NAs 
 -43.00   -5.00   -2.00   12.64   11.00 1301.00    8255 
flights |>
  group_by(month) |>
  summarize(avg_dep_delay = 
              mean(dep_delay, na.rm = TRUE))
# A tibble: 12 × 2
  month avg_dep_delay
  <int>         <dbl>
1     1          10.0
2     2          10.8
3     3          13.2
4     4          13.9
5     5          13.0
# ℹ 7 more rows

Of course, you can do multiple summaries

flights |>
  group_by(month, origin) |>
  summarize(avg_dep_delay = mean(dep_delay, na.rm = TRUE),
            avg_arr_delay = mean(arr_delay, na.rm = TRUE))
# A tibble: 36 × 4
# Groups:   month [12]
  month origin avg_dep_delay avg_arr_delay
  <int> <chr>          <dbl>         <dbl>
1     1 EWR            14.9          12.8 
2     1 JFK             8.62          1.37
3     1 LGA             5.64          3.38
4     2 EWR            13.1           8.78
5     2 JFK            11.8           4.39
# ℹ 31 more rows

Separate summarize() into two commands means summarizing avg_dep_delay first, then take the result of the first summary, summarize avg_arr_delay:

flights |>
  group_by(month, origin) |>
  summarize(avg_dep_delay = mean(dep_delay, na.rm = TRUE)) |> 
  summarize(avg_arr_delay = mean(arr_delay, na.rm = TRUE))
Error in `summarize()`:
ℹ In argument: `avg_arr_delay = mean(arr_delay, na.rm = TRUE)`.
ℹ In group 1: `month = 1`.
Caused by error:
! object 'arr_delay' not found

flights |>
  group_by(month, origin) |>
  summarize(avg_dep_delay = mean(dep_delay, na.rm = TRUE)) 
# A tibble: 36 × 3
# Groups:   month [12]
  month origin avg_dep_delay
  <int> <chr>          <dbl>
1     1 EWR            14.9 
2     1 JFK             8.62
3     1 LGA             5.64
4     2 EWR            13.1 
5     2 JFK            11.8 
# ℹ 31 more rows

Your time (3/5)

Apply the group_by() + summarize() syntax to calculate the average flight distance for each of the three origins. Your results should look like this:

# A tibble: 3 × 2
  origin distance
  <chr>     <dbl>
1 EWR       1057.
2 JFK       1266.
3 LGA        780.

Hint:

flights |> group_by(...) |> summarize(...)

Solution

flights |> group_by(origin) |> summarize(distance = mean(distance, na.rm = TRUE))
# A tibble: 3 × 2
  origin distance
  <chr>     <dbl>
1 EWR       1057.
2 JFK       1266.
3 LGA        780.

arrange()

Sometimes, we may wish to have the result sorted by a variable, this can be done with arrange():

flights |>
  group_by(carrier) |>
  summarize(avg_dep_delay = 
              mean(arr_delay, na.rm = TRUE))
# A tibble: 16 × 2
   carrier avg_dep_delay
   <chr>           <dbl>
 1 9E              7.38 
 2 AA              0.364
 3 AS             -9.93 
 4 B6              9.46 
 5 DL              1.64 
 6 EV             15.8  
 7 F9             21.9  
 8 FL             20.1  
 9 HA             -6.92 
10 MQ             10.8  
# ℹ 6 more rows
flights |>
  group_by(carrier) |>
  summarize(avg_dep_delay = 
              mean(arr_delay, na.rm = TRUE)) |> 
  arrange(avg_dep_delay)
# A tibble: 16 × 2
   carrier avg_dep_delay
   <chr>           <dbl>
 1 AS             -9.93 
 2 HA             -6.92 
 3 AA              0.364
 4 DL              1.64 
 5 VX              1.76 
 6 US              2.13 
 7 UA              3.56 
 8 9E              7.38 
 9 B6              9.46 
10 WN              9.65 
# ℹ 6 more rows

Sort by decreasing order can be done with either - or desc():

flights |>
  group_by(carrier) |>
  summarize(avg_dep_delay = 
              mean(arr_delay, na.rm = TRUE)) |> 
  arrange(-avg_dep_delay)
# A tibble: 16 × 2
   carrier avg_dep_delay
   <chr>           <dbl>
 1 F9             21.9  
 2 FL             20.1  
 3 EV             15.8  
 4 YV             15.6  
 5 OO             11.9  
 6 MQ             10.8  
 7 WN              9.65 
 8 B6              9.46 
 9 9E              7.38 
10 UA              3.56 
11 US              2.13 
12 VX              1.76 
13 DL              1.64 
14 AA              0.364
15 HA             -6.92 
16 AS             -9.93 
flights |>
  group_by(carrier) |>
  summarize(avg_dep_delay = 
              mean(arr_delay, na.rm = TRUE)) |> 
  arrange(desc(avg_dep_delay))
# A tibble: 16 × 2
   carrier avg_dep_delay
   <chr>           <dbl>
 1 F9             21.9  
 2 FL             20.1  
 3 EV             15.8  
 4 YV             15.6  
 5 OO             11.9  
 6 MQ             10.8  
 7 WN              9.65 
 8 B6              9.46 
 9 9E              7.38 
10 UA              3.56 
11 US              2.13 
12 VX              1.76 
13 DL              1.64 
14 AA              0.364
15 HA             -6.92 
16 AS             -9.93

Your time (4/5)

Can you improve the previous group_by() and summarize() code to show the distance sorted from small to large as this:

# A tibble: 3 × 2
  origin distance
  <chr>     <dbl>
1 LGA        780.
2 EWR       1057.
3 JFK       1266.

?

Hint

flights |> 
  group_by(origin) |> 
  summarize(distance = mean(distance, na.rm = TRUE)) |> 
  arrange(...)

Solution

flights |> 
  group_by(origin) |>
  summarize(distance = mean(distance, na.rm = TRUE)) |> 
  arrange(distance)

Decompose a data analysis problem with dplyr

Do you know that US Department of Transportation (DoT) says you’re entitled to a refund if the consumer is scheduled to arrive at the destination airport 3 hours more for more domestic itineraries?

https://www.transportation.gov/individuals/aviation-consumer-protection/refunds

Decompose a data analysis problem with dplyr

Airline: How many of our flights are 3 hrs+ late?

Task: We want to g the proportion of flights with arrival delay larger than 3 hours.

Thought process:

  1. I need to know for each rows whether they are delay more than 3 hrs or not.

This sounds like a mutate() and we learn the “larger than 3 hrs” part in filter()

flights |> 
  mutate(over_3h_delay = arr_delay > 180, .keep = "used") 
# A tibble: 336,776 × 2
   arr_delay over_3h_delay
       <dbl> <lgl>        
 1        11 FALSE        
 2        20 FALSE        
 3        33 FALSE        
 4       -18 FALSE        
 5       -25 FALSE        
 6        12 FALSE        
 7        19 FALSE        
 8       -14 FALSE        
 9        -8 FALSE        
10         8 FALSE        
11        -2 FALSE        
12        -3 FALSE        
13         7 FALSE        
14       -14 FALSE        
15        31 FALSE        
16        -4 FALSE        
17        -8 FALSE        
18        -7 FALSE        
19        12 FALSE        
20        -6 FALSE        
# ℹ 336,756 more rows

Task: We want to summarize the proportion of flights with departure delay larger than 3 hours.

Thought process:

  1. I need to know for each rows whether they are delay more than 3 hrs or not.
  2. Then I need to count the number of observations for over and below 3h delay.

This sounds like group_by() and summarize(), but I want what’s inside summarise() to be the count

flights |> 
  mutate(over_3h_delay = arr_delay > 180, .keep = "used") |> 
  group_by(over_3h_delay) |>
  summarise(n = length(over_3h_delay))
# A tibble: 3 × 2
  over_3h_delay      n
  <lgl>          <int>
1 FALSE         323503
2 TRUE            3843
3 NA              9430

Task: We want to summarize the proportion of flights with departure delay larger than 3 hours.

Thought process:

  1. I need to know for each rows whether they are delay more than 3 hrs or not.
  2. Then I need to count the number of observations for over and below 3hrs delay.
  3. Then I need to calculate proportion based on n.

This sounds like mutate() again

flights |> 
  mutate(over_3h_delay = arr_delay > 180, .keep = "used") |> 
  group_by(over_3h_delay) |>
  summarise(n = length(over_3h_delay)) |> 
  mutate(prop = n/ sum(n) * 100)
# A tibble: 3 × 3
  over_3h_delay      n  prop
  <lgl>          <int> <dbl>
1 FALSE         323503 96.1 
2 TRUE            3843  1.14
3 NA              9430  2.80

Remark 1

What do these NA rows look like?

Remark 1

Can we look at those NAs? Yes!

flights |> filter(is.na(arr_delay))
# A tibble: 9,430 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
1  2013     1     1     1525           1530        -5     1934           1805
2  2013     1     1     1528           1459        29     2002           1647
3  2013     1     1     1740           1745        -5     2158           2020
4  2013     1     1     1807           1738        29     2251           2103
5  2013     1     1     1939           1840        59       29           2151
# ℹ 9,425 more rows
# ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
#   hour <dbl>, minute <dbl>, time_hour <dttm>

Do you wonder why the following happens?

flights |> filter(dep_delay == NA)
# A tibble: 0 × 19
# ℹ 19 variables: year <int>, month <int>, day <int>, dep_time <int>,
#   sched_dep_time <int>, dep_delay <dbl>, arr_time <int>,
#   sched_arr_time <int>, arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
#   hour <dbl>, minute <dbl>, time_hour <dttm>

Remark 1

This is not legit because comparing anything with NA gives an NA (rather than giving FALSE):

1 == NA
[1] NA
2!= NA
[1] NA

You should always use is.na(VAR) or !is.na(VAR) to check whether a variable contains NA values.

Remark 2

group_by(over_2h_delay) |> summarise(n = length(over_3h_delay)) can be simplified with a single command dplyr::count(over_2h_delay).

flights |> 
  mutate(over_3h_delay = arr_delay > 180, 
         .keep = "used") |> 
  group_by(over_3h_delay) |>
  summarise(n = n()) |> 
  mutate(prop = n/ sum(n) * 100)
# A tibble: 3 × 3
  over_3h_delay      n  prop
  <lgl>          <int> <dbl>
1 FALSE         323503 96.1 
2 TRUE            3843  1.14
3 NA              9430  2.80
flights |> 
  mutate(over_3h_delay = arr_delay > 180, 
         .keep = "used") |> 
  count(over_3h_delay) |> 
  mutate(prop = n/ sum(n) * 100)
# A tibble: 3 × 3
  over_3h_delay      n  prop
  <lgl>          <int> <dbl>
1 FALSE         323503 96.1 
2 TRUE            3843  1.14
3 NA              9430  2.80

Summary

Data analysis problem rarely tell you exactly what steps to use: you need to understand the context and solve it using the appropriate dplyr verbs. You may also want to ask

  • Are the delays coming from all three origins, or are they concentrated at a particular origin?
  • For flights delayed by 3+ hours, how many flights fall into each hourly delay category?
  • What are the records for flights with delays of more than 10 hours?

It is neither practical nor necessary to cover all the dplyr and related syntax upfront, so we will learn more as we go. In our example, we pick up the following:

  • n() and count()
  • how to handle NAs

Your time (5/5)

Can you reproduce the code we just walked through?

Task: We want to summarize the proportion of flights with departure delay larger than 3 hours.


Here is the step breakdown if you need:

  1. I need to know for each rows whether they are delay more than 3 hrs or not.
  2. Then I need to count the number of observations for over and below 2h delay.
  3. Then I need to calculate proportion based on n.

Extra

Are the delays coming from all three origins, or are they concentrated at a particular origin? 

flights |> filter(arr_delay > 180) |> group_by(origin) |> count()
# A tibble: 3 × 2
# Groups:   origin [3]
  origin     n
  <chr>  <int>
1 EWR     1515
2 JFK     1158
3 LGA     1170

For flights delayed by 3+ hours, how many flights fall into each hourly delay category?

flights |> 
  filter(arr_delay > 180) |> 
  mutate(hr_braket = arr_delay %/% 60) |> 
  group_by(hr_braket) |> 
  count()
# A tibble: 16 × 2
# Groups:   hr_braket [16]
  hr_braket     n
      <dbl> <int>
1         3  2272
2         4   945
3         5   371
4         6   151
5         7    39
# ℹ 11 more rows

Extra

What are the records for flights with delays of more than 10 hours

flights |> 
  filter(arr_delay > 600)
# A tibble: 39 × 19
   year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
  <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
1  2013     1     1      848           1835       853     1001           1950
2  2013     1     9      641            900      1301     1242           1530
3  2013     1    10     1121           1635      1126     1239           1810
4  2013     1    13     1809            810       599     2054           1042
5  2013    10    14     2042            900       702     2255           1127
# ℹ 34 more rows
# ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
#   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
#   hour <dbl>, minute <dbl>, time_hour <dttm>

Your time (bonus)

With the mtcars data, try the following:

  1. Convert the mpg variable into kpl (1 mpg = 0.425144 km/l)
  2. Only look at V-shaped engine Hint: look at the documentation to see what this means
  3. Find the average kpl and disp for each number of cylinders
  4. Arrange the result by disp in descending order

Upcoming assessment

Item Available Due Mode
Lab 1 Week 3 Tue
Sep 8 12am		 Week 3 Tue
Sep 8 11:59pm		 Complete during the lab session in your lab group
HW 1 Week 3 Thur
Sep 10 12am		 Week 4 Thur
Sep 17 11:59pm		 Complete individually
Lab 2 Week 4 Tue
Sep 15 12am		 Week 4 Tue
Sep 15 11:59pm		 Complete during the lab session in your lab group
HW 2 Week 4 Thur
Sep 17 12am		 Week 5 Thur
Sep 24 11:59pm		 Complete individually
Lab 3 Week 5 Tue
Sep 22 12am		 Week 5 Tue
Sep 22 11:59pm		 Complete during the lab session in your lab group
HW 3 Week 5 Thur
Sep 24 12am		 Week 6 Thur
Oct 01 11:59pm		 Complete individually
  • See canvas for lab group allocation