“Apply” functions keep you from having to write loops to perform some operation on every row or every column of a\u00a0matrix<\/a>\u00a0or\u00a0data frame<\/a>, or on every element in a\u00a0list<\/a>. For example, the built-in data setstate.x77<\/tt>\u00a0contains eight columns of data describing the 50 U.S. states in 1977. If you wanted the average of each of the eight columns, you could do this:<\/p>\n This is comparatively slow, much more so in large datasets. R is bad at looping. A more vectorized way to do this is to use the\u00a0apply()<\/tt>\u00a0function. In this example,\u00a0apply<\/tt>\u00a0extracts each column as a vector, one at a time, and passes it to the\u00a0median()<\/tt>\u00a0function.<\/p>\n The 2 means “go by column” — a 1 would have meant “go by row.” Of course, if we had used a 1, we would have computed 50 averages, one for each row. If we had had a\u00a0three-dimensional array<\/a>\u00a0we could have used a 3 there. The third argument specifies the function to be applied to each column. We can use any function that makes sense there. We can use our own function or even pass in a function that we write on the spot. If your function returns a vector of constant length, S-Plus will stick the vectors together into a matrix. However, if your function returns vectors of different lengths, S-Plus will have to create a list (see more details\u00a0below<\/a>).<\/p>\n The special cases of mean and sum have been taken care of already with the built-in\u00a0colMeans<\/tt>,\u00a0ColSums<\/tt>,\u00a0rowMeans<\/tt>, and\u00a0rowSums<\/tt>\u00a0functions. These are highly efficient and worth using.<\/p>\n In this example, we construct a function “on the fly” and pass it to apply. This particular function computes the median and maximum of each column of\u00a0state.x77<\/tt>.<\/p>\n If you pass additional arguments to apply, those arguments get passed down to the function you’re having apply call. So if you wanted to calculate the mean of each column after trimming the highest and lowest 10%, you could do this:<\/p>\n This is particularly handy for passing the\u00a0na.rm=T<\/tt>\u00a0argument to functions like\u00a0max<\/tt>.<\/p>\n Yes.\u00a0apply()<\/tt>\u00a0calls\u00a0lapply<\/tt><\/a>\u00a0and\u00a0lapply()<\/tt>\u00a0loops. Clearly\u00a0something<\/em>\u00a0has to loop. The reason that the apply family of functions is fast is that the looping is done in compiled code (C or Fortran), not in R’s own interpreted code. The difference can be the difference between finishing and crashing.\u00a0Note:<\/b>\u00a0After writing this I got curious about the extent to which\u00a0apply()<\/tt>\u00a0increases speed. I used commands like this:<\/p>\n expecting the last one to be reported as the slowest. Actually, though, the middle one was. I’m not sure what the story is here.<\/a><\/a><\/p>\n <\/a><\/a>I include this topic because it has bedeviled me in the past. Suppose I have this matrix\u00a0a<\/tt>, and I want to find the smallest number in each row. This is easy:<\/p>\n <\/a><\/a>So\u00a0apply()<\/tt>\u00a0works on each row, one at a time, to tell me the smallest number in each row. What if I want the\u00a0index<\/em>of the smallest number in each row? That is, I want the answer to the question “in which column can the minimum value be found”? That sounds easy, too: we’ll use the\u00a0which()<\/tt>\u00a0function, which returns the indices within a vector for which the vector holds the value TRUE.<\/p>\n <\/a><\/a>What has happened here is that there’s a tie in the second row.\u00a0apply()<\/tt>\u00a0returns a single value for rows 1 and 3, but two values for row 2, and R doesn’t know how to arrange those, so it makes a list. The[[1]]<\/tt>\u00a0tells us that the first element of the list has no name.<\/p>\n If we needed to do this we might impose a rule like “if there’s a tie pick out the first one.”<\/p>\n <\/a><\/a><\/p>\n <\/a><\/a>The regular\u00a0apply()<\/tt>\u00a0function can be used on a data frame since a data frame is a type of matrix. When you use it on the columns of a data frame, passing the number 2 for the second argument, it does what you expect. It will work on the rows of a data frame, too, but remember: apply extracts each row as a vector, one at a time. Every element of a vector must have the same kind of data, so unless every column of the data frame has the same kind of data, R will end up converting the elements of the row to a common format (like character).<\/p>\n The\u00a0lapply()<\/tt>\u00a0function works on any list, not just a rectangular one. (The “l” in “lapply” stands for “list.”) In that way it’s more general than\u00a0apply()<\/tt>, although it does not work on matrices or higher-dimensional arrrays. You don’t need to specify the “direction” as you do with\u00a0apply()<\/tt>; just pass the function.\u00a0However,\u00a0lapply()<\/tt>\u00a0always returns a list.<\/b>\u00a0Usually I want a vector, and that’s what\u00a0sapply()<\/tt>tries to do. The “s” in “sapply” stands for “simplify.” Here’s an example using the built-in\u00a0barley<\/tt>\u00a0data frame. My question is, how many levels of each variable are there? We can count the number by seeing how many unique entries there are: so\u00a0length(unique(x))<\/tt>\u00a0will do the trick.<\/p>\n <\/a><\/a><\/p>\n <\/a><\/a>tapply()<\/tt>\u00a0is a very powerful function that lets you break a vector into pieces, and then apply some function to each of the pieces. (For you Excel users,\u00a0tapply()<\/tt>\u00a0produces things that correspond to Excel’s pivot tables.) It’s sort of like\u00a0sapply()<\/tt>, except that with\u00a0sapply()<\/tt>\u00a0the pieces are always elements of a list. With\u00a0tapply()<\/tt>\u00a0you get to specify how the breakdown is done. For example, suppose I want to find the average yield for each variety of barley in the last example.<\/p>\n <\/a><\/a>tapply()<\/tt>\u00a0returns a vector with one element for each unique value of\u00a0barley$variety<\/tt>. The element for Grand Rapids, for example, gives the average of all the elements of\u00a0barley$yield<\/tt>\u00a0for whichbarley$variety == \"Grand Rapids\"<\/tt>. I have found\u00a0tapply()<\/tt>\u00a0to be incredibly useful. If you want to cross-tabulate by more than one variable, construct a list of your tabulating variables and pass that totapply()<\/tt>. Here we break yields down by year and site.<\/p>\n <\/a>We’ve learned something: 1931 was a much better year, except in Morris. (There’s some suspicion that Morris was in fact incorrectly recorded in this well-known data set.) 1932 appears before 1931 in the table because that’s how the levels of “year” were set up in S-Plus. (If this bothers you see\u00a0Reordering the levels of a factor<\/a>.) Years appear in the rows because they came first in the list. Of course a three- or higher-way table can be made in this way as well.<\/p>\n","protected":false},"excerpt":{"rendered":" Lapply and sapply: avoiding loops on lists and data frames Tapply: avoiding loops when applying a function to subsets “Apply” functions keep you from having… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[],"class_list":["post-166","post","type-post","status-publish","format-standard","hentry","category-r"],"_links":{"self":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts\/166","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/comments?post=166"}],"version-history":[{"count":0,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts\/166\/revisions"}],"wp:attachment":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/media?parent=166"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/categories?post=166"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/tags?post=166"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}> avgs <- numeric (8)\n> for (i in 1:8)\n+ avgs[i] <- mean (state.x77[,i]) # The \"+\" is R's continuation character; don't type it\n> avgs\n[1] 4246.4200 4435.8000 1.1700 70.8786 7.3780 53.1080 104.4600 70735.8800<\/pre>\n
> apply (state.x77, 2, median)\n Population Income Illiteracy Life Exp Murder HS Grad Frost Area \n 2838.5 4519 0.95 70.675 6.85 53.25 114.5 54277<\/pre>\n
> apply (state.x77, 2, function(x) c(median (x), max(x)))\n Population Income Illiteracy Life Exp Murder HS Grad Frost Area \n[1,] 2838.5 4519 0.95 70.675 6.85 53.25 114.5 54277\n[2,] 21198.0 6315 2.80 73.600 15.10 67.30 188.0 566432<\/pre>\n
> apply (state.x77, 2, mean, trim=.1)\n Population Income Illiteracy Life Exp Murder HS Grad Frost Area \n 3384.27500 4430.07500 1.09750 70.91775 7.29750 53.33750 106.80000 56575.72500<\/pre>\n
Does\u00a0apply()<\/tt>\u00a0loop?<\/h4>\n
> system.time (for (j in 1:20000) colMeans (state.x77))\n> system.time (for (j in 1:20000) apply (state.x77, 2, mean))\n> system.time (for (j in 1:20000) for (i in 1:8) mean (state.x77[,i]))<\/pre>\n
Sometimes you expect\u00a0apply()<\/tt>\u00a0to return a vector but you get a list<\/h4>\n
> a <- matrix (c(5, 2, 7, 1, 2, 8, 4, 5, 6), 3, 3)\n> a\n [,1] [,2] [,3] \n[1,] 5 1 4\n[2,] 2 2 5\n[3,] 7 8 6\n> apply (a, 1, min)\n[1] 1 2 6<\/pre>\n
> which (c(F, F, T, F, T, T, F)) # Example of \"which\" : where are the Trues?\n[1] 3 5 6\n#\n# For each row, find the column in which that row has its smallest value.\n# \n> apply (a, 1, function(x) which(x == min(x)))\n[[1]]\n[1] 2\n\n[[2]]\n[1] 1 2\n\n[[3]]\n[1] 3<\/pre>\n
> apply (a, 1, function(x) which(x == min(x))[1])\n[1] 2 1 3<\/pre>\n
<\/a><\/a>Lapply and sapply: avoiding loops on lists and data frames<\/h2>\n
library (lattice) # Make this data available\n> dim (barley) # Barley has 120 rows\n[1] 120 4\n> lapply (barley, function(x) length(unique(x))) # returns a list\n$yield:\n[1] 114\n\n$variety:\n[1] 10\n\n$year:\n[1] 2\n\n$site:\n[1] 6\n\n> sapply (barley, function(x) length(unique(x))) # Simplifies output to a vector\n yield variety year site \n 114 10 2 6\n> apply (barley, 2, function(x) length(unique(x))) # Also works on data frames (but not non-data frame lists).\n yield variety year site \n 114 10 2 6<\/pre>\n
<\/a><\/a>Tapply: avoiding loops when applying a function to subsets<\/h2>\n
> tapply (barley$yield, barley$site, mean)\n Grand Rapids Duluth University Farm Morris Crookston Waseca \n 24.93167 27.99667 32.66667 35.4 37.42 48.10833<\/pre>\n
> tapply (barley$yield, list (barley$year, barley$site), mean)\n Grand Rapids Duluth University Farm Morris Crookston Waseca \n1932 20.81000 25.70000 29.50667 41.51333 31.18 41.87000\n1931 29.05334 30.29333 35.82667 29.28667 43.66 54.34667<\/pre>\n