\n
\n Practical Guide to Principal Component Methods in R\n <\/a><\/p>\n\n\n\nBasics<\/h2>\n\n\n\n
Understanding the details of PCA requires knowledge of linear \nalgebra. Here, we\u2019ll explain only the basics with simple graphical \nrepresentation of the data.<\/p>\n\n\n\n
In the Plot 1A below, the data are represented in the X-Y coordinate \nsystem. The dimension reduction is achieved by identifying the principal\n directions, called principal components, in which the data varies.<\/p>\n\n\n\n
PCA assumes that the directions with the largest variances are the most \u201cimportant\u201d (i.e, the most principal).<\/p>\n\n\n\n
In the figure below, the PC1 axis<\/code> is the first principal direction<\/code> along which the samples show the largest variation. The PC2 axis<\/code> is the second most important direction<\/code> and it is orthogonal<\/code> to the PC1 axis.<\/p>\n\n\n\nThe dimensionality of our two-dimensional data can be reduced to a \nsingle dimension by projecting each sample onto the first principal \ncomponent (Plot 1B)<\/p>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-scatter-plot-data-mining-1.png\")
<\/figure>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-scatter-plot-data-mining-2.png\")
<\/figure>\n\n\n\nTechnically speaking, the amount of variance retained by each principal component is measured by the so-called eigenvalue<\/strong>.<\/p>\n\n\n\nNote that, the PCA method is particularly useful when the variables \nwithin the data set are highly correlated. Correlation indicates that \nthere is redundancy in the data. Due to this redundancy, PCA can be used\n to reduce the original variables into a smaller number of new variables\n ( = principal components<\/strong>) explaining most of the variance in the original variables.<\/p>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-unnamed-chunk-3-1.png\")
<\/figure>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-unnamed-chunk-3-2.png\")
<\/figure>\n\n\n\n\nTaken together, the main purpose of principal component analysis is to:\n<\/p>\n\n\n\n
- \nidentify hidden pattern in a data set,\n<\/li>
- \nreduce the dimensionnality of the data by removing the noise and redundancy in the data,\n<\/li>
- \nidentify correlated variables\n<\/li><\/ul>\n\n\n\n
Computation<\/h2>\n\n\n\nR packages<\/h3>\n\n\n\n
Several functions from different packages are available in the R software<\/em> for computing PCA:<\/p>\n\n\n\n- prcomp<\/em>() and princomp<\/em>() [built-in R stats<\/em> package],<\/li>
- PCA<\/em>() [FactoMineR<\/em> package],<\/li>
- dudi.pca<\/em>() [ade4<\/em> package],<\/li>
- and epPCA<\/em>() [ExPosition<\/em> package]<\/li><\/ul>\n\n\n\n
No matter what function you decide to use, you can easily extract and\n visualize the results of PCA using R functions provided in the factoextra<\/code> R package.<\/p>\n\n\n\n\nHere, we\u2019ll use the two packages FactoMineR (for the analysis) and factoextra (for ggplot2-based visualization).\n<\/p>\n\n\n\n
Install the two packages as follow:<\/p>\n\n\n\n
install.packages(c(\"FactoMineR\", \"factoextra\"))<\/code><\/pre>\n\n\n\nLoad them in R, by typing this:<\/p>\n\n\n\n
library(\"FactoMineR\")\nlibrary(\"factoextra\")<\/code><\/pre>\n\n\n\nData format<\/h3>\n\n\n\n
We\u2019ll use the demo data sets decathlon2<\/code> from the factoextra<\/em> package:<\/p>\n\n\n\ndata(decathlon2)\n# head(decathlon2)<\/code><\/pre>\n\n\n\nAs illustrated in Figure 3.1, the data used here describes athletes\u2019 \nperformance during two sporting events (Desctar and OlympicG). It \ncontains 27 individuals (athletes) described by 13 variables.<\/p>\n\n\n\n![\"Principal](\"http:\/\/www.sthda.com\/english\/sthda-upload\/images\/principal-component-methods\/pca-decathlon-300dpi.png\")
<\/figure>\n\n\n\n\nNote that, only some of these individuals and variables will be used to \nperform the principal component analysis. The coordinates of the \nremaining individuals and variables on the factor map will be predicted \nafter the PCA.\n<\/p>\n\n\n\n
In PCA terminology, our data contains :<\/p>\n\n\n\n
- \nActive individuals<\/em> (in light blue, rows 1:23) : Individuals that are used during the principal component analysis.\n<\/li>
- \nSupplementary individuals<\/em> (in dark blue, rows 24:27) : The \ncoordinates of these individuals will be predicted using the PCA \ninformation and parameters obtained with active individuals\/variables\n<\/li>
- \nActive variables<\/em> (in pink, columns 1:10) : Variables that are used for the principal component analysis.\n<\/li>
- \nSupplementary variables<\/em>: As supplementary individuals, the coordinates of these variables will be predicted also. These can be:\n
- \nSupplementary continuous variables<\/em> (red): Columns 11 and 12 corresponding respectively to the rank and the points of athletes.\n<\/li>
- \nSupplementary qualitative variables<\/em> (green): Column 13 \ncorresponding to the two athlete-tic meetings (2004 Olympic Game or 2004\n Decastar). This is a categorical (or factor) variable factor. It can be\n used to color individuals by groups.\n<\/li><\/ul>\n<\/li><\/ul>\n\n\n\n
We start by subsetting active individuals and active variables for the principal component analysis:<\/p>\n\n\n\n
decathlon2.active <- decathlon2[1:23, 1:10]\nhead(decathlon2.active[, 1:6], 4)<\/code><\/pre>\n\n\n\n## X100m Long.jump Shot.put High.jump X400m X110m.hurdle\n## SEBRLE 11.0 7.58 14.8 2.07 49.8 14.7\n## CLAY 10.8 7.40 14.3 1.86 49.4 14.1\n## BERNARD 11.0 7.23 14.2 1.92 48.9 15.0\n## YURKOV 11.3 7.09 15.2 2.10 50.4 15.3<\/code><\/pre>\n\n\n\nData standardization<\/h3>\n\n\n\n
In principal component analysis, variables are often scaled \n(i.e. standardized). This is particularly recommended when variables are\n measured in different scales (e.g: kilograms, kilometers, centimeters, \n\u2026); otherwise, the PCA outputs obtained will be severely affected. <\/p>\n\n\n\n
The goal is to make the variables comparable. Generally variables are\n scaled to have i) standard deviation one and ii) mean zero.<\/p>\n\n\n\n
The standardization of data is an approach widely used in the context\n of gene expression data analysis before PCA and clustering analysis. We\n might also want to scale the data when the mean and\/or the standard \ndeviation of variables are largely different.<\/p>\n\n\n\n
When scaling variables, the data can be transformed as follow:x<\/em>i<\/em>\u2212m<\/em>e<\/em>a<\/em>n<\/em>(x<\/em>)s<\/em>d<\/em>(x<\/em>)<\/p>\n\n\n\nWhere m<\/em>e<\/em>a<\/em>n<\/em>(x<\/em>) is the mean of x values, and s<\/em>d<\/em>(x<\/em>)<\/p>\n\n\n\n is the standard deviation (SD).<\/p>\n\n\n\n
The R base function `scale() can be used to standardize the data. It \ntakes a numeric matrix as an input and performs the scaling on the \ncolumns.<\/p>\n\n\n\n
\nNote that, by default, the function PCA<\/strong>() [in FactoMineR<\/em>], standardizes the data automatically during the PCA; so you don\u2019t need do this transformation before the PCA.\n<\/p>\n\n\n\nR code<\/h3>\n\n\n\n
The function PCA<\/em>() [FactoMineR<\/em> package] can be used. A simplified format is :<\/p>\n\n\n\nPCA(X, scale.unit = TRUE, ncp = 5, graph = TRUE)<\/code><\/pre>\n\n\n\nX<\/code>: a data frame. Rows are individuals and columns are numeric variables<\/li>scale.unit<\/code>: a logical value. If TRUE<\/em>,\n the data are scaled to unit variance before the analysis. This \nstandardization to the same scale avoids some variables to become \ndominant just because of their large measurement units. It makes \nvariable comparable.<\/li>ncp<\/code>: number of dimensions kept in the final results.<\/li>graph<\/code>: a logical value. If TRUE a graph is displayed.<\/li><\/ul>\n\n\n\nThe R code below, computes principal component analysis on the active individuals\/variables:<\/p>\n\n\n\n
library(\"FactoMineR\")\nres.pca <- PCA(decathlon2.active, graph = FALSE)<\/code><\/pre>\n\n\n\nThe output of the function PCA<\/em>() is a list, including the following components :<\/p>\n\n\n\nprint(res.pca)<\/code><\/pre>\n\n\n\n## **Results for the Principal Component Analysis (PCA)**\n## The analysis was performed on 23 individuals, described by 10 variables\n## *The results are available in the following objects:\n## \n## name description \n## 1 \"$eig\" \"eigenvalues\" \n## 2 \"$var\" \"results for the variables\" \n## 3 \"$var$coord\" \"coord. for the variables\" \n## 4 \"$var$cor\" \"correlations variables - dimensions\"\n## 5 \"$var$cos2\" \"cos2 for the variables\" \n## 6 \"$var$contrib\" \"contributions of the variables\" \n## 7 \"$ind\" \"results for the individuals\" \n## 8 \"$ind$coord\" \"coord. for the individuals\" \n## 9 \"$ind$cos2\" \"cos2 for the individuals\" \n## 10 \"$ind$contrib\" \"contributions of the individuals\" \n## 11 \"$call\" \"summary statistics\" \n## 12 \"$call$centre\" \"mean of the variables\" \n## 13 \"$call$ecart.type\" \"standard error of the variables\" \n## 14 \"$call$row.w\" \"weights for the individuals\" \n## 15 \"$call$col.w\" \"weights for the variables\"<\/code><\/pre>\n\n\n\n\nThe object that is created using the function PCA<\/em>() contains many information found in many different lists and matrices. These values are described in the next section.\n<\/p>\n\n\n\nVisualization and Interpretation<\/h2>\n\n\n\n
We\u2019ll use the factoextra<\/em> R package to help in the \ninterpretation of PCA. No matter what function you decide to use \n<\/p>\n\n\n[stats::prcomp(), FactoMiner::PCA(), ade4::dudi.pca(),
\nExPosition::epPCA()]<\/p>\n\n\n\n
, you can easily extract and visualize the results \nof PCA using R functions provided in the factoextra<\/em> R package.\n<\/p>\n\n\n\nThese functions include:<\/p>\n\n\n\n
get_eigenvalue(res.pca)<\/code>: Extract the eigenvalues\/variances of principal components<\/li>fviz_eig(res.pca)<\/code>: Visualize the eigenvalues<\/li>get_pca_ind(res.pca)<\/code>, get_pca_var(res.pca)<\/code>: Extract the results for individuals and variables, respectively.<\/li>fviz_pca_ind(res.pca)<\/code>, fviz_pca_var(res.pca)<\/code>: Visualize the results individuals and variables, respectively.<\/li>fviz_pca_biplot(res.pca)<\/code>: Make a biplot of individuals and variables.<\/li><\/ul>\n\n\n\nIn the next sections, we\u2019ll illustrate each of these functions.<\/p>\n\n\n\n
Eigenvalues \/ Variances<\/h3>\n\n\n\n
As described in previous sections, the eigenvalues<\/em> measure the amount of variation retained by each principal component. Eigenvalues<\/em>\n are large for the first PCs and small for the subsequent PCs. That is, \nthe first PCs corresponds to the directions with the maximum amount of \nvariation in the data set. <\/p>\n\n\n\nWe examine the eigenvalues to determine the number of principal \ncomponents to be considered. The eigenvalues and the proportion of \nvariances (i.e., information) retained by the principal components (PCs)\n can be extracted using the function get_eigenvalue<\/em>() [factoextra<\/em> package].<\/p>\n\n\n\nlibrary(\"factoextra\")\neig.val <- get_eigenvalue(res.pca)\neig.val<\/code><\/pre>\n\n\n\n## eigenvalue variance.percent cumulative.variance.percent\n## Dim.1 4.124 41.24 41.2\n## Dim.2 1.839 18.39 59.6\n## Dim.3 1.239 12.39 72.0\n## Dim.4 0.819 8.19 80.2\n## Dim.5 0.702 7.02 87.2\n## Dim.6 0.423 4.23 91.5\n## Dim.7 0.303 3.03 94.5\n## Dim.8 0.274 2.74 97.2\n## Dim.9 0.155 1.55 98.8\n## Dim.10 0.122 1.22 100.0<\/code><\/pre>\n\n\n\nThe sum of all the eigenvalues give a total variance of 10.<\/p>\n\n\n\n
The proportion of variation explained by each eigenvalue is given in \nthe second column. For example, 4.124 divided by 10 equals 0.4124, or, \nabout 41.24% of the variation is explained by this first eigenvalue. The\n cumulative percentage explained is obtained by adding the successive \nproportions of variation explained to obtain the running total. For \ninstance, 41.242% plus 18.385% equals 59.627%, and so forth. Therefore, \nabout 59.627% of the variation is explained by the first two eigenvalues\n together.<\/p>\n\n\n\n
Eigenvalues can be used to determine the number of principal components to retain after PCA (Kaiser 1961): <\/p>\n\n\n\n
- \n\nAn eigenvalue<\/em> > 1 indicates that PCs account for more \nvariance than accounted by one of the original variables in standardized\n data. This is commonly used as a cutoff point for which PCs are \nretained. This holds true only when the data are standardized.\n\n<\/li>
- \n\nYou can also limit the number of component to that number that accounts \nfor a certain fraction of the total variance. For example, if you are \nsatisfied with 70% of the total variance explained then use the number \nof components to achieve that.\n\n<\/li><\/ul>\n\n\n\n
Unfortunately, there is no well-accepted objective way to decide how \nmany principal components are enough. This will depend on the specific \nfield of application and the specific data set. In practice, we tend to \nlook at the first few principal components in order to find interesting \npatterns in the data.<\/p>\n\n\n\n
In our analysis, the first three principal components explain 72% of the variation. This is an acceptably large percentage.<\/p>\n\n\n\n
An alternative method to determine the number of principal components\n is to look at a Scree Plot, which is the plot of eigenvalues ordered \nfrom largest to the smallest. The number of component is determined at \nthe point, beyond which the remaining eigenvalues are all relatively \nsmall and of comparable size (Jollife 2002, Peres-Neto, Jackson, and Somers (2005)).<\/p>\n\n\n\n
The scree plot can be produced using the function fviz_eig()<\/code> or fviz_screeplot()<\/code> [factoextra<\/em> package]. <\/p>\n\n\n\nfviz_eig(res.pca, addlabels = TRUE, ylim = c(0, 50))<\/code><\/pre>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-eigenvalue-screeplot-1.png\")
<\/figure>\n\n\n\n\nFrom the plot above, we might want to stop at the fifth principal \ncomponent. 87% of the information (variances) contained in the data are \nretained by the first five principal components.\n<\/p>\n\n\n\n
Graph of variables<\/h3>\n\n\n\nResults<\/h4>\n\n\n\n
A simple method to extract the results, for variables, from a PCA output is to use the function get_pca_var()<\/code> [factoextra<\/em>\n package]. This function provides a list of matrices containing all the \nresults for the active variables (coordinates, correlation between \nvariables and axes, squared cosine and contributions)<\/p>\n\n\n\nvar <- get_pca_var(res.pca)\nvar<\/code><\/pre>\n\n\n\n## Principal Component Analysis Results for variables\n## ===================================================\n## Name Description \n## 1 \"$coord\" \"Coordinates for the variables\" \n## 2 \"$cor\" \"Correlations between variables and dimensions\"\n## 3 \"$cos2\" \"Cos2 for the variables\" \n## 4 \"$contrib\" \"contributions of the variables\"<\/code><\/pre>\n\n\n\nThe components of the get_pca_var()<\/code> can be used in the plot of variables as follow:<\/p>\n\n\n\nvar$coord<\/code>: coordinates of variables to create a scatter plot<\/li>var$cos2<\/code>: represents the quality \nof representation for variables on the factor map. It\u2019s calculated as \nthe squared coordinates: var.cos2 = var.coord * var.coord.<\/li>var$contrib<\/code>: contains the \ncontributions (in percentage) of the variables to the principal \ncomponents. The contribution of a variable (var) to a given principal \ncomponent is (in percentage) : (var.cos2 * 100) \/ (total cos2 of the \ncomponent).<\/li><\/ul>\n\n\n\n\nNote that, it\u2019s possible to plot variables and to color them according \nto either i) their quality on the factor map (cos2) or ii) their \ncontribution values to the principal components (contrib).\n<\/p>\n\n\n\n
The different components can be accessed as follow:<\/p>\n\n\n\n
# Coordinates\nhead(var$coord)\n# Cos2: quality on the factore map\nhead(var$cos2)\n# Contributions to the principal components\nhead(var$contrib)<\/code><\/pre>\n\n\n\nIn this section, we describe how to visualize variables and draw \nconclusions about their correlations. Next, we highlight variables \naccording to either i) their quality of representation on the factor map\n or ii) their contributions to the principal components.<\/p>\n\n\n\n
Correlation circle<\/h4>\n\n\n\n
The correlation between a variable and a principal component (PC) is \nused as the coordinates of the variable on the PC. The representation of\n variables differs from the plot of the observations: The observations \nare represented by their projections, but the variables are represented \nby their correlations (Abdi and Williams 2010). <\/p>\n\n\n\n
# Coordinates of variables\nhead(var$coord, 4)<\/code><\/pre>\n\n\n\n## Dim.1 Dim.2 Dim.3 Dim.4 Dim.5\n## X100m -0.851 -0.1794 0.302 0.0336 -0.194\n## Long.jump 0.794 0.2809 -0.191 -0.1154 0.233\n## Shot.put 0.734 0.0854 0.518 0.1285 -0.249\n## High.jump 0.610 -0.4652 0.330 0.1446 0.403<\/code><\/pre>\n\n\n\nTo plot variables, type this:<\/p>\n\n\n\n
fviz_pca_var(res.pca, col.var = \"black\")<\/code><\/pre>\n\n\n\n![\"\"\/](\"http:\/\/www.sthda.com\/english\/sthda-upload\/figures\/principal-component-methods\/006-principal-component-analysis-variables-correlation-circle-1.png\")
<\/figure>\n\n\n\nThe plot above is also known as variable correlation plots. It shows \nthe relationships between all variables. It can be interpreted as \nfollow:<\/p>\n\n\n\n
- Positively correlated variables are grouped together.<\/li>
- Negatively correlated variables are positioned on opposite sides of the plot origin (opposed quadrants).\n<\/li>
- The distance between variables and the origin measures the quality \nof the variables on the factor map. Variables that are away from the \norigin are well represented on the factor map.<\/li><\/ul>\n\n\n\n
Quality of representation<\/h4>\n\n\n\n
The quality of representation of the variables on factor map is called cos2<\/strong> (square cosine, squared coordinates) . You can access to the cos2 as follow:<\/p>\n\n\n\nhead(var$cos2, 4)<\/code><\/pre>\n\n\n\n## Dim.1 Dim.2 Dim.3 Dim.4 Dim.5\n## X100m 0.724 0.03218 0.0909 0.00113 0.0378\n## Long.jump 0.631 0.07888 0.0363 0.01331 0.0544\n## Shot.put 0.539 0.00729 0.2679 0.01650 0.0619\n## High.jump 0.372 0.21642 0.1090 0.02089 0.1622<\/code><\/pre>\n\n\n\nYou can visualize the cos2 of variables on all the dimensions using the corrplot package:<\/p>\n\n\n\n
library(\"corrplot\")\ncorrplot(var$cos2, is.corr=FALSE)<\/code><\/pre>\n\n\n\n