# PCA analysis with centered variables

If I center my variables and then run a PCA analysis, do I need to interpret negative eigenvectors different than positive eigenvectors?

-

2
What does it mean that a vector is negative? If $\mathbf{x}$ is an eigenvector of the matrix $\mathbf{A}$ then $\mathbf{Ax}=\lambda\mathbf{x}$ for some $\lambda$. But then $A(-\mathbf{x})=-A\mathbf{x}=\lambda(-\mathbf{x})$ so $-\mathbf{x}$ is also an eigenvector. – MånsT Apr 12 '12 at 12:08

1
– whuber♦ Apr 12 '12 at 14:36
I don't mind, thank you @whuber. I think I need to rephrase my question. In my PCA analysis I have in a component both negative and positive variables. This is probably very basic, but I've been told different things about the interpretation of the variables within a component, so I just need some clarification. First; Is it so that the negative values within a component moves in the same direction and the positive in the opposite? Or is it so that I should only look at the absolute values of the variables within a component? I hope this makes sense. – Hanne Apr 12 '12 at 22:30
Ah, so what you mean is how to interpret the relative signs of the coefficients of the eigenvectors. (Even the signs are meaningless because negating an eigenvector will reverse all the signs, but the relative signs do not change, no matter how the eigenvector is scaled, and therefore are meaningful.) Because not everybody reads all comments, it would be good for you to edit your question to make this clear. – whuber♦ Apr 13 '12 at 13:33
– whuber♦ Aug 27 '12 at 18:06

## 2 Answers

I think you have it backwards. If the value is positive, then a higher score on that variable is associated with a higher score on the component, if the value is negative, then a higher score implies a lower score on the component.

In addition, people sometimes use PCA to determine whether to keep or combine certain variables for a subsequent analysis. This is not, strictly speaking, an appropriate use of PCA. Factor analysis should be used for this purpose, but at any rate, people do it. In such a case, people will look at the absolute value to see if it is above some arbitrary threshold, such as .5, and if so, retain (or combine), and if not, drop. For what it's worth, I don't recommend this.

Update: I can't tell if I answered the right question or not. @whuber's second comment, in my opinion, is right on the money, and also consistent with my first paragraph above. However, the question is now different than before, and different from how I understand @whuber's comment, so I am a little confused. Essentially, PCA solves for the eigenvectors and eigenvalues. Neither will be negative whether or not you centered your variables first. The eigenvalues are the lengths of the corresponding eigenvectors. Just as I cannot buy a board -10 feet (i.e., -3 meters) long to build a patio, you cannot have a negative eigenvalue. The eigenvector returned will also be positive. You could negate it by multiplying all the signs by -1, but as @whuber notes, that would be meaningless. Once again as @whuber notes, the relative signs are meaningful, and their relation to the component is as I stated in my first paragraph above. That is, the relative signs (negative vs. positive) will denote the same relationship between higher (/ lower) scores on the variable and the component whether the variables were centered first or not.

-

centering your variables shouldn't change the PCA results, as PCA first determines a correlation matrix and goes on from there. The correlations between your variables should be the same regardless, so the PCA results should not be affected by any mean centering you perform.

-
I think in practice that is not always the case. Usually, $\mathbf X \mathbf X'$ is decomposed. If $\mathbf X$ is mean centered, $\mathbf X \mathbf X' = COV (\mathbf X)$, if it is mean centered and variance scaled, then $\mathbf X \mathbf X' = COR (\mathbf X)$. many (most) PCA functions will do the mean centering and/or variance scaling as part of their default pre-processing, but many also allow to switch this off. See e.g. the arguments `center` and `scale.` of `prcomp` in R. – cbeleites Apr 13 '12 at 12:48
@cbeleites that is interesting, thanks. With regard to the original question, the centering should not change the PCA results, unless you specify alternative options when you call the PCA function? – Luke Apr 13 '12 at 12:53
iff the default option is to center, then the results should not change (but they may flip as Måns explained). If you don't know whether your PCA algorithm does the centering and scaling, you can actually test it that way. – cbeleites Apr 13 '12 at 13:14
To be precise: the above formulae are not exactly correct for $COV$ and $COR$ but are what PCA usually does. If you really want to calculate correlation or covariance, you need to divide by the degrees of freedom $COV(\mathbf X) = \frac{1}{df} \mathbf X_{centered} \mathbf X_{centered}^′$; e.g. for the sample covariance $df = n - 1$ unless you have extra knowledge like mean must be zero for theoretical reasons. – cbeleites Apr 13 '12 at 13:23

1
@whuber: Sure! Depending on the data, the uncentered first PC may actually be very close to the mean vector. However, most PCA functions will do the centering by default, and the results do not change whether the mean centering is done outside the PCA function or inside or both outside and again inside. And it may be quite hidden in the results, e.g. R's `prcomp` has the center but by default doesn't print it. I've also seen it reported it as the "$0^{th}$ component". So it may not be very obvious that it is actually done. – cbeleites Apr 13 '12 at 14:28
show 1 more comment
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 15, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 20, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9310716390609741, "perplexity_flag": "middle"}