# How can I represent R squared in matrix form?

This question is a follow-up to a prior question.

Basically, I wanted to study under what conditions when we regress the residuals to $x_1$, we will get $\small R^2$ of 20%.

As a first step to attack this problem, my question is, how do I express $\small R^2$ in matrix form?

Then I will try to express "$\small R^2$ of regressing residuals to $x_1$" using matrix form.

Also, how can I add regression weights into the expression?

-

Re: Basically I wanted to study under what conditions when we regress the residuals to x1, we will get a R-squared of 20%, if the regression is an ordinary least squares regression, and $x_1$ was included in the model, then the answer is never, and I showed you in my answer here. If you're actually asking something else then please clarify. – Macro Jul 27 '12 at 15:36
Hi Macro, because I have weights in the regression. I wanted to be able to derive something show study the R^2. That's the reason for asking for the matrix form expression. Thank you! – Luna Jul 27 '12 at 19:06
Hi @Luna - OK but there is no mention of weights in this post. What kind of weights? Can you edit the post to clarify? – Macro Jul 27 '12 at 19:10
I had mentioned weights in that thread... here I am looking for a generic form - once I learn the generic form, I can add weights myself... right? – Luna Jul 27 '12 at 21:29

## 1 Answer

We have $$\begin{align*} R^2 = 1 - \frac{\sum{e_i^2}}{\sum{(y_i - \bar{y})^2}} = 1 - \frac{e^\prime e}{\tilde{y}^\prime\tilde{y}}, \end{align*}$$ where $\tilde{y}$ is a vector $y$ demeaned.

Recall that $\hat{\beta} = (X^\prime X)^{-1} X^\prime y$, implying that $e= y - X\hat{\beta} = y - X(X^\prime X)^{-1}X^\prime y$. Regression on a vector of 1s, written as $l$, gives the mean of $y$ as the predicted value and residuals from that model produce demeaned $y$ values; $\tilde{y} = y - \bar{y} = y - l(l^\prime l)^{-1}l^\prime y$.

Let $H = X(X^\prime X)^{-1}X^\prime$ and let $M = l(l^\prime l)^{-1}l^\prime$, where $l$ is a vector of 1's. Also, let $I$ be an identity matrix of the requisite size. Then we have

$$\begin{align*} R^2 &= 1- \frac{e^\prime e}{\tilde{y}^\prime\tilde{y}} \\ &= 1 - \frac{y^\prime(I - H)^\prime(I-H)y}{y^\prime (I - M)^\prime(I-M)y} \\ &= 1 - \frac{y^\prime(I-H)y}{y^\prime (I-M)y}, \end{align*}$$

where the second line comes from the fact that $H$ and $M$ (and $I$) are idempotent.

In the weighted case, let $\Omega$ be the weighting matrix used in the OLS objective function, $e^\prime \Omega e$. Additionally, let $H_w = X \Omega^{1/2} (X^\prime \Omega X)^{-1} \Omega^{1/2} X^\prime$ and $M_w = l \Omega^{1/2}(l^\prime \Omega l)^{-1} \Omega^{1/2} l^\prime$. Then, $$\begin{align*} R^2 &= 1 - \frac{y^\prime \Omega^{1/2} (I-H_w) \Omega^{1/2} y}{y^\prime \Omega^{1/2}(I-M_w) \Omega^{1/2}y}, \end{align*}$$

-
+1 It's nice (and elegant) to see the variance in the denominator pop out from regression against a constant. – whuber♦ Jul 27 '12 at 20:08
thanks a lot! I upvoted for you. Also, how to add regression weights to the expression? Thank you! – Luna Jul 27 '12 at 21:42

1
@Luna, if this post answers your question & provides the info you need, you should consider accepting it (by clicking the check mark to its left) as well as upvoting it. – gung Aug 11 '12 at 18:13

1
@Charlie, I think you may have accidentally dropped $y$ from your first equation for $\hat\beta$. Also, I don't quite follow your equation for $R^2$, I'm used to seeing $\sum(\hat y_i-\bar y)^2/\sum(y_i -\bar y)^2$ or $1-(\sum(y_i-\hat y_i)^2/\sum(y_i-\bar y)^2)$. I'm interpreting your $e^2$ as $\sum(y_i-\hat y_i)^2$, so I'm confused; is there a way that can make this clearer for me? – gung Aug 12 '12 at 3:45
@gung, You're right, I had $R^2$ defined incorrectly. I hope that it is correct now. Thanks! – Charlie Aug 13 '12 at 17:59
show 1 more comment
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 33, "mathjax_display_tex": 3, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9456334114074707, "perplexity_flag": "middle"}