# Thread:

1. ## Linear Regression, Residual Sum of Squares

Suppose the columns of a rank 4 design matrix
$X = [X_{1}, X_{2}, X_{3}, X_{4}]$ come in three groups
Group 1: $\{X_{1}\}$,
Group 2: $\{X_{2}, X_{3}\}$ and
Group 3: $\{X_{4}\}$ so that $X^T_{i}X_{j} = 0$ if i and j are indicies from different groups.
Next consider 4 models, with design matricies:
Model 1: $X_{(1)} = [X_{1}]$
Model 2: $X_{(2)} = [X_{1}, X_{2}, X_{3}]$
Model 3: $X_{(3)} = [X_{1}, X_{2}, X_{3}, X_{4}]$
Model 4: $X_{(4)} = [X_{1}, X_{4}]$

Now then the question asks to show that $SS_{2} - SS_{3} = SS_{1} - SS_{4}$ where $SS_{i}$ is the residual sum of squares (SS error) for the ith model.

$SS_{error} = Y^T Y - \hat\theta X^T Y$ is the only formula for SS error that I know, which I believe does not apply in this situation. So it means that I'm absolutely stuck! I think I'm looking at this the wrong way to be honest. Where should I look into? Any hints?
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 12, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.919986367225647, "perplexity_flag": "middle"}