# Regression with categorical factor variable and the correlation among the variables

````lm(y~x1 + x2 -1)
````

where `x1` is a continuous numerical variable and `x2` is a categorical factor variable with 4 levels.

Is there a way to measure the "correlation" between the `x1` variable and each level of the factor variable `x2`? By putting correlation into double quotes, I admit that I don't really know what is a good definition for the associatedness between a continuous variable and a specific level of the factor variable. Hopefully readers get my intuition. I mean that some levels of `x2` may associated with `x1` more actively than other levels of `x2`.

Not knowing how to measure it, I am thinking of the following procedure:

1. run `lm(y~x2 +1)`

2. run `lm(y~x2 + x1 -1)`

i.e. replace the intercept in "Step 1" by the continous variable `x1` in "Step 2" and then see which $\beta$ (of associated factor level) changed most.

My questions are:

1. Does my approach make sense?
2. How do I measure if a beta (of a specific associated factor level) changed and by how much? Is there a way to make fair comparison and draw some meaningful conclusions?

Could anybody please shed some lights on me?

Thanks a lot!

-

For the purposes of this question, are you only interested in the association between `x1` and `x2`? – Macro Jun 25 '12 at 23:50
Luna, I am struggling to understand what role $y$ plays in this question about association between $x_1$ and $x_2$. Could you enlighten us about this? – whuber♦ Jun 26 '12 at 0:06
@whuber, it could be sensible to only be interested in the association between $x_1, x_2$ insofar as it affects the regression coefficient estimates when they are both entered into the same regression model. This seems like an ad hoc method of diagnosing confounding. Although, if $y$ doesn't play any role at all, then of course the problem becomes much simpler. – Macro Jun 26 '12 at 0:10
Thanks, @Macro. I'm still confused, because I understand "association" quite generally to refer to a relationship between two things whereas "affects the regression coefficients" appears to refer to something else altogether. To circumvent guessing and possible misinterpretation of what is being asked, I am hoping that Luna can make this all clear with a suitable edit. – whuber♦ Jun 26 '12 at 0:13
Hi Macro and whuber, thanks a lot for your help. The more complete picture is: initially we did the regression lm(y~x1 + x2 -1)... since our end goal was to fit the data y~(x1, x2) and it seems that whenver factor is involved, we should do y~x1+x2-1, so did we. But then we wanted the further study the relationship between the two variables x1 and x2. And what impact do they have on the regression, partially and jointly... and do they have a lot of collinearity between them and how does that impact the regression, etc. These are typical data exploring steps... – Luna Jun 26 '12 at 2:26
show 1 more comment

## 2 Answers

The two variables' degree of association (put another way, the extent to which x1 means differ by level of x2) can be tested with an anova, among other ways. And ordering up an eta-squared statistic as part of an anova procedure will tell you the percent of variance in x1 that can be explained by level of x2.

Also, in places it seems that you are seeking to learn about this relationship between x1 and x2, but in other places you say things such as

[...] some levels of x2 may associated with x1 more actively than other levels of x2.

This seems to indicate a misunderstanding. If x2 is fixed at a single level, it cannot be meaningful to talk about its association or correlation with anything. It is only when x2 is free to vary that it can covary with something else.

-

If the covariate `x1` is relevant to explaining variation in `y`, it should be included in your model. Adding it in and then taking it out does not do you much good...if should definitely always be in your model. Theory will help guide you in this decision.

Once you've decided to include or exclude the regressor `x1`, then you can move on to 'diagnostics' such as exploring multicollinearity. I highly doubt that multicollinearity will be a problem, since your categorical variable can only take on four values. For multicollinearity diagnostics, google "variance inflation factor" which for you, using R's 'HH' package, would be `vif(lm(y ~ x1 + x2))`.

1) Does my approach make sense?

No. The first regression `lm(y~x2)` merely will report the mean of `y` in each category of `x2`. The second regression will report the intercepts of each category, and an additional slope parameter on `x1` for the entire sample. Do you understand how these are different?

So, all the parameters will likely change, because they represent different things.

2) How do I measure if a beta (of a specific associated factor level) changed and by how much? Is there a way to make fair comparison and draw some meaningful conclusions? Could anybody please shed some lights on me?

See my answer to your first question. You do not want to approach the problem in this way.

Also, it would help clarify things if you (1) stated what your variables represent (e.g., apples, temperature, ??) and (2) what research question you are attempting to answer.

-

1
(-1) Model is selection involves too many considerations for your "definitely always" statement in paragraph 1 to hold true. Also, it doesn't seem right to conclude that just because x2 only takes 4 values it cannot be closely associated with x1. For all we know, the mean of x1 might depend in large part on the level of x2, which could be group membership, region, industry, type of apple, etc. – rolando2 Sep 24 '12 at 13:31
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 6, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9515940546989441, "perplexity_flag": "middle"}