# What if interaction wipes out my direct effects in regression?

In a regression, the interaction term wipes out both related direct effects. Do I drop the interaction or report the outcome? The interaction was not part of the original hypothesis.

-

2
you could probably get a better answer if you provided more details about your experimental design, research question, and statistical model. – David Dec 13 '10 at 23:52
I have survey data, v1 and v2 predict the outcome, as I expected; however, the interaction between v1 (dichotomous) and v2 (5 groups) is not significant -- and (my question) it makes my v1 and v2 direct effects non-significant too. I can't find an example on reporting this in the literature. – Jen Dec 14 '10 at 0:03
If the v1:v2 interaction is not significant, do you need to have it included in the model? – Christopher Aden Dec 14 '10 at 1:40
– Glen Dec 14 '10 at 3:00

## 3 Answers

I think this one is tricky; as you hint, there's 'moral hazard' here: if you hadn't looked at the interaction at all, you'd be free and clear, but now that you have there is a suspicion of data-dredging if you drop it.

The key is probably a change in the meaning of your effects when you go from the main-effects-only to the interaction model. What you get for the 'main effects' depends very much on how your treatments and contrasts are coded. In R, the default is treatment contrasts with the first factor levels (the ones with the first names in alphabetical order unless you have gone out of your way to code them differently) as the baseline levels.

Say (for simplicity) that you have two levels, 'control' and 'trt', for each factor. Without the interaction, the meaning of the 'v1.trt' parameter (assuming treatment contrasts as is the default in R) is "average difference between 'v1.control' and 'v1.trt' group"; the meaning of the 'v2.trt' parameter is "average difference between 'v2.control' and 'v2.trt'".

With the interaction, 'v1.trt' is the average difference between 'v1.control' and 'v1.trt' in the 'v2.control' group, and similarly 'v2.trt' is the average difference between v2 groups in the 'v1.control' group. Thus, if you have fairly small treatment effects in each of the control groups, but a large effect in the treatment groups, you could easily see what you're seeing.

The only way I can see this happening without a significant interaction term, however, is if all the effects are fairly weak (so that what you really mean by "the effect disappeared" is that you went from p=0.06 to p=0.04, across the magic significance line).

Another possibility is that you are 'using up too many degrees of freedom' -- that is, the parameter estimates don't actually change that much, but the residual error term is sufficiently inflated by having to estimate another 4 [ = (2-1)*(5-1)] parameters that your significant terms become non-significant. Again, I would only expect this with a small data set/relatively weak effects.

One possible solution is to move to sum contrasts, although this is also delicate -- you have to be convinced that 'average effect' is meaningful in your case. The very best thing is to plot your data and to look at the coefficients and understand what's happening in terms of the estimated parameters.

Hope that helps.

-
Thanks Ben! I'll let you know what works out. – Jen Dec 14 '10 at 4:24

1
There's no moral hazard. The calculation of the main effects with the interaction included is quite different from the calculation without it. You have to do the additive model to report the main effects and then include the interaction in a separate model anyway. You ignore the main effects in the model that includes the interaction because they're not really main effects, they're effects at specific levels of the other predictors (including the interaction). – John Jul 10 '12 at 14:08

Are you sure the variables have been appropriately expressed? Consider two independent variables $X_1$ and $X_2$. The problem statement asserts that you are getting a good fit in the form

$$Y = \beta_0 + \beta_{12} X_1 X_2 + \epsilon$$

If there is some evidence that the variance of the residuals increases with $Y$, then a better model uses multiplicative error, of which one form is

$$Y = \beta_0 + \left( \beta_{12} X_1 X_2 \right) \delta$$

This can be rewritten

$$\log(Y - \beta_0) = \log(\beta_{12}) + \log(X_1) + \log(X_2) + \log(\delta);$$

that is, if you re-express your variables in the form

$$\eqalign{ \eta =& \log(Y - \beta_0) \cr \xi_1 =& \log(X_1)\cr \xi_2 =& \log(X_2)\cr \zeta =& \log(\delta) \sim N(0, \sigma^2) }$$

then the model is linear and likely has homoscedastic residuals:

$$\eta = \gamma_0 + \gamma_1 \xi_1 + \gamma_2 \xi_2 + \zeta,$$

and it may just so happen that $\gamma_1$ and $\gamma_2$ are both close to 1.

The value of $\beta_0$ can be discovered through standard methods of exploratory data analysis or, sometimes, is indicated by the nature of the variable. (For instance, it might be a theoretical minimum value attainable by $Y$.)

Alternatively, suppose $\beta_0$ is positive and sizable (within the context of the data) but $\sqrt{\beta_0}$ is inconsequentially small. Then the original fit can be re-expressed as

$$Y = (\theta_1 + X_1) (\theta_2 + X_2) + \epsilon$$

where $\theta_1 \theta_2 = \beta_0$ and both $\theta_1$ and $\theta_2$ are small. Here, the missing cross terms $\theta_1 X_2$ and $\theta_2 X_1$ are presumed small enough to be subsumed within the error term $\epsilon$. Again, assuming a multiplicative error and taking logarithms gives a model with only direct effects and no interaction.

This analysis shows how it is possible--even likely in some applications--to have a model in which the only effects appear to be interactions. This arises when the variables (independent, dependent, or both) are presented to you in an unsuitable form and their logarithms are a more effective target for modeling. The distributions of the variables and of the initial residuals provide the clues needed to determine whether this may be the case: skewed distributions of the variables and heteroscedasticity of the residuals (specifically, having variances roughly proportional to the predicted values) are the indicators.

-

Hmmm. This all seems plausible but more complex than my solution (the comments on the original question suggest that the predictors are both categorical). But as usual, the answer is "look at the data" (or the residuals). – Ben Bolker Dec 14 '10 at 15:34
@Ben I agree but I don't understand where the perception of "more complex" comes from, because analysis of univariate distributions and post-hoc analysis of residuals are essential in any regression exercise.  The only extra work required here is to think about what these analyses mean. – whuber♦ Dec 14 '10 at 15:53
Perhaps by "more complex" I just mean "In my experience, I have seen the issues I referred to in my answer (contrast coding) arise more frequently than those you referred to (non-additivity)" -- but this is really a statement about the kinds of data/people I work with rather than about the world. – Ben Bolker Dec 14 '10 at 16:23

In a regular multiple regression with two quantitative predictor variables, including their interaction just means including their observation-wise product as an additional predictor variable: $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 (X_1 \cdot X_2) = (b_0 + b_2 X_2) + (b_1 + b_3 X_2) X_1$

This typically introduces high multicollinearity since the product will strongly correlate with both original variables. With multicollinearity, individual parameter estimates depend strongly on which other variables are considered - like in your case. As a counter-measure, centering the variables often reduces multicollinearity when the interaction is considered.

I'm not sure if this directly applies to your case since you seem to have categorical predictors but use the term "regression" instead of "ANOVA". Of course the latter case is essentially the same model, but only after choosing the contrast coding scheme as Ben explained.

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 16, "mathjax_display_tex": 6, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 20, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9423273205757141, "perplexity_flag": "middle"}