# Chi-square test for equality of distributions: how many zeroes does it tolerate?

I am comparing two groups of mutants each of which can have only one out of 21 different phenotypes. I would like to see whether distribution of these outcomes is similar between two groups. I found an online test that calculates the "Chi-square test for equality of distributions" and gives me some plausible results. However, I have quite a few zeroes in this Table, so can I use chi-square in this case at all?

Here is the table with two groups and counts of particular phenotypes:

2 1
2 3
1 6
1 4
13 77
7 27
0 1
0 4
0 2
2 7
2 3
1 5
1 9
2 6
0 3
3 0
1 3
0 3
1 0
1 2
0 1
-

The Table did not come out right. Every odd number is a count from group 1 and every even number is the respective count from group 2 – Membran Oct 27 '10 at 14:42
I've reformatted your question. Is the table now correct? – csgillespie Oct 27 '10 at 15:04
yes, many thanks! – Membran Oct 27 '10 at 15:50

## 2 Answers

Perfectly feasible these days to do Fisher's 'exact' test on such a table. I just got p = 0.087 using Stata (tabi 2 1 \ 2 3 \ .... , exact. Execution took 0.19 seconds).

EDIT after chl's comment below (tried adding as a comment but can't format):

It works in R 2.12.0 for me, though i had to increase the 'workspace' option over its default value of 200000:

> fisher.test(x)
Error in fisher.test(x) : FEXACT error 7.
LDSTP is too small for this problem.
Try increasing the size of the workspace.
> system.time(result<-fisher.test(x, workspace = 400000))
user  system elapsed
0.11    0.00    0.11
> result\$p.value
[1] 0.0866764

(The execution time is slightly quicker than in Stata, but that's of dubious relevance given the time taken to work out the meaning of the error message, which uses 'workspace' to mean something different from R's usual meaning despite the fact that fisher.test is part of R's core 'stats' package.)

-

1
Interesting, Fisher's test crashed on R. – chl♦ Oct 27 '10 at 18:51
Cannot upvote more, sorry. It seems I hadn't increase the wksp enough :) – chl♦ Oct 27 '10 at 20:59
Isn't it that Fisher's "exact" test actually addresses slightly different question: "...it is used to examine the significance of the association (contingency) between the two kinds of classification" (wiki page). In my case I sought to confirm (or refute) the hypothesis that distributions of phenotypes between 2 groups are similar (equal). When I found that online test (see the first post) named "Chi-square test for equality of distributions" I thought it was precisely for my problem... – Membran Oct 27 '10 at 22:38
Also, if you think that mentioned version of Fisher's test is fine for comparing two distributions, can it also be used for checking uniformity of distribution (i.e. to say that phenotypes within one group were distributed non-uniformly between a finite number of possible phenotypes)? One can do this even in Excel using CHITEST function, but what if I have a distribution similar to the ones above, with lots of phenotypes observed less than 5 times? – Membran Oct 27 '10 at 22:42
@Membran # 1: It is a slightly different question as Fisher's exact test conditions on both sets of marginal totals. This seems something of an academic statistical nicety to me though, and I'm a statistician in academia. (BTW could you clarify to which wiki you refer?) @Membran #2: I would not call the conditional exact test "Fisher's exact test" in the case of a one-way table, but such a test should be possible.and I would have thought more straightforward for one-way tables, but I can't currently find software to assist and I don't have time to perform the calculation without. – onestop Oct 28 '10 at 4:07

The usual guidelines are that the expected counts should be greater than 5, but it can be somewhat relaxed as discussed in the following article:

Campbell , I, Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations, Statistics in Medicine (2007) 26(19): 3661–3675.

See also Ian Campbell's homepage.

Note that in R, there's always the possibility to compute $p$-value by a Monte Carlo approach (chisq.test(..., sim=TRUE)), instead of relying on the asymptotic distribution.

In you case, it appears that about 80% of the expected counts are below 5, and 40% are below 1. Would it make sense to aggregate some of the observed phenotypes?

-
Thank you for suggestions. Logically, it is not quite possible to merge phenotypes as each of them is a unique combination of three recorded parameters. Since each of these parameters can go "up", "down" or stay "unchanged" as a result of a mutation, so there can be 3^3=27 distinct phenotypes. In the example above I removed those phenotypes for which both groups scored "0", so there were only 21 of them. I do see the prevalence of certain phenotypes but i would like to have some statistical proof that distributions of such phenotypes in various groups of mutants is similar (or not). Thank you! – Membran Oct 27 '10 at 16:03

1
@Membran Aggregation doesn't have to be meaningful: you're free to combine bins any way you please. A subtle problem, though, is that post-facto aggregation casts the p-values in doubt; the aggregation ought to be independent of the data. – whuber♦ Oct 27 '10 at 21:16
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": false, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9403173923492432, "perplexity_flag": "middle"}