## Why doesn’t Stein effect happen for multinomial distributions?

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

(Medeen, et all, 1998)" show that Maximum Likelihood estimate is admissible for multinomial distribution under squared error. On other hand, James and Stein showed that arithmetic average is not an admissible estimator of Gaussian location parameter in 3 dimensions. But since maximum likelihood estimates of multinomial parameters are averages of observed counts, which become normally distributed for large sample sizes, why doesn't Stein effect happen here?

$\hat{p}$ is an inadmissible estimator of $\theta$ if there's an estimator that is no worse for every $\theta$ and better for at least one

-

Minor comment: I believe that was James and Stein, not "James Stein". – Mark Meckes Sep 17 2010 at 1:01

## 1 Answer

This is not an answer, but maybe worth thinking about (and I cannot yet leave comments). My intuition about the Stein phenomenon is that while the individual coordinates of the Gaussian random variable are independent, the loss function involves all of the location parameters jointly. Stein type estimators take this into account and by doing so outperform the MLE, making it inadmissible.

In the case of the multinomial parameters, they inherently have dependence via the sum-to-one constraint as a probability vector and you take this into account when averaging over possible parameter values. So a question related to yours, which may shed some light on it, is whether or not the MLE is admissible for a Gaussian location vector $\mu$ under the restriction that $\|\mu\| = c$ for some positive constant $c$.

UPDATE: "Admissibility and complete class results for the multinomial estimation problem...", Ighodaro, Thomas & Brown (Journal of Mult. Analysis '82) shows the MLE for the multinomial parameter becomes inadmissible if you remove the vertices of the simplex from the action space. It is a property of the risk behavior of the MLE at these extremal points that makes it admissible, then. Since the corresponding Gaussian problem has no such extremal points, this may constitute an explanation to your question.

-

I think normalization issue is minor because you can just drop the last parameter, and measure admissibility on the reduced parameter vector. To make it more in line with Gaussian example, let your random variable be an average of k observations with last component dropped. For large k it becomes distributed as a Gaussian. Then MLE estimate of the mean of your distribution (first d-1 components) is an arithmetic average of a sample of (approximately) Gaussian-distributed random variables – Yaroslav Bulatov Sep 17 2010 at 4:58
It isn't the normalization per se, it's the restriction of the action space.  I've edited my answer to clarify this and included a reference.   – R Hahn Sep 17 2010 at 6:25
Nice find! So I guess Stein effect does happen there.  – Yaroslav Bulatov Sep 18 2010 at 19:08
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 6, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8883407115936279, "perplexity_flag": "middle"}