Prove that entropy is maximized when probability is $1/n$

How can be proven that the entropy of a dice roll is maximized when the probability of each of its $6$ faces is equal, $1/6$?

-

2 Answers

Suprise is defined as -log(p{X=x}). A good way to think of entropy is the "expected surprise". In this sense, its easy to see that the uniform distribution maximizes the expected surprise.

-

not sure how to do it... can you help? – whynot Oct 26 '12 at 5:55

The entropy is given by $-\sum p_i\ln p_i$. Use Jensen's inequality with the logarithm function.

-

not sure how to do it... can you help? – whynot Oct 26 '12 at 4:52
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 4, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": false, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9440367221832275, "perplexity_flag": "head"}