# Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given $y$ in terms of $W$, i.e. we want to minimize the entropy $H(x|y,W)$ as a function of $W$. Suppose that, during learning, we know $x$ for every observed state $y$: what is the optimal supervised update of the model parameters $W$?

The problem is: I can't wrap my head around the question how I come - in a rigorous way - from the estimation problem that I want to solve to the parameter updates that depend on the variable I want to estimate. It's probably a standard procedure but the problem is hard to nail down to Google-friendly buzzwords.

Thanks in advance!! blue2script

-

## 1 Answer

I dont have answer to your question but i am interested in knowing what exactly does this statement in your question mean :

"Our aim is to minimize the estimation error of x given y in terms of W, i.e. we want to minimize the entropy $H(x|y,W)$ as a function of W"

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 12, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9214874505996704, "perplexity_flag": "head"}