## Could SVD be used to optimize the partial inner-products?

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

Suppose a set $N$ of $n$ distinct points in $m-$dimensional space is given in $X\in\mathbb{R}^{n\times m}$. Also, suppose a subset $L\subset N$, `$|L|=l<m<n$`, with $m-$dimensional coordinates in $X_l\in\mathbb{R}^{l\times m}$.

Now, given a inner-product matrix $M=XX_l^T\in\mathbb{R}^{n\times l}$, could the solution $Z$ minimizing `$$||M-ZZ_l^T||~~~\equiv~~~||M-ZZ_l^T||^2,$$` where $||\cdot||$ denotes the Frobenius matrix norm, over all $Z\in\mathbb{R}^{n\times k}$, `$k<m$`, be obtained in a closed form. Note that $Z_l\in\mathbb{R}^{l\times k}$ is a configuration corresponding to the subset $L\subset N$ defined above.

With the Eckart-Young theorem in mind, I'm inclined to think that the solution $Z$ might be $$Z=U\Sigma_k^{1/2},$$ where $\Sigma_k^{1/2}$ is a diagonal matrix containing square roots of only $k$ dominant singular values of $M$, with $U$ containing the corresponding left-singular vectors as its columns. The plain Eckart-Young theorem might be used by showing that $Z_l$ is related to right-singular vectors $V$, as in $$Z_l=V\Sigma_k^{1/2},$$ but I believe the answer might not be easily expressible in a closed-form. Feel free to post an existing theorem that might be helpful for proving the above, either exactly or with certain guarantees on the approximation, ie. "by taking $$Z=U\Sigma_k^{1/2},$$ as above, we are guaranteed that the error wrt $$\min_Z||M-ZZ_l^T||~~~\equiv~~~\min_Z||M-ZZ_l^T||^2$$ is not larger than ..." Also, notes on some related problems already present in the literature will be appreciated.

Note that in case $L=N$, the solution indeed translates to the Eckart-Young theorem, thence with optimal solution being based (now) on spectral decomposition of $M$ relying only on its $k$ dominant eigenvalues, hence with $$Z=E\Lambda_k^{1/2},$$ where $\Lambda_k^{1/2}$ is a diagonal matrix with square roots of only $k$ dominant eigenvalues of $M$, and $E$ is a matrix containing the corresponding eigenvalues.

-

Could someone edit my question? The preview seems not to capture everything I wrote. – usero Mar 14 2012 at 16:25
Done. As a rule of thumb, put backticks (...) around formulas when they are displayed incorrectly (as is reminded in the "How to write math" box on the right of every page). – Federico Poloni Mar 14 2012 at 16:30
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 27, "mathjax_display_tex": 5, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.3, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9113718867301941, "perplexity_flag": "head"}