# Finding a fit to a multi-dimensioned function

I have a model function $f:\mathbb{R}^2\rightarrow\mathbb{R}^2$, and a bunch of data points for which I'd like Mathematica to fit for me. Unfortunately `FindFit` seems to only deal with functions $\mathbb{R}^n\rightarrow\mathbb{R}$. I guess I could make my own square difference from the data to the model and use `NMinimize` on that, but I wondering if there was an easier way?

### Edit - toy example with answers

$g\left(x,y\right)=\left<abx+(2c+d)y,(2a+b)x+cdy\right>$

````g[{x_, y_}, {a_, b_, c_, d_}] := {a b x + (2 c + d) y, (2 a + b) x + c d y};
points = Flatten[Table[{{x, y}, g[{x, y}, {1, 2, 3, 4}]},
  {x, -1, 1, 0.5}, {y, -1, 1, 0.5}], 1];
````

Finding a fit for each component separately doesn't work:

````data1 = points /. {{x_, y_}, {u_, v_}} -> {x, y, u};
data2 = points /. {{x_, y_}, {u_, v_}} -> {x, y, v};
FindFit[data1, g[{x, y}, {a, b, c, d}][[1]], {a, b, c, d}, {x, y}]
FindFit[data2, g[{x, y}, {a, b, c, d}][[2]], {a, b, c, d}, {x, y}]
````

`{a -> 0.729723, b -> 2.74077, c -> 3.35659, d -> 3.28681}`

`{a -> 1.48555, b -> 1.02891, c -> 2.81936, d -> 4.25629}`

Jens suggestion works well:

````data3 = points /. {{x_, y_}, {u_, v_}} -> Sequence[{x, y, 0, u}, {x, y, 1, v}];
FindFit[data3, g[{x, y}, {a, b, c, d}].{1 - s, s}, {a, b, c, d}, {x, y, s}]
````

`{a -> 1., b -> 2., c -> 3., d -> 4.}`

My original suggestion (which is similar to user840's links)

````error = Plus @@ (points /.
    {{x_, y_}, {u_, v_}} -> Norm[g[{x, y}, {a, b, c, d}] - {u, v}]^2);
NMinimize[error, {a, b, c, d}]
````

Gives a different, but still correct, solution (or at least a close approximation thereof)

`{1.41754*10^-13, {a -> 1.00004, b -> 1.99993, c -> 2., d -> 6.}}`

-

How about using NonlinearModelFit? Take a look into the documentation and the examples. You will see that under Scope the first example shows how one can fit a model of more than one variable. If you provide example data, it would be possible to help you more detailed. – partial81 May 9 '12 at 5:53
@partial81 More than one variable, not but not a vector value.  That example is for $\mathbb{R}^2 \rightarrow \mathbb{R}$ while the OP wants  $\mathbb{R}^2 \rightarrow \mathbb{R}^2$. `NonlinearModelFit` doesn't seem to support this. – Szabolcs May 9 '12 at 8:11
ok, I assume you are right. I would love to try it with `NonlinearModelFit`, but without example data, I do not want to spend more time on this problem. – partial81 May 9 '12 at 10:54
Why not two fits, one for each dependent variable? – Daniel Lichtblau May 9 '12 at 18:30
@DanielLichtblau Each variable depends on all the model parameters, so I end up with different parameter values for the different fits. – wxffles May 9 '12 at 21:55
show 2 more comments

## 3 Answers

Without a specific example, I can only make general suggestions here.

The first thing that comes to mind is that you could fit each of the two components of your vector field independently. I.e., if $\vec{f}: \mathbb{R}^2\to\mathbb{R}^2$ is split up into $\vec{f} = \{f_x, f_y\}$ with $f_{1,2}: \mathbb{R}^2\to\mathbb{R}$, then `FindFit` would work on each of these component functions.

If you don't want the fits to be determined independently, it could still be possible to use `FindFit` by introducing an auxiliary variable $s$ that labels the two component functions above, $f_s$, and then provide the fitting data with this variable `s` included. Here, `s` can only have two discrete values (I borrowed the idea from spin-half quantum mechanics). Let's choose `s = 0` for the function $f_x$ and `s = 1` for $f_y$. So you'd have data in the form

````data = {{x1, y1, 0, fx1}, {x1, y1, 1, fy1}, ...}
````

where `fx1` is the first value of $f_x$ and `fy1` the first value of $f_y$, both at the point `x1, y1`.

Your model could then look something like this:

````model[x_, y_, s_] := modely[x, y]*s + modelx[x, y]*(1 - s)
````

where `modely[x, y]` is the model for $f_y$ and `modelx[x, y]` is the model for $f_x$. The variables in the `FindFit` call would be `x, y, s`, and the parameters in the models `modelx`, `modely` could be the same (or different).

There are probably many other reasonable ways to do such a fit, but these are some ideas.

-

Thanks.  This is working well for my problem.  `FindFit[data /. {{a_, b_}, {p_, q_}} -> Sequence[{a, b, 0, p}, {a, b, 1, q}], model.{1 - s, s}, ...]` – wxffles May 9 '12 at 21:33

Multi-layer perceptrons have this ability. The Neural network package from Wolfram offers this functionality.

Neural Networks

-

Since that isn't part of the standard suite of packages, could you provide a link? – rcollyer May 9 '12 at 13:14
True, but now you have the additional problem of structuring the neural net appropriately (which is by no means trivial if you don't have any experience with neural nets). And after that you have to train it, which is equivalent to the original problem... – Oleksandr R. May 9 '12 at 13:46
Yes there is some effort involved, but it offers a different approach to the solution, which I thought might fill in parts of the landscape of the large selection of approximation techniques that might be deployed outside of the half a dozen or so that `FindFit` offers and MLPs have inherent support for output vectors in higher dimensions. – image_doctor May 9 '12 at 15:35

You can check Fitting Multiple-Response Data from Bruce Miller. He gives a step-by-step example of how to perform a simultaneous fit to multiple functions, and provides a fairly general function for performing such fits.

Alternately, Multiple-Response Fitting notes from wolframThis package contains functions for simultaneously fitting multiple functions to data with a possibly shared set of parameters. It has flexible output options and the first part of the accompanying demonstration notebook can serve as an introduction to writing one's own fitting codes using FindMinimum.

-
lang-mma
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 16, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.905121386051178, "perplexity_flag": "middle"}