# Tagged Questions

Given multivariate data split into several subsamples (classes) the analysis finds linear combinations of variables, called discriminant functions, which discriminate between classes and are uncorrelated. The functions are applied then to assign old or new observations to the classes. Discriminant ...

1answer
620 views

### Treatment of outliers produced by Kurtosis

I was wondering if anyone could help me with information about Kurtosis (i.e. is there any way to transform your data to reduce it?) I have a questionnaire dataset with a large number of cases and ...
1answer
677 views

### Cluster Analysis followed by Discriminant Analysis

What is the rationale, if any, to use Discriminant Analysis (DA) on the results of a clustering algorithm like k-means, as I see it from time to time in the literature (essentially on clinical ...
2answers
431 views

### Why prediction of a predicted variable from a discriminant analysis is imperfect

I am puzzled by something I found using Linear Discriminant Analysis. Here is the problem - I first ran the Discriminant analysis using 20 or so independent variables to predict 5 segments. Among the ...
2answers
592 views

### How does linear discriminant analysis reduce the dimensions?

There are words from "The Elements of Statistical Learning" on page 91: The K centroids in p-dimensional input space span at most K-1 dimensional subspace, and if p is much larger than K, this ...
1answer
157 views

### Variant of discriminant analysis for known multiple independent classifications?

I have a large data set: over 100,000 data points, each with 60 dimensions. I want to display the data in 2D to visibly maximize the separation between classes, which I know for each point. I asked a ...
1answer
217 views

### Classifying clusters using discriminant analysis

Suppose I've data for 100 individuals for 5 variables, say Var1, Var2,...Var5. I run the cluster analysis using these 5 variables on these 100 rows & got 3 clusters. Now, I want to differentiate ...
3answers
1k views

### What is the relationship between regression and linear discriminant analysis?

Is their a relationship between regression and linear discriminant analysis? What are their similarities and differences? Does it make any difference for two classes and more than two classes?
1answer
405 views

### Multi class LDA vs 2 class LDA

The problem of designing a multi-class classifier using LDA can be expressed as a 2 class problem(one vs everything else) or a multi-class problem. Why is it that in certain cases Multi-class LDA ...
2answers
162 views

### How do you identify the variables that separate several groups?

I don't have much background on statistics. I am working on multivariate morphometrics of a sample of frogs. I have a data matrix of 19 variables (continuous characteristics) for around 250 samples. ...
0answers
134 views

### Using QDA for Non-Gaussian distributions

I am evaluating a Quadratic Discriminant Analysis (QDA) classifier on a high-dimensionality feature set. The features come from highly non-Gaussian distributions. However, when I transform the ...
2answers
567 views

### Why are Gaussian “discriminant” analysis models called so?

Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative ...
1answer
209 views

### What do “real values” refer to in supervised classification?

I'm using supervised classification algorithms from mlpy to classify things into two groups for a question-answering system. I don't really know how these algorithms work, but they seem to be doing ...
2answers
488 views

### Linear discriminant analysis and Bayes rule

What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
1answer
84 views

### Is discriminant analysis supervised learning?

Is linear discriminant analysis, specifically Linear Programming Discriminant Analysis (LPDA), supervised learning? Can you provide a valid reference that states so if possible. My study supervisor ...
2answers
667 views

### How to interpret a predictor with a positive structure coefficient and a negative standardised coefficient in discriminant function analysis?

I am doing a discriminant function analysis and I have four continous independent variables and one categorical dependent variable (that has 3 groups). I have chosen to do this analysis to see how ...
1answer
260 views

### Plotting a discriminant as line on scatterplot

Given a data scatterplot I can plot the data's principal components on it, as axes tiled with points which are principal components scores. You can see an example plot with the cloud (consisting of 2 ...
1answer
1k views

### Deriving total (within class + between class) scatter matrix

I was fiddling with PCA and LDA methods and I am stuck at a point, I have a feeling that it is so simple that I can't see it. Within-class ($S_W$) and between-class ($S_B$) scatter matrices are ...
2answers
312 views

### Theory on discriminant analysis in small sample size conditions

I see a similarity between a problem I'm working on and Linear (or Quadratic) Discriminant Analysis when the sample size is smaller than $p+1$. I'm interested in theory bounding the generalization ...
1answer
166 views

### Choosing variables for Discriminant Analysis

I've 110 variables & 200 data points. Of this 110 variables, one is group variable (say "brown eye","blue eye"). I want to use discriminant analysis to classify the groups based on remaining 119 ...
0answers
129 views

### LDA, Significance of orthonormality- Trace Ratio Maximization

The objective of fisher linear discriminant analysis can be formulated as maximizing $\frac{Tr[X^TAX]}{Tr[X^TBX]}$ over $X$ where $A$ and $B$ are positive semi-definite with orthonormality constraints ...
2answers
125 views

### Where does the definition of the hyperplane in a simple SVM come from?

I'm trying to figure out support vector machines using this resource. On page 2 it is stated that for linearly separable data the SVM problem is to select a hyperplane such that \$\vec{x}_i\vec{w} + b ...
2answers
191 views

### Can you use discriminant analysis to classify new observations into categories generated by a previous $k$-means clustering?

After doing k-means clustering on a set of observations, I would like to construct a discriminant function so as to classify new observations into the categories I found after k-means. Is this at all ...
1answer
149 views

### What exactly the ROC curve can tell us or can be inferred?

(I post this originally at http://stackoverflow.com/questions/15477282/what-exactly-the-roc-curve-can-tell-us-or-can-be-inferred, but people directed me to here. Sorry about posting this twice.) I ...
3answers
243 views

### Does PCA followed by LDA make sense?

This is a question about classification. I am a neuroscience student with little experience of classification methods and I'd be grateful for any advice about the best way to implement a linear ...
1answer
306 views

### Non-parametric discriminant analysis in R

I want to use Discriminant Analysis between two non normal populations in R. Can anybody tell me the name of the R function to do so? Could also anybody tell me how accurate my results will be if I ...
3answers
301 views

### Usage of LDA with more than two classes

I'm reading about the Linear Discriminant Analysis by Fisher and I have a couple of questions about its usage. If you have k>2 classes in a two-dimensional space you find k−1 vectors that you need ...
1answer
121 views

### Fisher discrimination power of a variable and Discriminant analysis

Apparently, the Fisher analysis aims at simultaneously maximising the between-class separation, while minimising the within-class dispersion. A useful measure of the discrimination power of a ...
1answer
222 views

### Decision boundaries from coefficients of linear discriminants?

I have a data set with four variables and 3000+ observations on which I performed an LDA. I was wondering how I can use the scaled coefficients of linear discriminants (output of R shown below as ...
1answer
154 views

### How to estimate the deposit mix of a bank using interest rate as the independent variable?

Let's say a bank has 5 different types of deposits. One type is certificates of deposits (CD), and the other 4 types are different checking and savings account products with various interest rates ...
0answers
85 views

### How do you detect if a given dataset has multivariate normal distribution?

I'm looking at Fisher's LDA on various datasets on UCI ML repository and trying to see where LDA might perform badly. One reason I can think of is if the data distribution is not a multi-variate ...
1answer
102 views

### Sample size and documentation for discriminant analysis

Does anybody have good documentation for discriminant analysis? I have 9 variables (measurements), 60 patients and my outcome is good surgery, bad surgery. Also, is my sample size too small? Thank ...
1answer
42 views

### Inconsistency in cross-validation results

I have a set of dataset recorded from subjects as they perform some particular cognitive task. The data consists of 16 channels and a number of sample points per channel and I want to classify this ...
1answer
184 views

### Dealing with high dimension in principal component analysis

For very extreme high dimensions in PCA, the number of dimensions $p$ is larger than the sample size $N$, does PCA work well or does it work at all? By 'work' I mean does it work mathematically If ...
1answer
122 views

### How do I test whether I can properly apply LDA?

I have some data which works nicely with JMP's canned linear discriminant analysis (LDA), but after reading about LDA I'm not sure if the analysis is valid. The Wiki article notes a fundamental ...
1answer
40 views

### Why is there a sharp elbow in my ROC curves?

I have some EEG data sets that I am testing against two classes. I can get a decent error rate from LDA (the class-conditional distributions aren't Gaussian, but have similar tails and good enough ...
1answer
170 views

### How does Fisher LDA work?

Intuitively, how does Fisher LDA work? From this Linear discriminant analysis and Bayes rule I completely understood the Bayesian approach but I'm not able to relate it to the Fisher's one described ...
1answer
182 views

### LDA projection for classification

I am dealing with 2 class LDA classification problem. During a test phase (after training), I'm trying to project a feature vector to lower dimensional space. How do we get the projected test ...
0answers
58 views

### Using linear discriminant analysis to validate the cluster groups resulting from kmeans

I'm currently working on a cluster analysis project and ran kmeans on the data for k=2. I was reading similar articles on similar experiments, and the investigators used discriminant analysis to ...
0answers
73 views

### How to interpret this output in Dimension Reduction?

I'm running Linear Discriminant Analysis on a dataset and then performing clustering on it. I'm reducing it to dimensions 2,6,10. On comparing metrics like Accuracy and Normalized Mutual Information, ...
0answers
83 views

### How to overcome singularity problem in Linear Discriminant Analysis?

I've code for LDA which is failing as the matrices passed to calculate eigen values are not singular and hence lead to infinite eigen values? Can anyone recommend what can be done to fix this? I've ...
0answers
47 views

### Overfitting a linear Linear Discriminant Function

I am estimating a Linear Discriminant function with 250 input variables over 4000 data records. Should I consider feature selection, am I over fitting the model? How do I know when feature selection ...
0answers
88 views

### Discriminant analysis with random effects

Is it possible to do discriminant analysis with random effects? Is there an R package for this? Context: I have habitat use data for two species of frogs from radio telemetry, but nested within ...
0answers
93 views

### Different approaches of linear discriminant analysis

What are the differences in three different approaches of classification of LDA for two groups? The three are: 1. Fisher's approach 2. regression approach 3. Bayes's approach They will be exactly ...
0answers
196 views

### How to combine features extracted by PCA, LDA and LBP?

What I'm thinking is to combine PCA features, LDA features and LBP features together to get a higher accuracy, since I think the three features are all kind of histogram vectors and when we decide the ...
0answers
127 views

### LDA with a categorical predictor variable

I have a dataset with a categorical treatment (1 if treated, 0 if control) and many columns (34) of response variables. Each column represents a species and its response (some measured abundance) to ...
0answers
55 views

### Difference and connection between generative learning, discriminative learning and max-margin learning

I once heard that, generative learning, discriminative learning and max-margin learning can be separated in terms of their respective definition of loss function. I am not sure how to achieve that?
0answers
359 views

### How to classify new cases in discriminant analysis exactly as SPSS does?

What I have: There is one base with already classified cases. There are 23 independent variables that were used in this classification and 10 groups. Another base has new unclassified cases. There ...
0answers
106 views

### Range of standardized coefficients in a discriminant analysis

I want to run a discriminant analysis on different motion capture measures to see which of the measures distinguishes best between my two conditions. The problem is that some of the standardized ...
2answers
126 views

### How can you make linear discriminant analysis reduce dimensions to the number of dimensions you are looking for?

Let's say I have a $m \times n$ matrix where $m$ is the number of points and $n$ is the number of dimensions. I would like to give a target dimension parameter which is let's say d. d can be a set of ...
1answer
60 views

### Is “discriminant function” a synonym for “classification function”

"discriminant function" and "classification function" are two terms used in literature to denote a a function that maps a feature vector into a discrete class variable. I presume "discriminant ...
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 14, "mathjax_display_tex": 1, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9170128703117371, "perplexity_flag": "middle"}