# Tagged Questions

Methods and principles of building "computer systems that automatically improve with experience."

12answers
13k views

### The Two Cultures: statistics vs. machine learning?

Last year, I read a blog post from Bendan O'Connor entitled "Statistics vs. Machine Learning, fight!" that discussed some of the differences between the two fields. Andrew Gelman responded to ...
10answers
2k views

### Machine learning cookbook / reference card / cheatsheet?

I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my ...
10answers
13k views

### What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
5answers
2k views

### Can you recommend a book to read before Elements of Statistical Learning?

Based on this post, http://quant.stackexchange.com/questions/111/how-can-i-go-about-applying-machine-learning-algorithms-to-stock-markets, I want to digest Elements of Statistical Learning. ...
2answers
1k views

### Proper way of using recurrent neural network for time series analysis

Recurrent neural networks differ from "regular" ones by the fact that they have a "memory" layer. Due to this layer, recurrent NN's are supposed to be useful in time series modelling. However, I'm not ...
0answers
1k views

### Machine learning self-learning book? [duplicate]

Possible Duplicate: Machine learning cookbook / reference card / cheatsheet? I wonder if there is a good self-learning textbook for machine learning? I am particularly looking for those in ...
6answers
3k views

### What skills are required to perform large scale statistical analyses?

Many statistical jobs ask for experience with large scale data. What are the sorts of statistical and computational skills that would be need for working with large data sets. For example, how about ...
1answer
1k views

### Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
5answers
2k views

### Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after k-fold cross-validation? Or is it better instead to stick with one of the models learned in one of the cross-validation splits for ...
6answers
2k views

### Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
1answer
697 views

### Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
2answers
694 views

### Variance estimates in k-fold cross-validation

K-fold cross-validation can be used to estimate the generalization capability of a given classifier. Can I (or should I) also compute a pooled variance from all validation runs in order to obtain a ...
2answers
336 views

### Supervised learning with “rare” events, when rarity is due to the large number of counter-factual events

Suppose you get to observe "matches" between buyers and sellers in a market. You also get to observe characteristics of both buyers and sellers which you would like to use to predict future matches ...
1answer
2k views

### Clustering: Should I use the Jensen-Shannon Divergence or its square?

I am clustering probability distributions using the Affinity Propagation algorithm, and I plan to use Jensen-Shannon Divergence as my distance metric. Is it correct to use JSD itself as the distance, ...
1answer
209 views

### What do “real values” refer to in supervised classification?

I'm using supervised classification algorithms from mlpy to classify things into two groups for a question-answering system. I don't really know how these algorithms work, but they seem to be doing ...
2answers
380 views

### Trouble applying hidden Markov models

Edit: I updated the question to hopefully make it more easy to understand. I think it was overly complex. I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
2answers
397 views

### How to get started with neural networks

I'm completely new to neural networks but highly interested in understanding them. However it's not easy at all to get started. Could anyone recommend a good book or any other kind of resource? Is ...
7answers
6k views

### Having a job in data-mining without a PhD

I've been very interested in data-mining and machine-learning for a while, partly because I majored in that area at school, but also because I am truly much more excited trying to solve problems that ...
2answers
765 views

### When is “Nearest Neighbor” meaningful, today?

In 1999, Beyer et al. asked, When is "Nearest Neighbor" meaningful? Are there better ways of analyzing and visualizing the effect of distance flatness on NN search since 1999? Does [a given] data ...
6answers
1k views

### Data mining: How should I go about finding the functional form?

I'm curious about repeatable procedures that can be used to discover the functional form of the function y = f(A, B, C) + error_term where my only input is a set of ...
2answers
574 views

### Sites for predictive modeling competitions

I participate in predictive modeling competitions on Kaggle, TunedIt, and CrowdAnalytix. I find that these sites are a good way to "work-out" for statistics/machine learning. Are there any other ...
2answers
367 views

### What happens when you apply SVD to a collaborative filtering problem? What is the difference between the two?

In Collaborative filtering, we have values that are not filled in. Suppose a user did not watch a movie then we have to put an 'na' in there. If I am going to take an SVD of this matrix, then I have ...
1answer
666 views

### Bayesian classifier with multivariate normal densities

Supposing a Bayesian classifier with multivariate normal densities, how do I find the error rate of the classifier when we have two classes? I am using this: When dimension $d = 1$: P(x | \mu , ...
8answers
6k views

### Machine Learning using Python

I am considering using Python libraries for doing my Machine Learning experiments. Thus far, I had been relying on WEKA but have been pretty dissatisfied on the whole. This is primarily because I have ...
8answers
13k views

### Detecting a given face in a database of facial images

I'm working on a little project involving the faces of twitter users via their profile pictures. A problem I've encountered is that after I filter out all but the images that are clear portrait ...
12answers
8k views

### Best books for an introduction to statistical data analysis?

I bought this book: How to Measure Anything: Finding the Value of Intangibles in Business and Head First Data Analysis: A Learner's Guide to Big Numbers, Statistics, and Good Decisions What ...
6answers
2k views

### What machine learning algorithm can be used to predict the stock market?

Alternatively, to predict foreign exchange markets. I know this can get pretty complicated, so as an introduction, I'm looking for a simple prediction algorithm that has some accuracy. (It's for a ...
5answers
965 views

### Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
5answers
1k views

### Programmer looking to break into machine learning field

I am a software developer (mostly .NET and Python about 5 years experience). What can I do to help me get a job in the machine learning field or really anything that will get me started in that field? ...
3answers
5k views

### How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
3answers
820 views

### Understanding Naive Bayes

To demonstrate the concept of Naïve Bayes Classification, consider the example displayed in the illustration above. As indicated, the objects can be classified as either GREEN or RED. My task is to ...
6answers
1k views

### Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
4answers
1k views

### What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
2answers
500 views

### How to choose between learning algorithms

I need to implement a program that will classify records into 2 categories (true/false) based on some training data, and I was wondering at which algorithm/methodology I should be looking at. There ...
2answers
1k views

### Restricted Boltzmann machines vs multilayer neural networks

I've been wanting to experiment with a neural network for a classification problem that I'm facing. I ran into papers that talk of RBMs. But from what I can understand, they are no different from ...
7answers
458 views

### How can I help ensure testing data does not leak into training data?

Suppose we have someone building a predictive model, but that someone is not necessarily well-versed in proper statistical or machine learning principles. Maybe we are helping that person as they are ...
2answers
454 views

### On the  “strength” of weak learners

I have several closely-related questions regarding weak learners in ensemble learning (e.g. boosting). This may sound dumb, but what are the benefits of using weak as opposed to strong learners? ...
2answers
1k views

### Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
4answers
599 views

### Can SVM do stream learning one example at a time?

I have a streaming data set, examples are available one at a time. I would need to do multi class classification on them. As soon as I fed a training example to the learning process, I have to discard ...
3answers
7k views

### What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
1answer
512 views

### Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
4answers
3k views

### GLMNET or LARS for computing LASSO solutions?

I would like to get the coefficients for the LASSO problem $$||Y-X\beta||+\lambda ||\beta||_1.$$ The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
0answers
141 views

### Updating classification probability in logistic regression through time

I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
1answer
236 views

### Naive Bayes for two continuous features

I have two features which are both continuous. How to perform a classification task based on them? I've read the Wikipedia entry on Naive Bayes, but this is only for discrete outcome and one feature.
2answers
583 views

### Building background for machine learning for CS student

I am a CS graduate student and I am starting to get really interested in Machine Learning (and Predictive Analytics). I have started working on a text classification project with a professor to learn ...
5answers
354 views

### Is using the same data for feature selection and cross-validation biased or not?

We have a small dataset (about 250 samples * 100 features) on which we want to build a binary classifier after selecting the best feature subset. Lets say that we partition the data into: Training, ...
3answers
802 views

### How does neural network recognise images?

I am trying to learn how Neural Network works on image recognition. I have seen some examples and become even more confused. In the example of letter recognition of a 20x20 image, the values of each ...
2answers
1k views

### Comparing two classifier accuracy results for statistical significance with t-test

I want to compare the accuracy of two classifiers for statistical significance. Both classifiers are run on the same data set. This leads me to believe I should be using a one sample t-test from what ...
0answers
170 views

### Behavior of a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ ...
4answers
627 views

### Is a strong background in maths a total requisite for ML?

I'm starting to want to advance my own skillset and I've always been fascinated by machine learning. However, six years ago instead of pursuing this I decided to take a completely unrelated degree to ...
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 7, "mathjax_display_tex": 1, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9315620064735413, "perplexity_flag": "middle"}