Presentation on theme: "An Overview of Machine Learning"— Presentation transcript:
1 An Overview of Machine Learning Speaker: Yi-Fan ChangAdviser: Prof. J. J. DingDate: 2011/10/21
2 Outline & Content What is machine learning? Learning system model Training and testingPerformanceAlgorithmsMachine learning structureWhat are we seeking?Learning techniquesApplicationsConclusion
3 What is machine learning? A branch of artificial intelligence, concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data.As intelligence requires knowledge, it is necessary for the computers to acquire knowledge.
4 Learning system model Testing Input Samples Learning Method System Training
5 Training and testing Data acquisition Practical usage Universal set (unobserved)Training set (observed)Testing set(unobserved)
6 Training and testingTraining is the process of making the system able to learn.No free lunch rule:Training set and testing set come from the same distributionNeed to make some assumptions or bias
7 Performance There are several factors affecting the performance: Types of training providedThe form and extent of any initial background knowledgeThe type of feedback providedThe learning algorithms usedTwo important factors:ModelingOptimization
8 AlgorithmsThe success of machine learning system also depends on the algorithms. The algorithms control the search to find and build the knowledge structures.The learning algorithms should extract useful information from training examples.
9 Algorithms Supervised learning ( ) Unsupervised learning ( ) PredictionClassification (discrete labels), Regression (real values)Unsupervised learning ( )ClusteringProbability distribution estimationFinding association (in features)Dimension reductionSemi-supervised learningReinforcement learningDecision making (robot, chess machine)
13 What are we seeking?Supervised: Low E-out or maximize probabilistic termsUnsupervised: Minimum quantization error, Minimum distance, MAP, MLE(maximum likelihood estimation)E-in: for training setE-out: for testing set
14 What are we seeking? Under-fitting VS. Over-fitting (fixed N) error (model = hypothesis + loss functions)
15 Learning techniques Supervised learning categories and techniques Linear classifier (numerical functions)Parametric (Probabilistic functions)Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden Markov models (HMM), Probabilistic graphical modelsNon-parametric (Instance-based functions)K-nearest neighbors, Kernel regression, Kernel density estimation, Local regressionNon-metric (Symbolic functions)Classification and regression tree (CART), decision treeAggregationBagging (bootstrap + aggregation), Adaboost, Random forest
16 Learning techniques Linear classifier PerceptronLogistic regressionSupport vector machine (SVM)Ada-lineMulti-layer perceptron (MLP)Linear classifier, where w is an d-dim vector (learned)
17 Learning techniques Using perceptron learning algorithm(PLA) Training TestingError rate: 0.10Error rate: 0.156
18 Learning techniques Using logistic regression Training Testing Error rate: 0.11Error rate: 0.145
19 Learning techniques Non-linear case Support vector machine (SVM): Linear to nonlinear: Feature transform and kernel function
21 Applications Face detection Object detection and recognition Image segmentationMultimedia event detectionEconomical and commercial usage
22 ConclusionWe have a simple overview of some techniques and algorithms in machine learning. Furthermore, there are more and more techniques apply machine learning as a solution. In the future, machine learning will play an important role in our daily life.
23 Reference W. L. Chao, J. J. Ding, “Integrated Machine Learning Algorithms for Human Age Estimation”, NTU, 2011.