»úÆ÷ѧϰ¼°ÆäÓ¦ÓÃ(Ó¢ÎÄ°æ)

Ŀ¼1. Introduction ÒýÑÔ¡­ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­11.1 Towards Intelligent Machines ×ßÏòÖÇÄÜ»úÆ÷ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­11.2 Well-Posed Machine Learning Problems Êʶ¨µÄ»úÆ÷ѧϰÎÊÌâ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­51.3 Examples of Applications in Diverse Fields ²»Í¬ÁìÓòµÄÓ¦ÓÃʵÀý ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­71.4 Data Representation Êý¾Ý±íʾ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­121.4.1 Time Series Forecasting ʱ¼äÐòÁÐÔ¤²â ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­151.4.2 Datasets for Toy (Unreastically Simple) and Realistic Problems³õ¼¶ÎÊÌâºÍÏÖʵÎÊÌâÊý¾Ý¼¯ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­171.5 Domain Knowledge for Productive use of Machine Learningʹ»úÆ÷ѧϰÓÐЧӦÓõÄÁìÓò֪ʶ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­181.6 Diversity of Data: Structured/Unstructured Êý¾Ý¶àÑùÐÔ£º½á¹¹»¯/·Ç½á¹¹»¯ ¡­¡­¡­¡­¡­201.7 Forms of Learning ѧϰÐÎʽ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­211.7.1 Supervised/Directed Learning ¼à¶½/Ö¸µ¼Ñ§Ï° ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­211.7.2 Unsupervised/Undirected Learning ·Ç¼à¶½/ÎÞÖ¸µ¼Ñ§Ï° ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­221.7.3 Reinforcement Learning Ç¿»¯Ñ§Ï° ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­221.7.4 Learning Based on Natural Processes: Evolution, Swarming, and Immune Systems»ùÓÚ×ÔÈ»´¦ÀíµÄѧϰ£º½ø»¯¡¢¼¯ÈººÍÃâÒßϵͳ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­231.8 Machine Learning and Data Mining »úÆ÷ѧϰºÍÊý¾ÝÍÚ¾ò ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­251.9 Basic Linear Algebra in Machine Learning Techniques»úÆ÷ѧϰ¼¼ÊõÖеĻù´¡ÏßÐÔ´úÊý ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­261.10 Relevant Resources for Machine Learning »úÆ÷ѧϰµÄÏà¹Ø×ÊÔ´ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­342. Supervised Learning: Rationale and Basics ¼à¶½Ñ§Ï°£º»ù±¾Ô­ÀíºÍ»ù´¡¡­ ¡­¡­¡­¡­¡­¡­362.1 Learning from Observations ´Ó¹Û²âÖÐѧϰ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­362.2 Bias and Variance Æ«²îºÍ·½²î ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­422.3 Why Learning Works: Computational Learning TheoryѧϰΪʲôÓÐЧ£º¼ÆËãѧϰÀíÂÛ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­462.4 Occam's Razor Principle and Overfitting Avoidance°Â¿¨Ä·Ì굶ԭÀíºÍ·ÀÖ¹¹ýÄâºÏ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­492.5 Heuristic Search in Inductive Learning ¹éÄÉѧϰÖеÄÆô·¢Ê½ËÑË÷ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­512.5.1 Search through Hypothesis Space ¼ÙÉè¿Õ¼äËÑË÷ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­522.5.2 Ensemble Learning ¼¯³Éѧϰ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­532.5.3 Evaluation of a Learning System ѧϰϵͳµÄÆÀ¼Û ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­552.6 Estimating Generalization Errors ¹À¼Æ·º»¯Îó²î ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­562.6.1 Holdout Method and Random Subsampling Áô³ö·¨ºÍËæ»úϲÉÑù ¡­¡­¡­¡­¡­¡­¡­562.6.2 Cross-validation ½»²æÑéÖ¤ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­572.6.3 Bootstrapping ×ÔÖú·¨ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­582.7 Metrics for Assessing Regression (Numeric Prediction) AccuracyÆÀ¼Û»Ø¹é£¨ÊýÖµÔ¤²â£©¾«¶ÈµÄÖ¸±ê ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­592.7.1 Mean Square Error ¾ù·½Îó²î ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­602.7.2 Mean Absolute Error ƽ¾ù¾ø¶ÔÎó²î ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­602.8 Metrics for Assessing Classification (Pattern Recognition) AccuracyÆÀ¼Û·ÖÀࣨģʽʶ±ð£©¾«¶ÈµÄÖ¸±ê ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­612.8.1 Misclassification Error Îó·ÖÀàÎó²î ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­612.8.2 Confusion Matrix »ìÏý¾ØÕó ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­622.8.3 Comparing Classifiers Based on ROC Curves »ùÓÚROCÇúÏߵķÖÀàÆ÷±È½Ï ¡­¡­662.9 An Overview of the Design Cycle and Issues in Machine Learning»úÆ÷ѧϰÖеÄÉè¼ÆÖÜÆÚºÍÎÊÌâ¸ÅÊö ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­683. Statistical Learning ͳ¼Æѧϰ¡­ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­733.1 Machine Learning and Inferential Statistical Analysis »úÆ÷ѧϰÓëÍƶÏͳ¼Æ·ÖÎö ¡­¡­¡­¡­733.2 Descriptive Statistics in Learning Techniques ѧϰ¼¼ÊõÖеÄÃèÊöͳ¼Æѧ ¡­¡­¡­¡­¡­¡­¡­743.2.1 Representing Uncertainties in Data: Probability Distributions±íʾÊý¾ÝÖеIJ»È·¶¨ÐÔ£º¸ÅÂÊ·Ö²¼ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­753.2.2 Descriptive Measures of Probability Distributions ¸ÅÂÊ·Ö²¼µÄÃèÊö·½·¨ ¡­¡­¡­¡­803.2.3 Descriptive Measures from Data Sample Êý¾ÝÑù±¾µÄÃèÊö·½·¨ ¡­¡­¡­¡­¡­¡­¡­¡­833.2.4 Normal Distributions Õý̬·Ö²¼ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­843.2.5 Data Similarity Êý¾ÝÏàËÆÐÔ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­853.3 Bayesian Reasoning: A Probabilistic Approach to Inference±´Ò¶Ë¹ÍÆÀí£ºÍƶϵĸÅÂÊ·½·¨ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­873.3.1 Bayes Theorem ±´Ò¶Ë¹¶¨Àí ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­883.3.2 Naive Bayes Classifier ÆÓËر´Ò¶Ë¹·ÖÀàÆ÷ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­933.3.3 Bayesian Belief Networks ±´Ò¶Ë¹ÐÅÄîÍøÂç ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­983.4 k-Nearest Neighbor (k-NN) Classifier k½üÁÚ£¨k-NN£©·ÖÀàÆ÷ ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­1023.5 Discriminant Functions and Regression Functions Åбðº¯ÊýºÍ»Ø¹éº¯Êý ¡­¡­¡­¡­¡­¡­1063.5.1 Classification and Discriminant Functions ·ÖÀàºÍÅбðº¯Êý ¡­¡­¡­¡­¡­¡­¡­¡­1073.5.2 Numeric Prediction and Regression Functions ÊýÖµÔ¤²âºÍ»Ø¹éº¯Êý ¡­¡­¡­¡­¡­1083.5.3 Practical Hypothesis Functions ʵ¼ùÓ¦ÓÃÖеļÙÉ躯Êý ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­1093.6 Linear Regression with Least Square Error Criterion»ùÓÚ*С¶þ³ËÎó²î×¼ÔòµÄÏßÐԻع鷨 ¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­¡­1123.6.1 Minimal Sum-of-Error-Squares and the Pseudoinverse *СÎó²îƽ·½ºÍÓëαÄæ 1133.6.2 Gradient Descent Optimization Schemes ÌݶÈϽµ·¨ÓÅ»¯·½°¸ ¡­¡­¡­¡­¡­¡­¡­1153.6.