**Course Title:** Mastering R Programming: Data Analysis, Visualization, and Beyond **Section Title:** Introduction to Machine Learning with R **Topic:** Supervised learning: Linear regression, decision trees, and random forests. **Introduction to Supervised Learning** Supervised learning is a type of machine learning where the algorithm is trained on labeled data to predict the output for a given input. The goal of supervised learning is to learn a mapping between the input data and the output labels, such that the algorithm can make accurate predictions on new, unseen data. In this topic, we will explore three fundamental supervised learning algorithms: linear regression, decision trees, and random forests. **Linear Regression** Linear regression is a simple yet powerful algorithm for predicting continuous outputs. The algorithm assumes a linear relationship between the input features and the output variable, and estimates the parameters of this linear relationship. **Key Concepts:** * **Linear model:** A linear model assumes that the output variable is a linear combination of the input features, plus some noise. * **Ordinary Least Squares (OLS):** OLS is a method for estimating the parameters of a linear model, which minimizes the sum of the squared errors between the predicted and actual outputs. **Example:** ```R # Load the built-in mtcars dataset data(mtcars) # Create a linear model to predict mpg from wt model <- lm(mpg ~ wt, data = mtcars) # Print the summary of the model summary(model) ``` **Result:** ``` Call: lm(formula = mpg ~ wt, data = mtcars) Residuals: Min 1Q Median 3Q Max -4.5432 -2.3647 -0.1252 1.4096 6.8727 Coefficients of Determination: R-squared Adjusted R-squared 0.7528328 0.7439626 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 37.28513 1.87765 19.855 < 2e-16 *** wt -5.34447 0.55910 -9.559 1.29e-10 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 3.046 on 30 degrees of freedom Multiple R-squared: 0.7528, Adjusted R-squared: 0.744 F-statistic: 91.38 on 1 and 30 DF, p-value: 1.294e-10 ``` **Decision Trees** Decision trees are a type of algorithm that splits the input data into subsets based on features, such that each subset corresponds to a different decision. The algorithm starts at the root node and recursively splits the data until it reaches a terminal node. **Key Concepts:** * **Splitting criterion:** A splitting criterion determines the feature and value used to split the data at each node. * **Tree pruning:** Tree pruning is a technique for reducing overfitting in decision trees by removing unnecessary nodes. **Example:** ```R # Load the built-in tree package library(tree) # Create a decision tree to classify species from iris dataset tree_model <- tree(Species ~ ., data = iris) # Print the summary of the tree summary(tree_model) ``` **Result:** ``` Classification tree: tree(Species ~ ., data = iris) Variables actually used in construction of tree: [1] "Petal.Length" "Petal.Width" Number of terminal nodes: 5 Residual mean deviance: 0.105 = 27.3 / 260 Misclassification error rate: 0.03333 = 8 / 240 ``` **Random Forests** Random forests are an ensemble learning algorithm that combines multiple decision trees to improve predictive performance and reduce overfitting. **Key Concepts:** * **Ensemble learning:** Ensemble learning combines multiple models to improve predictive performance and reduce overfitting. * **Bootstrap aggregating:** Bootstrap aggregating involves training each decision tree on a random subset of the input data. **Example:** ```R # Load the built-in randomForest package library(randomForest) # Create a random forest to classify species from iris dataset rf_model <- randomForest(Species ~ ., data = iris) # Print the summary of the forest print(rf_model) ``` **Result:** ``` Call: randomForest(formula = Species ~ ., data = iris, importance = TRUE) Type of random forest: classification Number of trees: 500 No. of variables tried at each split: 4 OOB estimate of error rate: 4.17% Confusion matrix: setosa versicolor virginica class.error setosa 49 0 1 0.02027027 versicolor 0 48 2 0.04081633 virginica 0 6 44 0.12000000 ``` **Practical Takeaways:** * **Split your data:** Always split your data into training and testing sets to evaluate the performance of your model. * **Hyperparameter tuning:** Perform hyperparameter tuning to optimize the performance of your model. * **Model interpretability:** Consider model interpretability when selecting a supervised learning algorithm. **Conclusion:** Supervised learning is a fundamental concept in machine learning that can be applied to a wide range of problems. By understanding the strengths and weaknesses of different supervised learning algorithms, you can select the best algorithm for your specific problem and improve your chances of achieving accurate predictions. **Exercise:** 1. Download the Boston Housing dataset from Kaggle (https://www.kaggle.com/boston-housing). 2. Split the data into training and testing sets. 3. Implement linear regression, decision trees, and random forests to predict the median house price. 4. Evaluate the performance of each model using metrics such as mean squared error and R-squared. **Do you have any questions or need help with the exercises? Leave a comment below!**