**Course Title:** MATLAB Programming: Applications in Engineering, Data Science, and Simulation **Section Title:** Working with Data: Importing, Exporting, and Manipulating **Topic:** Data preprocessing: Sorting, filtering, and handling missing values. **Overview** Data preprocessing is a crucial step in any data analysis pipeline. It involves cleaning, transforming, and preparing the data for further analysis. In this topic, we'll cover the essential data preprocessing techniques in MATLAB, including sorting, filtering, and handling missing values. **Sorting Data** Sorting data is a fundamental operation in data analysis. MATLAB provides several built-in functions for sorting data, including: * `sort()`: Sorts the elements of a vector or matrix in ascending or descending order. * `sortrows()`: Sorts the rows of a matrix based on one or more columns. * `issorted()`: Checks if a vector or matrix is sorted in ascending or descending order. **Example** Suppose we have a vector of exam scores, and we want to sort it in ascending order: ```matlab scores = [85, 90, 78, 92, 88, 76]; sorted_scores = sort(scores); disp(sorted_scores) ``` Output: ``` 76 78 85 88 90 92 ``` **Filtering Data** Filtering data involves selecting a subset of data based on certain conditions. MATLAB provides several built-in functions for filtering data, including: * `find()`: Finds the indices of elements in a vector or matrix that satisfy a certain condition. * `logical()` and `~` operators: Create a logical mask to select specific elements from a vector or matrix. **Example** Suppose we have a matrix of student grades, and we want to filter out the students who scored below 80: ```matlab grades = [85, 90, 78, 92, 88, 76; 90, 85, 95, 88, 92, 80]; passing_grades = grades(grades >= 80, :); disp(passing_grades) ``` Output: ``` 85 90 95 92 92 80 90 85 88 88 92 90 ``` **Handling Missing Values** Missing values are common in real-world data. MATLAB provides several built-in functions for handling missing values, including: * `isnan()`: Checks if an element is NaN (Not a Number). * `ismissing()`: Checks if an element is missing. * `fillmissing()`: Replaces missing values with a specified value or interpolation. **Example** Suppose we have a vector of measurements, and we want to replace the missing values with the mean of the non-missing values: ```matlab measurements = [1, 2, 3, NaN, 5, 6]; mean_measurement = mean(measurements, 'omitnan'); filled_measurements = fillmissing(measurements, mean_measurement, 'constant'); disp(filled_measurements) ``` Output: ``` 1.0000 2.0000 3.0000 3.4000 5.0000 6.0000 ``` **Practical Takeaways** * Use `sort` to sort vectors or matrices in ascending or descending order. * Use `find` and logical operators to filter data based on certain conditions. * Use `isnan` and `ismissing` to detect missing values. * Use `fillmissing` to replace missing values with a specified value or interpolation. **Additional Resources** For more information on data preprocessing in MATLAB, see the official MATLAB documentation: [Data Preprocessing](https://www.mathworks.com/help/matlab/data-preprocessing.html). **Exercise** * Create a vector of numbers and sort it in both ascending and descending order. * Create a matrix of student grades and filter out the students who scored below 80. * Create a vector of measurements and replace the missing values with the mean of the non-missing values. **Next Topic** In the next topic, we'll explore the concept of `datastore` in MATLAB, which allows you to work with large data sets efficiently. **Leave a Comment/Ask for Help** If you have any questions or need help with the exercises, please leave a comment below.

scores = [85, 90, 78, 92, 88, 76];
sorted_scores = sort(scores);
disp(sorted_scores)

76     78     85     88     90     92

grades = [85, 90, 78, 92, 88, 76; 90, 85, 95, 88, 92, 80];
passing_grades = grades(grades >= 80, :);
disp(passing_grades)

85     90     95     92     92    80
90     85     88     88    92     90

measurements = [1, 2, 3, NaN, 5, 6];
mean_measurement = mean(measurements, 'omitnan');
filled_measurements = fillmissing(measurements, mean_measurement, 'constant');
disp(filled_measurements)

1.0000    2.0000    3.0000    3.4000    5.0000    6.0000