**Course Title:** Modern Python Programming: Best Practices and Trends **Section Title:** Data Science and Visualization with Python **Topic:** Pandas for data manipulation and analysis. Pandas is a powerful library for data manipulation and analysis in Python. It provides data structures and functions to efficiently handle structured data, including tabular data such as spreadsheets and SQL tables. **Why Use Pandas?** 1. **Efficient Data Handling**: Pandas provides two primary data structures: Series (1-dimensional labeled array of values) and DataFrames (2-dimensional labeled data structure with columns of potentially different types). These data structures enable efficient data handling and manipulation. 2. **Handling Missing Data**: Pandas offers several options for handling missing data, such as filling or replacing missing values. 3. **Data Merging and Joining**: Pandas provides various methods for combining DataFrames based on a common column, similar to SQL joins. 4. **Data Reshaping and Pivoting**: Pandas offers several functions for reshaping and pivoting data, such as stacking and unstacking. **Installing Pandas** You can install Pandas using pip: ```bash pip install pandas ``` **Importing Pandas** To use Pandas, you need to import it in your Python script or code: ```python import pandas as pd ``` **Creating a DataFrame** You can create a DataFrame from a dictionary, where the keys are column names and the values are lists of data: ```python data = { 'Name': ['John', 'Anna', 'Peter', 'Linda'], 'Age': [28, 24, 35, 32], 'Country': ['USA', 'UK', 'Australia', 'Germany'] } df = pd.DataFrame(data) print(df) ``` Output: ``` Name Age Country 0 John 28 USA 1 Anna 24 UK 2 Peter 35 Australia 3 Linda 32 Germany ``` **Selecting Data** You can select specific columns or rows from a DataFrame using the following methods: * **Column selection**: `df['column_name']` * **Row selection**: `df.loc[row_index]` * **Multiple column selection**: `df[['column1', 'column2']]` * **Multiple row selection**: `df.loc[[row_index1, row_index2]]` **Filtering Data** You can filter data in a DataFrame using conditional statements: ```python # Select rows where Age is greater than 30 filtered_df = df[df['Age'] > 30] print(filtered_df) ``` Output: ``` Name Age Country 2 Peter 35 Australia 3 Linda 32 Germany ``` **Grouping and Aggregating Data** You can group data by one or more columns and apply aggregate functions using the `groupby` method: ```python # Group by Country and calculate the mean Age grouped_df = df.groupby('Country')['Age'].mean() print(grouped_df) ``` Output: ``` Country Australia 35.0 Germany 32.0 UK 24.0 USA 28.0 Name: Age, dtype: float64 ``` **Handling Missing Data** You can handle missing data in a DataFrame using the following methods: * **Detecting missing values**: `df.isnull().sum()` * **Dropping missing values**: `df.dropna()` * **Filling missing values**: `df.fillna(value)` **Merging and Joining DataFrames** You can merge two DataFrames based on a common column using the `merge` method: ```python # Create another DataFrame data2 = { 'Name': ['John', 'Anna', 'Peter', 'Linda'], 'Score': [85, 90, 78, 92] } df2 = pd.DataFrame(data2) # Merge df and df2 on the 'Name' column merged_df = pd.merge(df, df2, on='Name') print(merged_df) ``` Output: ``` Name Age Country Score 0 John 28 USA 85 1 Anna 24 UK 90 2 Peter 35 Australia 78 3 Linda 32 Germany 92 ``` **Reshaping and Pivoting Data** You can reshape and pivot data using the `pivot_table` method: ```python # Create a pivot table pivot_df = pd.pivot_table(df, values='Age', index='Country', columns='Name') print(pivot_df) ``` Output: ``` Name Anna John Linda Peter Country Australia NaN NaN NaN 35.0 Germany NaN NaN 32.0 NaN UK 24.0 NaN NaN NaN USA NaN 28.0 NaN NaN ``` That's it for this topic. Practice the concepts and techniques demonstrated here to become proficient in using Pandas for data manipulation and analysis. **What's Next?** In the next topic, we will cover 'Visualizing data with Matplotlib and Seaborn.' Do you have any questions about this topic?