{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "view-in-github"
},
"source": [
"![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KlrhzBXMyHsX"
},
"source": [
"# (Exercise) Earthquake Data Analysis"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8s95rbr6yQgI"
},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3oiUPDZryR1a"
},
"source": [
"**2014 South Napa CA M6 Earthquake - August 24**\n",
"\n",
"Continuous \"mole-track\" running parallel to the strike of the fault indicates some E-W compression in addition to [right-lateral faulting](https://www.usgs.gov/faqs/what-fault-and-what-are-different-types). Photo taken near Buhman Rd.\n",
"\n",
"Source: *USGS*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vSUo3ZWAyWeZ"
},
"source": [
"In this assignment, we will review `pandas` fundamentals, such as how to\n",
"\n",
"* Open csv files\n",
"\n",
"* Manipulate dataframe indexes\n",
"\n",
"* Parse date columns\n",
"\n",
"* Examine basic dataframe statistics\n",
"\n",
"* Manipulate text columns and extract values\n",
"\n",
"* Plot dataframe contents using\n",
"\n",
"- Bar charts
\n",
"- Histograms
\n",
"- Scatter plots
\n",
"
\n",
"\n",
"Data for this assignment in .csv format downloaded from the USGS Earthquakes Database is available at:\n",
"\n",
"https://unils-my.sharepoint.com/:x:/g/personal/tom_beucler_unil_ch/Efg089STo25Gq6N_BBn_qGoBIsAOd2yUNBgeTfPR2wxw4g?download=1\n",
"\n",
"You don’t need to download this file. You can open it directly with Pandas, with a little help from Pooch (don't worry about reading into the [Pooch documentation](https://www.fatiando.org/pooch/latest/), unless you really want to! 😃).\n",
"\n",
"We'll load the datafile into memory and store the path to the file in the variable `datafile`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3rG9WenMyaLf",
"outputId": "4268169f-7316-4af4-f657-dd17b210c2c6"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Downloading data from 'https://unils-my.sharepoint.com/:x:/g/personal/tom_beucler_unil_ch/Efg089STo25Gq6N_BBn_qGoBIsAOd2yUNBgeTfPR2wxw4g?download=1' to file '/root/.cache/pooch/06e3b7c8b8cd93e39fea0cbfc17a3966-Efg089STo25Gq6N_BBn_qGoBIsAOd2yUNBgeTfPR2wxw4g'.\n"
]
}
],
"source": [
"# Pooch Code\n",
"import pooch\n",
"datafile = pooch.retrieve('https://unils-my.sharepoint.com/:x:/g/personal/tom_beucler_unil_ch/Efg089STo25Gq6N_BBn_qGoBIsAOd2yUNBgeTfPR2wxw4g?download=1',\n",
" known_hash='84d455fb96dc8f782fba4b5fbe56cb8970cab678f07c766fcba1b1c4674de1b1')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1qNEv-74yZGu"
},
"source": [
"**Q1) First, import `Numpy`, `Pandas` and `Matplotlib` and (optional) set the display options.**\n",
"\n",
"Hint: Display options are documented [at this link](https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "xreaix9UpPl9"
},
"outputs": [],
"source": [
"# Import all libraries here\n",
"import numpy as __\n",
"import pandas as __\n",
"import matplotlib.pyplot as ___"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UP5P-WvGypfO"
},
"source": [
"**Q2) Use Pandas’ `read_csv` function directly on the datafile to open it as a `DataFrame`**"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QZ9j5F44lAVE"
},
"source": [
"The dataframe should look something like this"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DUwTIEILk5PA"
},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FqboiSfTFCNG"
},
"source": [
"To display the first few rows of the table, you will use `.head()` function. To display information, use `.info()` function.\n",
"\n",
"Check out these tutorials if you have doubts about what these functions do.\n",
"\n",
"[.head()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.head.html)\n",
"\n",
"[.info()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.info.html?highlight=info#pandas.DataFrame.info)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VI-QLxH2z5Jc"
},
"outputs": [],
"source": [
"# Open the URL as a Pandas' DataFrame\n",
"df = pd.read_csv(__)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "X30QdMJbz_QS"
},
"outputs": [],
"source": [
"# Display first few rows\n",
"df.____()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BI0IVVQi0CkR"
},
"outputs": [],
"source": [
"# Display DataFrame info\n",
"df.____()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nYm2I_em0RBW"
},
"source": [
"The dates were not automatically parsed into `datetime` types!\n",
"What can we do?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5ir5o2Za0Y6W"
},
"source": [
"**Q3) Re-read the data in such a way that all date columns are identified as dates and the earthquake ID is used as the index**\n",
"\n",
"Recreate the screenshot below.\n",
"\n",
"Hint: The documentation for `.read_csv()` function is [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html?highlight=read_csv)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J7dCSAeClT9x"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PddmZRM1074f"
},
"outputs": [],
"source": [
"# Re-read the URL\n",
"df = pd.read_csv(__,__=__)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "dIyfgB8M1Ht-"
},
"outputs": [],
"source": [
"# Use the `head` function to check that it worked\n",
"df.___()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZaXdY7Gv1IRx"
},
"outputs": [],
"source": [
"# Use the `info` function to check that it worked\n",
"df.___()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N4B5jey01PER"
},
"source": [
"**Q4) Use `describe` to get the basic statistics of all the columns**\n",
"\n",
"Hint: The documentation of `describe` is [at this link](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.describe.html)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "KKWJuNUe1Pqi"
},
"outputs": [],
"source": [
"# Use the `describe` function\n",
"___.___()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4SO_TnBm1q-p"
},
"source": [
"**Q5) Use `nlargest` to get the top 20 earthquakes by magnitude**\n",
"\n",
"Hint: The documentation of `nlargest` is [at this link](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.nlargest.html)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fn2A03s0lt-M"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wAYTpvLu1aMl"
},
"outputs": [],
"source": [
"# Use `nlargest`\n",
"df.___(___,___)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "l6xxgocC2H6U"
},
"source": [
"Examine the column titled 'place'.\n",
"It seems to contain both state and country information. How would you get it out?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VIPm4Zc72Xg_"
},
"source": [
"**Q6) Extract the state or country using `Pandas` text data functions, and add it as a new column to the `DataFrame`**\n",
"\n",
"Hint 1: The documentation for Pandas' text data functions is [here](https://pandas.pydata.org/pandas-docs/stable/text.html)\n",
"\n",
"Hint 2: You will use `.split()` to extract the country names. The documentation of this function can be found [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.split.html?highlight=split#pandas.Series.str.split)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VLrRxe8dmQf0"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "xYMaJQWp3gx0"
},
"outputs": [],
"source": [
"# Extract the state or country\n",
"country = df.__.str.__(',',expand=True,n=_)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LuvIHtxx5FhN"
},
"outputs": [],
"source": [
"# Add it as a new column to the `DataFrame` called `country`"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qgHw-fnb5cXN"
},
"source": [
"**Q7) Display each unique value from the new `country` column**\n",
"\n",
"Hint: You may use the `unique` function documented [at this link](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.unique.html)\n",
"\n",
"You should see an array with different country and state names (e.g., array(['Alaska','Nevada',...]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0DsgkKi829z1"
},
"outputs": [],
"source": [
"# Display unique values\n",
"df.___.___()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HfDv2UEq5k1H"
},
"source": [
"**Q8) Create a filtered dataset that only has earthquakes larger than magnitude 4**\n",
"\n",
"Hint: Print the table to see the name of the column containing the earthquake magnitude.\n",
"\n",
"Check out this [link](https://www.geeksforgeeks.org/ways-to-filter-pandas-dataframe-by-column-values/) to find examples of filtering by column values"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "EriZJEFj_OWk"
},
"outputs": [],
"source": [
"# Filter the dataset based on the earthquakes' magnitudes"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IcsVAjqk_VHm"
},
"source": [
"**Q9) Using the filtered dataset (magnitude > 4), count the number of earthquakes whose magnitudes >4 [Num 1], count the number of earthquakes in each country/state [Num 2]. Make a bar chart of Num2 for the top 5 locations with the most earthquakes**\n",
"\n",
"Hint 1: To get Num 1, `Pandas` has a `count` function documented [at this link](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.count.html)\n",
"\n",
"Hint 2: Check out the `value_counts` function to get Num 2\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3RmwkdtZnAH7"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pINwgzUXCC6U"
},
"outputs": [],
"source": [
"# Count the number of earthquakes whose magnitudes are larger than 4\n",
"df_filt.__()\n",
"# Count the number of >4 earthquakes in each country/state\n",
"df_filt.__.__()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LBB86IuMnMMA"
},
"source": [
"Recreate the bar chart below.\n",
"\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0Lo1fKkHCIWu"
},
"outputs": [],
"source": [
"# print the first 5 rows of num2\n",
"num2.__[:_]\n",
"\n",
"# convert what you just printed to a DataFrame with 2 columns: 'country' (text) and 'earthquake_num' (number)\n",
"# Hint: separate the text and numbers and store them separately in lists\n",
"top5_df = pd.DataFrame({'country':list(num2.__[__].__),'earthquake_num':list(num2.__[__]._____)})\n",
"top5_df"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "iLKNY2gKEDIQ"
},
"outputs": [],
"source": [
"# Now plot the numbers in the top5_df dataset with bar chart.\n",
"__,__ = plt._______(figsize=(_,_))\n",
"top5_df.plot.bar(x=_____,y=_____,rot=0,ax=__)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7Dim2tWqHjXa"
},
"source": [
"**Q10) Make a histogram for the distribution of the earthquakes' magnitudes**\n",
"\n",
"Hint: `Pandas` has a histogram function documented [at this link](https://pandas.pydata.org/pandas-docs/version/0.23/generated/pandas.DataFrame.hist.html) and `Matplotlib` has one documented [at this link](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.hist.html)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6gTeOHFYnjq7"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "vk98XG-bJKVw"
},
"outputs": [],
"source": [
"__,__ = plt.subplots(figsize=(9,4))\n",
"# Make the histogram\n",
"____.__.____\n",
"__.__________('______') # X axis label\n",
"__._______('_____') # Y axis label\n",
"__._____(alpha=0.2,c='b',ls='--')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lvVMIsUMoVjk"
},
"source": [
"##### Use a logarithmic scale for y axis\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "91fb_hVEQIKz"
},
"source": [
"Hint: Here you can find a [tutorial](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.yscale.html) for how to change axis scale."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "sFSrizEQJM2P"
},
"outputs": [],
"source": [
"__,__ = plt.subplots(figsize=(9,4))\n",
"# Make the histogram\n",
"____.__.____\n",
"__.__________('______') # X axis label\n",
"__._______('_____') # Y axis label\n",
"__._____(alpha=0.2,c='b',ls='--')\n",
"# Use a logarithmic scale for y axis\n",
"__.______(___)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KXVHXhxFosJ9"
},
"source": [
"##### Make one histogram for the filtered dataset, and one for the unfiltered dataset\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "KuuiVifvJYRF"
},
"outputs": [],
"source": [
"# Make one histogram for the filtered dataset, and one for the unfiltered dataset\n",
"fig,ax = plt.subplots(1,2,figsize=(12,4))\n",
"______.__.____(___=____,color='#1f77b4')\n",
"__.__________('______') # X axis label\n",
"__._______('_____') # Y axis label\n",
"__._____(alpha=0.2,c='b',ls='--')\n",
"__.________(________) # Add title for the filtered figure on the left\n",
"\n",
"______.__.____(___=____,color='#ff7f0e')\n",
"__.__________('______') # X axis label\n",
"__._______('_____') # Y axis label\n",
"__._____(alpha=0.2,c='b',ls='--')\n",
"__.________(________) # Add title for the unfiltered figure on the right\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QGX3tUNXJgft"
},
"source": [
"**Q11) Visualize the locations of earthquakes by making a scatterplot of their latitude and longitude**\n",
"\n",
"Hint: Consider reading the documentation for [`plt.scatter`](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.scatter.html) to make the scatter plot and that of [`plt.colorbar`](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.colorbar.html) to color the points by magnitude."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_LD3KtpCpV2V"
},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "B_4clHfHLrtO"
},
"outputs": [],
"source": [
"# You can use a two-column subplot with\n",
"# both the filtered/unfiltered datasets\n",
"# to facilitate their comparison.\n",
"__,__ = plt.subplots(_,_,_______=(__,__))\n",
"# Filtered data\n",
"_____.plot.scatter(x=______,y=_______,c=____,ax=____,cmap=____,vmin=0,vmax=8)\n",
"# Unfilted data\n",
"df.plot.scatter(x=______,y=_______,c=____,ax=____,cmap=____,vmin=0,vmax=8)\n",
"ax[0].set_title('Filtered')\n",
"ax[1].set_title('Unfiltered')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UaTR67heJqCm"
},
"source": [
"Do you notice a difference between filtered and unfiltered datasets?"
]
}
],
"metadata": {
"colab": {
"include_colab_link": true,
"name": "W3_S1.ipynb",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}