![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
]
},
{
"cell_type": "markdown",
"source": [
"# Exercise 4: Mapping Wildfire Susceptibility in the Liguria Region with Simple Machine Learning Classifiers"
],
"metadata": {
"id": "HN8VFRGJeDsF"
}
},
{
"cell_type": "markdown",
"source": [
"\n",
"\n",
"---\n",
"\n",
"\n",
"\n",
"**Credits**\n",
"\n",
"This online tutorial would not be possible without invaluable contributions from [Andrea Trucchia](https://www.researchgate.net/profile/Andrea-Trucchia) (reduced data, methods), [Giorgio Meschi](https://www.linkedin.com/in/giorgio-meschi-86216b180/) (code, methods), and [Marj Tonini](https://www.researchgate.net/profile/Marj-Tonini-2) (presentation, methods). The methodology builds upon the following article:\n",
"\n",
"[Trucchia, A.; Meschi, G.; Fiorucci, P.; Gollini, A.; Negro, D., Defining Wildfire Susceptibility Maps in Italy for Understanding Seasonal Wildfire Regimes at the National Level, *Fire*, (2022)](https://www.mdpi.com/2571-6255/5/1/30)\n",
"\n",
"which generalizes the study below from the Liguria region (our case study) to all of Italy:\n",
"\n",
"[Tonini, Marj, et al. \"A machine learning-based approach for wildfire susceptibility mapping. The case study of the Liguria region in Italy.\" *Geosciences* 10.3 (2020): 105.](https://www.mdpi.com/2076-3263/10/3/105)\n",
"\n",
"\n",
"\n",
"---\n",
"\n"
],
"metadata": {
"id": "BaCACOGXLhCu"
}
},
{
"cell_type": "markdown",
"source": [
"In week 3's final notebook, we will train classifiers on real [wildfire](https://en.wikipedia.org/wiki/Wildfire) data to map the fire risk in different regions of Italy. To keep the data size manageable, we will focus on the coastal [Liguria region](https://en.wikipedia.org/wiki/Liguria) that experiences a lot of wildfires, especially during the winter."
],
"metadata": {
"id": "bstlA2R4LZcb"
}
},
{
"cell_type": "markdown",
"source": [
"## Machine Learning for Environmental Risk Analysis\n",
"\n",
"For environmental sciences practioners, one of the scenarios where machine learning can be particularly useful is risk analysis. Environmental risk analysis involves predicting where potential hazards may occur; it also involves understanding why some regions are more vulnerable to hazards than others. Machine learning models can be useful for these tasks because they can analyze large datasets containing different environmental predictors (e.g., weather conditions, soil conditions etc.) in an effective manner. By learning the hidden links between predictors and hazard risks, we may also gain new insights on what predictors or patterns are useful for creating early hazard warning systems.\n",
"\n",
"In this exercise, we ask you to use the machine learning classifiers we learned in this chapter to recreate wildfire susceptibility maps for the Liguria region of Italy. The basic idea is to use ML classifiers to analyze a dataset with observations in weather conditions, vegetation cover, and topography information. The goal will be to understand how different factors enhance or reduce the probability of firefire in Liguria, which can help authorities and decision-makers to apply resources to critical areas for hazard prevention."
],
"metadata": {
"id": "cEsFUVuJIUWP"
}
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "DPTrE6Fsde2M"
}
},
{
"cell_type": "markdown",
"source": [
"**Caption**: A wildfire in Italy. Can we predict which locations are most susceptible to wildfires using simple classifiers? 🔥\n",
"\n",
"**Source**: [ANSA](https://www.ansa.it/english/news/2015/08/14/wildfires-in-italy-up-120-this-summer-on-past-two-years_f6e4eff7-ac07-4f9e-aa5a-e61f8c2ad7f2.html)"
],
"metadata": {
"id": "2Wajb6bWdkz1"
}
},
{
"cell_type": "markdown",
"source": [
"Let's start by downloading and loading the datasets into memory using the [`pooch`](https://pypi.org/project/pooch/), [`pickle5`](https://pypi.org/project/pickle5/), and [`GeoPandas`](https://geopandas.org/en/stable/) libraries:"
],
"metadata": {
"id": "I5YF6S6kjWoK"
}
},
{
"cell_type": "code",
"source": [
"# Install geopandas and pickle5\n",
"%pip install geopandas\n",
"%pip install pickle5"
],
"metadata": {
"id": "EvUUjxk2QQox",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3f17bd83-f725-4fea-8fdd-3f8e98a183b1"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting geopandas\n",
" Downloading geopandas-0.10.2-py2.py3-none-any.whl (1.0 MB)\n",
"\u001b[K |████████████████████████████████| 1.0 MB 5.2 MB/s \n",
"\u001b[?25hRequirement already satisfied: shapely>=1.6 in /usr/local/lib/python3.7/dist-packages (from geopandas) (1.8.5.post1)\n",
"Collecting pyproj>=2.2.0\n",
" Downloading pyproj-3.2.1-cp37-cp37m-manylinux2010_x86_64.whl (6.3 MB)\n",
"\u001b[K |████████████████████████████████| 6.3 MB 53.6 MB/s \n",
"\u001b[?25hCollecting fiona>=1.8\n",
" Downloading Fiona-1.8.22-cp37-cp37m-manylinux2014_x86_64.whl (16.7 MB)\n",
"\u001b[K |████████████████████████████████| 16.7 MB 865 kB/s \n",
"\u001b[?25hRequirement already satisfied: pandas>=0.25.0 in /usr/local/lib/python3.7/dist-packages (from geopandas) (1.3.5)\n",
"Collecting munch\n",
" Downloading munch-2.5.0-py2.py3-none-any.whl (10 kB)\n",
"Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas) (57.4.0)\n",
"Collecting click-plugins>=1.0\n",
" Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)\n",
"Requirement already satisfied: click>=4.0 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas) (7.1.2)\n",
"Collecting cligj>=0.5\n",
" Downloading cligj-0.7.2-py3-none-any.whl (7.1 kB)\n",
"Requirement already satisfied: attrs>=17 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas) (22.1.0)\n",
"Requirement already satisfied: six>=1.7 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas) (1.15.0)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas) (2022.9.24)\n",
"Requirement already satisfied: numpy>=1.17.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->geopandas) (1.21.6)\n",
"Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->geopandas) (2022.5)\n",
"Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->geopandas) (2.8.2)\n",
"Installing collected packages: munch, cligj, click-plugins, pyproj, fiona, geopandas\n",
"Successfully installed click-plugins-1.1.1 cligj-0.7.2 fiona-1.8.22 geopandas-0.10.2 munch-2.5.0 pyproj-3.2.1\n",
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting pickle5\n",
" Downloading pickle5-0.0.12-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (256 kB)\n",
"\u001b[K |████████████████████████████████| 256 kB 4.9 MB/s \n",
"\u001b[?25hInstalling collected packages: pickle5\n",
"Successfully installed pickle5-0.0.12\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"import geopandas as gpd\n",
"import numpy as np\n",
"import pickle5 as pickle\n",
"import pooch"
],
"metadata": {
"id": "ZlylDdDDkmTt"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Function to load the data\n",
"def load_data(path):\n",
"\n",
" # Load the content of the pickle file (using pickle5 for Google Colab)\n",
" with open(path, \"rb\") as fh:\n",
" points_df = pickle.load(fh)\n",
"\n",
" # Convert it to a Geopandas `GeoDataFrame` for spatial analysis\n",
" points_df = gpd.GeoDataFrame(points_df,\n",
" geometry=gpd.points_from_xy(np.float64(points_df.x),\n",
" np.float64(points_df.y)))\n",
" return points_df"
],
"metadata": {
"id": "7wpeP_KwjVJp"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Path to the data in UNIL OneDrive\n",
"variables_path = pooch.retrieve('https://unils-my.sharepoint.com/:u:/g/personal/tom_beucler_unil_ch/EU4FQkuYknFDiDfd7droyAcBP0qFOR5-c-_Oq74gjhTGwQ?download=1',\n",
" known_hash='e8ebc70f972b5af4ef3d6110dcd61ce01ce5a830dcdb7d2c9e737aeab781606c')\n",
"wildfires_path = pooch.retrieve('https://unils-my.sharepoint.com/:u:/g/personal/tom_beucler_unil_ch/EcjqeERsnIRHjhcx1ZFVNggBS7nPUkW530XRrpVUB-qnOw?download=1',\n",
" known_hash='361f067aafbac8add8f8a9a5c630df3c962cd37a2f125f420e7b9330fd0a1a4c')"
],
"metadata": {
"id": "B4U806prkx5Q",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "15234d82-83de-47b7-9b3c-ebce1623b43b"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"Downloading data from 'https://unils-my.sharepoint.com/:u:/g/personal/tom_beucler_unil_ch/EU4FQkuYknFDiDfd7droyAcBP0qFOR5-c-_Oq74gjhTGwQ?download=1' to file '/root/.cache/pooch/07ef52a70acfc4230ffc4d0ca1624c7a-EU4FQkuYknFDiDfd7droyAcBP0qFOR5-c-_Oq74gjhTGwQ'.\n",
"Downloading data from 'https://unils-my.sharepoint.com/:u:/g/personal/tom_beucler_unil_ch/EcjqeERsnIRHjhcx1ZFVNggBS7nPUkW530XRrpVUB-qnOw?download=1' to file '/root/.cache/pooch/083c82ff55a8a8a6c98e76bf347a96ef-EcjqeERsnIRHjhcx1ZFVNggBS7nPUkW530XRrpVUB-qnOw'.\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"# Load the data and convert it to a GeoPandas `GeoDataFrame`\n",
"# This can take a minute\n",
"variables = load_data(variables_path)\n",
"wildfires = load_data(wildfires_path)"
],
"metadata": {
"id": "r4oQ_Vq6lPMG"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Part I: Pre-Processing the Dataset for Classification"
],
"metadata": {
"id": "QXEbGT2plrYU"
}
},
{
"cell_type": "markdown",
"source": [
"**Q1) After analyzing the topography and land cover data provided in `variables`, create your input dataset `inputs` from `variables` to predict the occurence of wildfires (`wildfires`). Keep at least one categorical variable (`veg`, `bioclim`, or `phytoclim`).**\n",
"\n",
"Hint 1: Refer to the documentation [at this link](https://unils-my.sharepoint.com/:t:/g/personal/tom_beucler_unil_ch/EREf92h-43VBtgPjwveMfrQBRahitmHxBKAtNx04lleh7Q?e=Fx6nad) to know what the different `keys` of `variables` refer to.\n",
"\n",
"Hint 2: You may refer to Table 1 of [Tonini et al.](https://www.mdpi.com/2076-3263/10/3/105), copied below, to choose your input variables, although we recommend starting with less inputs at first to build a simpler model and avoid overfitting."
],
"metadata": {
"id": "Sx0LlrRxG3sO"
}
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "5B9LqO4ykATs"
}
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "YUnRGFvPZNiE"
}
},
{
"cell_type": "markdown",
"source": [
"Here are some pandas commands you could use to explore your data.\n",
"[.head()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.head.html)\n",
"[.columns()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.columns.html)\n",
"[.describe()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.describe.html)"
],
"metadata": {
"id": "7YzgTgOIQaXM"
}
},
{
"cell_type": "code",
"source": [
"# Explore the `variables` dataset\n",
"#########################################################################################################\n",
"# 1. What kind of data is provided in the \"variables\" panda DataFrame?\n",
"#########################################################################################################\n",
"# Can you print the variable names?\n",
"print(variables._______)"
],
"metadata": {
"id": "yof-5oY_lxKT"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Can you print a table with all the statistics of each variable in the dataframe?\n",
"print(variables._________)"
],
"metadata": {
"id": "PK7q0aDCRCL9"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"There are 25 columns (variables) in the DataFrame. It is probably best to start simple and just use a few variables to make the wildfire prediction.\n",
"\n",
"Can we use a very simple model to predict wildfires ❓\n",
"\n",
"A simple model might contain `dem`, `slope`, `veg` and `bioclim`. We can use it as a baseline to evaluate model performance when you use other combinations to train the model.\n",
"\n",
"However, we cannot tell you which combination would perform the best as we have not done an extensive search while preparing the notebook."
],
"metadata": {
"id": "FZYhaClmRT1U"
}
},
{
"cell_type": "code",
"source": [
"# Here you will filter the DataFrame so that only the variables you want are in the input\n",
"############################################################################################################\n",
"# 1. Drop down the names of variables you want here so that pandas can do the filtering for you.\n",
"############################################################################################################\n",
"vars_tokeep = [___,____,____,__name__]"
],
"metadata": {
"id": "bFxOTEU6JH0t"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"############################################################################################################\n",
"# 2. Create a new dataframe 'inputs' with just the variables you want\n",
"############################################################################################################\n",
"inputs = variables[_______] # Filter 'variables' dataframe\n",
"inputs.head # print the first few columns to make sure everything works fine"
],
"metadata": {
"id": "KsxoaGQoZ2Mt",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9d19ae32-198c-46bd-fc16-41a774853d82"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"| \n", " | 0 | \n", "1 | \n", "2 | \n", "3 | \n", "4 | \n", "5 | \n", "6 | \n", "7 | \n", "8 | \n", "9 | \n", "10 | \n", "11 | \n", "12 | \n", "13 | \n", "14 | \n", "15 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "545 | \n", "15.253697 | \n", "34 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 1 | \n", "213 | \n", "19.419361 | \n", "32 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 2 | \n", "484 | \n", "22.660355 | \n", "37 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 3 | \n", "376 | \n", "17.878534 | \n", "32 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 4 | \n", "1241 | \n", "8.114972 | \n", "32 | \n", "26 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 396496 | \n", "695 | \n", "21.491549 | \n", "34 | \n", "15 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 396497 | \n", "254 | \n", "14.585821 | \n", "32 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 396498 | \n", "237 | \n", "8.356695 | \n", "32 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
| 396499 | \n", "917 | \n", "14.110499 | \n", "33 | \n", "15 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 396500 | \n", "556 | \n", "24.595417 | \n", "32 | \n", "8 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "1.0 | \n", "0.0 | \n", "
396501 rows × 16 columns
\n", "