In this notebook and the corresponding Medium blogpost, I analyzed the correlation between social indicators (such as poverty, distance from next pharmacy, income, etc.), Google mobility data and the number of confirmed Covid-19 cases and fatalities in Chile.
After understanding the correlation, I built a Machine Learning model to predict the new cases per day for cases and fatalities caused by Covid-19 in Chile.
Combining information from different data sources, I tried to find answers to the following questions:
- How is the situation within Chile and what differences are there in the different regions?
- How did mobility in the country change?
- Which correlation does exist between the socioeconomic indicators and the mobility data with the Covid-19 numbers?
- How does a Machine Learning model perform with socioeconomic and mobility data perform?
- Covid-19 Chile Dashboard(GitHub repository)
- Medium article: How do social indicators drive the spread of Covid-19 in Chile?
- Medium article: Social Vulnerability and Covid-19 in the United States(GitHub repository)
You simply fork or copy the code, the notebook includes directly reads in the Covid-19 data from its sources. The CASEN socioeconomic data has to be download separately (see below), as well as the Google mobility data (see below).
Apart from this, the notebook runs on Python 3.7 and includes the libraries pandas, numpy, matplotlib, seaborn and sklearn.
This repository has been made with the help of many very important and helpful data sources:
- The Chilean Ministry of Social Developmentprovides a database with a wide range of socioeconomic indicators (Caracterización Socioeconómica Nacional, CASEN). You can download the compressed file here, I relied on the STATA data (unpacked, its about 630 MB); you will have to download it - I did not upload it to my repository.
- For data about Covid-19 in Chile, the Chilean Ministry of Science, Technology, Knowledge and Innovation and the Chilean Data Observatory provides on GitHub more than 50 data products, including daily updated information about Covid-19 confirmed cases and fatalities in Chilean regions and districts. I included data about the Covid-19 cases per region (data product 3) and Covid-19 fatalities per region (data product 14).
- Finally, Google provides quite interesting mobility reports for a lot of countries. On their website you can download the Global Mobility Report as CSV which includes the data for Chile and the Chilean regions.
Please not that the data about cases and fatalities is updated daily.
The CASEN database includes answers from more than 216.000 individuals from all regions in Chile to questions from the following topics: education, work, income, health, identity, housing and poverty. Each category includes around 30 different questions, just like "How far is the next pharmacy?"
The CASEN study is realized every two years, e.g. to track the percentage of families that are still living in poverty.
As Google writes on the mobility homepage:
As global communities respond to COVID-19, we've heard from public health officials that the same type of aggregated, anonymized insights we use in products such as Google Maps could be helpful as they make critical decisions to combat COVID-19.
These Community Mobility Reports aim to provide insights into what has changed in response to policies aimed at combating COVID-19. The reports chart movement trends over time by geography, across different categories of places such as retail and recreation, groceries and pharmacies, parks, transit stations, workplaces, and residential.
The Google mobility data show the change in percentage from a baseline (before Covid-19) in the amount of visitors in different categories, just like parks, retail, work etc. Only the "residential" category is different: It is measured in duration rather than visitors.
You can find my analysis and my findings in the Medium blogpost.
- Covid Chile Prediction.ipynb: The jupyter notebook that includes all the code.
- Libro_de_Codigos_Casen_2017.pdf: A PDF that explains all indicators that appear in the CASEN database (Spanish)
- Media: A directory that includes the data (pictures and data) for the Medium blogpost
Every contribution is welcome. In the Medium blogpost, I have written some ideas and suggestions for further investigations.
Thanks to the Chilean Ministry of Science, Technology, Knowledge and Innovation and the Chilean Data Observatory for providing daily updated data about Covid-19 on their GitHub. Furthermore the socio-economic CASEN data, provided by the Chilean Ministry of Social Development was very useful and enabled me to make this investigation. I also relied on WorldBank population data. Thanks to Google for providing the mobility reports. I also read a lot of articles about time series prediction and would like to especially thank Varshita Sher, her article about "Time Series Modeling using Scikit, Pandas, and Numpy" was quite useful to get an understanding of time series prediction. Also a lot of my evaluation code (RMSE or the model selection etc.) was inspired by her code. Also very interesting was Mario Filhos description about "How To Predict Multiple Time Series At Once With Scikit-Learn" which helped me to understand the concept of multiple time series prediction.
Maximilian Müller, Senior Business Development Manager for Smart Metering and Data Analytics in the Renewable Energy sector. Now diving into the field of Data Science.
Copyright 2020 Maximilian Müller
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
From opensource.org