Finding Local COVID Rates
I live in the U.S. state of Minnesota. Recently the state government provided recommendations for how and when to open schools in the fall. The guidance was based on the infection rates of the COVID-19 disease. In simple terms, the higher the rates, the less in-person the school should be. The actual calculation I needed was:
total number of cases in your area over the past 2 weeks per 10,000 residents
I have three kids in school, so when I heard this recommendation I went to find out this value for my area. It turned out to be a difficult statistic to find. Along with the announcement my state government released a set of data, but it was about three weeks behind the current rates. I found different sets of data available, but they either reported the daily case rate or a total count of infections, not this very specific calculation.
See Also:
GitLab Pages to the Rescue
I started by manually calculating the values with the data that I could find. This worked, but every time there were updated statistics, I had to go back and recalculate the value. I wanted a way to have this information available for me and others with the up-to-date information whenever I looked at it.
My process and output quickly evolved:
- I decided I would just write a script to download the data and do the calculation for me
- Once I had this working I thought the next step would be to automatically graph the results
- Then I thought I could publish the graphs on a website
- If I was going to publish them, I thought the best thing to do to get this done quickly would be to use GitLab Pages.
The result is a simple static website: https://mattnohr.gitlab.io/covid-county/
How It Works
The basic flow for my new “system” is:
(*) --> "Download data"
--> "Calculate the rates"
--> "Create a new .csv file with daily calculated values"
--> "Publish .csv file to GitLab pages"
--> "Use GitLab pages to serve static website that reads .csv"
--> (*)
The first few steps are done with a simple Kotlin script that is run using the Gradle build tool. I used GitLab CI/CD pipelines to run a job to do that automatically. You can find an example gradle .gitlab-ci.yml file here. The basics of this step for me look like:
build:
stage: build
script: gradle --build-cache run
The next step was to get it published to GitLab pages. That also used a GitLab CI/CD job. It simply needed to move the .csv results out of the build directory into the “public” directory that is used to host GitLab pages.
pages:
stage: deploy
dependencies:
- build
script:
- mv build/data.csv public/
The actual static webpage uses d3.js that is able to read the data from a .csv file and graph it.
My GitLab project can be found here: https://gitlab.com/mattnohr/covid-county
Running on a Schedule
Once I had the system up and running with GitLab CI, I was able to use GitLab Pipeline Schedules to run the script a few times a day to get updated data. Now I do not have to worry about when the data is updated, I can just review my GitLab Pages site to see the latest values.
Pipeline Scheudles let you easily schedule pipelines daily, weekly, or monthly. Since I wanted this to run multiple times a day, I used a cron schedule to run the pipeline 4 times a day on weekdays:
0 8,12,16,20 * * 1-5
Result
Now I have a simple website that has the most up-to-date calculations for this specific value for my local area. Now I just need to wait for our local school board to make a final decision on how schools will look!
Cover image by @ThisisEngineering RAEng on Unsplash