Getting data off the web

March 25-27, April 1: lectures on APIs and web scraping

March 25-27, April 1: lectures on APIs and web scraping

Lec 1 - Introduction to data-wrangling
Lec 2-3 - Create arrays, array shapes
Lec 4-5 - pandas
Lec 4 - Parsing XML
Lecture 6: advanced pandas
versioning with git
- Intro to git 1
- Intro to git 2
Lecture 10: visualization with seaborn
Lec 12 - Split-apply-combine paradigm
Lec 13 - Enriching your dataframe with external information and join/merge
Lecture 16 - Getting data: API calls and HTTP requests
Lecture 18 - Web scraping
Lec 20 - Continuous integration/testing with github actions
Lecture 21 - Working with text data:
Lecture 22 - Poster that includes files from notebooks
Lecture 23 - Data format conversion and backups to cloud services
- Conversions between data format with visidata
- Write to different format with pandas
- backup_to_different_format with_pandas.qmd
- git Large File Storage (LFS)
- Setting up python and install packages in requirements.txt

March 25-27, April 1: lectures on APIs and web scraping

See the sidebar for many exampels regarding getting data off the web with APIs and web scraping tools.