HTTP requests

What is an HTTP request

Or with raw headers on the right…

Making HTTP requests in python

The most popular package to make HTTP requests in python, for instance to fetch the unviersity homepage or to retrieve data through APIs, is the requests package.

import requests

req = requests.get('https://www.rutgers.edu')
req
<Response [200]>

Here, let us print the first 1000 characters of the response to this request.

req.content[:1000]
b'<!DOCTYPE html>\n<html lang="en" dir="ltr" prefix="content: http://purl.org/rss/1.0/modules/content/  dc: http://purl.org/dc/terms/  foaf: http://xmlns.com/foaf/0.1/  og: http://ogp.me/ns#  rdfs: http://www.w3.org/2000/01/rdf-schema#  schema: http://schema.org/  sioc: http://rdfs.org/sioc/ns#  sioct: http://rdfs.org/sioc/types#  skos: http://www.w3.org/2004/02/skos/core#  xsd: http://www.w3.org/2001/XMLSchema# ">\n  <head>\n    <meta charset="utf-8" />\n<meta name="description" content="Rutgers, The State University of New Jersey, is one of America&#039;s leading public research universities. Consistently top-ranked, Rutgers offers a range of undergraduate and graduate degree programs and continuing education opportunities." />\n<link rel="shortlink" href="https://www.rutgers.edu/" />\n<link rel="canonical" href="https://www.rutgers.edu/" />\n<link rel="image_src" href="https://www.rutgers.edu/sites/default/files/legacy-inline/2020/Feb/Screen%20Shot%202020-02-19.png" />\n<meta name="referrer" '

This is the HTML file that produces the university homepage when visited using a web browser.