Blog Post 5 Min Read

Using Postgres to store job description data

Hai Bao

Data Science Enthusiast • May 10, 2026

In today's very competitive job market, it is very important to be able to be an early applicant as each position receives significant amount of applicants. That is why I created a Python script to scrape job posting data (job description and job requirements) that runs daily. All the scraped data needs to be stored somewhere and that somewhere is a Postgres database.

Storing the data serves two purposes. First thing it does is that it allows us to keep track of new postings by storing older ones. The second purpose is to utilize the data to create insights about the current job market.

The Postgres database runs on a local machine. I originally had it running on AWS but the upkeep costs were too high in my opinion so I migrated to a local machine.

High-resolution performance benchmark graph

Fig 1.1: Line graph monitoring number of job postings over time, updated daily, stored in a AWS S3 bucket. Click on the graph for the full screen view.