bazos-car-scraper

Python, requests, BeautifulSoup4, SQLite, cron · source on GitHub

What is it

An automated data collection tool that scrapes used-car listings frombazos.sk, cleans the data and stores it in a structured SQLite database.

It's more than a simple page downloader: it filters out noise (e.g. spare parts disguised as cars) and deduplicates entries, so the dataset stays clean for later analysis.

Features

Parsing: extracts price, location, title and direct link from unstructured HTML.
Noise filtering: listings under €500 are ignored to skip wrecks, spare parts and accessories. Listings without a numeric price (e.g. "Dohodou") are kept.
Deduplication: the listing link is the primary key, so an already stored listing is skipped (INSERT OR IGNORE).
Storage: a local SQLite database (bazos_cars.db, next to the script).
Configurable search: the search query is a command-line argument.
Analysis: analyze.py draws charts from the collected data.

How it works

┌──────────┐  requests  ┌───────────────┐   clean    ┌──────────────┐
│ bazos.sk │──────────▶│ BeautifulSoup │──────────▶│ SQLite: cars │
│ page 1…n │  1 s pause │  parse HTML   │  filter    └──────┬───────┘
└──────────┘            └───────────────┘  dedupe           │
                                                    analyze.py → charts

The scraper goes through the search results page by page until there are no more listings:

Request & parse: fetches the results with requests (10 s timeout) and parses the HTML with BeautifulSoup4. Network errors are logged and stop the run cleanly.
Data cleaning: normalizes prices (removes "€" and spaces) into an integer price_eur, keeps the original text in price_raw so non-numeric prices are not lost, and turns relative links into absolute URLs.
Storage: inserts the listings with parameterized INSERT OR IGNORE queries and logs how many new listings each page added.
Rate limiting: waits 1 second between pages and identifies itself with an honest User-Agent (bazos-car-scraper/1.0).

Database schema

Table cars:

column	type	description
`link`	TEXT	Absolute URL of the listing (primary key)
`title`	TEXT	Listing title
`price_eur`	INTEGER	Price in EUR, NULL if the price is not a number (e.g. "Dohodou")
`price_raw`	TEXT	Price exactly as shown on the site
`location`	TEXT	Location text from the listing
`first_seen`	TEXT	ISO timestamp (local time) of when the scraper first stored it

How to run

# create and activate a virtual environment
python3 -m venv .venv
source .venv/bin/activate

# install dependencies
pip install -r requirements.txt

# scrape with the default query ("honda civic 1.8 vtec")
python scraper.py

# scrape with a custom query
python scraper.py "skoda octavia"

# generate charts into docs/
python analyze.py

# look at the data
sqlite3 bazos_cars.db "SELECT title, price_eur, location FROM cars ORDER BY first_seen DESC LIMIT 10;"

analyze.py saves a price histogram and a chart of new listings per day into docs/. Charts will be added here once enough data has been collected.

Deployment

Runs on my Debian homelab, scheduled by cron every hour. The database path is resolved relative to the script, so it doesn't matter which directory cron starts it from.

0 * * * * /home/user/scripts/bazos_scraper/.venv/bin/python /home/user/scripts/bazos_scraper/scraper.py >> /home/user/scripts/bazos_scraper/scraper.log 2>&1