Bollette Auto
A self-hosted system that watches my inbox, automatically downloads household utility bills from provider portals, files them on my home NAS, and extracts their data into a web dashboard — with Telegram notifications.
The problem
Every month, the same chore: an email arrives from the provider, log into the portal, download the PDF, rename it, save it to the right folder. Multiply that by electricity, gas, phone and internet and it becomes a tedious, repetitive task — exactly the kind of thing a computer should be doing for me.
What it does
Bollette Auto (“auto bills”) runs in the background inside a small Docker container and, on its own:
- Checks the inbox over IMAP (Gmail and other mailboxes) at regular intervals.
- Recognises bill emails by their sender.
- Gets the PDF — from the attachment, from a link in the email, or from the provider’s portal.
- Files it on the NAS in a tidy
year/PROVIDER/structure. - Extracts the data from the PDF into a database so it can be explored.
- Notifies the outcome on Telegram — bill downloaded, or an error.
How it works
At its core is an IMAP polling loop: each mailbox gets its own thread that checks for new mail, routes it to the right provider, and triggers the matching scraper. The downloaded PDF is handed to the archiver, which stores it on the NAS (mounted over CIFS) under the folder for that year and provider.
Two download strategies
- Browser (Playwright) — for portals that require interactive navigation. The
scraper is configuration-driven: the CSS selectors for the login page and the
download are described in a YAML file, so adding a “standard” provider needs no
code. For robustness the download tries several strategies in sequence (URL in
the
onclick, native browser download, opening a new tab, direct link). - Direct API — for portals that expose REST APIs. A dedicated scraper logs in and downloads the PDF without launching a browser: faster and more reliable.
Three ways a bill arrives
Not every provider behaves the same, and the system has learned them one by one:
- PDF attachment — the simplest case: the email already carries the file. A configuration block is enough, no code. That’s how the home insurance and the water bill work.
- Link in the email — no attachment, but a link to a public page. For the motorbike road tax the receipt is an HTML page (or a PagoPA PDF) that the system downloads and, if needed, prints to PDF. For Sorgenia electricity the link leads to a page from which the PDF is picked up. Either way, no login and no SMS codes: I had actually built a portal-scraping version with saved cookies, and scrapped it once I saw the link in the email was enough.
- Portal — when there’s no other way, a login through a browser or the provider’s API. For TIM, protected by a reCAPTCHA, I log in by hand once and the system reuses the saved session, telling me on Telegram when it expires.
A per-provider subject filter keeps policies and notices that come from the same sender from being filed as “receipts”.
What happens to the email
After processing, the email is marked as read and moved to a “processed” folder; those older than twelve months are deleted, but only from that folder, never from the inbox (the monitored mailboxes include my personal Gmail). Identical error notifications aren’t repeated, and stay quiet at night.
Configuration-driven design
A single YAML file describes mailboxes, providers, selectors and notifications.
Most providers can be added without touching code; only the “hard” portals —
single-page apps with tokens in sessionStorage, unusual APIs — need a small
dedicated scraper in Python.
From PDFs to a dashboard
The bills don’t just sit as PDFs in a folder. Each file is parsed (text extraction with pdfplumber) to pull out the amount, billing period, invoice date, consumption and provider, and the data lands in a SQLite database.
On top of that data runs a web dashboard — a separate FastAPI + Uvicorn service with a static frontend and Chart.js charts:
- Overview — summary cards, monthly spend and consumption charts, a per-provider breakdown and annual spend.
- Bills — a paginated table with provider / year / month filters.
- Supplies — a card per provider with the supply details (POD / customer code).
- Trends — commodity price over time and consumption per provider.
The filters (provider, year, month) are global and persist as you move between tabs.
Security matters
- NAS credentials kept out of the code, in environment variables.
- A dedicated NAS user with read/write access to the bills folder only (principle of least privilege).
- An App Password for Gmail instead of the account password.
- No secrets in version control: everything sensitive stays out of the repository.
When the TIM session expires
The saved session lasts a few weeks. When it ran out in October, renewing it turned out harder than the first time: the browser Playwright drives gets stuck in an endless reCAPTCHA loop, and even launching the real Chrome with a dedicated profile didn’t help, because TIM silently rejects a login from a brand-new profile. In my everyday Chrome the login works. So the method that reliably works now is to log in there as usual and import the cookies: export them with a browser extension and run a small script that keeps only TIM’s cookies, tests that the session really loads the bills page, and writes the file only if it does. That check exists because an earlier helper had overwritten a good session with a logged-out one. One last detail: the browser identity the scraper presents had to be aligned with the version of Chrome I actually use, or the imported session risked being refused. After the renewal, the October bill downloaded on its own.
Status
In production and still growing: active providers cover electricity, gas, phone, internet, water, home insurance and motorbike tax, and the archive now holds over 280 bills. The system moved from Docker Desktop (which kept failing on reboot) to native Docker inside a Linux distribution on WSL2 on the mini PC, with automatic startup. The latest provider, water (10 October 2026), was just one line of configuration. TIM remains the fragile one: every few weeks the session needs renewing by hand (see above).
Stack
Python · Playwright · IMAP · Docker (WSL2) · SQLite · FastAPI · Synology NAS (CIFS) · Telegram Bot API
What I learned
-
Portals change their HTML often: keeping selectors in configuration rather than in code turns maintenance into a matter of seconds.
-
Splitting “generic YAML-driven scraper” from “dedicated scraper” keeps the system simple for the common case and flexible for the hard one.
-
Known limitation: portals with two-factor authentication aren’t handled. When a portal asks for an SMS code, it’s worth looking for another route (like the link in the email).
-
Emails that slip through: the system only reads unread mail in the inbox, so an email that skips the inbox (a filter archives it) or that I’ve already opened is never seen. For API providers it’s enough to re-run the scraper; for attachment-only ones the email has to be marked unread again. A periodic safety scan is an idea, not yet done.
-
The Python code is baked into the Docker image: after a change you have to rebuild, a restart isn’t enough.