RoleLens

About the project

Job search usually starts with a title. Backend Engineer. Product Engineer. Solutions Engineer. Technical Analyst. But titles are weak signals: two companies can use the same title for completely different work, and two roles with very different titles can require almost the same underlying skills. RoleLens starts from the person instead. It reads a structured professional profile, reads the vacancy, infers what the work actually involves, separates career fit from practical viability, and only asks for my attention when something is genuinely worth looking at. Underneath, it is a lightweight semantic agent: an LLM supplies the interpretation, while final authority stays in deterministic Python backed by a durable SQLite queue.

Stack

Stack by layer

Language

Python, standard library only at runtime

Persistent state

SQLite (jobs, evaluations, durable queue, retry state)

Job source (my deployment)

Platsbanken via the Arbetsförmedlingen JobSearch API

Primary evaluator

Gemini 3.7 Flash, structured JSON output

Provider fallback

GPT-5 mini via Azure, transport failure only

Semantic batch size

Maximum 10 jobs per request

Scheduling (my deployment)

Hermes on a cron/systemd tick, twice daily

Delivery (my deployment)

Telegram, adaptable to email or any other channel

Runtime

Small always-on Linux VPS

Distribution

Self-hosted, open source, private config kept out of the repo

Noticeable changes

• Reads the advertisement instead of the title, surfacing roles that keyword alerts hide entirely.
• Separates Career Fit from Opportunity Score, so a perfect-work role with a hard blocker stays legible instead of vanishing into one blended number.
• Treats a valid but incomplete LLM response as pending work, never as a negative evaluation: omitted jobs stay in the durable queue and are retried.
• Keeps final authority in deterministic Python, with the model contributing interpretation rather than decisions.
• Replaying stored model responses through the real production policy code exposed two Swedish-language logic bugs in my own application, not in the model.
• Runs a full 45-job snapshot end to end in about three and a half minutes on a small VPS, with zero fallback calls and no backlog left behind.

The problem

Traditional job search works well only when you already know exactly which title to search for, and that assumption breaks down constantly. A backend engineer with AI experience might be a strong candidate for a Product Engineer role. Someone working across software, automation and business systems may fit a Solutions Engineer position. A vacancy called Systems Engineer may in practice be platform engineering, cloud infrastructure, CI/CD and cross-functional implementation work. Keyword alerts fail in both directions at once: they flood you with every advertisement containing the word engineer, and they hide the one role that describes exactly your work under a title you never thought to search for. The false positives are annoying. The false negatives are expensive, because you never learn the role existed. Both have the same root cause: the title is not the job. The description is the job.

The story behind it

RoleLens started as a personal tool, not as an AI product. I had a practical problem in my own job search and wanted to automate the part that consumed the most attention: opening dozens of descriptions a day, interpreting what the work really was, comparing it against my background, and checking language, seniority, location and eligibility barriers. The first realisation was that a CV is not enough. A CV describes what someone has already done; job discovery also depends on where they want to go. So I wrote a structured professional profile covering experience, projects, technical skills, education, preferred kinds of work, career direction, languages, geography, practical constraints, and the areas where my evidence is strong or thin. That profile became the reference every evaluation is measured against. The second realisation followed immediately: sending the whole job market to an LLM would make no sense. Most vacancies are obviously irrelevant, and a software profile does not need an expensive semantic reading of every healthcare or hospitality advertisement. RoleLens therefore became a staged system rather than a single prompt.

How a job becomes a notification

Discovery is deliberately over-inclusive, because it is free: 60 to 100 small queries per run against a public job API, costing zero tokens. Every hit is normalised into a job record and upserted into SQLite under a unique source key, with a SHA-256 content hash deciding whether an advertisement has genuinely changed. Cheap deterministic logic then removes the clearly irrelevant and orders the rest, without ever deciding inclusion on its own. Only the survivors reach semantic evaluation, in batches of at most ten, and the model is never asked for a simple match or no-match. It interprets what the role really is, identifies the requirements that matter, compares them with the profile, cites supporting evidence, names gaps, and distinguishes hard blockers from soft preferences. That structured output then passes through a deterministic policy layer before anything is ranked or delivered. The model contributes judgment. The application retains control.

System execution topology & provider routing

Two scores, not one

The most important design decision was refusing to collapse two different questions into a single match score. Career Fit asks whether the actual work matches my background, capabilities and direction, judged independently of whether I could practically get this particular job; language, location, citizenship, clearance and seniority never reduce it. Opportunity Score asks whether this specific vacancy is realistically worth pursuing right now, after the practical reality of the advertisement is accounted for. Kept apart, they say something precise: a role at career fit 95 and opportunity 30 is exactly my work with something in the advertisement blocking me, and that is actionable information a single blended number destroys.

Why semantic understanding matters

Job descriptions are full of nuance a keyword system cannot see. An advertisement written in Swedish does not automatically mean Swedish is mandatory. A company mentioning background screening does not automatically mean citizenship or clearance is required. A technology appearing in the text does not always mean years of experience with it are demanded. And missing information should not silently become a negative judgment: when the profile holds no evidence either way, the honest conclusion is unknown, not unqualified. These distinctions sound small, and they decide whether a good opportunity is surfaced or quietly discarded.

Structured evaluation output

A 200 OK that was still a failure

During a controlled provider evaluation, one model returned HTTP 200, valid JSON and no transport error. From an infrastructure perspective everything had succeeded. Seventeen of the twenty requested jobs were simply missing from the response. That single result changed how the whole system is designed. A missing job can never be read as a negative evaluation; it means the job was not evaluated, and those are completely different states. RoleLens now stores what came back, keeps the omitted IDs pending in the durable queue, and can retry them in one bounded cleanup pass. It never discards them and never pretends they were judged.

Testing the system, not just the model

Benchmarking providers improved semantic accuracy, but it also exposed something more uncomfortable. The model is only one component. When I replayed stored model responses through the real production policy code, the benchmark found two errors in my own Swedish-language logic: a statement equivalent to "Swedish is not required" could create a Swedish-language blocker, and a genuinely mandatory Swedish requirement written with a different sentence structure went undetected. The model output was not the problem; my application logic was. Because errors are weighted asymmetrically, a false negative counting triple against a false positive, and because the scored artefact is the deterministic decision rather than the model's own opinion, the benchmark measures what the system would actually do. Testing the LLM is not enough. Prompt behaviour, output completeness, parsing, deterministic rules, retries, persistence and final ranking have to be tested together.

Completeness before punctuality

An early version capped each run at 40 semantic candidates: bounded runtime, predictable cost, sensible on paper. The product behaviour was wrong. If 65 relevant vacancies appear and only the first 40 are evaluated, the best role of the day can be number 52, and the system is optimised for finishing quickly rather than for finding the best opportunity. I changed the invariant. RoleLens now freezes the complete relevant candidate snapshot for a run and processes all of it before reporting. The scheduled time means start searching now, not deliver at this minute regardless of completeness. That reads like an implementation detail. It is a product decision.

Why SQLite was enough

RoleLens deliberately avoids infrastructure it does not need: no Kubernetes, no vector database, no Redis, no queue broker, no browser automation, no third-party runtime dependencies. Python and SQLite carry the whole workload. SQLite holds the vacancies, the evaluations, the processing state and the retry information, and it acts as the durable queue itself. A provider outage does not erase pending jobs, a partial response does not erase the omitted ones, and a process restart does not lose the system's knowledge of what still needs evaluation. The database is not just storage. It is the reliability model.

Model routing and small batches

Gemini 3.7 Flash is the primary semantic evaluator; GPT-5 mini through Azure is retained as a provider and transport fallback, not as an automatic second opinion. The distinction matters: if the primary returns a valid response that omits one job, that job is not handed to another model to overrule the first, it simply stays pending for a bounded retry. Semantic disagreement and infrastructure failure are different problems and are handled differently. Batches stay at ten jobs or fewer, because structured-output completeness matters more than packing requests. Larger batches look cheaper right up to the moment the model starts silently omitting items, at which point the efficiency is imaginary.

Delivery

My own deployment runs unattended on a small VPS, scheduled through Hermes twice a day, delivering to Telegram because that is already part of my personal automation. A card carries exactly what I need to decide whether to look further: what the role really is, why it fits, the gap, what remains unknown, both scores, the deadline and the application link. Telegram and Hermes are deployment choices, not requirements of the engine; delivery can be pointed at email or any other notification workflow.

A production run

• 45 jobs selected for semantic evaluation, 45 evaluated, nothing left unresolved.
• 5 semantic batches, 0 fallback calls, 0 pending backlog after the run.
• 3 matches worth my attention, in roughly 3 minutes 24 seconds.
• Instead of opening 45 descriptions, I started with the three that had already survived both semantic evaluation and deterministic policy checks.

From personal tool to open source

The problem is not specific to software engineering. An accountant may fit roles they would never search by title. A mechanical engineer may map naturally onto product, systems or operations positions. A welder can describe TIG and MIG/MAG work, ISO certifications, materials, years of experience, language level and preferred geography, and the same engine asks the same question: what does this work actually involve, and how well does it fit this person? RoleLens is therefore being developed as an open-source, self-hosted system, with the public engine deliberately separated from my private production configuration. Profile data, production databases, secrets, raw vacancy data and private benchmark material stay outside the distribution. The engine can be shared; the personal context does not need to be.

What I learned

RoleLens began as an automation project and turned into an exercise in product design, LLM evaluation and reliability engineering. The interesting questions were never which model to use or how to call an API. They were what should happen when a model omits an item but reports success, what separates career potential from application viability, when a model should judge and when code should enforce, how a system should represent uncertainty, whether finishing on time matters more than not silently postponing the best vacancy, and how you test an entire decision pipeline rather than a single model. Those questions shaped the project far more than the original scraper did.