Projects

Seven projects at Audiense and five personal ones.

Audiense · 2021–present

Professional work

Machine learning & NLP

Replacing a 32B GPU model with a fine-tuned small one on CPU

Detecting brands, people and products in TikTok video transcripts ran as a Node service calling Qwen2.5-32B on GPU, in 20 replicas on EKS, the most expensive of our vLLM clusters. The replacement is a GLiNER service on CPU in EKS that consumes from Redpanda.

What I did. I fine-tuned GLiNER with LoRA on about 5,200 examples taken from Qwen's production output plus negatives from an audit, running the open weights locally because the hosted API had no data-processing agreement.

How I measured it. Because the training labels came from Qwen, I did not measure against Qwen's output. I used three references instead. The first is a held-out set of 1,017 posts and the second is 1,024 fresh posts from later days. Two Claude judges labelled both from scratch without seeing either system's output, and an adjudicating model settled their disagreements (the two judges agreed at about 0.9 F1). No person reviewed those two references. I set the bar beforehand for these two: no more than 0.03 F1 below Qwen on people and 0.05 on brands. The third reference is a validity score from 0 to 1 for every entity either system proposed, given by four independent decision models (models that answer a yes/no question with a score: TypeSafe's Jev and models from Perplexity, Liquid and Cloudflare). I averaged their scores and calibrated the average on 153 labels from my team. Before the switch it ran in shadow mode on live traffic, validated with a 568-case differential test.

Result. I shipped it knowing it missed my bar: against the two judge-labelled references it trails Qwen on brands by 0.10 and 0.17 F1 and, on the fresh posts, on people by 0.08 (95% intervals exclude zero). Brands are the bigger gap, and those are the cases most blurred with products; I accepted that trade-off because of the cost. It replaced Qwen in production, an order of magnitude cheaper (from thousands of euros a month to a few hundred) and faster than Qwen was on vLLM. The other measurements are less clear-cut. The validity score puts it within 0.02 F1 of Qwen overall (not statistically significant): better on people, worse on products, slightly behind on brands. A fourth check, using only human labels, tied them at 0.61 F1. The judge references count recall against everything the judges found; the validity score only against what the systems proposed, which flatters both, so the numbers can't be compared directly. I describe it as competitive, not equivalent.

GLiNERLoRA / PEFTAWS EKSRedpanda

Platform

VEGA: replacing per-team data copies with one query API

Every product team that needed audience data kept its own copy: an RDS here, a Mongo there, each fed by its own ETL and Spark jobs pulling from the lake. Each copy had its own pipeline to maintain and its own storage and compute bill.

What I did. I designed VEGA, an asynchronous query platform on top of the Trino work described below: a team submits an aggregation, gets an id, and fetches the result when it's ready. Jobs go through a queue, failed runs are retried and traced, and Redis rate-limits requests. I added a data catalog, query examples and a query builder, then worked with each team on their integration so they could drop their copy.

Result. Teams read from the lake through VEGA instead of maintaining local copies, and at least six local copies of the data were retired, together with several ETLs, a costly Spark process and the databases they filled. A schema change is now fixed in one place. Reports for very large audiences, which used to be out of reach because infrastructure cost and response times didn't scale, now run on VEGA.

FastAPITrinoAthenaRedisRedpanda

Machine learning & NLP

Interest classification with two models that must agree

Given a profile's bio and recent posts, assign the few interests that describe it from a taxonomy of 444 categories. The first version compared text embeddings against category embeddings above a cosine threshold. It looked fine in aggregate, but some categories acted as attractors (religious profiles landing in primary education, cricket looking like rugby), and the model was often confident when it was wrong.

What I did. I fine-tuned the embedding model with LoRA on labelled examples with hard negatives, which took F1 from 0.255 to 0.390 on a human-labelled test set. That wasn't enough, so I added a cross-encoder to rerank the bi-encoder's top 20 candidates. After hand-judging over a thousand assignments, agreement between the two models predicted correctness better than either score. A category is now kept only if it is in the top 5 of both rankings; if none survive, the profile is left unclassified. It runs on Kubernetes with vLLM, wired to a streaming consumer, so profiles are enriched as they arrive.

Funnel: 444 categories, bi-encoder keeps top 20 candidates in about 3 ms, cross-encoder reranks to top 5 in about 400 ms, intersection leaves about 2 final categories

Result (300 profiles, 1,150 human judgements). Strict precision 36% → 52%, at the cost of fewer categories per profile and more unclassified profiles. For audience segmentation that is the better trade.

The full story, with the failure cases and the numbers behind each decision, is going out on the Audiense engineering blog as Two models, one AND.

Sentence TransformersLoRA / PEFTMNRL + MatryoshkaCross-encodersvLLMQwenKubernetes

Data lake & streaming

Keeping a 15 TB lake current in real time

Audience profiles — their attributes and their social relationships — have to reflect new data as it arrives. Batch refreshes weren't keeping up, and the lake behind them is over 15 TB.

What I did. I created the Iceberg tables that streaming services now write to, and I maintain them. I led the migration of social-relationship data into Iceberg: schema changes were coordinated with every dependent consumer while they kept reading. I then completed the platform's move from batch to real-time enrichment. Geolocation was the last piece to go real time. One Redpanda consumer enriches every event with geolocation, and querying the lake was too slow for that. We kept a copy of that data in MySQL only for lookup performance. Trino's fault-tolerant execution coordinates tasks through S3, which is far too slow for per-event lookups, and these queries don't need it. We turned it off and added a local disk cache, lookups became fast enough to read Iceberg directly, and we decommissioned the MySQL copy.

Result. More than 20 streaming consumers read and write the same Iceberg tables in real time, and enrichment no longer waits for a batch window. Geolocation lookups now run on the lake itself, which retired one copy of the data to maintain and pay for.

Apache IcebergParquetKafkaRedpandaAWS ECSTrinoSpark

Query engine & performance

Tuning Trino report queries

Audience reports run as a sequence of heavy Trino queries over the lake. They shuffled enormous amounts of data between workers, and failed often enough that the services depending on them had to handle it themselves.

What I did. I profiled query patterns and S3 access, rewrote the worst steps, and added Iceberg table statistics so Trino's cost-based optimizer picked better join orders and shuffled less data. I also added a worker-allocation rule for a step that was waiting behind everything else.

Result (thousands of report queries the week before and the week after). Internal network per query is down 93%, and Trino query failures went from 2.2% at worst to under 0.2%.

TrinoApache IcebergS3

Machine learning & NLP

Resolving mentions to real-world entities

A mention in a post has to be linked to a real-world entity before it can enrich anything.

What I did. I built a system that matches mentions to entities in a knowledge base, with a validation step before other services consume the results. It publishes complete, versioned datasets to Iceberg; consumers keep reading a consistent version while the next one is prepared.

NERApache IcebergKubernetes CronJob

Machine learning & NLP

Extracting usable phrases from multilingual bios

Social media bios mix languages, abbreviations, emojis and hashtags. The interest classifier needs clean phrases.

What I did. I built a spaCy pipeline over nine languages that extracts meaningful phrases, filters noise and removes duplicates. Its output feeds the interest classifier above.

spaCyPython

Outside work

Personal projects

Built on my own time. Radar is a live pilot; the rest I built to learn how the technology works. In the agent prototypes the model only proposes actions; deterministic code owns the state and the rules.

Product

Radar: public tenders for small firms, in a Monday email

Public tenders for cleaning, building-maintenance and gardening SMEs are scattered across the feeds of the Spanish public procurement platform (PLACSP), in XML and PDF. Small firms find them late or never.

What I built. The whole product, on my own. It ingests both PLACSP Atom feeds with versioning, applies a high-recall prefilter by CPV code and title, and then a cost-aware classifier: a decision model (TypeSafe Jev) scores each tender yes/no per vertical, and only the uncertain zone (0.35 to 0.65) goes to an LLM (Claude Haiku) as a second engine, with a rules fallback if the models fail. On top of that: a weekly digest, daily alerts with a "closing soon" section, one-click feedback buttons in the email, and a web app.

Scope: launched in September 2026 as a pilot to see whether it gets traction. It covers Comunitat Valenciana, Castilla-La Mancha and Madrid.

PythonSupabaseNext.jsTypeSafe JevClaude Haiku

Voice agents

Voice agent prototype

I wanted to understand how voice agents work end to end. A simulated clinical pre-screening interview gave me a scenario with structure: questions, inclusion criteria, exclusion criteria.

What I built. A streaming pipeline — Deepgram for speech to text, a language model through OpenRouter for the dialogue, ElevenLabs for speech — connected over LiveKit/WebRTC. Conversation state moves through explicit transitions and is persisted to SQLite in WAL mode, which makes a crash mid-interview recoverable. Automated tests cover the state machine.

LiveKitDeepgramElevenLabsOpenRouterSQLite

Voice agents & harnesses

Math tutor with a validation harness

Can a language model hold a natural tutoring conversation while deterministic code decides what's correct and what comes next?

What I built. A Spanish-speaking voice tutor on LiveKit, same STT → LLM → TTS shape as above. The model proposes an interpretation of what the student said and a pedagogical action; a harness validates both before any domain code changes the learning state or a response is released. The educational rules are separated from the model and voice providers, state lives in SQLite, and the core loop is evaluated offline.

Scope: primary-school arithmetic, one student at a time. A prototype for engineering experiments.

PythonLiveKitFastAPISQLite

Agents & MCP

Hermes Expense Tracker

Shared household expenses, tracked by talking to an assistant.

What I built. A Python FastMCP server that owns the business rules and the SQLite persistence. Each household member has a Hermes Agent profile (Hermes Agent is an open-source agent runtime), over Telegram or the CLI, that calls the server's domain tools; totals and splits are computed by the server. Shared projects use membership-based visibility, so each person sees only what they belong to.

Hermes AgentFastMCPPythonSQLiteTelegram

Knowledge management

AI research wiki

An Obsidian vault I maintain with Hermes Agent, organised so an agent can read and update it: source papers and articles sit next to linked concept notes, technology profiles and architecture comparisons. Current topics are agent harnesses, tool use, RAG, evals, observability, and voice agents, including turn-taking. The voice prototypes above started as notes here.

ObsidianHermes AgentMarkdown