How I Used Agentic Engineering to Become a Top 5% Contributor to a Major Open-Source Project in Less Than a Month
19 merged PRs across three repos and three languages, in 25 days — and the machinery that made it possible.
In early July I opened my first pull request against LiteLLM. I kept opening them for about a week. Nothing merged. Nothing was reviewed. Nobody commented.
That’s not a complaint — maintainers owe strangers nothing, and a quiet queue is usually a bandwidth problem rather than a judgment. But it is information, and after a week I decided to treat it as information rather than as something to push harder against.
So I asked Claude to survey eighteen major open-source projects and answer a narrower question than “which project is best”: as an unknown outside contributor, where would my work most likely get recognized and merged? The results are in the article Your First Pull Request Used to Be a Coin Flip. Based on these results I moved on to Dify — an open-source platform for building LLM applications, where you assemble apps in a visual workflow editor rather than writing code.
Twenty-five days later I have 19 merged pull requests across three Dify repositories and three languages. A complete sweep of the organization’s 32 repositories and 18,661 merged pull requests puts me at #101 of 2,001 all-time external contributors, the top 5.05%, or #92 and top 4.62% once you exclude the nine accounts whose profiles openly declare they work there. The full ranking, and the script that produced it, are in a public gist — bring your own GitHub token and you’ll get today’s numbers, which will differ from these, because the queue keeps moving.
I want to be precise about what that measures, because rank statistics are easy to inflate. It counts merged pull requests, nothing else. It says nothing about the size or difficulty of what merged. And it’s contaminated in one direction only, and not in my favor: GitHub doesn’t expose private organization membership, so several people who plainly work as engineers on Dify’s team are classified as “external” and sit above me in that list. Contamination in the other direction — an actual Dify engineer buried below me on nineteen merges — is implausible enough that I didn’t chase it.
And it is a rank by merge count, not by status. I have not been granted write access anywhere, which is the rung that actually marks a trusted collaborator — as my own coin-flip piece argued, a merged pull request makes you a contributor, not an insider. This is nineteen merges, not a seat at the table.
This piece is about how I did it. Not the merge count — the machinery. Below are the four principles I used to get there.
Vibe coding and agentic engineering are not the same activity
The phrase “vibe coding” describes something real: you describe what you want in natural language, the model produces code, you look at the result, and you judge it by whether it seems to work. If it seems to work, you keep it. If it doesn’t, you describe the problem and try again. The loop is fast and the feedback signal is your own impression of the output.
That’s how I started coding with Claude Code back in 2025, when I asked it to build a chess game app for my iPhone with pieces shaped like the ones on chess.com. After Ckaude finished, I installed the app on my iPhone and tested it. When I noticed bugs such as the pieces being the wrong shape, I gave Claude another prompt to fix the bugs and then retested the app. After a couple rounds of this, Claude had finally delivered a competent game that satisfied me.
This works surprisingly well for things where you are the only person who bears the cost of being wrong. A script for yourself. A prototype. A weekend project.
It stops working the moment someone else bears that cost.
When I open a pull request on Dify, a maintainer spends their attention on it. If my fix is plausible but wrong, I’ve wasted the maintainer’s time and left a worse repository behind. “It seemed to work” is not a standard that survives contact with that. And the failure mode of a capable model is not gibberish — it’s something coherent, well-structured, and confidently incorrect. Vibe coding has no defense against exactly that, because its only test is whether the output looks right, and looking right is the failure’s whole nature.
Agentic engineering, as I mean it, is the practice of building the apparatus that delivers reliable output and catches the confident-but-wrong case before anyone else sees it.
Concretely, in my setup: one Claude.ai instance, which lives in Anthropic’s Claude app, plans any changes as the Architect. Another Claude Code instance, which lives in my Mac’s terminal, executes the plan as a Coder. My own private git repository (architect-cc-comms) is the communications bus that enables them to communicate with each other. And I make the decisions. The article I Built a Dashboard for the App That Guards My Money — Without Writing the Code provides a detailed description about how my two Claude setup built software for me on one of my own projects.
But the agents are the least interesting part. What makes it work is a set of standing rules about verification — preconditions the executing agent must check before it runs, stop conditions that halt it rather than letting it improvise, and a requirement that every claim be traced to something observed rather than inferred.
The difference in one line: vibe coding optimizes for generating output. Agentic engineering optimizes for rejecting it.
Here’s a number that makes that concrete. I keep a running tally of one specific failure — stating something as fact when running a check would have settled it, and not running the check. It stands at nineteen instances in twelve days — coincidentally the same as the merge count, and measuring the opposite thing. Not nineteen bugs shipped: nineteen times the apparatus caught a confident-but-wrong claim, usually Claude’s, before it reached a maintainer. Twice in two days, the caught error was the same class — a status flag inherited from its own notes that had gone stale and that it carried forward as fact rather than re-verifying.
That tally is not an embarrassment. It’s the product. If it were zero I’d assume the machinery had stopped working.
Principle one: use data to pick the project
This is the cheapest leverage available, and I hadn't done it when I picked LiteLLM. I'd looked at the project — its popularity, its activity, whether I found it interesting. I hadn't looked at whether it merges work from strangers.
Claude’s survey of 18 major AI projects measured, per repository, what fraction of first-time outside contributors get their first pull request merged, how long a first human response takes, and what fraction of pull requests never get touched by a human at all. Dify’s organization has four main repositories and they each behave like four different projects: the marketplace-edge repo (dify-official-plugins) was merging about two-thirds of first-time contributors, the platform core (dify) fewer than half but responds fastest, and one repository (graphon) is so internal that only eleven outsiders, including me, have ever opened a pull request against it.
The natural instinct is to contribute to the project you admire most. The better question is where an unknown contributor’s work gets recognized and merged. Those are frequently not the same place, and the gap between them is where months disappear quietly.
My LiteLLM pull requests are still open. Still zero merged, still zero reviewed. I don’t think that reflects badly on anyone there. I think it reflects a repository where maintainer attention is scarce and I had no way to earn any.
Principle two: use the product to find bugs
The survey told me where to knock. It said nothing about what to bring, and my first instinct was wrong.
I went to Dify’s issue queues, because that’s what you’re told to do: find an unclaimed bug, fix it, open a PR for your fix. Over one week I had Claude investigate eighteen concrete bugs from the issue queue. Fifteen of them already had an open pull request that fixed them by the time I looked. The other three died for reasons of their own: one was a maintainer’s design call rather than a defect, one couldn’t be reproduced, and one needed a vendor API key I couldn’t obtain. That’s how the tally reached 0-for-18 — eighteen investigated, eighteen verified, zero pull requests opened.
Fifteen of those eighteen I lost to someone faster; the other three were unwinnable for reasons that had nothing to do with speed. The ones I lost, I lost to other engineers — fast, organized contributors fixing bugs soon after they came up.
The visible, well-specified bug recorded in a GitHub issue is the most contested square on the board. It’s a public record of a bug that’s clear and unambiguous, which is precisely why it’s already claimed.
But I want to be careful about the lesson here, because it isn’t “the issue queue is worthless.” It’s that working the queue by hand is a losing race. I read an issue when I happen to look at it, form a plan, dispatch the work, verify the fix, and open a pull request some hours later. The contributors beating me to those bugs weren’t better engineers. They were earlier. I come back to the queue in principle four and win it — just not by hand.
What worked in the meantime was building real applications on Dify and paying attention when something broke. A bug I hit myself is structurally unraced, because nobody else was standing where I was standing.
The clearest case started with an API key.
I was configuring a SiliconFlow model plugin with a key I knew was valid. Dify rejected it. The key worked against SiliconFlow’s own API, so the credential wasn’t the problem — Dify was validating it against the wrong endpoint. The plugin had the Chinese domestic API endpoint URL hardcoded, with no way for a user on the international platform to point it elsewhere. A valid key, correctly entered, rejected with an error message blaming the key.
That’s one bug and one pull request. But it raised a better question, which is the third principle.
Principle three: when you find one bug, sweep the codebase for its siblings
Was this the only plugin that has this bug?
I had my Architect Claude conduct an audit across Dify’s entire plugin repository every time I found a bug worth fixing. In one case, I had Claude fix a bug that was caused by a hardcoded regional URL for authenticating an API key that users who lived outside China couldn’t use. After Claude fixed the bug in one plugin, I had the Architect scan every plugin to find ones that had hardcoded regional URLs. The result was a bounded, enumerable class — 102 plugins with a hardcoded URL and no way to override it, against 76 that already let the user supply their own. Within the locked group, four plugins were live instances of the same bug I’d hit.
I opened issues for all four plugins with matching pull requests, plus an umbrella issue laying out this class of bug and asking the maintainers how they wanted situations like this handled in the future. Each carried a verification receipt; the strongest was a successful credential validation against a freshly minted international key.
Two PRs (PR#3601 and PR#3604) merged within five hours — four hours seventeen minutes and four hours fifty-two minutes, each approved by the repository’s primary maintainer.
In another case (PR #3625), I had Claude fix a bug in one plugin and then had it do another audit that found the same bug in 2 other plugins. I then had Claude fix that bug in the other plugins, and I opened a single PR that covered all 3 plugins. That PR merged within 21 hours after I opened it.
Principle four: build a machine to find the rest, then fix the machine
Principle two said the issue queue is the most contested square on the board, and that my hand-worked record there is 0-for-18. This principle says go back and take it anyway — because what loses that race is the hand, not the queue.
Every one of those eighteen losses had the same shape. An issue gets filed. It sits there until I happen to open GitHub. By then someone else has read it, fixed it, and opened a pull request. The bottleneck was never my ability to produce a fix; it was the hours between an issue appearing and me noticing it existed.
The only way to be earlier is to stop being the one who notices. So I built an automated tool that lets me respond more quickly to new issues.
An automated poller watches the repositories and surfaces new issues every 30 minutes. A sweeper takes each issue and invokes Claude Code to classify the issue into three buckets — FIX, NEEDS HUMAN, or NO FIX — with a written rationale. For every issue classified as FIX, the sweeper emails me in real time and automatically invokes Claude Code to fix the issue and push the fix to my fork of the repo. Every morning, I get an email digest of the issues, their classification and a concise rationale about why the model classified each issue.
One rule sits above all of it: the machine never opens anything itself. No issues, no pull requests, no comments. It only produces candidates. For issues it classifies as FIX, it goes one step further: it builds the fix and pushes it to a branch on my own fork — but it stops there. It never opens the pull request. I open every PR myself — and the filter isn’t that I’ve read every line, because at this volume I haven’t. It’s that every claim in the pull request body has to trace back to something observed: a reproduction, a proven failure before the fix, a trace through the code. I essentially have to reproduce the bug myself so that I can take a before screenshot and then verify the fix so that I can take an after screenshot.
Six of my nineteen merges came from that pipeline — the same issue queue that produced zero when I worked it by hand. Nothing about the queue changed. What changed is that a machine reads it within thirty minutes of an issue appearing, and I arrive at the fix already built.
Those six merges came in over the full 25 days, not from the single daily email digest above. Here is how they break down by what the sweeper had thought of each issue:
Look at that table for a second, because it says something I didn’t expect.
Only one of the six merges came from an issue the sweeper had classified as FIX — the one class that Claude automatically fixes. Two merges came from issues the sweeper classified as NO FIX, meaning don’t bother.
So the judge was frequently wrong about which issues to act on. What it was reliably good at was surfacing and summarizing — reading every new issue overnight and handing me a triaged digest with a clear rationale for each call in the morning. Receiving the digests each morning allowed me to setup a daily routine. The concise rationales are what made the wrong calls cheap: I could disagree with it in ten seconds by clicking the link to the issue and verifying the rationale or lack thereof.
The most instructive merge on my record came from overriding it. The sweeper looked at an issue, concluded a fix already existed, and returned NO FIX. I audited that verdict, found it false, instructed Claude to fix it, and shipped the fix — merged in seven and a half hours, about eight hours from the digest landing in my inbox.
That prompted a real fix to the machine. The bug was completely deterministic and had nothing to do with the model: the triage module was scraping bare #1234-style references out of free-text issue comments and treating any closed pull request it found that way as evidence of a prior rejected attempt. My first instinct, and the planning agent’s, was that the judge had hallucinated. Reading the code showed otherwise. A prompt fix would have fixed nothing. So I had Claude fix the real bug in the triage module.
Before I touched the sweeper I also put it under version control — it had been running unversioned on a server for weeks — with a baseline commit first and the fix as a reviewable second commit. Without the baseline there’s no “before,” and no clean way to undo a bad change on a machine that runs on a timer.
What this looks like when it goes wrong
Two examples, because a piece like this is worthless without them.
I shipped a wrong sentence in a merged pull request. The body claimed a change improved error classification “plugin-wide” because the underlying mapping was shared across all model types. It isn’t. Several model types define their own mapping that shadows it. The real reach was two of five. That is exactly the failure the evidentiary gate exists to prevent, and it got through because the sentence sounded like it followed from the code rather than from anything I’d checked. A bot flagged it — and because that same bot had been confidently wrong four times on another thread, Claude verified the claim independently from the class hierarchy before accepting it. Then I posted a correction: conceding plainly, bounding the error to that one sentence, and crediting the person whose report exposed it. I posted it before opening my next pull request, deliberately. A correction you volunteer reads as receipts. One extracted from you later reads as damage control.
I tested a hypothesis about one of Dify’s maintainers and it was refuted. I’d noticed that my fastest merges all involved Chinese-provider plugins and wondered whether the maintainer specialized there — which, if true, would have been a routing advantage to exploit by focusing on fixing Chinese-provider plugins. Measured across his complete 1,329-merge history: Western-provider plugin PRs merged at a 15.2-hour median against 16.6 hours for Chinese ones. The plugin’s region had no effect on merge speed.
And the current honest limits: my LiteLLM pull requests remain at zero merged and zero reviewed. Two Dify issues I opened were closed by the maintainers as fixed by other PRs while the defects still reproduce, and my re-verification comments on both are still sitting without replies. Twelve other pull requests are open right now and may never merge.
The four principles, revisited
Use data to pick the project. Where does an unknown contributor’s work actually get recognized and merged? Measure it rather than guessing.
Use the product to find bugs. A defect you hit yourself is structurally unraced. The repo’s issue queue contains well-specified bugs, but they are the most contested squares on the board.
When you find one bug, sweep for its siblings. Fixing one rejected API key was one pull request. The audit that turned it into a bounded four-instance class was four pull requests and a demonstration that someone had enumerated the problem rather than just reporting a single bug.
Build a machine to find the rest — then keep fixing the machine. The queue that beat me 0-for-18 by hand produced six merges once a poller was reading it every thirty minutes: what loses that race is the hand, not the queue. Expect the machine’s classifications to be wrong — its surfacing and its written rationales are the value, and its wrong verdicts are worth auditing, since my fastest merge began by overriding one.
Underneath all four is the thing that separates this from vibe coding. Generating a plausible fix is nearly free now. Deciding which fixes are real, and catching yourself when you’re confidently wrong, is the entire remaining job. Eighteen queue candidates investigated and rejected without opening a PR, a machine whose most useful output is a verdict I overrode, and nineteen logged instances of Claude catching its own unchecked assertions — that isn’t overhead around the work.
That is the work. The nineteen merges are what falls out the other end.
Part of an ongoing series on building software with two Claude agents — one architect, one coder, me as operator. The series began with I Don’t Read My Own Specs Anymore. New pieces weekly.