Six people share one desk row above a bakery on Banerji Road, Kochi. They run a backwater tour-booking app: three engineers, a designer, a founder who still answers support tickets, and a part-time ops contractor who arrives twice a week. Deployments, before this story begins, were a Friday-evening ritual involving one laptop that held, in its terminal history, the only accurate description of production. Call that laptop the single point of failure it was. This is the story of their first ninety days running GitOps — a composite of teams we have watched make the jump, told straight, with the numbers that matter arriving at the end.
GitOps Days 1–30: Writing Things Down First
Nothing was installed in week one, which surprised everyone. The team started with a decision record — one page, four choices: which store would hold the declared configuration, how the folders inside it would be arranged, which secrets tool they would standardise on, and what the drift rule would be when live state disagreed with the record. Writing the decisions down, the contractor later admitted, surfaced an argument nobody knew they were postponing: two engineers had quietly incompatible ideas about where environment settings should live. Better to settle that in a document than in a production incident.
Their plan for the month matched the standard 30/60/90 roadmap: adopt manual sync on a staging environment, wire up drift alerts, and change nothing in production. Manual sync means the agent prepares and displays every change but a human presses apply — training wheels, deliberately chosen. The alerts went out over their existing chat. For four weeks, the only GitOps artefacts were decisions, one staging environment, and a slowly filling channel of drift notifications that taught them, more than any tutorial, where their undocumented habits lived. The founder’s summary at the month-end standup deserves quoting: “We didn’t buy software. We bought an argument settlement.”
GitOps Days 31–60: Hands Off the Button
Month two is where the roadmap asks for courage, in measured doses. Staging sync went automatic — the button was retired for the test environment while staying mandatory in production. Secrets moved off a shared spreadsheet into a proper encrypted setup. The reconciler’s reach across tenant namespaces was clamped down so one team’s workload could not disturb another’s. And two rehearsals happened that the team still talks about.
The first was break-glass practice: an engineer deliberately suspended reconciliation mid-week, made a manual fix as if an outage demanded it, then brought the recorded state back into agreement before resuming. The exercise took forty minutes and produced a one-page procedure that had not existed that morning. The second was less theatrical but bigger: the CI pipeline’s production credentials were deleted. Not rotated — deleted. Under the pull arrangement the build system no longer needed any route into production, so the credential that had kept the founder awake was simply retired. He reports that sleeping improved measurably.
Not everything in month two worked on the first try, and the stumbles are the useful part of any story. The drift alerts, tuned by an optimist, pinged the chat forty times a day until everyone muted the channel — exactly the failure mode that makes teams abandon monitoring. A quieter threshold, plus a daily digest, fixed it. The tenant lockdown was applied in the wrong order at first, briefly blocking the designer’s preview environment; the contractor wrote an apology in the decision record, which is where apologies belong. And one engineer missed the break-glass rehearsal for a client call, which taught the team that drills need two dates on the calendar, not one.
GitOps at Day 90 and the Week of Truth
Production got its turn gradually: automated sync for low-risk services first, prune guards on anything stateful, a dashboard on the office wall screen, and a first rebuild drill — the whole staging environment torn down and recreated from the record on a timer, to prove the documented state was complete. Reconciler upgrades and key rotation went onto a quarterly calendar, on the principle that an operation practiced quarterly is an operation available during an emergency. The rebuild drill itself finished in under an hour, surfaced three forgotten settings, and became the team’s favourite party trick — the one where they show a visitor the empty environment rebuilding itself while the biriyani arrives.
Then came the week the dashboard told the truth. Across one ordinary week of changes, the median time from an approved merge to a healthy, serving system was about three and three-quarter minutes; the slowest tenth of changes took up to twenty-six, usually waiting on a data migration rather than on the machinery. One deployment attempt in five failed on first apply — and every failure was caught by the pipeline and retried or rolled back without a human losing an evening. The platform spent 1.12 percent of its application-minutes out of agreement with the record, and inside that figure sits a sharper fact: two drift incidents produced 94 percent of the disagreement, both traced to a single hand-edit habit the team then retired. Four of the six drift occurrences detected were auto-reverted without discussion. And of five out-of-band changes someone attempted directly on the live system, four had no ticket at all — which is precisely the accounting hole the record now closes.
Those figures match, almost eerily, the sample metrics published in xOps School’s GitOps field guide, which is where the team’s contractor first studied the roadmap they followed. Teams elsewhere should expect their own numbers to differ; the point of measuring is that yours stop being anecdotes.
What the Team Would Tell You at the Bakery Downstairs
- The decision record earned its keep before any tool did. Half the benefit of the first month was discovering what the team had never agreed on.
- Manual sync is not cowardice; it is calibration. Watching the agent prepare changes for four weeks built exactly the trust the button-presses were later replaced with.
- Deleting the CI credential was the single best security day. Nothing enforces the pull model like removing the alternative.
- The drift channel is a mirror. Two hand-edit habits produced nearly all the divergence. The tool did not fix the habits; it exposed them, which turned out to be better.
- Rehearse the pause. The forty-minute break-glass drill looked silly on the calendar and priceless in the retrospective.
Ninety days after that first meeting above the bakery, the Friday-evening laptop is a spare Chromebook used for expense reports, and deployments happen the way ferry crossings do in the guides on this site — on a published schedule, with the conditions posted beforehand. When the team wants a day off from dashboards, they take one: the locals’ favourite recommendations are all in our piece on Kerala beyond the famous sights — local life, food, nature, and culture, the weekend shortlist stays fresh in our running guide to events, places, and experiences across Kerala, and the odd working holiday abroad is covered by the smarter way to rent and manage vehicles in Japan. The app, meanwhile, books tours while they are away, reconciles itself back to the record every few minutes, and saves the memory problems for humans — who have beaches for that.
