Obvious in Hindsight

I spent a good amount of time building an integration test generator for a product that spanned multiple repositories and services. The problem felt obvious to me because most of the bugs we worried about were not inside a single service. They appeared at the boundaries. A request would pass through several systems, each behaving correctly in isolation, and still fail when everything came together. The tool could generate integration tests, identify failures, suggest fixes, and even raise reviews. I assumed that if it could save teams effort and prevent bugs, people would naturally want to use it. What followed was a long lesson in adoption. I started with developers because they owned the code. They understood the idea, but integration quality was not something they were being measured on, so it rarely moved to the top of their list. I tried QA teams next. They saw the value immediately, but finding bugs later in the cycle was already part of how they demonstrated impact. Preventing t...

I maintain a logbook of things that slow me down. Not major architectural problems. Not company-level strategy. Small annoyances. Repeated code reviews. The same class of bugs appearing again. Manually tracing which build introduced a regression discovered two weeks later. Places where I find myself doing work that feels repetitive, mechanical, or unnecessarily difficult. I want to be lazy. So I write them down. If something repeatedly demands effort, I assume there is probably a better way. What surprised me is how often the solution does not appear immediately. Some problems sit in the notebook for months. Others for years. I remember manually investigating regressions after they were discovered internally weeks later. I remember comparing review styles to understand why some reviewers seemed faster and more effective than others. At the time I did not have a good answer. Then GenAI arrived and suddenly several disconnected notes started connecting. Problems that looked unrelated bec...

A five-year-old trying to open a front door can test anyone's patience. The parent has groceries in one hand, a bag in the other, and has just finished a long drive home from work. The child fumbles with the keys, puts them in the wrong way, drops them, tries again. The fastest solution is obvious. Take the keys. Open the door. Walk in. Yet something important is happening in those extra thirty seconds. I've noticed the same feeling while working with engineers. Sometimes a teammate is struggling through a problem that I know how to solve. I can see the answer. I know the shortcut. I know where the bug is. Part of me wants to take over and finish it. Sometimes I do. Sometimes the situation demands it. Production is broken. A deadline is near. The package needs to get inside the house. Other times, I wait. I let them struggle a little longer. Occasionally I ask another engineer to help, just as a parent might ask an older sibling to open the door. The goal is not the door. The g...

A lot of complexity comes from separating things that are destined to move together. At first, the symptoms look unrelated. Slow responses. Too many meetings. Knowledge gaps. Bugs that span multiple teams. Delays in delivering seemingly simple changes. The instinct is usually to optimize the symptoms individually. Add a process. Create a document. Schedule another sync. But the symptom is rarely the problem. I saw this play out in our organization. Over time, teams formed around the empires we had built. Royalties had a team. Reporting had a team. Payments had a team. Accounting had a team. Financial flows had a team. Every team was doing good work. Every team had smart people. Yet customer value flowed across all of them. Features crossed boundaries. Problems crossed boundaries. Context crossed boundaries. The organization was optimized around ownership of components while the customer experienced a single journey. The result was fragmentation, slower responses, duplicated effort, and...

One of the easiest ways to create future confusion is to put a decision somewhere people are unlikely to look. Security rules hidden inside controllers. Rate limits embedded in random services. Business policies buried in emails. At the time it feels convenient. The context is fresh. Everyone involved remembers why the decision was made. A year later, the context is gone but the decision remains. I saw this during a marketplace migration. One marketplace had very little traffic and the return on migrating it was not worth the effort. The team consciously decided not to migrate it. The decision was made. The problem was where the decision lived. It existed in an email thread. Later another announcement went out describing the migration as complete. Both statements were technically true, but the nuance lived only in people's memories. Months later an issue appeared. We searched documents, emails, and announcements trying to understand what was supposed to happen. The answer was not i...

One code smell I have learned to pay attention to is when an object exposes its state and expects everyone else to decide what that state means. If you find yourself reading a field off an object and immediately writing an if statement, there is a good chance a decision has escaped from where it belongs. The object should expose isAtLimit() , not expose count and trust every caller to implement the rule correctly. Today there may be one caller. Next year there may be ten. The rule changes, one gets updated, three are forgotten, and the bugs begin. I learned this the hard way in a pricing system. There was a service responsible for calculating the price of an item. At some point, a marketplace-specific plugin was introduced that read the computed values and tweaked them outside the original pricing logic. It worked perfectly at the time. Then the business evolved. Pricing rules changed. Vendor-specific exceptions were added. Two years later a vendor reported incorrect pricing and nobo...

For a long time, I treated monitoring as something that happened after the feature was built. Ship first. Add dashboards later. Add alerts when needed. Experience taught me otherwise. You cannot add meaningful observability to code that was not designed to be observable. If you do not decide upfront what success looks like, what failure looks like, and how each will be measured, you end up discovering failure modes through customer complaints, production incidents, and late-night investigations. Observability is not a deployment task. It is a design decision. One question changed how I think about systems: If this fails silently at 2am and I'm asleep, what wakes someone up?  If the answer is a customer ticket, a support escalation, or a confused user, the design is incomplete. The alarm should ship with the feature. The metric should exist before the endpoint. In one federated financial workflow, this became critical. Multiple teams owned different parts of the flow. Any change ma...

For a long time, I believed that a good solution would eventually win. If the architecture was cleaner, the process was better, or the quality improvement was obvious, people would naturally adopt it. Experience taught me otherwise. Most ideas do not fail because they are wrong. They fail because they answer the wrong question. Before people ask, “Is this a good solution?”, they ask something much simpler: “What is in it for me?” Until that question is answered, adoption rarely happens. A thirsty person will drink water, no matter how raw the container. Someone who is not thirsty will walk past the finest glass. Organizations work the same way. A software development manager who is measured on bug reduction and quality will care about integration tests. The same person may completely ignore the exact same proposal if it is presented as engineering purity or long-term goodness. Not because they are irrational, but because incentives focus attention. People do not adopt solutions for th...