Hard Problems in Contemporary Software Design


Silicon Valley spends most of its energy on what I consider the “easy problems” in software design. How do we get more people to try our app? How do we make people come back more often? How do we get them to stay longer each time?

These problems are easy because they involve relatively few, well-understood parts. There are only a handful of ways to reach users out in the world and engage them while they are using our apps.

To be fair, it still takes a ton of effort to work on these problems because everyone has to compete over the same audience. It’s actually getting harder for teams to differentiate on product strategy now, precisely because these problems have been solved over and over again.

I also don’t have a fundamental issue with people continuing to play the zero-sum game over people’s attention. It’s the same game media industries always play, and after all, software is a form of media. It’s perfectly rational for my colleagues to “get the money” as long as investors and advertisers are willing to reward them for continuing to push on these same old problems.

I just don’t find the easy problems interesting anymore. This game has been fully played out. Plus, “winning” along these lines doesn’t benefit society. It mainly enriches us here in Silicon Valley, which I guess would be a good enough outcome if we were here just to make money and enjoy the California lifestyle, but that’s not why I came to work in software in the first place. I personally don’t believe you should be considered successful as a product person in any field unless you’ve been able to make meaningful contributions to the lives of humans on the other end.

So, these days, I’m thinking a lot about what I consider the “hard problems” in software design. These complex problems arise from a convergence of technological, economic, and cultural forces that create structural barriers to building good products that actually improve people’s lives. These problems are hard because there are many interacting pieces, none of which can be easily removed because it serves some strong need.

For example, let’s say I want to create a new app that’s all about positive vibes, and I want to respect people’s personal space by not sending any unnecessary emails or notifications. It just feels wrong to spam our users.

But this app has to live side by side with other apps that constantly spam users. How would people even remember that my app exists? They are busy checking how many people searched them on LinkedIn! It doesn’t matter that other apps shouldn’t be abusing communication channels that are intended for important messages. The hard reality is, as long as sending spam improves usage metrics, spam will be sent. And why don’t users simply opt out? The truth is, they can’t. Most apps add social features like messaging just so that users have no choice but to enable notifications.

That’s them though, and I’m me, so let’s say I stick to my morals. I can just make sure that my app delivers so much value that it gains mind share even without spamming users. But I live in California, and it costs millions every year to run even a tiny startup here. So I have to go out and raise more money to continue growing my app. And now, investors are asking to see some usage metrics so they can compare them against other startups that are definitely spamming at least as much as LinkedIn.

Even a simple moral decision to not spam users involves solving a hard design problem with many constraints. Just to decide whether you can afford to be a good platform citizen, you have to navigate a maze of considerations -— marketing, onboarding, core functionalities, platform dynamics, business model, funding environment, and even the local cost of living.

Hard problems are always like this. What we want to achieve seems obvious and simply the right thing to do. We want to make a useful product, and charge a fair price for it. We want to empower people to do more, and focus on what’s really important in life. We want our products to have a positive impact on people’s overall quality of life. These are all things everybody says they want to do. But when we set out to actually make these things happen, at every step, we run into traps set by an evil genius who is a grandmaster at making you sell out.

But at the end of the day, hard problems are just design problems — they should be solvable with some careful analysis. And it’s the high degree of complexity that makes hard problems that much more intellectually stimulating and rewarding to solve.

My plan for the next stage of my career is to try to design solutions to a few hard problems that prevent software from serving us better in our daily lives. The solutions will be in the form of conceptual design patterns and software that implements these patterns. I hope you’ll find them useful.