I’ve been following along with the TDD is Dead series, and it has made me appreciate how far along I’ve come these past few years. If you haven’t tuned in already, it’s a very good listen full of insights at many different levels. It’s a battle between ideals and convention. On on side, you have Kent Beck and Martin Fowler: pioneers of TDD and unit testing. On the other, you have DHH, the creator of the Ruby on Rails framework. Each side has a wealth of experience, but look at programming from nearly polar opposite viewpoints.

The main argument DHH (and many others) have against testing is the insane lengths you have to go to isolate code to allow testing. In some cases, you are creating abstractions and layers for the sake of testing. In this sense, the code you are writing is actually worse off without testing.

I want to ignore all of the benefits of testing for a moment here, and focus on design, because I think that’s what the discussion really needs to be about. The underlying quesiton is: can you have well-tested code and well-designed code that concisely conveys its intent.

Of course you can, but it’s not easy. There is a natural progression of hitting extremes that is part of the learning process. With more experience, you find a good balance. What I see happen, and what turns people off of TDD, is hitting that extreme and stopping. This misses the refactor step of the TDD cycle, and with more experience, the refactorings are more valuable.

Wrapping things in artificial layers and introducing more intermediate results may make testing easier, but it doesn’t necessarily make your application any better. More often than not, you can’t fix your design issues without fixing things at a higher level. And even more dangerously, if you already think your design is perfect, you have no chance.

As you write new code, you can’t be afraid of improving the foundation it is built upon. When you are stuck, sometimes taking only a few steps back isn’t enough to find your way. You need a fresh mind and willingness to introduce real change. You need to change code at a higher level and improve how things communicate. Continuing to pass the same messages through a new system might not be enough. You need to improve the message as well. And to me, this is where most people fall short when trying to refactor their application.

If the message remains the same between each layer of your application, the boundaries aren’t adding any value. They may help with code isolation, but they are just artificial layers. Without boundaries, you are not gaining trust, and your layers will have cracks.

Software developers are hired to solve problems with technology. When the solution is known, we are implementing that solution with technology. The choice of technology is often more apparent, and the job is more comparable to following a manual. When the solution is unknown, things can become a little more challenging. Our emphasis should shift from technical issues to the expression of our intent.

Many programming languages, frameworks, databases and other tools, limit the way we can express our intent. We tend to unknowingly favor abstractions, meaningless components, and tools that meet technical problems we aren’t even facing yet. We focus on how data is stored, transmitted, and the components that allow us to do it quickly. We favor lossy technology that does not allow us to fully express the problem we are solving. Our intent is lost and becomes difficult to express.

With an emphasis on expression, development can become an iterative exploration of the problem domain. Language, boundaries, objects, actors, processes, transformations and workflows emerge as we try to crystallize our view, and our representation of it.

It’s important that you do not let your technology choice dilute your intent. Different programming paradigms will lend to different problem domains. Can you represent the journey from a request to a response through a series of clear transformations? Does each state between those transformations have a name, and clear properties? Can you model the messages between components and the boundaries they cross as objects/namespaces? How about as a long procedural script? Can you stuff a series of interactions into create/read/update/destroy commands? How lossy is it – and is that acceptable?

One added benefit of working this way is low amount of cognitive load it requires. There is always some internal translation or mapping of what you’re trying to express, and the code that makes it happen. The less loss that happens here, the easier it is to understand, and the easier it is to change.

If you subscribe to the idea of building your domain layer first, maintaining an effort to express your domain as articulately and losslessly as possible is very powerful. It becomes a natural process of exploration, and separates the programming from the problem solving. Each specific problem you need to solve becomes very isolated, and has a clear connection back to the overarching goal.

As you work your way outward to user interfaces, or various supporting services, define the most expressive and ideal interaction first. If you need to compromise later for technical reasons, that’s fine. Contrast with that the opposite: if what you built suffered major losses early on, it’s much harder to understand and enhance later.

Focus on the lossless expression of your intent. Good design will follow.

I’ve been told that writing tests ensures your software works properly. Unfortunately, no amount of tests will 100% tell you that your software is bug free. It can’t. Testing is all about adding confidence. Confidence that your efforts are going towards the right goal.

We tackle logical complexity with unit tests. Tests for every code branch, and ensuring our outputs match our expected outputs for every set of inputs. We test for catchable failures. These tests are lightning fast: they simply build and manipulate data structures to meet some assertion. At this level, we are making sure that this tiny unit does its job exactly as its been programmed to. The job you told it to do. Now, that job may be wrong in the first place, but even so, it’s successfully achieving your failure.

Now, that unit may not even be compatible with the rest of the system. So we write integration tests. This ensures that units are compatible with each other, and can successfully be wired together to perform tasks. These tests are simple: connect A and B, and hopefully they yield C. They are a little slower, because they require some form of a running system. Luckly, less of these are needed.

Even with perfect units doing what their told in perfect harmony, you may not be solving business problems yet. Working code is nice (or at least, slightly better than non-working code), but if it’s not serving its purpose, is it anything more than dead weight?

Confidence gives us power. The power not to worry, the power to move onto the next task. In other words: feedback. Letting us know when we are done, and when we still have plenty of work to do. Testing creates a feedback loop as we work to tell us if what we built actually works. TDD gives us a very tight feedback loop: failing test, write code, passing test, etc.

Why are so many developers leaving the most critical test until the end? The acceptance tests. Does this code actually solve the problems it was created to? If it doesn’t, how much time was wasted writing code before this was realized? That is the feedback loop you need to tighten.

It’s easy to work in a silo, and write beautiful code that is “bug free”, fully covered, and runs lightning fast. It’s easy to merge that into an application and wait for bug reports inevitably come in. You may think you are performing because you’re delivery high quality code and getting your job done. And you may be. Or maybe not. You won’t know unless you actually test against your requirements.

One way you can ensure you are on the right track is creating two nested feedback loops. The outer loop, testing against business requirements. You can formalize this with tools like Behat or Cucumber. As an example, these may drive use cases and check the results. It will depend on your domain. And that domain will depend on which layer you are working in.

The problem domain of your web layer is HTTP. Your acceptance test suite should, as expressively as possible, assert that various HTTP request messages are responded to with the corresponding HTTP response messages. That’s it.

Your business domain layer should, as expressively as possible, assert that your business objects can interact under the correct conditions, with the correct outcomes. With an expressive and accurate ubiquitous language, you can ensure those concepts drill all the way down to the code, rather than just at the periodic status updates.

While you are implementing these acceptance tests, you’ll write units of code that integrate with the rest of your system. And you’ll write tests to ensure those things do those jobs. When you are all done, your business tests should pass. And you know you’ll be done.

With these tests driving your progress, you can be confident that your solution is actually solving the business problems it was meant to. When your code leads you astray, you’ll quickly realize that they aren’t helping you achieve your goal, and you’ll find your way again.