Radyology

Continous Testing: with Ruby, Rails, and JavaScript -- Now in Print!

It's done! It's out. And I couldn't be happier.

As part of writing writing Continous Testing: with Ruby, Rails, and JavaScript, I had the opportunity to talk to a lot of people about what Rod and I were doing, and what they thought of it. Most of the people, once introduced to the idea, thought it was so obviously good that there just wasn't any reason not to follow this practice. By running your tests on every change, you create a tight feedback loop that makes it so much easier to find bugs and get a better understanding of what your code is doing. I'm really proud of what we've done with this book because I think the techniques we discuss in it will really open this practice up to a wide range of developers and projects.

Avoiding Asynchronous Programming in Node.JS

Node.js is supposed to be all about an asynchronous style of programming. Rather than executing functions and getting results, we use a lot of first order functions and callbacks to do things asynchronously. Here's an example of this style from the djangode project. makeApp is a function that takes an array of arrays that are regex/function pairs. Some of those function invoke other functions (such as setTimeout) that take other functions.

I find this kind of code rather tricky to test, and to reason about. Functions nested four levels deep makes my spidey-sense tingle. A good technique for cleaning it up can be extracting those anonymous functions into named functions, but if there are variables that are shared across scopes, that can get a little messy too.

However, I really like Node.js and the approach they've taken, simply because the asynchronous style is a rather elegant (if not new) solution to the kind of performance problems you encounter when working with single threaded code. Blocking operations, IO is particular, can really kill your app, and using threads to avoid blocking can lead to code that is nigh-impossible to test or reason about...so the Node.js asynchronous style merely being a bit tricky is a huge upgrade.

However, I haven't found myself really digging using that style all the time. I generally prefer a functional style when working in JavaScipt. In my opinion, function composition is a huge advantage in JavaScript, but callbacks generally don't have return values, so they're harder to compose. In fact, it really only seems appropriate to use callbacks when I would otherwise be tempted to use a thread to avoid onerous blocking IO. Otherwise, I like to stick a more functional style and stay away from the asynchronous code.

Dictionary of Algorithms and Data Structures

I gotta say, this is pretty neat.

You know, just in case you forget what Huffman encoding is.

What is Twitter?

Twitter is a personal denial of service attack. If you want to steal someone's productivity for a day, just troll them on Twitter.

Oh, and I'm not posting this on Twitter, because I have better things to do than defend my argument.

Is Unwarranted Overtime Theft?

After reading Tim Ottinger's excellent article on the meaning of "sustainable pace", I couldn't help but pose a question on Twitter. 140 characters just isn't enough to clarify this question, and so we move to the blog to fill in the details.

Let's assume that we have a customer (likely a proxy) who wants a development team to work overtime to complete a task. Let's further assume that this is really a task, that is, a day or two of work, to be done on a weekend or during a couple of really late nights. He thinks that this task requires an additional level of effort, and that working late is worth the sacrifice (to both morale and the later productivity of the team). To clarify, the customer isn't providing the budget for this project and he isn't any more of a stakeholder than the developers.

Let's also assume that we have a development team that has a decent understanding of the business concerns of the project. Their opinion is that this task is no more urgent or particularly valuable than anything else they've been asked to do. To use the terms from Tim's article, they don't think it's necessary to come to the rescue in this case.

My question is this: If the customer proxy forces strongly suggests that the development team work overtime to complete the task, even though the developers think overtime is unwarranted for this task, is that unethical?

On the one hand, the customer is likely benefiting from the delivery of additional features, and the morale of the team may be damaged. If this is 'the last straw' for a key developer that can easily find work elsewhere, the customer may have cost the company dearly to benefit himself. Even if the customer truly believes that the sacrifice is worth the benefit, is he caught in a conflict of interest?

On the other hand, what if the customer is right? A good development team understand the needs of the business, but you would think that the customer would have a better understanding. Disagreements happen. Our goal is always better communication and understanding, but there's little point in having an 8 hour meeting to avoid 12 hours of overtime.

What say you, Internets?

Continuous Testing with Ruby: Now in Beta!

"Continuous Testing with Ruby" is now available as a beta book on pragprog.com! This is the first beta, so there's still more to do, but Rod and I are very happy to have it out there so we can get some feedback.

Continuous Testing starts with a simple idea: Run your tests all the time. I think this book does a great job of communicating all the incremental improvements to that idea that Rod and I have come up with over the last few years. From using CT as an alternative to debugging, to finding ways to automatically check other project artifacts (like documentation and seed data), once you get hooked on having instant feedback for everything, you start to find all kinds of ways to apply it.

Probably the biggest effect, however, is how CT helps you write better tests. One of the points we made in the book goes like this:

An old axiom of Test Driven Development states that the tests test the correctness of the code; while the code, in turn, test the correctness of the tests. The tests also test the design of the code—code that’s hard to test is usually poorly designed. But what tests the design of the tests? In our experience, Continuous Testing is an effective way to test the design and overall quality of our tests...running our tests all the time creates a feedback loop that tells us when tests are misbehaving, as we create them. This means we can correct existing problems faster, and prevent bad tests from creeping into our system in the first place.

I'm very proud of what we've done with this book. I think it will be really valuable to anyone who's been writing automated tests and wants to get more out of them. I've been a doing TDD (and then BDD) since about 2003. My interest in continuous testing began in 2006 when I thought about all the time I was wasting figuring out which tests I needed to run after a change. After that, I wrote Infinitest, and now Continuous Testing with Ruby. I'm greatly looking forward to see where we go from here.

Testing Client Side Javascript with Jasmine and Node.js

After having spent a long time trying to run javascript tests with Rhino, I've finally found something better. Rhino was never really fast enough for me. Spinning up the JVM just takes too long when you're running your tests continuously. A few seconds really makes a difference when you do something hundreds (thousands?) of times per day.

On top of that, testing frameworks like ScrewUnit (aside from being abandonware) have scoping issues that make tests very brittle...unless you run each one in it's own process...again back to the JVM startup problem.

Recently, I started playing around with the Jasmine testing framework for Javascript. It was really nice, but I wasn't convinced it was really going to solve the problems I was experiencing since they were mostly due to global namespaces and Rhino.

Enlighentment came, however, when I tried running Jasmine tests from within Node.js, using a patched version of mhevery's jasmine-node module. If you don't know, Node.js is a Javascript runtime environment based on the V8 engine. It's actually designed for server-side development, but I wondered if I could use it to run tests for my in-browser javascript. The answer is mostly, but the results were very compelling and I'm now doing this on all my web application projects.

Firstly, here's an example project that shows how I set things up.

To understand what's going on here, lets begin at the html. In index.html, we're not doing anything fancy. Just a few tags to load jquery and the javascripts in our app:

The interesting bits start to show up in spec/specHelper.js

specHelper.js is actually a proper node.js module, meaning that it explicitly declares it's exports. What it's doing, actually, is exporting the namespace for our entire application (as the variable 'app'). This means that in our specs:

We can create a completely clean copy of the entire application and store it in a variable that the tests can access. Jasmine's beforeEach function can recreate it before every single test. This means that we don't have to create a new Rhino instance for each test to get a clean environment. As a result, these tests run fast (hundreds of tests per second).

They run so fast, in fact, that I've been using Watchr to run them. Here's my Watchr script

This automatically runs all the Jasmine specs on each change. The tests run so quickly I haven't had the need to optimize it yet. I've been really happy with this setup (despite a few problems with jquery and env.js) and I'm hoping to improve it as the tools mature.

A Bit of Heresy: Functional Languages are Overrated

Lets talk about problems. As professional software developers, we get paid to solve problems. Sometimes we do this by writing code. Sometimes we do this by finding creative ways to avoid writing code. In either case, it's important that the problem drives the solution, and not the other way around.

Enter: The Concurrency Problem. Moore's Law is starting to run out. We've all gotten very used to using hardware to solve scalability problems, and up until now, that's been a relatively easy thing to do. But chip manufacturers are starting to hit physical limits of transistor size and density. Now, the only way to scale up is to scale out by adding more cores, and that means concurrency. "But concurrent programming is hard!", they say. "And we have an army of semi-conscious Java developers in our company that wouldn't know a semaphore from a telegraph line! Oh noes! What will we do?"

The current fashionable solution to this problem are functional programming languages. They are the next big thing. Languages such as Scala, Clojure, Erlang, and Haskell are supposedly going to save us from the dread intermittent bugs and horrible deadlocks intrinsic in concurrent systems built with object oriented or procedural languages. The implication in this argument is that functional programming languages are more thread safe because (depending on the language) they either shy away from side effects or find ways to eliminate it completely.

Well, I'm sorry, but I'm just not buying it.

Don't get me wrong. Concurrent programming in a typical object oriented language is hard. Really hard. I've slogged through enough of it to learn to just avoid it unless you really, really need it. Especially when you're dealing with shared state. Michael Feathers once told me that he thought if threads had never been invented, and someone had proposed them at a conference today, they'd be laughed out of the building. He may have been kidding, but honestly, I would believe it.

My problem with the functional programming cure to this problem is that I haven't seen a cure yet that's better than the problem. Maybe it's me. Maybe my tiny Ben brain just can't comprehend the magic and majesty of OCaml, but every time I've tried to learn a new functional language, it's ended it disaster. First with Erlang, and then Haskell, I've now had two aborted attempts to understand the supposed cure to the concurrency problem. The languages are just too complex, too terse, and absolutely full of academic shitheaddery. I mean, seriously, what the hell.

If your concern with the death of Moore's law is that you won't be able to afford/find skilled programmers to scale your systems using multicore processors, these are not the droids you're looking for.

But even if you're got a bunch of really sharp developers on your team, I'm not sure this is the solution either. Message based architectures (especially asynchronous ones) are good, pretty easy to understand, and easy to test. Shared state in concurrent systems is a pain, and you shouldn't do that. Interprocess communication is relatively easy through various methods. Whether you're using an object oriented, pure functional, or hybrid language, these things are true.  And while it might be possible to ignore thread safety problems by just working in a purely functional language, I think you're just trading one kind of complexity for another. And if you're working in a hybrid language, what assurances do you have, really, that your system is really thread safe? Indeed, the risk compentation effect might give you a false sense of security, giving you a system that actually had more concurrency problems that one written in a "dangerous" language.

I think the downfall of some of these languages is that they get wrapped up in being "functional" rather than focusing on solving a particular class of problem. They abandon convention and common sense for complex type systems and functional purity. You can have an actor model architecture without using a functional programming language. You can avoid shared state in any modern language (even JavaScript). These things are not synonymous with functional programming.

I think the future of computing is people. Lots of people. Web applications, social networks, and millions (billions?) of mobile devices interacting over the net. I think the majority of these problems can be solved with multi-process, message passing architectures. In the end, a large scale system might wind up with one core for every 10 users. Maybe less. If multiple cores are the new gigahertz, I would not be the least surprised if computing gets that cheap. All it would have to do is follow the trend of the last 40 years, just along a different axis. Really, the only barrier to a process-oriented approach in that case would be memory, and that may not actually be a limitation at all.

So I think the cases where a functional programming language are really the best solution to the concurrency problem are, in fact, rather rare. It's worth noting that even after Twitter raised eyebrows by moving their backend processing from Ruby to Scala, they "reverted" to Java for some of the concurrent operations:

initially we had a number of actors inside the system...over time as we ran more and more system tests on it, we found that actors weren’t necessarily the ideal concurrency model for all parts of that system. Some parts of the concurrency model of that system are still actor based....but for other parts we’ve just gone back to a traditional Java threading model. The engineer working on that, John Kalucki, just found it was a little bit easier to test, a bit more predictable. 

So I need someone to explain this to me. The state of the art in functional languages scare me a lot more than intelligently-designed concurrent programming, but everyone is hopping on the bandwagon, so I feel I must be doing something wrong. So, in typical fashion, I've made a heretical claim in the hopes that someone will show me the light and prove me wrong. Because I seriously don't get it.

Language of the year: 2010

I learn a new language every year. This year, I had decided to learn a functional language. The contestants were many and varied, but I'm pleased to announce that the winner this year is...

Contract Tests in JUnit 4

As part of his talk on integration tests J.B. Rainsberger talked about how contract tests can be used to test the interaction between classes when using a mockist approach to developer testing. He wondered aloud if it would be possible to write these kinds of tests using abstract classes and JUnit 4. The answer is yes, with some caveats, as I demonstrate in the screencast below. Enjoy!

EDIT: If the embedded screencast below doesn't work for you, try this link.