Language of the year: 2010
Testing Client Side Javascript with Jasmine and Node.js

A Bit of Heresy: Functional Languages are Overrated

Lets talk about problems. As professional software developers, we get paid to solve problems. Sometimes we do this by writing code. Sometimes we do this by finding creative ways to avoid writing code. In either case, it's important that the problem drives the solution, and not the other way around.

Enter: The Concurrency Problem. Moore's Law is starting to run out. We've all gotten very used to using hardware to solve scalability problems, and up until now, that's been a relatively easy thing to do. But chip manufacturers are starting to hit physical limits of transistor size and density. Now, the only way to scale up is to scale out by adding more cores, and that means concurrency. "But concurrent programming is hard!", they say. "And we have an army of semi-conscious Java developers in our company that wouldn't know a semaphore from a telegraph line! Oh noes! What will we do?"

The current fashionable solution to this problem are functional programming languages. They are the next big thing. Languages such as Scala, Clojure, Erlang, and Haskell are supposedly going to save us from the dread intermittent bugs and horrible deadlocks intrinsic in concurrent systems built with object oriented or procedural languages. The implication in this argument is that functional programming languages are more thread safe because (depending on the language) they either shy away from side effects or find ways to eliminate it completely.

Well, I'm sorry, but I'm just not buying it.

Don't get me wrong. Concurrent programming in a typical object oriented language is hard. Really hard. I've slogged through enough of it to learn to just avoid it unless you really, really need it. Especially when you're dealing with shared state. Michael Feathers once told me that he thought if threads had never been invented, and someone had proposed them at a conference today, they'd be laughed out of the building. He may have been kidding, but honestly, I would believe it.

My problem with the functional programming cure to this problem is that I haven't seen a cure yet that's better than the problem. Maybe it's me. Maybe my tiny Ben brain just can't comprehend the magic and majesty of OCaml, but every time I've tried to learn a new functional language, it's ended it disaster. First with Erlang, and then Haskell, I've now had two aborted attempts to understand the supposed cure to the concurrency problem. The languages are just too complex, too terse, and absolutely full of academic shitheaddery. I mean, seriously, what the hell.

If your concern with the death of Moore's law is that you won't be able to afford/find skilled programmers to scale your systems using multicore processors, these are not the droids you're looking for.

But even if you're got a bunch of really sharp developers on your team, I'm not sure this is the solution either. Message based architectures (especially asynchronous ones) are good, pretty easy to understand, and easy to test. Shared state in concurrent systems is a pain, and you shouldn't do that. Interprocess communication is relatively easy through various methods. Whether you're using an object oriented, pure functional, or hybrid language, these things are true.  And while it might be possible to ignore thread safety problems by just working in a purely functional language, I think you're just trading one kind of complexity for another. And if you're working in a hybrid language, what assurances do you have, really, that your system is really thread safe? Indeed, the risk compentation effect might give you a false sense of security, giving you a system that actually had more concurrency problems that one written in a "dangerous" language.

I think the downfall of some of these languages is that they get wrapped up in being "functional" rather than focusing on solving a particular class of problem. They abandon convention and common sense for complex type systems and functional purity. You can have an actor model architecture without using a functional programming language. You can avoid shared state in any modern language (even JavaScript). These things are not synonymous with functional programming.

I think the future of computing is people. Lots of people. Web applications, social networks, and millions (billions?) of mobile devices interacting over the net. I think the majority of these problems can be solved with multi-process, message passing architectures. In the end, a large scale system might wind up with one core for every 10 users. Maybe less. If multiple cores are the new gigahertz, I would not be the least surprised if computing gets that cheap. All it would have to do is follow the trend of the last 40 years, just along a different axis. Really, the only barrier to a process-oriented approach in that case would be memory, and that may not actually be a limitation at all.

So I think the cases where a functional programming language are really the best solution to the concurrency problem are, in fact, rather rare. It's worth noting that even after Twitter raised eyebrows by moving their backend processing from Ruby to Scala, they "reverted" to Java for some of the concurrent operations:

initially we had a number of actors inside the system...over time as we ran more and more system tests on it, we found that actors weren’t necessarily the ideal concurrency model for all parts of that system. Some parts of the concurrency model of that system are still actor based....but for other parts we’ve just gone back to a traditional Java threading model. The engineer working on that, John Kalucki, just found it was a little bit easier to test, a bit more predictable. 

So I need someone to explain this to me. The state of the art in functional languages scare me a lot more than intelligently-designed concurrent programming, but everyone is hopping on the bandwagon, so I feel I must be doing something wrong. So, in typical fashion, I've made a heretical claim in the hopes that someone will show me the light and prove me wrong. Because I seriously don't get it.

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Colin Jones

"Try Clojure!" says the peanut gallery ;) But seriously, if you have a few hours, Rich Hickey has some lectures on at http://clojure.blip.tv that present a much better argument for Clojure than I could. Baked-in immutability for the win! That said, there's certainly a lot of working concurrent code in Java - I just wouldn't want to be responsible for maintaining much of it (see Java Concurrency in Practice for my fears...)

/b

another one from the peanut gallery here. you're right, ocaml and haskell and others ARE full of academic shitbaggery. clojure isn't.

.

"Maybe it's me."

Yep.

Paint Can

Your 'seriously' and 'what the hell' links are at worst information dense definitions of core language mechanisms. The ocaml one isn't even difficult to read. Maybe you just need to be a bit more patient and take time to read thoroughly.

Here's a book I like:

http://www.amazon.com/Purely-Functional-Structures-Chris-Okasaki/dp/0521663504

I've found that reading this has helped me learn & appreciate the functional approach to programming more. If you're like me and you have enjoyed reading Sedgewick or other algorithms text, it might be good for you as well.

dman

I would have loved this if the blog post contained more technical details and some code snippets about the tradeoffs involved. Functional languages cover a broad spectrum. There are often multiple concepts being explored in a language, then there is the issue of cultural heritage of the respective communities, and the fact that FP languages have traditionally had multiple implementations with different tradeoffs. as an example -> lisp and scheme are functional without being immutable. scheme favors a small spec while lisp includes large number of inbuilt primitives. Clojure introduces persistent datastructures and interesting new concepts like agents and integration with the jvm. I know too little about haskell and ML to make any informed comments about them but they appear espouse static typing unlike lisp / scheme. So if you have a particular beef about a functional language, look further and you will find one that doesnt share those traits.

Andy Kish

The strong type system is the best part of Haskell! The type system is really a happy medium where it's expressive enough (if your program compiles that actually means something) and is not burdensome for the programmer (like certain dependently typed languages). The types are also expressive enough that you can often figure out what's going on just from a function's type alone.

Functional languages may not be your cup of tea, but just because you don't understand them does not mean that they are overrated.

N

The problem aren't the languages, the problem is people that give up as soon as the language does look like java.

Loup Vaillant

First, learn what is a tagged union (more often called algebraic data type). learn where it can be useful (Option types compared to Null pointers, for instance). Okasaki's book makes a heavy use of tagged unions. You should try and read the first chapters. There is also his thesis, which is very like the book, available on line: http://www.cs.cmu.edu/~rwh/theses/okasaki.pdf

Second, don't be scared. Functional languages tend to be freakishly terse, but the programs written in them are correspondingly short. It may take time to read a line, but reading the whole program is easier.

Third, don't be offended. There are perfectly rational reasons for changing the syntax of those languages, or having a complex type system. For instance function application is written as "f x" instead of "f(x)" because it enables currying, and is most lightweight (function application is the most heavily used language construct, after all). Most of the time, the complexities you see are actually simple things, but unfamiliar.

Finally, think differently. When using a functional language, you don't want to instruct your computer what to *do*. Rather, you want to tell it about what, *is*. Stop to see your code as a string of instructions. Start to see it as a tree of descriptions.

M

Learning a functional language like Haskell for the first time 'might' give you the worst headache in your entire life. Many people who know Haskell admit that it was after a few months (or even years) of jumping back and forth that they actually "got the language." If you quit Haskell after trying to learn it the first time, it's ok. I did too. However, now I'm finding its complex features powerful, expressive and extremely logical.

Haskell can give grief to people programming in imperative languages for years. So, it's no surprise that newcomers to programming who have just understood imperative languages will need strong painkillers. That's why, don't blindly believe every "I don't under Haskell (or any other functional language)" post or comment. Just put in more time, it'll pay off.

Blame

Honestly, yet another comment about how functional programming languages are hype from somebody who never managed to learn one.

Of the two links you provided to support your "academic shitheaddery" claim (whatever that means), one is to the formal language definition for Haskell and the other is a bog standard explanation of common language features.

If you don't want to read the formal semantics of a language, then don't read it. Having this sort of precise definition is required for inventing new useful features. It's not supposed to be a tutorial.

The comments to this entry are closed.