The rise of Functional Programming

Posted on: Jun 07, 2012 by Thierry Caminel

For many of us, Functional Programming (FP) is related to some boring computer science courses about lambda calculus and strange programming language like Lisp, Haskell or OCaml, used only by academic people and not by real programmers, who obviously use C or Java.

But FP is one of the hidden key concepts behind the "big data" revolution.  It probably became mainstream when Google described, in their 2004 seminal paper[1], MapReduce as an “abstraction inspired by the map and reduce primitives present in Lisp and many other functional languages”. Now Twitter, LinkedIn, Akamai, TomTom, Erickson and others use modern functional languages such as Clojure (a Lisp dialect), Scala (a combination of Java and FP) or Erlang (a distant descendant of Prolog).

The main reason is that FP facilitates distributed programming. In FP, functions, not objects or procedures, are the fundamental building blocks of programs:  whereas object programming put emphasis on object composition, FP put it on composition of functions.  Functions can be combined, sent remotely, applied locally on distributed data sets, and that is the main paradigm shift of "big data". By eliminating side-effects and mutable variables and facilitating code distribution over several CPU, FP eases concurrent programming, a topic more and more important as the number of CPU core increases. FP makes also easier the development of internal Domain Specific Languages (DSL), these little languages embedded into applications simplifying the maintenance of business logic and improving programmer’s productivity.

Obviously FP will not replace imperative programming, first because few developers have to deal with scalability concerns or DSL development, and because tooling will tend to hide complexity of concurrent and distributed programming, through libraries or language evolution (for example, Java 8 will introduce lambda expressions). FP principles are also behind some widespread languages like SQL or XSLT, so most programmers already do some kind of FP but haven’t realized it. JavaScript also allows FP, and interestingly recent and popular libraries emphases this aspect for example to ease HTML DOM tree manipulation (jquery [2]).

Anyway it might be a good idea that software developers have explicit knowledge of principles of FP  in order to write better code, such as avoiding side effects and instances variables, using immutable data structures, combining functions to be applied to objects.  These skills might become more and more important in the future as FP patterns will spread.

So if during your next job interview the candidate appears to know what a monad is [2], you might consider hiring him with confidence for your next-generation framework.

[1] J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI’04, 6th Symposium on Operating Systems Design and Implementation, Sponsored by USENIX, in cooperation with ACM SIGOPS, pages 137–150, 2004. [2]

Share this blog article

About Thierry Caminel

Senior technical architect and business development manager and member of the Scientific Community
Thierry Caminel is senior technical architect and business development manager at Atos, and member of the Scientific Community. His main focus areas are in distributed systems, context aware computing, Cloud and event driven architecture. Before joining Atos he worked in several startups innovating in the field of Artificial Intelligence, Web 2.0 and Machine to Machine, and lead embedded systems development for space experiments. Thierry holds a software engineering degree, and lives in Toulouse.

Follow or contact Thierry