Showing posts with label Functional Languages. Show all posts
Showing posts with label Functional Languages. Show all posts

Tuesday, 30 April 2013

Lexical Closures

Today, I found the first confusing bug in the software I've been working on. It had me going for pretty much the whole day, not helped by the fact that my MonoDevelop also had a bug causing it to crash when the IKVM runtime threw a meaningless exception, meaning I couldn't actually get to see any of what was happening.

I've been using a fairly out of date MonoDevelop, as it's the one available from the ubuntu repos. Strangely enough, recent builds on the website are only available for Windows, OSX and Suse.
Luckily, there doesn't seem to be any proper reason for this, so I basically did what this page told me to and built MD 3 from source. So far, it works a treat and doesn't seem to feature the annoying bug I mentioned. Nice :)

Anyway, the bug. It seems really stupid now, so in order to feel less stupid I'm going to turn this into a post where I explain lexical closures in the context of C#.

So, what is a lexical closure? Why, it's a function object which captures the bindings of free variables (variables which aren't defined in the function or it's parameter list).
I'll elaborate:

Delegates

If you've written C# before, you've probably used function objects or delegates, as they call them:

// Declare the delegate type outside a function somewhere:
delegate int OneAdder(int a);

// Instantiate the delegate
OneAdder addOne = delegate(int a) { return a + 1; };

// Use the delegate to print a "4"
Console.WriteLine(addOne(3));


You can also use the lambda syntax to more succinctly create delegates:

OneAdder addOne = (a) => a + 1;

There's more; From .NET 3.5 there are some standard generic delegate types already predeclared:

Action<int> : void-returning function with an int parameter
Action<int, int> : void-returning function with two int parameters
Func<int, string> : string-returning function with an int parameter
Func<int, int, string> : string-returning function with two int parameters

As you might intuit, delegates are reference types just like classes and amenable to the same assignment/passing-around semantics. For instance, one could write a function:

static int CallMyDelegate(Func<int, int> func)
{
 return func(5);
}

Lexical Closure

Delegates in C# have 'Lexical closure'. That means that when you call this, passing it 5:

static Func<int, int> MakeAddingFunction(int number)
{
 return delegate(int a) { return a + number; };
}


What you get back is a function which adds 5 to things.
It's called a lexical closure, because the function 'closes over' the 'number' variable (which is in its lexical scope), so it can use it later. One can also say the delegate 'captures the lexical environment'.
Think on that for a while until you understand it. Try it out, even.

This whole delegate thing is really useful as it gives us a nice, succinct way to abstract some operation in terms of it's inputs and outputs, so it can be passed to code which can then apply it to some data without understanding the operation. C#'s "Linq" is based on this kind of thing. If you haven't heard of Linq go and read up on it once you've finished reading this.

Peculiarities

Lexical closure is about capturing the *bindings* of variables in scope, not the *values* of them.
The binding is the invisible thing which is given a name and associated with different values over the variable's lifetime.
This means that the delegate has full access to the variable, rather than just receiving some copy of its value.
Consider this code:

static void Main(string[] args)
{
 int someNumber = 3;
 Action<int> addThis = delegate(int a) { someNumber = someNumber + a; };
  
 addThis(10);
 
 Console.WriteLine(someNumber); // This prints "13"
}


That's right - it prints 13! The someNumber variable was modified by addThis.
Whenever you open up a function, a loop, or an if block, or anything with curly braces, that conceptually creates a new 'lexical environment' and a new set of bindings for all variables declared within those braces. In the case of loops, a new environment is created for each iteration, so you can bind to different variables each time round, even though they have the same name:

var setters = new List<Action<int>>();
var getters = new List<Func<int>>();

for(int i = 0; i < 3; i++)
{
 int closedOverVariable = 0;
 setters.Add(delegate(int newValue) { closedOverVariable = newValue; });
 getters.Add(() => closedOverVariable);
}

// Set the first int to 5
setters[0](5);

// Set the second int to 100
setters[1](100);

// Print the value of the first int : "5"
Console.WriteLine(getters[0]);

// Print the value of the second int : "100"
Console.WriteLine(getters[1]);

We just created three ints with no way to access them apart from via the appropriate getter and setter delegates. I always found that kinda spooky; It's like the ghost of a variable...

The Bug

So where did it all go wrong today? Here's a piece of code which demonstrates what I got wrong:

// A nice array of ints
int[] ints = new[] { 1, 2, 3 };

// A list of delegates
var delegates = new List<Func<int>>();

// Lets go through the ints, and make a 
// delegate for each one which returns the corresponding int.
foreach(int i in ints)
{
 delegates.Add (() => i);
}

// Now go through all three delegates, printing the return values.
// It should go "1, 2, 3", right?
Console.WriteLine (delegates[0]());
Console.WriteLine (delegates[1]());
Console.WriteLine (delegates[2]());


WRONG! It will go "3, 3, 3"!

That's because the foreach statement in C# doesn't create a new binding for 'i' at each iteration. It behaves more like 'i' is declared outside the loop and the value reassigned 3 times. Each new delegate we're creating is getting the same binding - accessing the same variable. I find this a little counter-intuitive, as I would expect there to be 3 different bindings created in 3 different lexical environments as we iterate over the ints, just as there would if 'i' had been declared inside the loop.
That last part is actually the way to fix this mess:

int[] ints = new[] { 1, 2, 3 };

var delegates = new List<Func<int>>();

foreach(int i in ints)
{
 // 3 different bindings will be created for 'tmp'
 // in 3 different lexical environments.
 // We're initializing it with the *value* of 'i'
 int tmp = i;

 // Close over the binding of 'tmp', rather than that of 'i'
 delegates.Add (() => tmp);
}

Console.WriteLine (delegates[0]());
Console.WriteLine (delegates[1]());
Console.WriteLine (delegates[2]());



And that's all, folks. Hopefully I've helped someone understand lexical closures better. Maybe I've even managed to steer someone away from falling into this trap.

Happy coding :)

Saturday, 20 April 2013

Being an OOP Expert

It's tough to be an expert at Object-oriented programming. There all kinds of things you have to know about, like singletons, visitors, abstract superclasses, methods, factories, facades, the bridge pattern...
If you don't know what these things are, how can you ever hope to produce scalable, maintainable software?

A Job Interview

At the interview for my current job, I was presented with a matrix to fill out. The rows were different skills or technologies, and the columns went from "Don't know about" to "Expert" or something similar.
Amongst others, there was a row for Unit testing, a row for .NET Framework and a row for Object Oriented Programming.
I ticked the "Expert" box for this row only.
Now you're probably thinking how arrogant I am. Some of you, maybe those versed in CLOSMultimethods or Haskell's typeclasses are probably already thinking of how best to blow my mind with a humbling comment.
That's why I'm explaining now, that I'm aware of the number of different technologies and practices which "Object Oriented Programming" can be made to refer to. I'm also aware that those technologies are almost certainly not the ones they mean when they're asking how much I know about OOP.
For the rest of this post when I'm speaking about OOP, I'm referring to what 90% of the software world know as OOP; That is a class-based, stateful object system with information hiding annotations and subclassing. Java, C# or C++ are the systems I speak of.


I'm not convinced

At the risk of sounding arrogant all the same, I'd say I've written plenty of code in Java and a reasonable amount of C#, I've been to hell and back with this idea of OOP.
I used to be an OOP proponent. A zealot.

Once, I accidentally used a 2D array to represent a matrix for a scholarship project. I was advised by my supervisor to do something more "object oriented" such as represent a row with a class and use a Java List<T> for the columns.
This greatly improved the scalability and modularity of my code, so much so that after the project, my code was bought from me for a large sum of money by a multinational firm, and it currently sits at the core of a complex probabilistic model used to drive the world's cruise missiles. Couldn't have done that with a mere 2D array, could I now?
Ok, I promise I'll stop the sarcasm now.

"More object oriented"...

What I was really being advised to do was to make use of a larger number of the OOP constructs provided by the language. I went ahead and did this, as it seemed like a good idea at the time.
Like any nerd I like shiny things and by god, the result was that.
It was also completely nonsensical to anyone who actually understood how the solution worked.
This is what happens when the solution you want to implement gets really departed from the terms you're implementing it in.
This is actually the idea behind Domain Specific Languages (DSLs), where the language is designed around concepts involved in solving the problem.

I had an excuse - I was 16 years old and I had just started really playing with OOP, falling into all the traps, spending most of my time figuring out how to represent my solution with sets of classes, and generally going nuts with the whole idea.
This is not a professional or productive way to work. This kind of thinking is not engineering.

Programming

To me, programming is about finding the route from the problem to the logic gates in the processor. It's a game of representations, that is, abstractions. Abstractions are necessary so you can make large problems manageable by putting them into simple terms. As a software engineer, your job is often to design these abstractions and make them useful in as many situations as possible, whether that's by extending a language like Java with a fancy library, or creating a new DSL over a host syntax like XML or even with its own custom parser.
This is a big part of what makes software engineering so hard; There are many ways to look at a particular problem.

Putting aside the awkwardness we know about, you might think representing that matrix as a list of column objects would be a reasonable idea. But just wait until someone wants to build a system on top of it which needs to look at matrices row by row or god forbid, even diagonals.
It's not intractable; You can indeed wrap it up in a "float[] GetMatrixRow(int rowIndex)" function, but that's another layer of abstraction to maintain, another layer between what the compiler reads and your actual intent(bye bye code optimizations), and more code to drown in when you can't figure out where that NaN is coming from.

Another problem with building a code tower like this is now that your actual "intent", whatever that was, is now just some part of this mass of abstraction. As far as the compiler is concerned, it's a lump of code. It'll compile it, but there's not going to be any reasoning about it.
What you're writing is not something which describes your solution particularly well, rather something which after being run will produce correct behaviour. This makes the implementation less valuable, in my opinion.
This all sounds a little extremist I'm sure, but keep in mind the problem I posed out was an extremely simple one - doing something columnish with a matrix of numbers.

erm... OOP?

Yeah yeah, I know, I've been banging on about domain specific languages and their benefits for a good number of paragraphs now; This was supposed to be about OOP.
OOP represents an extension of what gets commonly called the "imperative" programming model;
The "Do this, now do this to change that, now check if that's changed, do these things if it has" model, which should really be considered a pile of fairly sophisticated syntactic sugar for machine code, as it really isn't all that far departed from the basic Von-Neumann model which is what's actually happening in your computer case.

OOP is an extra layer of sugar up, and a slight move into another dimension - there is an introduction of some extra concepts which aren't just about sequenced operations.

Here's the thing: In order to solve most any real world problems, you have to build abstractions atop your processor. Unless your problem is to add 3 numbers in sequence together for which, you could say your processor supports hardware acceleration :P.

Some problems are solved by computing linear functions to calculate what direction your cruise missile is travelling in.
Some are solved by running an optimal sequence of dependent steps in the most efficient way, such as a build system (Makefiles are a good example of this).
Some need to keep track of groups of values which evolve and affect one another in predefined ways, such as a particle system.
Wikipedia tells me that OOP came about trying to solve problems similar to the last one.

It's no secret that as long as your machine abstraction(model) is Turing Complete (TC), you can represent any computable function. That goes for OOP too; There's no saying it's impossible to code certain things in an OOP or non-OOP TC language.

The point I'm making is that having the OOP abstraction as part of your language really isn't that applicable and useful by itself, and it's definitely not something which will increase productivity and scalability for most classes of problems.
Unless your problem is to represent the state of synchronous message-receiving groups of values, it won't solve much of your problem either. Yes, I know you can use those constructs to represent a solution to the problem, but you could also probably build a lot of furniture with hammers and nails. If you've got a hammer available, that's great, but it doesn't mean it's time to start either building nail-oriented furniture, or building more appropriate tools by nailing multiple hammers together to create some kind of saw and......
You get my drift; Go and buy more tools. :)

Patterns

So, you've got an OOP language: A language with first-class support for managing groups of values and synchronous messaging between those groups of state.
Great, Now you want to solve your problem with it.
Design patterns to the rescue!
Yes, conceptual adaptors if you will, to convert the model your OOP language gives you into something you can use to solve a common class of problem. There are some patterns which are only a little extra atop the OOP model. Other patterns however, are quite a jump. The Visitor pattern, for instance. Now I'm not going to start whining about the Visitor pattern and how complicated and unreadable it is to implement, because that's not important. What's important is how readable the result is, and in my humble opinion, the result works quite nicely once you look past the boilerplate. This is important to my point, as I can now point out that having that OOP model somewhere in there didn't really help us at all. It didn't bring us any closer to traversing that tree or whatever we used the Visitor pattern for, if anything it got in the way a bit.

It got in the way a bit

That's what I commonly find with OOP ways of thinking. I've seen some absolutely ghastly code which was written in a way which seemed to say: "Look, I managed to find a use for OOP here, here and here!", in fact I've written a fair amount of it. I'm not saying that OOP is the devil, because it is not. It's simply a model for solving certain classes of problems.
So if OOP isn't going to be the centrepiece what concepts should a language designer make first-class citizens in a "General Purpose" Programming Language?
You can go for pure functional, which can be a pretty close fit for some problems, but of questionable use for matters of state.
There's the C#/C++ approach of including as much as possible. Clojure is a really good example of this, but the rules are a bit different there as it's a Lisp, and the distinctions between language features and library features tend to disappear.

Wait!

I haven't mentioned a lot to do with OOP, I know. I haven't really talked about encapsulation that much. I haven't talked about subclassing or polymorphism.
How on earth can this be a post about OOP without those things?
My answer to that is that it's not really a post about OOP. It's a post about representing problems, the misunderstood role of OOP as the saviour of software complexity, and why I think I can write that I'm an "Expert" on OOP.
I can write that I'm an "Expert" on OOP, because I know what it is and what it's not. I know when it's appropriate, and I know when it just gets in the way a bit. I also know that it certainly doesn't get the engineer out of having to solve the actual problem!


This turned out longer than I expected, I'm now off to watch today's Doctor Who ep.
'Til next time.

Wednesday, 20 April 2011

A start

Right, first post, hmm...


I guess I'll start with what I've been up to at Uni...
On the last leg of my Comp Sci course, I'm finding it increasingly difficult not to check out of things.
I just want the damned thing to be over, and to go and do my Industrial Placement with ARM (which I may or may not get).
As of now, I've got a _sane_ database of laptop specs to pull out of my arse for my group statistics/AI project (essentially a product comparison engine). It's gonna be a long haul of ruby, sed, bash, regexes, and little manual fixups. Really, not fun stuff...


Now onto the fun stuff...
I like progamming languages. I LOVE progamming languages, I know a fair few. The newest members of my army, being scala, haskell and ruby.

Scala blew me away when I first met it. It had so many features.
Seemed like it had all the best language features, created to this date.
Then, I hated it. I hated it for it's seemingly nonsensical type-inference mechanism, which surely did little more than to make code unreadable.
A few days went by, A few days writing 1600 lines of Java for my Games AI project, and lo-and-behold, I didn't feel so bad about it.
I concluded that, aside from a few choice features (pattern matching, traits), it should be thought of as no more, and no less than Java, but with less of the verbose crap.

I still wasn't satisfied, I wanted something different. MORE different.
I attempted to write my own stack-based 'functional assembler', if you will, which actually turned out pretty nifty.
A lovechild of Lisp(as I understood it then) & FORTH, I hope to implement it on an FPGA someday when I have money.

Speaking of Lisp; yesterday, I finally managed to wrap my head around Common Lisp!
Or at least it's syntax and actual operation.
I feel somehow liberated, even though I don't know enough about the libraries, or have had enough practice, to do anything really useful with it.
I may just be doing my next Games AI project in Clojure in an attempt to get to grips with the Lisp way of thinking.

Industrial Placement...
Last week, I finally applied for the year's placement/internship at ARM, in Cambridge, that I've had my eye on for over a year now. I've yet to hear back from them, but i've got a vote of confidence from my cousin, Ben, who did this placement a few years ago.
After I complete that, I'll have - aside from experience working for one hell of a name - a designated BSc Hons in Computer Science; Hopefully enough to kick-start my career into a...direction of some kind.


That's about all for now, I believe.
I'm now off to depress myself with HTML.
Till next time :)

Lee