Showing posts with label Programming. Show all posts
Showing posts with label Programming. Show all posts

Tuesday, 30 April 2013

Lexical Closures

Today, I found the first confusing bug in the software I've been working on. It had me going for pretty much the whole day, not helped by the fact that my MonoDevelop also had a bug causing it to crash when the IKVM runtime threw a meaningless exception, meaning I couldn't actually get to see any of what was happening.

I've been using a fairly out of date MonoDevelop, as it's the one available from the ubuntu repos. Strangely enough, recent builds on the website are only available for Windows, OSX and Suse.
Luckily, there doesn't seem to be any proper reason for this, so I basically did what this page told me to and built MD 3 from source. So far, it works a treat and doesn't seem to feature the annoying bug I mentioned. Nice :)

Anyway, the bug. It seems really stupid now, so in order to feel less stupid I'm going to turn this into a post where I explain lexical closures in the context of C#.

So, what is a lexical closure? Why, it's a function object which captures the bindings of free variables (variables which aren't defined in the function or it's parameter list).
I'll elaborate:

Delegates

If you've written C# before, you've probably used function objects or delegates, as they call them:

// Declare the delegate type outside a function somewhere:
delegate int OneAdder(int a);

// Instantiate the delegate
OneAdder addOne = delegate(int a) { return a + 1; };

// Use the delegate to print a "4"
Console.WriteLine(addOne(3));


You can also use the lambda syntax to more succinctly create delegates:

OneAdder addOne = (a) => a + 1;

There's more; From .NET 3.5 there are some standard generic delegate types already predeclared:

Action<int> : void-returning function with an int parameter
Action<int, int> : void-returning function with two int parameters
Func<int, string> : string-returning function with an int parameter
Func<int, int, string> : string-returning function with two int parameters

As you might intuit, delegates are reference types just like classes and amenable to the same assignment/passing-around semantics. For instance, one could write a function:

static int CallMyDelegate(Func<int, int> func)
{
 return func(5);
}

Lexical Closure

Delegates in C# have 'Lexical closure'. That means that when you call this, passing it 5:

static Func<int, int> MakeAddingFunction(int number)
{
 return delegate(int a) { return a + number; };
}


What you get back is a function which adds 5 to things.
It's called a lexical closure, because the function 'closes over' the 'number' variable (which is in its lexical scope), so it can use it later. One can also say the delegate 'captures the lexical environment'.
Think on that for a while until you understand it. Try it out, even.

This whole delegate thing is really useful as it gives us a nice, succinct way to abstract some operation in terms of it's inputs and outputs, so it can be passed to code which can then apply it to some data without understanding the operation. C#'s "Linq" is based on this kind of thing. If you haven't heard of Linq go and read up on it once you've finished reading this.

Peculiarities

Lexical closure is about capturing the *bindings* of variables in scope, not the *values* of them.
The binding is the invisible thing which is given a name and associated with different values over the variable's lifetime.
This means that the delegate has full access to the variable, rather than just receiving some copy of its value.
Consider this code:

static void Main(string[] args)
{
 int someNumber = 3;
 Action<int> addThis = delegate(int a) { someNumber = someNumber + a; };
  
 addThis(10);
 
 Console.WriteLine(someNumber); // This prints "13"
}


That's right - it prints 13! The someNumber variable was modified by addThis.
Whenever you open up a function, a loop, or an if block, or anything with curly braces, that conceptually creates a new 'lexical environment' and a new set of bindings for all variables declared within those braces. In the case of loops, a new environment is created for each iteration, so you can bind to different variables each time round, even though they have the same name:

var setters = new List<Action<int>>();
var getters = new List<Func<int>>();

for(int i = 0; i < 3; i++)
{
 int closedOverVariable = 0;
 setters.Add(delegate(int newValue) { closedOverVariable = newValue; });
 getters.Add(() => closedOverVariable);
}

// Set the first int to 5
setters[0](5);

// Set the second int to 100
setters[1](100);

// Print the value of the first int : "5"
Console.WriteLine(getters[0]);

// Print the value of the second int : "100"
Console.WriteLine(getters[1]);

We just created three ints with no way to access them apart from via the appropriate getter and setter delegates. I always found that kinda spooky; It's like the ghost of a variable...

The Bug

So where did it all go wrong today? Here's a piece of code which demonstrates what I got wrong:

// A nice array of ints
int[] ints = new[] { 1, 2, 3 };

// A list of delegates
var delegates = new List<Func<int>>();

// Lets go through the ints, and make a 
// delegate for each one which returns the corresponding int.
foreach(int i in ints)
{
 delegates.Add (() => i);
}

// Now go through all three delegates, printing the return values.
// It should go "1, 2, 3", right?
Console.WriteLine (delegates[0]());
Console.WriteLine (delegates[1]());
Console.WriteLine (delegates[2]());


WRONG! It will go "3, 3, 3"!

That's because the foreach statement in C# doesn't create a new binding for 'i' at each iteration. It behaves more like 'i' is declared outside the loop and the value reassigned 3 times. Each new delegate we're creating is getting the same binding - accessing the same variable. I find this a little counter-intuitive, as I would expect there to be 3 different bindings created in 3 different lexical environments as we iterate over the ints, just as there would if 'i' had been declared inside the loop.
That last part is actually the way to fix this mess:

int[] ints = new[] { 1, 2, 3 };

var delegates = new List<Func<int>>();

foreach(int i in ints)
{
 // 3 different bindings will be created for 'tmp'
 // in 3 different lexical environments.
 // We're initializing it with the *value* of 'i'
 int tmp = i;

 // Close over the binding of 'tmp', rather than that of 'i'
 delegates.Add (() => tmp);
}

Console.WriteLine (delegates[0]());
Console.WriteLine (delegates[1]());
Console.WriteLine (delegates[2]());



And that's all, folks. Hopefully I've helped someone understand lexical closures better. Maybe I've even managed to steer someone away from falling into this trap.

Happy coding :)

Monday, 29 April 2013

More TestVisor Development

Firstly...

Git

I'd like to point out that Git is pretty darn good.
I picked GitHub for the hosting because I liked the look of it and had heard ravings about Git; This is probably one of the best decisions I made for this project.
I've only had extended contact with Mercurial and Subversion in the past and I can definitely appreciate the difference in target markets:
Git is targeted toward people who value a holistic view of their environment; It's a more toolbox than black box. Mercurial(or Hg), is more of a nice extensible black box with a good set of buttons on the top.
Any software dev team should really be familiar with at least one of these tools.

IKVM

IKVM seems like a magical piece of software. It's basically a JVM running on CLI/.NET, and it works!
I had a problem - I'm developing in mono, but I was lusting after Mozilla Rhino, a nice Javascript engine written in Java. After a quick look around, I didn't really find anything that worked as well for my platform, so I went ahead and downloaded Rhino. One call to "ikvmc", and my problem seems well and truly solved! I got a Rhino.dll that I can link into my mono project with the worst of the hassle being that I had to also include /usr/lib/ikvm/IKVM.OpenJDK.Core.dll (to use of some jvm types like java.lang.Class), which took around a minute to locate and reference.
I've gotta say, it felt weird typing this into a C# project:
return Context.javaToJS(new java.lang.Boolean(func(testParams, testKey)), s1);

TestVisor

It's been an exciting few days for this project. I'm still on holiday, so I've been tinkering with it pretty much nonstop.
I'm hopefully getting some help with the work from a University chum before long.
Here's a little demo of Javascript test plans:



Saturday, 20 April 2013

Being an OOP Expert

It's tough to be an expert at Object-oriented programming. There all kinds of things you have to know about, like singletons, visitors, abstract superclasses, methods, factories, facades, the bridge pattern...
If you don't know what these things are, how can you ever hope to produce scalable, maintainable software?

A Job Interview

At the interview for my current job, I was presented with a matrix to fill out. The rows were different skills or technologies, and the columns went from "Don't know about" to "Expert" or something similar.
Amongst others, there was a row for Unit testing, a row for .NET Framework and a row for Object Oriented Programming.
I ticked the "Expert" box for this row only.
Now you're probably thinking how arrogant I am. Some of you, maybe those versed in CLOSMultimethods or Haskell's typeclasses are probably already thinking of how best to blow my mind with a humbling comment.
That's why I'm explaining now, that I'm aware of the number of different technologies and practices which "Object Oriented Programming" can be made to refer to. I'm also aware that those technologies are almost certainly not the ones they mean when they're asking how much I know about OOP.
For the rest of this post when I'm speaking about OOP, I'm referring to what 90% of the software world know as OOP; That is a class-based, stateful object system with information hiding annotations and subclassing. Java, C# or C++ are the systems I speak of.


I'm not convinced

At the risk of sounding arrogant all the same, I'd say I've written plenty of code in Java and a reasonable amount of C#, I've been to hell and back with this idea of OOP.
I used to be an OOP proponent. A zealot.

Once, I accidentally used a 2D array to represent a matrix for a scholarship project. I was advised by my supervisor to do something more "object oriented" such as represent a row with a class and use a Java List<T> for the columns.
This greatly improved the scalability and modularity of my code, so much so that after the project, my code was bought from me for a large sum of money by a multinational firm, and it currently sits at the core of a complex probabilistic model used to drive the world's cruise missiles. Couldn't have done that with a mere 2D array, could I now?
Ok, I promise I'll stop the sarcasm now.

"More object oriented"...

What I was really being advised to do was to make use of a larger number of the OOP constructs provided by the language. I went ahead and did this, as it seemed like a good idea at the time.
Like any nerd I like shiny things and by god, the result was that.
It was also completely nonsensical to anyone who actually understood how the solution worked.
This is what happens when the solution you want to implement gets really departed from the terms you're implementing it in.
This is actually the idea behind Domain Specific Languages (DSLs), where the language is designed around concepts involved in solving the problem.

I had an excuse - I was 16 years old and I had just started really playing with OOP, falling into all the traps, spending most of my time figuring out how to represent my solution with sets of classes, and generally going nuts with the whole idea.
This is not a professional or productive way to work. This kind of thinking is not engineering.

Programming

To me, programming is about finding the route from the problem to the logic gates in the processor. It's a game of representations, that is, abstractions. Abstractions are necessary so you can make large problems manageable by putting them into simple terms. As a software engineer, your job is often to design these abstractions and make them useful in as many situations as possible, whether that's by extending a language like Java with a fancy library, or creating a new DSL over a host syntax like XML or even with its own custom parser.
This is a big part of what makes software engineering so hard; There are many ways to look at a particular problem.

Putting aside the awkwardness we know about, you might think representing that matrix as a list of column objects would be a reasonable idea. But just wait until someone wants to build a system on top of it which needs to look at matrices row by row or god forbid, even diagonals.
It's not intractable; You can indeed wrap it up in a "float[] GetMatrixRow(int rowIndex)" function, but that's another layer of abstraction to maintain, another layer between what the compiler reads and your actual intent(bye bye code optimizations), and more code to drown in when you can't figure out where that NaN is coming from.

Another problem with building a code tower like this is now that your actual "intent", whatever that was, is now just some part of this mass of abstraction. As far as the compiler is concerned, it's a lump of code. It'll compile it, but there's not going to be any reasoning about it.
What you're writing is not something which describes your solution particularly well, rather something which after being run will produce correct behaviour. This makes the implementation less valuable, in my opinion.
This all sounds a little extremist I'm sure, but keep in mind the problem I posed out was an extremely simple one - doing something columnish with a matrix of numbers.

erm... OOP?

Yeah yeah, I know, I've been banging on about domain specific languages and their benefits for a good number of paragraphs now; This was supposed to be about OOP.
OOP represents an extension of what gets commonly called the "imperative" programming model;
The "Do this, now do this to change that, now check if that's changed, do these things if it has" model, which should really be considered a pile of fairly sophisticated syntactic sugar for machine code, as it really isn't all that far departed from the basic Von-Neumann model which is what's actually happening in your computer case.

OOP is an extra layer of sugar up, and a slight move into another dimension - there is an introduction of some extra concepts which aren't just about sequenced operations.

Here's the thing: In order to solve most any real world problems, you have to build abstractions atop your processor. Unless your problem is to add 3 numbers in sequence together for which, you could say your processor supports hardware acceleration :P.

Some problems are solved by computing linear functions to calculate what direction your cruise missile is travelling in.
Some are solved by running an optimal sequence of dependent steps in the most efficient way, such as a build system (Makefiles are a good example of this).
Some need to keep track of groups of values which evolve and affect one another in predefined ways, such as a particle system.
Wikipedia tells me that OOP came about trying to solve problems similar to the last one.

It's no secret that as long as your machine abstraction(model) is Turing Complete (TC), you can represent any computable function. That goes for OOP too; There's no saying it's impossible to code certain things in an OOP or non-OOP TC language.

The point I'm making is that having the OOP abstraction as part of your language really isn't that applicable and useful by itself, and it's definitely not something which will increase productivity and scalability for most classes of problems.
Unless your problem is to represent the state of synchronous message-receiving groups of values, it won't solve much of your problem either. Yes, I know you can use those constructs to represent a solution to the problem, but you could also probably build a lot of furniture with hammers and nails. If you've got a hammer available, that's great, but it doesn't mean it's time to start either building nail-oriented furniture, or building more appropriate tools by nailing multiple hammers together to create some kind of saw and......
You get my drift; Go and buy more tools. :)

Patterns

So, you've got an OOP language: A language with first-class support for managing groups of values and synchronous messaging between those groups of state.
Great, Now you want to solve your problem with it.
Design patterns to the rescue!
Yes, conceptual adaptors if you will, to convert the model your OOP language gives you into something you can use to solve a common class of problem. There are some patterns which are only a little extra atop the OOP model. Other patterns however, are quite a jump. The Visitor pattern, for instance. Now I'm not going to start whining about the Visitor pattern and how complicated and unreadable it is to implement, because that's not important. What's important is how readable the result is, and in my humble opinion, the result works quite nicely once you look past the boilerplate. This is important to my point, as I can now point out that having that OOP model somewhere in there didn't really help us at all. It didn't bring us any closer to traversing that tree or whatever we used the Visitor pattern for, if anything it got in the way a bit.

It got in the way a bit

That's what I commonly find with OOP ways of thinking. I've seen some absolutely ghastly code which was written in a way which seemed to say: "Look, I managed to find a use for OOP here, here and here!", in fact I've written a fair amount of it. I'm not saying that OOP is the devil, because it is not. It's simply a model for solving certain classes of problems.
So if OOP isn't going to be the centrepiece what concepts should a language designer make first-class citizens in a "General Purpose" Programming Language?
You can go for pure functional, which can be a pretty close fit for some problems, but of questionable use for matters of state.
There's the C#/C++ approach of including as much as possible. Clojure is a really good example of this, but the rules are a bit different there as it's a Lisp, and the distinctions between language features and library features tend to disappear.

Wait!

I haven't mentioned a lot to do with OOP, I know. I haven't really talked about encapsulation that much. I haven't talked about subclassing or polymorphism.
How on earth can this be a post about OOP without those things?
My answer to that is that it's not really a post about OOP. It's a post about representing problems, the misunderstood role of OOP as the saviour of software complexity, and why I think I can write that I'm an "Expert" on OOP.
I can write that I'm an "Expert" on OOP, because I know what it is and what it's not. I know when it's appropriate, and I know when it just gets in the way a bit. I also know that it certainly doesn't get the engineer out of having to solve the actual problem!


This turned out longer than I expected, I'm now off to watch today's Doctor Who ep.
'Til next time.

Thursday, 21 April 2011

Starting Lisp

First off, Lisp isn't a language. At least, it's not just 1 language - it's a family of them which all share a similar syntax and manner of operation.
The variant I've been acquainting myself with is Common Lisp.
It's intimidating for several reasons, the main one being the elitist attitude attached to many of those who use it. Maybe it *is* the best, most advanced language in the world(I can't say), but hearing that makes it no easier to learn.
The syntax looks....bloated. One would tend to skim though a line of code and assume there is some kind of unnecessary, 'yet beautiful' paradigm at work which could only be appreciated by an academic, but in all makes a less useful language.
It definitely helps to have a background in functional languages to understand just how useful Lisp can be. 

After downloading and installing a Lisp runtime (clisp, for example), you will notice that the front-end takes the form of a Read-eval-print loop, something very familiar if you come from a ruby/python/similar background.

For those unfamiliar with this concept, it's exactly what it says it is. You type text in the correct syntax, which gets *read* into an internal format, then gets *eval*-uated, and the resulting data *print*-ed on the screen.

Now to the code :)

In the loop try evaluating some data, like 1 or 3 or "sandwiches", and hitting enter to evaluate. That shouldn't do anything really very surprising at all.
What's happening here is that the reader is converting the text, "1", into the internal Lisp object which represents an integer.
This object is then passed to the evaluator, which does nothing with the object.
Still the integer object representing 1, it gets passed to the printer, which converts it to a textual representation "1".

Thrilling stuff, eh?
Not really, no. In that example, the evaluator did absolutely nothing, but it's in fact the most complex part which makes up a Lisp system, as we shall see.

But, how do we write out a Lisp list?
Like this: (1 2 3 4), this is a list containing 4 objects.
Lists can also contain other lists: (1 2 3 4 (1 2 3))

Now lets try to evaluate a list.
Type (1 2 3 4) into the read-eval-print prompt, and hit enter.
This will now throw a potentially very ugly error(ugliness is implementation-dependant).
clisp responds with the following insult:


*** - EVAL: 1 is not a function name; try using a symbol instead
The following restarts are available:
USE-VALUE      :R1      You may input a value to be used instead.
ABORT          :R2      Abort main loop

Not nice to look at, but it's point is that the object '1' isn't the name of a function.
Why was it expecting a function name? Because that's how Lisp code is written. It's a list. It's all lists.
Want to call a function? Create a list beginning with the function name, and the parameters as the other list elements.

Here's a practical function call, the seminal "hello world" program:
(print "Hello world")
This is a list containing two objects. One is a name, the other is a string. The reader, takes this, creates these objects in memory, and passes the list to the evaluator. The evaluator notices it's been given a list, and after evaluating all of the other elements, calls the function named by 'print' with the "Hello world" string as an argument(remember, simple objects like strings do nothing when evaluated).

But what if we feel we need to stop evaluation for some reason? What if we just want to tell the evaluator to stop and, for example, not evaluate a list. Rather, leave it as a list?

This is where the special name "quote" is used.
When the evaluator encounters a list to evaluate whose first element is the name: "quote", it simply returns the other element in the list without trying to evaluate it.
Given this, we can do what we described, and get the evaluator to leave our list alone by writing the following:
(quote (1 2 3 4))
After the reader has done it's job, and turned everything into objects(2 lists, one inside the other), the evaluator sees the first element of the list passed to it is the name 'quote'. This makes it stop in it's tracks, and simply evaluate to whatever object was given as the argument, in this case, a list of 1, 2, 3 and 4.
This is then passed to the printer, and is displayed as:
(1 2 3 4)
on the screen.

As the reader(you, not the Lisp reader) will hopefully now understand, this quote thing is useful. Most obviously, when you want to write out lists. This assumption is true enough, in fact, that in Common Lisp, there is a shorthand in the form of the single-quote, eg:
(quote someobject)
is completely and utterly equivalent to
'someobject

and (quote (1 2 3 4))
is totally and thoroughly the same as
'(1 2 3 4)


Now, a quick test. If you want the printer to show a list, why can't you type:
(print (1 2 3 4))
when instead you must type:
(print '(1 2 3 4)) ?


It may not say much, but I feel this tutorial clearly outlines the workings of the language, which are easily misunderstood.

More Common Lisp:
Make a greeting function, 'greet':
(defun greet () (print "hello"))

Make a better greeting function, which takes a string as a parameter:
(defun greet2 (name) (print (concatenate 'string "Hello, " name)))

Call the greet2 function:
(greet2 "Lee")
prints => "Hello, Lee"


That's all for now :)