C# in Depth

Cover of C# in Depth
Order now (3rd edition)

(You are currently looking at the first edition version of this page. This page is also available for the second and third editions.)

Notes for Chapter 9: Lambda expressions and expression trees

9.1.5: Another potential shortcut - lambdas with no arguments

Although C# 3 has a shortcut for the case of a single argument, it has no parallel for a lambda which doesn't need any arguments. For instance, a Func<int> might be created from: () => 5. There are numerous possible representations. Two which spring to mind are |> 5 and |=> 5, using the vertical bar as a sort of "no data here" symbol.

9.2.1: No need for a lambda!

The code in listing 9.4 can be abbreviated even further than just removing the braces from round the body of the lambda expression. We don't need a lambda expression at all - we can use a simple method group conversion, converting this:

Action<Film> print = film => Console.WriteLine(film);

to this:

Action<Film> print = Console.WriteLine;

Of course as it's now so brief, we're not getting as much benefit from the extra variable. We can eliminate it completely, leaving this as the rest of the listing:

films.ForEach(Console.WriteLine);

films.FindAll(film => film.Year < 1960)
     .ForEach(Console.WriteLine);

films.Sort((f1,f2) => f1.Name.CompareTo(f2.Name));
films.ForEach(Console.WriteLine);

It's a judgement call as to which form is most readable - I quite like keeping the extra variable - but it's worth taking note of this. When you write a "simple" lambda expression, just think that it might not need to be a lambda at all. I'll readily admit this possibility had completely passed me by in this case - thanks are due to Joe Albahari for pointing it out.

9.2.2: Security in lambdas

Eric raised an interesting point which I not only hadn't covered - I hadn't even thought about it.

A lambda converted to a delegate becomes a method on the class in question (or a nested child class). If the delegate accesses a private field on the class, that's OK because the code all lives in the class.

What happens when a lambda is converted to an expression tree and it accesses a private? We want that scenario to still work, even though when you compile the expression tree, the resulting method is NOT on any method associated with the class.

In the desktop CLR, what we do is rely upon a new feature called Restricted Skip Visibility, whereby partially or fully trusted code is allowed to view the private state of other code provided that the viewing code is at least as or more trusted than the code that owns the private state.

Note that hoisted locals are considered private state.

The implications here are interesting, and things get odder in Silverlight (which doesn't have Restricted Skip Visibility) and in the SQL Server version of the CLR, which doesn't grant the required permission to any code. However, there's good news:

We are presently attempting to design a more general mechanism into all versions of the CLR so that this notion of "this object possesses a license to mess with the private state of that object" is cleanly represented.

9.3.1: Expression trees and parameters

There was some confusion on the forum around why the expression shown in listing 9.6 doesn't have any parameters. After all, it performs addition on two numbers - doesn't that mean it has two parameters? Well, not quite...

Think of the expression as a black box - you don't know what goes on inside it. You just feed it some number of parameters, and it will give you a result (assuming it doesn't have a void return type, effectively). In this case the addition is always performed with 2 and 3 as the operands - nothing varies, so there are no parameters. That's why when we compile the expression tree in listing 9.7 we end up with a Func<int> - something which accepts no parameters but returns an integer.

An alternative way of thinking about this is as a method. Consider this code:

public static int Add2And3()
{
    return 2 + 3;
}

Again, it's adding 2 and 3 - but the method doesn't take any parameters.

If we want to create an expression tree which does take parameters, we need to change the code a bit. Here's the code to do that, courtesy of Marc Gravell:

// Declare the parameters
ParameterExpression firstParam = Expression.Parameter(typeof(int), "x");
ParameterExpression secondParam = Expression.Parameter(typeof(int), "y");

// Create an Expression that performs an operation on the parameters
Expression add = Expression.Add(firstParam, secondParam);
        
// Create an Expression<T> using Expression.Lambda
// Note that we specify the parameters again
Expression<Func<int,int,int>> lambda = 
    Expression.Lambda<Func<int,int,int>>(add, firstParam, secondParam);
        
// Compile the expression into a delegate
Func<int,int,int> compiled = lambda.Compile();
        
// Execute it
Console.WriteLine(compiled(2,3));

As you can see, this time when we call compiled we give it two parameters.

That may help to clear up some misunderstandings, but reader feedback has suggested that expression trees are best understood by just trying them. Write a small test program and play around. Have fun - and don't worry too much if you find expression trees tricky: if you understand the basic concept that they're expressing logic as data, and that you can create them using lambda expressions, that's as much as most developers will ever need to know.

9.3.3: Lambda to expression tree conversion restrictions

I asked Eric for more information about what kind of lamdba expressions couldn't be turned into expression trees. Bearing in mind the following list, aren't you glad I didn't include it all in the book?

The following are illegal in an expression tree:

  • anonymous methods
  • statement lambdas
  • expression lambdas with ref/out parameters
  • all the assignment operators
  • the event += -= operators
  • any use of a removed partial method
  • a call to a virtual method via "base"
  • almost anything unsafe that would require unsafe codegen when converted to a delegate:
    • any binary operator where either operand has pointer type
    • any unary operator where the operand has pointer type
    • any sizeof operator on a non-built-in type
    • any conversion from or to a pointer type
  • multidimensional array initializers
  • any method that uses the undocumented "__arglist" feature
  • any "naked" method group
  • any "null typed" expression other than the literal null

The last four warrant a bit more explanation:

  • Support for multi-d initializers was left out of the expression tree API for no good reason and by the time we realized it, it was too edge-case a scenario to warrant defining/testing/documenting/etc new apis for it.
  • Both declaring and calling methods with C-style variable-number-of-arguments parameter lists are legal in C# for interop purposes; hardly anyone knows that. Search for "__arglist" for details. It's not legal to call one in an expression tree.
  • There are supposed to be no "naked" method groups in C# expressions but there are compiler bugs that make them legal. For example, you can say "M is object", and it will always be false, but not a compiler error, even though it really should be. It's not legal to do this in an expression tree because we have no way of describing the type of the operand. (We could bless this and just make it the constant false, but I would rather not compound our earlier error by blessing it further.)
  • Again, compiler bugs. The spec implies that (null ?? null) is not legal, but the compiler allows it. The type of that expression is "the null type", which again, is not a type according to the specification. Again, it's not legal to do this in an expression tree because we have no way of describing the type. (And again, we could make this the constant null, but let's not bless bad code, that just makes it harder to take the breaking change if we ever fix the bug.)

Given the size of the list, it seems amazing that expression trees are useful for anything - but as I noted in the book, the limitations very rarely crop up in real code.

9.3.3: infoof and tricks with expression trees

Eric reveals a feature which has been under discussion:

We have for many years considered creating an infoof (pronounced "in foof", of course) operator that would be to MethodInfos as typeof is to Types. The problem is, how do you specify the arguments so that overload resolution can select the correct method out of the method group? There’s no good way to do it, and there have always been way more important things to do, so it keeps getting cut.

However, the fact that lambda expression conversions can use the IL methodinfo operator can be used to your advantage:

If you want to get a methodinfo for a particular method, you can just say

Expression<Func<int,string,double>> e = (arg1, arg2)=>M(arg1, arg2);

And then pull the MethodInfo out of the generated Call node! Of course you end up with a few wasted object allocations along the way, but you are guaranteed to get the same method out that the compiler would have picked for overload resolution.

Interestingly, I'm pretty sure Marc Gravell came up with exactly the same idea after I'd read Eric's note, but before I included it on this web site.

9.4.3: Hindley-Milner type inference

I won't pretend to fully understand this comment, as I hadn't heard of Hindley-Milner type inference (Wikipedia link) before, other than it being briefly mentioned in Eric's blog:

A number of people have asked me why we didn’t simply use Hindley-Milner type inference, a la F#, OCAML, etc. Two reasons.
  1. HM type inference is not guaranteed to terminate in a reasonable amount of time; our type inference algorithm guarantees progress every time through the loop and therefore runs in polynomial time. (Though of course, overload resolution can then be exponential in C#, which is unfortunate.)
  2. HM type inference works poorly in a language which has class inheritance; it was designed for functional languages like Haskell with pattern matching rather than inheritance.