Tuesday, April 23, 2013

Why no extension methods in Delphi?

I have been wondering this for a long time now. Why does Delphi not have this great language feature? Sure, we have helpers but they are not the same. First they are still officially considered a feature you should not use when designing new code. However it opens up some interesting possibilities - some might call it hacks... but that is a topic for another day. Second they don't work on interfaces or generic types. And interestingly though that is what I want to talk about today.

But first - if you don't know it already - I suggest you read what an extension method is - really I could not explain it any better. Oh, and to all readers that want to jump the "stop the dotnetification of Delphi - keep these new language features away" wagon - this post is not for you, sorry.

Ever had a class or a set of classes you wanted to add some functionality to? Sure, there are ways to do so like the decorator pattern. But did you see the problem there if you have a type you cannot inherit from because either you cannot modify the code or it's not a class but an interface? Well, then create a new interface and add that functionality there, someone might say. How, if you cannot extend the given type? Use the adapter or bridge pattern? You can see where this is going. You might end having to change existing code or introduce lots of code to apply your additional functionality.

The most prominent example of extension methods (and not surprisingly the reason they were introduced in C# 3.0) are the extension methods for IEnumerable<T>. If you want to use the foreach (or for..in loop in Delphi) all you have to implement is the GetEnumerator method (and actually the only method that IEnumerable<T> got). So if you ever need to implement that in some of your classes you implement just one method and got access to almost any query operation you can imagine - not saying they all make sense in every context, but you get the idea.

Extension methods are great. You don't clutter your class with things that don't belong there directly but apply to an aspect of your class (in our case being enumerable). They follow good principles like the dependency inversion principle. The way you are using them is more natural and makes more sense than having static methods (or routines) where you pass in the instance you want to call the method on as first parameter.

Even without the fancy LINQ Syntax without question it is much more readable to write

for c in customers.Where(HasBillsToPay).OrderBy<string>(GetCompanyName) do

instead of

for c in EnumerableHelper.OrderBy<string>(
  EnumerableHelper.Where(customers, HasBillsToPay), GetCompanyName) do

And that statement has only two chained calls - imagine how that grows in length if you got a more complex query with grouping or something else. Also in that case it is the order on how the query gets processed - easier to read and to write.

But - you remember - no helpers for interfaces and generics! Well, we can implement these methods in our base TEnumerable<T> class and/or put it on our IEnumerable<T> interface, no? Yes, we can. And everything would be fine if there wasn't one tiny detail - how generics are implemented in Delphi and how the compiler handles them: it generates a type for every specialization. Which means the same code compiled for every possible T in your application, for TCustomer, TOrder, TCategory and so on. Only with a small set of methods implemented (and possible classes for more complex operations like GroupBy for example) this means you get hundreds of KB added for each TList<T> you will ever use - even if you never touch these methods. That is because the linker cannot remove any method inside an interface even if never called.

So how to work around that problem (which is what I have been doing in the Spring4D refactoring branch lately)? Let's take a look again on how extension methods are defined. Nothing prevents us from creating that syntax in Delphi, so a Where method could look like this:

  Enumerable = record
    class function Where<TSource>(source: IEnumerable<T>;
      predicate: TPredicate<TSource>): IEnumerable<T>;

Simple, isn't it? But how to call it? We need a little trick here, let's see:

  Enumerable<T> = record
    fThis: IEnumerable<T>;
    function GetEnumerator: IEnumerator<T>;

    function Where(predicate: TPredicate<TSource>): Enumerable<T>

    class operator Implicit(const value: IEnumerable<T>): Enumerable<T>;

As you can see we use a record type that wraps the interface we want to extend and add the method there. We can implement the "extension methods" there or direct the call to our extension method type if we want to keep it seperatly.

We now have a nice way to do our query just like we wrote it above if customers where from Enumerable<T>. Or we can perform a cast (since we have an implicit operator that will get used). Also notice how the result of the Where method is of the record type. That way we can chain the calls easily. And because we implemented GetEnumerator we can use it in a for..in loop just like any IEnumerable<T>.

What's also nice about the record type is that the linker can now be smart and remove any method that we never call and save us dozens of megabytes in our binary (not kidding).

So our life could be so much easier if we had extension methods (or call them helper) for interfaces and generic types. But as long as we don't have that, we have to find some clever workarounds.

If you are a Spring4D user, check out the changes in the refactoring branch and let me know what you think.


  1. could not agree more...

  2. Just switch to dynamic typing and all these problems go away! :-)

    1. And how could I do that with Delphi? The only dynamic typing in Delphi I can think of are invokable variants. But they don't support generic methods and are not very nice to use (no code completion and so on)

  3. I have been trying out some of this stuff in a small datasnap REST server I am working on. The new Select method was exactly what I needed for a particular problem I had, so was very happy to see that.

    Could you explain a bit further on the recommended usage of this?

    Right now I just use the AsEnumerable method to get the record wrapper and chain the extension methods I need. Are you saying there is a better way?

    1. It also was causing the compile time to explode (and actually freeze the IDE sometimes) as it has to compile the complete Enumerable record type including all the adapter classes just to let the linker remove them again after (as you are not using every extension method on every list you have).

      The suggested way to access the extension methods from the interface then most likely will be to cast the interface to the record type - like this:

      for i in Enumerable<TCustomer>(customers).Where ...)

  4. Sincerely, being used with extensions with VB.Net and C#, I don't see that much difference in workings of class helpers and extensions.
    As of the reason 2, I believe Embarcadero could make helpers 1st class citizens and update its' uses..... Despite the fact that scope rules are an much tighter concept on Delphi than in dotNet languages environment.

    1. They are similar but there are no helpers for interfaces and generic types which was the subject. Also the limitation of only having one helper at a time and the official statement that helpers should not be considered when designing new code limits their use and shows their low ranking as a language feature.

      Also as I pointed out their implementation should just be as syntax sugar, while the implementation in Delphi is way beyond that - have you ever debugged through a class helper method call? It's a nightmare. And that is because you actually can have virtual helper methods an inherit helpers.

    2. Actually the disclaimer in newer Delphi versions is much softer that the one when I first met with this concept (circa Delphi 8 launch)... When the helpers were only to use in binding in VCL.NET (where it were used to exaustion) and discouraged in ANY CODE.
      Almost an documented "outcast feature", much like GOTOs. ;-)

      And yes, they should not be used when design new code, since you can control that code ;-). As I won't design new classes in C# with extension methods for that new class - it's silliness.

      But I create some extensions for String and other library classes (which cannot be extended) - for example, like a Contains that use an case insensitive comparer for strings...

    3. Considering extension methods for design is not silliness especially not for interfaces (which was the subject here you remember).

      Think again about the IEnumerable example. All you have to implement is GetEnumerator. If all these extension methods were not extension methods but part of the interface you would have to implement every single one of them.

    4. They would end being convenience methods on the IEnumerable client and/or implementator (ex System.Linq.ParallelEnumerable). They are not part of the design of IEnumerable AT ALL.

      AsParallel extension is an implementation detail that are convenient to expose for using IEnumerable with parallel queries.

      I agree with Embarcadero disclaimer: fist design main functionality and detect if any operation on used types/classes are good for an general-purpose convenience method.

      If so is the case - and it should be called from the type/class for clarity/simplicity of use - but you can't (or is inconvenient to) subclass it for some reason, extend it. It doesn't matter if the instrument for it is an extension method or a class helper. Applies to both.

    5. I still don't get your point.

      AsParallel is just another extension method for IEnumerable.

      My point was that currently Delphi does not support helpers/extension methods for interfaces/generic types, how an implementation would just be some kind of syntax sugar/compiler magic and how to achieve something similar without waiting for Embarcadero to implement it.