How to create an operator overload that only accepts nil

Since the introduction of Nullable types in Spring4D they had a problem: assigning null to them was cumbersome. Some time ago you could implicitly assign a Variant to them which made it possible to assign Null. However that caused some issue because of implicit Variant type conversion.

Imagine you had a Nullable<Integer> which should only allow Null or some Integer. But in fact you could assign a string to it - not good. Why was that the case? Well because it had an implicit operator for Variant to Nullable<Integer> and the compiler was too smart converting the string into a Variant to pass that to the operator overload. Result was an exception at runtime - very bad considering that Nullable<T> should be type safe at compile time.

Next idea was making an overload for Pointer to Nullable<T> but that made it possible to pass all kinds of reference types to a nullable - another source for bugs.

Before anyone asks, a Clear method was also never an option because the type should be immutable. Think of a property. If you would call Clear on that you would just clear the value returned from the getter.

For 1.2 I used an explicit type with no state and a static property Null so you can assign or compare with Nullable.Null (suggestion by Sir Rufo). An option but it always bugged me because it was not as crisp as just assigning nil like in Swift or null in C#.

Today it struck me. Use some reference type that you cannot assign anything else but nil to. How is that possible? Make it private so you cannot declare any variable of that type.

My first attempt was this:

type
  Nullable<T> = record
  strict private type
    Null = class end;
  public
    class operator Implicit(const value: Null): Nullable<T>;
  end;

That kinda worked. It is impossible to declare a variable of Null and thus you cannot assign anything but nil to it. But wait. You can assign any untyped pointer to it (and declaring $TYPEDADDRESS in the unit the Nullable<T> type is declared in did not work). Well, are there any types that don't allow assigning some untyped pointer to? Yes, events or managed types like dynamic arrays or interfaces. I tried them all and if I did not miss anything it works with a private interface type - I was not able to assign any typed or untyped pointer, interface or object to it.

So this is what worked (it looks a bit different in Spring.pas because I did not put the type into the generic Nullable<T> type but somewhere else where it is not accessible except from the same unit)

type
  Nullable<T> = record
  strict private type
    Null = interface end;
  public
    class operator Implicit(const value: Null): Nullable<T>;
  end;

Now (starting with Spring4D 1.2.1) you can finally simply assign nil to a Nullable<T> without losing type safety.

P.S. By default the Nullable<T> is still in compatibility mode to 1.1 which makes the implicit Variant conversion possible. Take a look into Spring.inc to disable it and make the Nullable<T> completely type safe.

I wish I had dcc32 -dontmakemycodeslow

Quick, what is wrong with this code performance wise?

function TWhereIterator<T>.MoveNext: Boolean;
var
  current: T;
begin
  Result := False;

  if fState = STATE_ENUMERATOR then
  begin
    fEnumerator := fSource.GetEnumerator;
    fState := STATE_RUNNING;
  end;

  if fState = STATE_RUNNING then
  begin
    while fEnumerator.MoveNext do
    begin
      current := fEnumerator.Current;
      if fPredicate(current) then
      begin
        fCurrent := current;
        Exit(True);
      end;
    end;
    fState := STATE_FINISHED;
    fEnumerator := nil;
  end;
end;

Maybe you have read this article in the past (if not, feel free to read it first, I'll be waiting) and know what to look for.

There is no exception or string here but a function returning an interface. Unfortunately the compiler does not just pass the field to GetEnumerator (Remember: the result of a function where the return type is managed is actually being passed as hidden var parameter) but preserves space on the stack for a temporary variable and makes sure it is properly cleared before returning, every time the method is called. In a tight loop this causes more overhead for every call to MoveNext than the actual work being done in the STATE_RUNNING block.

Always be aware of code executed under some condition and possible variables created by the compiler. I had another case in the past where I was working with TValue in a big case statement (over TTypeKind) and I ended up with a dozen of temporary variables (for each case label) generated by the compiler, although only one of them was used each time.

I solved this by putting the STATE_ENUMERATOR block into a method. That plus moving around the state checking a bit caused this code to run almost twice as fast as before. This was the test code:

TEnumerable.Range(1, 100000000)
  .Where(
  function(const x: Integer): Boolean
  begin
    Result := x mod 3 = 0
  end)
  .Count;

Which now takes around 1 second on my machine - a for-to loop with the same condition takes around 250 ms. Not bad after all given that you hardly do that for some simple integers and a bit more complex filters. By the way it is just as fast - or slow ;) as the same code in C#.

As you might have noticed development of Spring4D 1.2.1 is ongoing and will contain some bugfixes, a few small new features and significant performance optimization for some collection use cases.

Generics, modules and typeinfo

The good

When working in a loosely coupled software architecture you might be using different modules: packages, dynamic link libraries and typically a host executable that loads them.

If you are using a DI Container it is very easy to extend functionality by just putting that into one module and once its loaded it feeds the container all the nice things it contains and another module might consume these. That all works fine and well.

The bad

The fun begins as so often when generic types join the game. Let's say we have a generic interface that we are using and this is contained in a runtime package that two other modules require. Everything is fine and both modules can use that interface. However IList<Integer> in one of the modules is not exactly the same as IList<Integer> in the other modules because when you use a generic type the Delphi compiler compiles it for every unit you are using it in and during the linking phase all duplicates are removed. So far so good. But if you have two different modules both now contain IList<Integer> and with that bring their own typeinfo which was compiled into them. When a module is being loaded and uses runtime packages which typically includes the rtl.bpl also they get registered there. Now you have two (or more) of identical types in your application (given you compiled them from the same source state of course).

Now the DI container comes into play which identifies types by their typeinfo pointer - which as I explained is different across multiple modules.

You can test this yourself by creating two small projects, an executable and a DLL with the following code:

library MyLibrary;

uses
  Spring.Container,
  Spring.Collections;

procedure RegisterStuff;
begin
  GlobalContainer.RegisterType<IList<Integer>>.DelegateTo(
    function: IList<Integer>
    begin
      Result := TCollections.CreateList<Integer>([1, 2, 3, 4, 5, 6, 7, 8, 9]);
    end).AsSingleton;
  GlobalContainer.Build;
end;

begin
  RegisterStuff;
end.

program MyProgram;

{$APPTYPE CONSOLE}

uses
  SysUtils, Windows,
  Spring.Container,
  Spring.Collections;

var
  i: Integer;
begin
  LoadLibrary('MyLibrary.dll');

  try
    for i in GlobalContainer.Resolve<IList<Integer>> do
      Writeln(i);
  except
    on E: Exception do
      Writeln(E.Message);
  end;
  Readln;
end.

If you execute MyProgram you will get a nice  EResolveException with message 'Cannot resolve type: IList<System.Integer>'. Now that we know what is going on this is no surprise. But how can we solve that?

The ugly

In our software I used an admittedly scary hack. If you know a bit about the DI container architecture you know that it is very extensible and that you can add so called sub dependency resolvers. These resolvers are being asked if they can resolve a requested type. There are a few builtin ones that are used to resolve dynamic arrays, lists or a TFunc of a registered type. But you can also add your own ones. I wrote a resolver that checks if the requested type could not be resolved by the container already and then looks through its known types to find one with the same full qualified name. Since we are using very explicit named units there is no chance that we accidentally have 2 different types that have the same full qualified name.

I will not post the code here to not scare you away but since others might also run into this problem (I know at least one person that was building a plugin system by using the DI container and also ran into that problem) this issue will be addressed during the development and refactoring for the DI container in version 1.3 of Spring4D.

Speaking of version 1.3 - the release of 1.2 is almost there! We are working hard to deliver it shortly after the next Delphi release which will happen ... soonish, I think. ;)

The release/1.2 branch has been around for a while - please take a look if you haven't already. It contains a lot of new features and bugfixes. I will tell you more about some of the amazing features next time which won't take as long as it took since the last blog post - promise. :)