Class and instance semantics

Python supports multiple inheritance, however, since the .NET study came out Python has implemented new style classes which, while they still support multiple inheritance have a restriction: they cannot inherit from more than one builtin type.

A similar restriction can be imposed for .NET classes. They could support multiple inheritance provided they only inherit from a single .NET base class.

Type declarations

Type declarations, or type inferencing would help for speed. I hope that sometime soon the PyPy project will deliver a wonderful new parsers and compiler for Python that performs all the necessary optimisations (and I believe in the tooth fairy!) I don't believe that this is needed to make a .NET Python competitive with C-Python. Of course, if our target is to beat C# we might need some help.

Another reason why some form of type declaration would be useful will be to interact fully with the other .NET languages. It is to be hoped that PEP318 will provide a basis for introducing statements that have a declarative effect.

What you can do

You can run existing Python scripts, and you can use any existing Python libraries.

You can import CLR classes, access their methods and attributes. You can't subclass a CLR class (or at least, I couldn't get it to work). It partly runs under Mono, but Mono needs to improve significantly before it can fully support Python for .Net

What you cannot do

You cannot use a Python class from the CLR. You cannot script ASP.Net pages using Python.

There is an interesting issue concerning the difference between value and reference types. Specifically, if you have an array of value types, you cannot modify elements by subscripting into the array from Python:

items = CLR.System.Array.CreateInstance(Point, 3)
  for i in range(3):
    items[i] = Point(0, 0)

items[0].X = 1 # won't work!!

Instead you have to write something like this:

items = CLR.System.Array.CreateInstance(Point, 3)
  for i in range(3):
    items[i] = Point(0, 0)

# This _will_ work. We get 'item' as a boxed copy of the Point
# object actually stored in the array. After making our changes
# we re-set the array item to update the bits in the array.

item = items[0]
item.X = 1
items[0] = item

I expect that the same issue will arise with a direct implementation of Python under .Net as well. If Python evaluates the partial expression as to a reference, then it will implicitly box the value at that point and any changes to the boxed value will be lost.

How fast is it?

I haven't done extensive timing tests, but it isn't fast. A simple assignment into a CLR hashtable is about 100 times slower than the equivalent assignment into a Python dictionary. On this basis the original Activestate Python port really flies, but of course the comparison is unfair as any actual processing done in Python will run at normal C-Python speeds (which it wouldn't do on a natively ported Python).

Today, when you use Python to drive COM objects, you must try to avoid unnecessary calls through the COM layer. Exactly the same holds true for Python for .Net. You can run code on either side of the bridge at full speed, but try to avoid crossing that bridge more often than absolutely necessary.

The problem with delegates

The real bottleneck though, is the use of the reflection library on every attribute lookup or method call. Using reflection to find a method is slow, but the real hit comes when you try to call a method using the object that was returned. A call through a made through the reflection classes is at least 100 times slower than a direct call or a call through a delegate. Creating a delegate from the method object is also possible, but is just as slow if you only use the delegate once. There might be a benefit if the delegate could be cached and reused, but there are other problems with delegates.

The most significant callable objects in Python are: functions, unbound methods, bound methods, and type objects.

.Net doesn't have pointers to functions ², instead it uses delegates. Delegates are cool: they can be combined to give multicast delegates, and any delegate may also be called asynchronously allowing for interesting threaded applications. However, delegates also have major limitations.

1. You can create a delegate analagous to a Python bound method. i.e. one that refers to a non static method in a specific instance of an object.

2. You can create a delegate analagous to a Python function pointer, i.e. one that points to a static method with no associated instance.

But: you cannot create a delegate analagous to an unbound method, i.e. one where we have a reference to a non static method, but no specific instance.

And: you cannot create a delegate analagous to a type object, i.e. one that creates an object. In .Net, the call to a constructor uses a different instruction sequence than a call to an ordinary method, and there is no way to wrap such a call in a delegate without providing a factory function.

N.B. There are no equivalents to native methods required in .Net Python, since all Python code is compiled to native functions.

Function objects

In the model I tried to use, a def statement creates a PyFunction object that will call the original code. (This is another reason why delegates on their own are not sufficient: Python functions have writeable attributes, so we need something more than a delegate.) The PyFunction object stores the default arguments, and handles all processing of default and keyword arguments.

It is easy to overlook the fact that although in Python it looks as though all calls are completely dynamic, there are some things fixed in a function call.

When calling a function, you cannot generally tell in advance how many arguments the function will accept, nor what their names are. However, at the point of call you can almost always tell how many arguments you are passing.

When a function is entered, it doesn't know how many arguments it was given when called, nor which were positional and which keyword. However, every function actually receives a fixed number of arguments, and there is no distinction between positional and keyword.

This function always receives 3 arguments no matter how it is called:

def foo(a=1, b=2, c=3): pass

This function receives exactly two arguments, p and q:

def bar(*p, **q): pass

It makes sense therefore, to compile the code for these functions to accept exactly the expected number of arguments, and leave it to the function object to perform the require mapping from the arguments actually provided by the caller.

If PyFunction actually implements several overloads, then we can minimise the work done on a function call:

foo(3)  # Python

expands to the equivalent of:

foo.__call__(3)

the PyFunction3 class implements this as:

PyObject __call__(PyObject arg1)
{
    return this.__call__(arg1, defaults[1], defaults[2])
}

PyObject __call__(PyObject arg1, PyObject arg2, PyObject arg3)
{
    return this.fDelegate(arg1, arg2, arg3)
}

and the function foo was actually compiled as:

static PyObject foo(PyObject a, PyObject b, PyObject c)
{
    return PyNone;
}

We can handle bound methods in exactly the same way, using a PyBoundMethod class instead of a PyFunction class. However, there is no way to use delegates to bind an instance to a method at runtime without using the reflection classes, so we need another mechanism for unbound methods.

The only answer is to use automatically generated static methods.

class X
{
    int foo() { ... }
    void bar(int iVal) { ... }
}

At runtime, when we first try to use an instance of an X in Python, we can create an anonymous wrapper class:

class Wrap_X
{
    static PyObject foo(PyObject self)
    {
        return (PyObject)
            ((X)((PyExternalObject)foo).getObject())();
    }
    static PyObject bar(PyObject self, PyObject iVal)
    {
        ((X)((PyExternalObject)foo).getObject())((int) iVal);
        return PyNone;
    }
}

at the same time, the code can create a dictionary mapping the method names 'foo', and 'bar' onto PyUnboundMethod1 and PyUnboundMethod2 instances.

Similar wrapper methods may be created for properties, and factories to wrap the constructors.

Setting a global should not involve a dictionary lookup

Direct assignments to global variables should not involve a dictionary lookup. We know when we are assigning to a global, we know which global is being assigned, therefore we can simply store the value in a static variable in the module.

Of course, we have to continue to support setting a global from outside the module, and also setting it indirectly through globals()

Accessing a global should not involve a dictionary lookup

As with assignment, if we know the variable we are accessing we can pick it straight up from the static variable. There may be an issue if the global could be unset or masking a builtin of the same name, so we may need more code than a simple load of a variable, but not much more.

globals() dictionary

We have to allow access to the global variables through the dictionary interface, but this doesn't have to be a real dictionary. It could be a subclass of dictionary which directly reads and writes the variables for known globals, and falls back on a dictionary only when accessing variables not referred to directly in the program source.

Optimising builtins

References to builtins may be optimised at compile time, but only when there is no global variable of the same name which could be hiding the builtin. It would seem prudent also, if optimising access to a builtin, to ensure that the globals() dictionary for the module will throw an exception on any attempt to mask the builtin at runtime.

Readonly globals

This is a bit more speculative, and might break too much of Python's dynamic nature for some people, but it would also be possible to optimise access to some global values that are set only once. For example an imported module is rarely reassigned, so (provided there is no obvious reuse of the variable) you could optimise access to the module's attributes and use the same mechanism as for builtins to reject any attempt to modify the variable indirectly.

However, this might be a step too far for some of the Python community. I can think of rare situations where I have resorted to rebinding a name in a different module where the name was set once by an import statement (this was to stub out the module for unit tests), but it isn't generally a good practice.