Python metaclass

the base article is available here:  http://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python 

 

 

to understand the metaclass in Python, there is some background that you should be aware in order to grasp the idea of metaclass and its foundation.

 

 

Part I: 

Classes as object

 

Given an class declaration:

 

 

>>> class ObjectCreator(object):
  ...       pass
  ... 

  >>> my_object = ObjectCreator()
  >>> print my_object
  <__main__.ObjectCreator object at 0x8974f2c>

 

 

Classes in Python has more... put it simple. that Classes in Python are objects too. 

 

 

As soon as keyword class, Python execute it and craete an OBJECT. the instruction 

 

 

 

>>> class ObjectCreator(object):
  ...       pass
  ... 

 

 

create in memory an object with name ObjectCreator.

 

 

what you can do with the Class object.

 

 

 

• you can assign it to a variable
• you can copy it
• you can add attributes to it
• you can pass it as a function parameter

 

here is an example of how you can do with the class object.

 

 

 

 

>>> print ObjectCreator # you can print a class because it's an object
  <class '__main__.ObjectCreator'>
  >>> def echo(o):
  ...       print o
  ... 
  >>> echo(ObjectCreator) # you can pass a class as a parameter
  <class '__main__.ObjectCreator'>
  >>> print hasattr(ObjectCreator, 'new_attribute')
  False
  >>> ObjectCreator.new_attribute = 'foo' # you can add attributes to a class
  >>> print hasattr(ObjectCreator, 'new_attribute')
  True
  >>> print ObjectCreator.new_attribute
  foo
  >>> ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable
  >>> print ObjectCreatorMirror.new_attribute
  foo
  >>> print ObjectCreatorMirror()
  <__main__.ObjectCreator object at 0x8997b4c>

 

 

Part II:

 

Creating Classes Dynamically

 

 

In the base article, there is a example demonstrate how to create classes inside a function.  But that is not dynamic. 

 

 

>>> def choose_class(name):
  ...     if name == 'foo':
  ...         class Foo(object):
  ...             pass
  ...         return Foo # return the class, not an instance
  ...     else:
  ...         class Bar(object):
  ...             pass
  ...         return Bar
  ...     
  >>> MyClass = choose_class('foo') 
  >>> print MyClass # the function returns a class, not an instance
  <class '__main__.Foo'>
  >>> print MyClass() # you can create an object from this class
  <__main__.Foo object at 0x89c6d4c>

 

 

Since classes are object, they must be generated by something. the somehting here is the metaclass. 

 

 

But before we drill down to the Metaclass, let's close exame type keyword .

 

 

The following shows what typically you do with type, you get what type an objects is.

 

 

 

>>> print type(1)
<type 'int'>
>>> print type("1")
<type 'str'>
>>> print type(ObjectCreator)
<type 'type'>
>>> print type(ObjectCreator())
<class '__main__.ObjectCreator'>

 

 

However, type has more, it has a completely different ability, it can also create classes on the fly...

 

the function prototyp of type that create types on the fly is as follow.

 

 

 

type(name of the class, 
       tuple of the parent class (for inheritance, can be empty), 
       dictionary containing attributes names and values)

 

e.g

 

 

>>> class MyShinyClass(object):
...     pass
...
>>> print type(MyShinyClass)
<type 'type'>
>>> MyShinyClass2 = type('MyShinyClass2', (), {})
>>> print MyShinyClass2



>>> # and then you can create the instance of the dynamically created class
...
>>> print MyShinyClass2()
<__main__.MyShinyClass2 object at 0x0110CCB0># and then you can cre

 

So, the following are equivalent.

 

 

 

>>> class Foo(object):
...       bar = True

 

 

and 

 

 

 

>>> Foo = type('Foo', (), {'bar':True})

 

 

and after you created Foo (with the type keyword), you uses it as if you have decalred it. 

 

 

 

  >>> print Foo
  <class '__main__.Foo'>
  >>> print Foo.bar
  True
  >>> f = Foo()
  >>> print f
  <__main__.Foo object at 0x8a9b84c>
  >>> print f.bar
  True

 

Part III:

 

What are metaclass (finally)

The metaclass, boils down to the following definition:

 

Metaclasses are the 'stuff' that creates classes, since classes create objects (as well as being objects), so you take that meta classes as class's class

 

 

  MyClass = MetaClass()
  MyObject = MyClass()

 

the following shows step by step by examing the internals of the type system in Python.

 

A speical note on type method itself.

 

 

 写道

It's because the function type is in fact a metaclass. type is the metaclass Python uses to create all classes behind the scenes.

 

 

Checking the __class__ attribute.

 

 

>>> age = 35
  >>> age.__class__
  <type 'int'>
  >>> name = 'bob'
  >>> name.__class__
  <type 'str'>
  >>> def foo(): pass
  >>> foo.__class__
  <type 'function'>
  >>> class Bar(object): pass
  >>> b = Bar()
  >>> b.__class__
  <class '__main__.Bar'>

 

 

But what is the __class__ of the __class__, see following..

 

 

 

  >>> a.__class__.__class__
  <type 'type'>
  >>> age.__class__.__class__
  <type 'type'>
  >>> foo.__class__.__class__
  <type 'type'>
  >>> b.__class__.__class__
  <type 'type'>

 

So, they are 'type'. 

 

From the examles above. we can see that a meta class is a "class factory", "type" is the built-in metaclass pythons ues, but of course, you can create your own metaclasses.

 

 

Part IV:

 

the __metaclass__ attribute

the form

 

 

class Foo(object):
  __metaclass__ = something...
  [...]

 

 

remember we said before that Pyton will translate the class declaration to an object, but wait, when there is a __metaclass__ attribute.

 

 

 

Python willl look for __metaclass__ in the class definition. If it finds it, if will use it to create the object classFoo. If it doesn't, it will use type to create the class.

 

The order of searching the __metaclass__ is as follow.

 

 

 

• Is there a __metaclass__ attribute in Foo?
• If yes, create in memory a class object (I said a class object, stay with me here), with the nameFoo by using what is in __metaclass__.
• If Python can't find __metaclass__, it will look for a __metaclass__ in Bar (the parent class), and try to do the same.
• If Python can't find __metaclass__ in any parent, it will look for a __metaclass__ at the MODULE level, and try to do the same.
• Then if it can't find any __metaclass__ at all, it will use type to create the class object.

 

 

Part V: 

What to put in a custom metaclass

Let's see a stupid example.

 

Suppose we are creating a metaclasses which will alternate all the attributes of classes that it create to have attributes of the classes to be upper cases.

 

 

The following shows how to do that via creating a function as the __metaclass__ (__metaclass__ does not necessary mean it has to be a class)  at the module level, 

 

 

 

# the metaclass will automatically get passed the same argument
# that you usually pass to `type`
def upper_attr(future_class_name, future_class_parents, future_class_attr):
  """
    Return a class object, with the list of its attribute turned 
    into uppercase.
  """

  # pick up any attribute that doesn't start with '__'
  attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
  # turn them into uppercase
  uppercase_attr = dict((name.upper(), value) for name, value in attrs)

  # let `type` do the class creation
  return type(future_class_name, future_class_parents, uppercase_attr)

__metaclass__ = upper_attr # this will affect all classes in the module

class Foo(object): 
  # we can define __metaclass__ here instead to affect only this class
  bar = 'bip'

print hasattr(Foo, 'bar')
# Out: False
print hasattr(Foo, 'BAR')
# Out: True

f = Foo()
print f.BAR
# Out: 'bip'

 

 

Later, let's do that via the class defintinition

 

 

 

# remember that `type` is actually a class like `str` and `int`
# so you can inherit from it
class UpperAttrMetaclass(type): 
    # __new__ is the method called before __init__
    # it's the method that creates the object and returns it
    # while __init__ just initializes the object passed as parameter
    # you rarely use __new__, except when you want to control how the object
    # is created.
    # here the created object is the class, and we want to customize it
    # so we override __new__
    # you can do some stuff in __init__ too if you wish
    # some advanced use involves overriding __call__ as well, but we won't
    # see this
    def __new__(upperattr_metaclass, future_class_name, 
                future_class_parents, future_class_attr):

        attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        return type(future_class_name, future_class_parents, uppercase_attr)

 

If we apply all the OOP technique (like the super keyword, calling to base __new__ and applies the convention names such as "cls", "base" and etcc..), then we can get the following code .

 

 

 

class UpperAttrMetaclass(type): 

    def __new__(cls, name, bases, dct):

        attrs = ((name, value) for name, value in dct.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        return super(UpperAttrMetaclass, cls).__new__(cls, name, bases, uppercase_attr)

 

 

So in a conclusion, about __metaclass__

 

 

indeed, metaclasses are especially useful to do black magic, and therefore complicated stuff. But by themselves, they are simple:

• intercept a class creation
• modify the class
• return the modified class

there is also a discussion on why to use metaclass rather instead of functions. for brevity, I 'd like to ignore this part. Pease see the page http://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python for more details.

Part VI:

why the hell would you use metaclass?

Normally you won't use metaclass, but there is occasions that you will find metaclass handy.

A common cases is creating of an API. A typical one is the Django ORM:

if you have 

  class Person(models.Model):
    name = models.CharField(max_length=30)
    age = models.IntegerField()

 

and if you do this:

guy = Person(name='bob', age='35')
  print guy.age

 

It won't return an IntegerField object. It will return an int, and can even take it directly from the database.

This is possible because models.Model defines __metaclass__ and it uses some magic that will turn the Person you just defined with simple statements into a complex hook to a database field.

Part VII:

the last word:

Classes are object to create instance

Everything is an object in Python, and they are all either instances of classes or instances of metaclasses.

Exception: type;

type is its own metaclass. this is not reproducible in pure python, and is done by cheating a little bit in the implementation level.

metaclass is not absolutely necessary, youcan achieve the goal of alternating a class with 

• monkey patching
• class decorators