Making the double type¶
In Theano’s framework, a
is any object which defines the following
methods. To obtain the default methods described below, the Type should
be an instance of
Type or should be an instance of a
Type. If you will write all methods yourself,
you need not use an instance of
Methods with default arguments must be defined with the same signature, i.e. the same default argument names and values. If you wish to add extra arguments to any of these methods, these extra arguments must have default values.
filter(value, strict=False, allow_downcast=None)¶
This casts a value to match the Type and returns the cast value. If
valueis incompatible with the Type, the method must raise an exception. If
filtermust return a reference to
value(i.e. casting prohibited). If
strictis False, then casting may happen, but downcasting should only be used in two situations:
Noneand the default behavior for this type allows downcasting for the given
value(this behavior is type-dependent, you may decide what your own type does by default)
We need to define
filterwith three arguments. The second argument must be called
strict(Theano often calls it by keyword) and must have a default value of
False. The third argument must be called
allow_downcastand must have a default value of
filter_inplace(value, storage, strict=False, allow_downcast=None)¶
If filter_inplace is defined, it will be called instead of filter() This is to allow reusing the old allocated memory. As of this writing this is used only when we transfer new data to a shared variable on the gpu.
storagewill be the old value. i.e. The old numpy array, CudaNdarray, …
Returns True iff the value is compatible with the Type. If
filter(value, strict = True)does not raise an exception, the value is compatible with the Type.
Default: True iff
filter(value, strict=True)does not raise an exception.
Returns True iff
a == b
Returns True iff
bare approximately equal, for a definition of “approximately” which varies from Type to Type.
Makes a Variable of this Type with the specified name, if
None, then the Variable does not have a name. The Variable will have its
typefield set to the Type object.
Default: there is a generic definition of this in Type. The Variable’s
typewill be the object that defines this method (in other words,
Syntactic shortcut to
Used to compare Type instances themselves
Types should not be mutable, so it should be OK to define a hash function. Typically this function should hash all of the terms involved in
Optional. Only needed to profile the memory of this Type of object.
Return the information needed to compute the memory size of
The memory size is only the data, so this excludes the container. For an ndarray, this is the data, but not the ndarray object and other data structures such as shape and strides.
get_size()work in tandem for the memory profiler.
get_shape_info()is called during the execution of the function. So it is better that it is not too slow.
get_size()will be called on the output of this function when printing the memory profile.
obj – The object that this Type represents during execution
Python object that
Number of bytes taken by the object represented by shape_info.
Optional. Only needed to profile the memory of this Type of object.
shape_info – the output of the call to get_shape_info()
the number of bytes taken by the object described by
Optional, for TensorType-alikes.
Return a copy of the type with a possibly changed value for dtype and broadcastable (if they aren’t None).
dtype – New dtype for the copy.
broadcastable – New broadcastable tuple for the copy.
Optional to run, but mandatory for DebugMode. Return True if the Python objects a and b could share memory. Return False otherwise. It is used to debug when Ops did not declare memory aliasing between variables. Can be a static method. It is highly recommended to use and is mandatory for Type in Theano as our buildbot runs in DebugMode.
For each method, the default is what
for you. So, if you create an instance of
Type or an
instance of a subclass of
filter. You might want to override
as well as
values_eq. The other defaults generally need not be
For more details you can go see the documentation for Type.
For certain mechanisms, you can register functions and other such things to plus your type into theano’s mechanisms. These are optional but will allow people to use you type with familiar interfaces.
To plug in additional options for the transfer target, define a function which takes a theano variable and a target argument and returns eitehr a new transferred variable (which can be the same as the input if no transfer is nessecary) or returns None if the transfer can’t be done.
Then register that function by calling
with it as argument.
We are going to base Type
double on Python’s
filter and shall override
# Note that we shadow Python's function ``filter`` with this # definition. def filter(x, strict=False, allow_downcast=None): if strict: if isinstance(x, float): return x else: raise TypeError('Expected a float!') elif allow_downcast: return float(x) else: # Covers both the False and None cases. x_float = float(x) if x_float == x: return x_float else: raise TypeError('The double type cannot accurately represent ' 'value %s (of type %s): you must explicitly ' 'allow downcasting if you want to do this.' % (x, type(x)))
strict is True we need to return
strict is True and
x is not a
float (for example,
x could easily be an
int) then it is
incompatible with our Type and we must raise an exception.
strict is False then we are allowed to cast
x to a
x is an
int it we will return an equivalent
However if this cast triggers a precision loss (
x != float(x)) and
allow_downcast is not True, then we also raise an exception.
Note that here we decided that the default behavior of our type
allow_downcast is set to
None) would be the same as
allow_downcast is False, i.e. no precision loss is allowed.
def values_eq_approx(x, y, tolerance=1e-4): return abs(x - y) / (abs(x) + abs(y)) < tolerance
The second method we define is
values_eq_approx. This method
allows approximate comparison between two values respecting our Type’s
constraints. It might happen that an optimization changes the computation
graph in such a way that it produces slightly different variables, for
example because of numerical instability like rounding errors at the
end of the mantissa. For instance,
a + a + a + a + a + a might not
actually produce the exact same output as
6 * a (try with a=0.1),
values_eq_approx we do not necessarily mind.
We added an extra
tolerance argument here. Since this argument is
not part of the API, it must have a default value, which we
chose to be 1e-4.
values_eq is never actually used by Theano, but it might be used
internally in the future. Equality testing in
DebugMode is done using
Putting them together
What we want is an object that respects the aforementioned
contract. Recall that Type defines default implementations for all
required methods of the interface, except
filter. One way to make
the Type is to instantiate a plain Type and set the needed fields:
from theano import gof double = gof.Type() double.filter = filter double.values_eq_approx = values_eq_approx
Another way to make this Type is to make a subclass of
values_eq_approx in the subclass:
from theano import gof class Double(gof.Type): def filter(self, x, strict=False, allow_downcast=None): # See code above. ... def values_eq_approx(self, x, y, tolerance=1e-4): # See code above. ... double = Double()
double is then an instance of Type
Double, which in turn is a
There is a small issue with defining
double this way. All
Double are technically the same Type. However, different
Double Type instances do not compare the same:
>>> double1 = Double() >>> double2 = Double() >>> double1 == double2 False
Theano compares Types using
== to see if they are the same.
This happens in DebugMode. Also, Ops can (and should) ensure that their inputs
have the expected Type by checking something like
if x.type == lvector.
There are several ways to make sure that equality testing works properly:
Double.__eq__so that instances of type Double are equal. For example:def __eq__(self, other): return type(self) is Double and type(other) is Double
Double.__new__to always return the same instance.
Hide the Double class and only advertise a single instance of it.
Here we will prefer the final option, because it is the simplest.
Ops in the Theano code often define the
__eq__ method though.
Untangling some concepts¶
Initially, confusion is common on what an instance of Type is versus
a subclass of Type or an instance of Variable. Some of this confusion is
syntactic. A Type is any object which has fields corresponding to the
functions defined above. The Type class provides sensible defaults for
all of them except
filter, so when defining new Types it is natural
to subclass Type. Therefore, we often end up with Type subclasses and
it is can be confusing what these represent semantically. Here is an
attempt to clear up the confusion:
An instance of Type (or an instance of a subclass) is a set of constraints on real data. It is akin to a primitive type or class in C. It is a static annotation.
An instance of Variable symbolizes data nodes in a data flow graph. If you were to parse the C expression
intwould be a Type instance and
xwould be a Variable instance of that Type instance. If you were to parse the C expression
c = a + b;,
cwould all be Variable instances.
A subclass of Type is a way of implementing a set of Type instances that share structural similarities. In the
doubleexample that we are doing, there is actually only one Type in that set, therefore the subclass does not represent anything that one of its instances does not. In this case it is a singleton, a set with one element. However, the
TensorTypeclass in Theano (which is a subclass of Type) represents a set of types of tensors parametrized by their data type or number of dimensions. We could say that subclassing Type builds a hierarchy of Types which is based upon structural similarity rather than compatibility.
from theano import gof class Double(gof.Type): def filter(self, x, strict=False, allow_downcast=None): if strict: if isinstance(x, float): return x else: raise TypeError('Expected a float!') elif allow_downcast: return float(x) else: # Covers both the False and None cases. x_float = float(x) if x_float == x: return x_float else: raise TypeError('The double type cannot accurately represent ' 'value %s (of type %s): you must explicitly ' 'allow downcasting if you want to do this.' % (x, type(x))) def values_eq_approx(self, x, y, tolerance=1e-4): return abs(x - y) / (abs(x) + abs(y)) < tolerance def __str__(self): return "double" double = Double()
We add one utility function,
__str__. That way, when we print
double, it will print out something intelligible.