- Instances of
Applyrepresent the application of an Op to some input Variable (or variables) to produce some output Variable (or variables). They are like the application of a [symbolic] mathematical function to some [symbolic] inputs.
Broadcasting is a mechanism which allows tensors with different numbers of dimensions to be used in element-by-element (elementwise) computations. It works by (virtually) replicating the smaller tensor along the dimensions that it is lacking.
A variable with an immutable value. For example, when you type
>>> x = tensor.ivector() >>> y = x + 3
Then a constant is created to represent the
3in the graph.
- Elemwise (i.e. element-wise)
An element-wise operation
fon two tensor variables
Nis one such that:
f(M, N)[i, j] == f(M[i, j], N[i, j])
In other words, each element of an input matrix is combined with the corresponding element of the other(s). There are no dependencies between elements whose
[i, j]coordinates do not correspond, so an element-wise operation is like a scalar operation generalized along several dimensions. Element-wise operations are defined for tensors of different numbers of dimensions by broadcasting the smaller ones.
- See Apply
- Expression Graph
A directed, acyclic set of connected Variable and Apply nodes that express symbolic functional relationship between variables. You use Theano by defining expression graphs, and then compiling them with theano.function.
An Op is destructive (of particular input[s]) if its computation requires that one or more inputs be overwritten or otherwise invalidated. For example, inplace Ops are destructive. Destructive Ops can sometimes be faster than non-destructive alternatives. Theano encourages users not to put destructive Ops into graphs that are given to theano.function, but instead to trust the optimizations to insert destructive ops judiciously.
Destructive Ops are indicated via a
destroy_mapOp attribute. (See
- see expression graph
- Inplace computations are computations that destroy their inputs as a side-effect. For example, if you iterate over a matrix and double every element, this is an inplace operation because when you are done, the original input has been overwritten. Ops representing inplace computations are destructive, and by default these can only be inserted by optimizations, not user code.
- Part of a function Mode – an object responsible for ‘running’ the compiled function. Among other things, the linker determines whether computations are carried out with C or Python code.
- An object providing an optimizer and a linker that is passed to theano.function. It parametrizes how an expression graph is converted to a callable object.
.opof an Apply, together with its symbolic inputs fully determines what kind of computation will be carried out for that
Applyat run-time. Mathematical functions such as addition (
T.add) and indexing
x[i]are Ops in Theano. Much of the library documentation is devoted to describing the various Ops that are provided with Theano, but you can add more.
- An instance of
Optimizer, which has the capacity to provide an optimization (or optimizations).
- A graph transformation applied by an optimizer during the compilation of a graph by theano.function.
- An Op is pure if it has no destructive side-effects.
- The memory that is used to store the value of a Variable. In most cases storage is internal to a compiled function, but in some cases (such as constant and shared variable the storage is not internal.
- A Variable whose value may be shared between multiple functions. See
- The interface for Theano’s compilation from symbolic expression graphs
to callable objects. See
The the main data structure you work with when using Theano. For example,
>>> x = theano.tensor.ivector() >>> y = -x**2
yare both Variables, i.e. instances of the
Some Tensor Ops (such as Subtensor and Transpose) can be computed in constant time by simply re-indexing their inputs. The outputs from [the Apply instances from] such Ops are called Views because their storage might be aliased to the storage of other variables (the inputs of the Apply). It is important for Theano to know which Variables are views of which other ones in order to introduce Destructive Ops correctly.
View Ops are indicated via a
view_mapOp attribute. (See