IfElse vs Switch¶
Both ops build a condition over symbolic variables.
IfElsetakes a boolean condition and two variables as inputs.
Switchtakes a tensor as condition and two variables as inputs.
switchis an elementwise operation and is thus more general than
switchevaluates both output variables,
ifelseis lazy and only evaluates one variable with respect to the condition.
from theano import tensor as T from theano.ifelse import ifelse import theano, time, numpy a,b = T.scalars('a', 'b') x,y = T.matrices('x', 'y') z_switch = T.switch(T.lt(a, b), T.mean(x), T.mean(y)) z_lazy = ifelse(T.lt(a, b), T.mean(x), T.mean(y)) f_switch = theano.function([a, b, x, y], z_switch, mode=theano.Mode(linker='vm')) f_lazyifelse = theano.function([a, b, x, y], z_lazy, mode=theano.Mode(linker='vm')) val1 = 0. val2 = 1. big_mat1 = numpy.ones((10000, 1000)) big_mat2 = numpy.ones((10000, 1000)) n_times = 10 tic = time.clock() for i in range(n_times): f_switch(val1, val2, big_mat1, big_mat2) print('time spent evaluating both values %f sec' % (time.clock() - tic)) tic = time.clock() for i in range(n_times): f_lazyifelse(val1, val2, big_mat1, big_mat2) print('time spent evaluating one value %f sec' % (time.clock() - tic))
In this example, the
IfElse op spends less time (about half as much) than
since it computes only one variable out of the two.
$ python ifelse_switch.py time spent evaluating both values 0.6700 sec time spent evaluating one value 0.3500 sec
linker='cvm' are used,
ifelse will compute both
variables and take the same computation time as
switch. Although the linker
is not currently set by default to
cvm, it will be in the near future.
There is no automatic optimization replacing a
switch with a
broadcasted scalar to an
ifelse, as this is not always faster. See
If you use test values, then all branches of the IfElse will be computed. This is normal, as using test_value means everything will be computed when we build it, due to Python’s greedy evaluation and the semantic of test value. As we build both branches, they will be executed for test values. This doesn’t cause any changes during the execution of the compiled Theano function.