boardcasting in tensorflow

ref:

Broadcasting Tensorflow - Learning Tensorflow

 

When we operate on arrays of different dimensionality, they can combine in different ways, either elementwise or through broadcasting.

# This is known as an elementwise operation, where the elements from each list are considered in turn, added together and then the results combined.

a = tf.constant([[1,2,3], [4,5,6]])
b = tf.constant([[1,2,1], [1,2,1]])
c = a + b

#output
[[2 4 4]
[5 7 7]]

# multi dimension boardcasting
a = tf.constant([[1, 2, 3], [4, 5, 6]], name='a')
b = tf.constant([[100], [101]], name='b')
c = a + b

#output
[[101 102 103]
[105 106 107]]

# To understand this, let’s look at matrix shapes.
a.shape
TensorShape([Dimension(2), Dimension(3)])
b.shape
TensorShape([Dimension(2), Dimension(1)])

You can see from these two examples that a has two dimensions, the first of size 2 and the second of size 2.
In other words, it has two rows, each with three scalars in it.
Our b constant also has two dimensions, two rows with one scalar in each.
This is not the same as a list, nor is it the same as a matrix if one row of two scalars.

Due to the fact that the shapes match on the first dimension but not the second, the broadcasting happened across columns instead of rows.
For more on broadcasting rules, see
https://www.tensorflow.org/api_docs/python/tf/broadcast_to