What is a Tensor in TensorFlow? Shape, Type & Operators

What is a Tensor?

The name TensorFlow is derived directly from its core object: the tensor. In TensorFlow, every computation involves tensors. A tensor is a vector or matrix of n dimensions that represents all types of data. All values in a tensor hold an identical data type with a known (or partially known) shape, and the shape of the data is the dimensionality of the matrix or array.

A tensor can originate from the input data or from the result of a computation. In TensorFlow, all the operations are conducted inside a graph. The graph is a set of computations that take place successively. Each operation is called an op node, and the nodes are connected to each other.

The graph outlines the ops and the connections between the nodes. However, it does not display the values. The edge between the nodes is the tensor, i.e., a way to populate the operation with data.

In machine learning, models are fed with a list of objects called feature vectors. A feature vector can be of any data type, and it is usually the primary input that populates a tensor. These values flow into an op node through the tensor, and the result of that operation creates a new tensor, which in turn is used in a new operation. All these operations can be viewed in the graph.

Representation of a Tensor

In TensorFlow, a tensor is a collection of feature vectors (i.e., an array) of n dimensions. For instance, if we have a 2×3 matrix with values from 1 to 6, we write it as the matrix shown below:

TensorFlow represents this matrix as:

[[1, 2, 3], 
   [4, 5, 6]]

If we create a three-dimensional matrix with values from 1 to 8, the same data is drawn as a box with two stacked faces:

TensorFlow represents this matrix as:

[ [[1, 2],  
       [[3, 4],  
       [[5, 6],  
       [[7,8] ]

Note: A tensor can be represented with a scalar or can have a shape of more than three dimensions. It is simply harder to visualize the higher dimension levels.

Types of Tensor

In TensorFlow, all the computations pass through one or more tensors. A tf.Tensor is an object with three properties:

• A unique label (name)
• A dimension (shape)
• A data type (dtype)

Each operation you perform with TensorFlow involves the manipulation of a tensor. There are four main tensor types you can create:

• tf.Variable
• tf.constant
• tf.placeholder
• tf.SparseTensor

In this tutorial, you will learn how to create a tf.constant and a tf.Variable.

One point to keep in mind before you run the code: this tutorial follows the original TensorFlow 1.x API, where a graph is built first and executed later inside a session. TensorFlow 2 runs eagerly by default, so several of the symbols below were moved or removed. Every one of them is still reachable through the tf.compat.v1 namespace, and the table maps each to its modern counterpart.

Before you go through the tutorial, make sure you activate the conda environment that has TensorFlow installed. This environment is named hello-tf.

For macOS users:

source activate hello-tf

For Windows users:

activate hello-tf

After you have done that, you are ready to import TensorFlow.

# Import tf
import tensorflow as tf

Create a Tensor of n-Dimension

You begin with the creation of a tensor with one dimension, namely a scalar.

To create a tensor, you can use tf.constant() as shown in the TensorFlow tensor shape example below:

tf.constant(value, dtype, name = "")
arguments

- `value`: Value of n dimension to define the tensor. Optional
- `dtype`: Define the type of data:    
    - `tf.string`: String variable    
    - `tf.float32`: Float variable    
    - `tf.int16`: Integer variable
- "name": Name of the tensor. Optional. By default, `Const_1:0`

To create a tensor of dimension 0, run the following code.

## rank 0
# Default name
r1 = tf.constant(1, tf.int16) 
print(r1)			

Output

Tensor("Const:0", shape=(), dtype=int16)

The three parts of that output map onto the three tensor properties, as the annotated version below makes clear.

You can also give the tensor a name of your own with the name argument.

# Named my_scalar
r2 = tf.constant(1, tf.int16, name = "my_scalar") 
print(r2)

Output

Tensor("my_scalar:0", shape=(), dtype=int16)

Each tensor is displayed by its tensor name. Each tensor object is defined with attributes such as a unique label (name), a dimension (shape) and a TensorFlow data type (dtype).

You can define a tensor with decimal values or with a string by changing the type of data.

# Decimal
r1_decimal = tf.constant(1.12345, tf.float32)
print(r1_decimal)
# String
r1_string = tf.constant("Guru99", tf.string)
print(r1_string)

Output

Tensor("Const_1:0", shape=(), dtype=float32)
Tensor("Const_2:0", shape=(), dtype=string)

A tensor of dimension 1 can be created as follows:

## Rank 1
r1_vector = tf.constant([1,3,5], tf.int16)
print(r1_vector)
r2_boolean = tf.constant([True, True, False], tf.bool)
print(r2_boolean)

Output

Tensor("Const_3:0", shape=(3,), dtype=int16)
Tensor("Const_4:0", shape=(3,), dtype=bool)

You can notice that the TensorFlow shape is composed of only 1 column.

To create an array of 2 tensor dimensions, you need to close the brackets after each row. Check the tensor shape example below.

## Rank 2
r2_matrix = tf.constant([ [1, 2],
                          [3, 4] ],tf.int16)
print(r2_matrix)

Output

Tensor("Const_5:0", shape=(2, 2), dtype=int16)

The matrix has 2 rows and 2 columns filled with the values 1, 2, 3 and 4.

A matrix with 3 dimensions is constructed by adding another level of brackets.

## Rank 3
r3_matrix = tf.constant([ [[1, 2],
                           [3, 4], 
                           [5, 6]] ], tf.int16)
print(r3_matrix)

Output

Tensor("Const_6:0", shape=(1, 3, 2), dtype=int16)

The result matches the three-dimensional box diagram shown earlier in this tutorial.

Shape of Tensor

When you print a tensor, TensorFlow infers the shape. However, you can read the shape directly with the TensorFlow shape property.

Below, you construct a matrix filled with the numbers from 10 to 15 and you check the shape of m_shape.

# Shape of tensor
m_shape = tf.constant([ [10, 11],
                        [12, 13],
                        [14, 15] ]                      
                     ) 
m_shape.shape

Output

TensorShape([Dimension(3), Dimension(2)])

The matrix has 3 rows and 2 columns.

TensorFlow has useful commands to create a vector or a matrix filled with 0 or 1. For instance, if you want to create a 1-D tensor with a specific shape of 10, filled with 0, you can run the code below:

# Create a vector of 0
print(tf.zeros(10))

Output

Tensor("zeros:0", shape=(10,), dtype=float32)

The property works for a matrix as well. Here, you create a 10×10 matrix filled with 1.

# Create a vector of 1
print(tf.ones([10, 10]))

Output

Tensor("ones:0", shape=(10, 10), dtype=float32)

You can use the shape of a given matrix to make a vector of ones. The matrix m_shape has 3×2 dimensions, so the following code creates a tensor with 3 rows filled with ones:

# Create a vector of ones with the same number of rows as m_shape
print(tf.ones(m_shape.shape[0]))

Output

Tensor("ones_1:0", shape=(3,), dtype=float32)

If you pass the value 1 into the bracket, you construct a vector of ones equal to the number of columns in the matrix m_shape.

# Create a vector of ones with the same number of column as m_shape
print(tf.ones(m_shape.shape[1]))

Output

Tensor("ones_2:0", shape=(2,), dtype=float32)

Finally, you can create a 3×2 matrix filled entirely with ones.

print(tf.ones(m_shape.shape))

Output

Tensor("ones_3:0", shape=(3, 2), dtype=float32)

Type of Data

The second property of a tensor is the type of data. A tensor can hold only one type of data at a time, and you can read that type with the dtype property.

print(m_shape.dtype)

Output

<dtype: &#x27;int32'>

On some occasions, you want to change the type of data. In TensorFlow, that is possible with the tf.cast method.

Example

Below, a float tensor is converted to an integer with the cast method.

# Change type of data
type_float = tf.constant(3.123456789, tf.float32)
type_int = tf.cast(type_float, dtype=tf.int32)
print(type_float.dtype)
print(type_int.dtype)

Output

<dtype: &#x27;float32'>
<dtype: &#x27;int32'>

TensorFlow chooses the type of data automatically when the argument is not specified during the creation of the tensor. TensorFlow guesses the most likely type of data: if you pass text, for example, it infers a string.

Creating Operator

Some Useful TensorFlow Operators

You know how to create a tensor with TensorFlow. It is time to learn how to perform mathematical operations.

TensorFlow contains all the basic operations. You can begin with a simple one and use a TensorFlow method to compute the square root of a number. This operation is straightforward because only one argument is required to construct the tensor.

The square root of a number is constructed with tf.sqrt(x), with x as a floating-point number.

x = tf.constant([2.0], dtype = tf.float32)
print(tf.sqrt(x))

Output

Tensor("Sqrt:0", shape=(1,), dtype=float32)

Note: The output returned a tensor object and not the square root of 2. In this example, you print the definition of the tensor and not the actual evaluation of the operation. In the Session section below, you will learn how TensorFlow executes the operations.

Following is a list of commonly used operations. The idea is the same: each operation requires one or more arguments.

• tf.add(a, b)
• tf.subtract(a, b)
• tf.multiply(a, b)
• tf.divide(a, b)
• tf.pow(a, b)
• tf.exp(a)
• tf.sqrt(a)

Example

# Add
tensor_a = tf.constant([[1,2]], dtype = tf.int32)
tensor_b = tf.constant([[3, 4]], dtype = tf.int32)

tensor_add = tf.add(tensor_a, tensor_b)
print(tensor_add)

Output

Tensor("Add:0", shape=(1, 2), dtype=int32)

Code Explanation

Create two tensors:

• one tensor with 1 and 2
• one tensor with 3 and 4

You add up both tensors.

Notice that both tensors need to have the same shape. You can also execute a multiplication over the two tensors.

# Multiply
tensor_multiply = tf.multiply(tensor_a, tensor_b)
print(tensor_multiply)

Output

Tensor("Mul:0", shape=(1, 2), dtype=int32)

Variables

So far, you have only created constant tensors, which is of limited use on its own. Data always arrives with different values, and to capture that you can use the Variable class. A variable represents a node whose values change over time.

To create a variable, you can use the tf.get_variable() method.

tf.get_variable(name = "", values, dtype, initializer)
argument
- `name = ""`: Name of the variable
- `values`: Dimension of the tensor
- `dtype`: Type of data. Optional
- `initializer`: How to initialize the tensor. Optional
If initializer is specified, there is no need to include the `values` as the shape of `initializer` is used.

For instance, the code below creates a two-dimensional variable with two random values. By default, TensorFlow returns a random value. You name the variable var.

# Create a Variable
## Create 2 Randomized values
var = tf.get_variable("var", [1, 2])
print(var.shape)

Output

(1, 2)

In this example, you create a variable with one row and two columns, so you pass [1, 2] as the dimension of the variable.

The initial values of a tensor can also be zero. When you train a model, for instance, you need initial values before computing the weight of the features. Below, you set those initial values to zero.

var_init_1 = tf.get_variable("var_init_1", [1, 2], dtype=tf.int32,  initializer=tf.zeros_initializer)
print(var_init_1.shape)

Output

(1, 2)

You can also pass the values of a constant tensor into a variable. You create a constant tensor with the method tf.constant() and use that tensor to initialize the variable.

The first values of the variable are 10, 20, 30 and 40. The new tensor has a shape of 2×2.

# Create a 2x2 matrix
tensor_const = tf.constant([[10, 20],
[30, 40]])
# Initialize the first value of the tensor equals to tensor_const
var_init_2 = tf.get_variable("var_init_2", dtype=tf.int32,  initializer=tensor_const)
print(var_init_2.shape)

Output

(2, 2)

Placeholder

A placeholder has the purpose of feeding the tensor. A placeholder initializes the data that will flow inside the tensors. To supply a placeholder, you use the feed_dict argument, and the placeholder is fed only within a session.

In the next example, you will see how to create a placeholder with the method tf.placeholder. In the Session section, you will learn to feed a placeholder with an actual tensor value.

The syntax is:

tf.placeholder(dtype,shape=None,name=None )
arguments:
- `dtype`: Type of data
- `shape`: dimension of the placeholder. Optional. By default, shape of the data
- `name`: Name of the placeholder. Optional			
data_placeholder_a = tf.placeholder(tf.float32, name = "data_placeholder_a")
print(data_placeholder_a)

Output

Tensor("data_placeholder_a:0", dtype=float32)

Session

TensorFlow works around 3 main components:

• Graph
• Tensor
• Session

Graphs and sessions are independent. You can run a session and keep the values for further computations later.

In the example below, you will:

• Create two tensors
• Create an operation
• Open a session
• Print the result

Step 1) You create two tensors x and y

## Create, run  and evaluate a session
x = tf.constant([2])
y = tf.constant([4])

Step 2) You create the operator by multiplying x and y

## Create operator
multiply = tf.multiply(x, y)

Step 3) You open a session. All the computations happen within the session, and when you are done you need to close it.

## Create a session to run the code
sess = tf.Session()
result_1 = sess.run(multiply)
print(result_1)
sess.close()

Output

[8]

Code explanation

• tf.Session(): Open a session. All the operations flow within the session
• run(multiply): execute the operation created in step 2
• print(result_1): Finally, you can print the result
• close(): Close the session

The result shows 8, which is the multiplication of x and y.

Another way to create a session is inside a block. The advantage is that it closes the session automatically.

with tf.Session() as sess:
    result_2 = multiply.eval()
print(result_2)

Output

[8]

In the context of a session, you can use the eval() method to execute the operation. It is equivalent to run() and makes the code more readable.

You can create a session and see the values inside the tensors you created so far.

## Check the tensors created before
sess = tf.Session()
print(sess.run(r1))
print(sess.run(r2_matrix))
print(sess.run(r3_matrix))

Output

1
[[1 2] 
 [3 4]]
[[[1 2]  
  [3 4]  
  [5 6]]]

Variables are empty by default, even after you create the tensor. You need to initialize a variable before you can use it. The object tf.global_variables_initializer() is called to initialize the values of every variable at once, which is helpful before you train a model.

You can check the values of the variables you created before. Note that you need to use run to evaluate the tensor.

sess.run(tf.global_variables_initializer())
print(sess.run(var))
print(sess.run(var_init_1))
print(sess.run(var_init_2))

Output

[[-0.05356491  0.75867283]]
[[0 0]]
[[10 20] 
 [30 40]]

You can use the placeholder you created before and feed it with an actual value. You need to pass the data into the feed_dict argument.

For example, you will take the power of 2 of the placeholder data_placeholder_a.

import numpy as np
power_a = tf.pow(data_placeholder_a, 2)
with tf.Session() as sess:
    data = np.random.rand(1, 10)
    print(sess.run(power_a, feed_dict={data_placeholder_a: data}))  # Will succeed.

Code Explanation

• import numpy as np: Import the NumPy library to create the data
• tf.pow(data_placeholder_a, 2): Create the ops
• np.random.rand(1, 10): Create a random array of data
• feed_dict={data_placeholder_a: data}: Feed the placeholder with data

Output

[[0.05478134 0.27213147 0.8803037  0.0398424  0.21172127 0.01444725  0.02584014 0.3763949  0.66022706 0.7565559 ]]

Graph

TensorFlow renders every operation with a dataflow scheme. The dataflow graph was designed to expose the data dependencies between individual operations. A mathematical formula or an algorithm is made of a number of successive operations, and a graph is a convenient way to visualize how those computations are coordinated.

The graph shows nodes and edges. A node is the representation of an operation, i.e., the unit of computation. An edge is the tensor: it can produce a new tensor or consume the input data, depending on the dependencies between the individual operations.

The structure of the graph connects the operations (i.e., the nodes) together and shows how each one is fed. Note that the graph does not display the output of the operations; it only helps to visualize the connections between them.

Let us see an example. Imagine you want to evaluate the following function:

TensorFlow builds a graph to execute the function. The graph looks like this:

You can easily see the path that the tensors will take to reach the final destination.

For instance, the add node cannot run before both Mul and Pow have produced their results. The graph shows that TensorFlow will:

1. compute x·z in the Mul node and x² in the Pow node
2. add those two results together in the add node
3. add z to that sum in the add_1 node
4. add the constant 5 in the add_2 node

x = tf.get_variable("x", dtype=tf.int32,  initializer=tf.constant([5]))
z = tf.get_variable("z", dtype=tf.int32,  initializer=tf.constant([6]))
c = tf.constant([5], name =	"constant")
square = tf.constant([2], name =	"square")
f = tf.multiply(x, z) + tf.pow(x, square) + z + c

Code Explanation

• x: Initialize a variable called x with a constant value of 5
• z: Initialize a variable called z with a constant value of 6
• c: Initialize a constant tensor called c with a constant value of 5
• square: Initialize a constant tensor called square with a constant value of 2
• f: Construct the operator

In this example, the values of the variables are kept fixed. The constant tensor c is the constant parameter in the function f and takes a fixed value of 5; in the graph you can see this parameter in the tensor called constant.

A constant tensor was also created for the power in the operator tf.pow(). It is not necessary, but it means the exponent appears in the graph as the circle called square.

From the graph, you can follow what happens to the tensors and why the function returns 66: 5 × 6 plus 5² plus 6 plus 5 equals 66.

The code below evaluates the function inside a session.

init = tf.global_variables_initializer() # prepare to initialize all variables
with tf.Session() as sess:
    init.run() # Initialize x and y
    function_result = f.eval()
print(function_result)

Output

[66]

TensorFlow Basics Quick Reference

The four tables below collect every command used in this tutorial so you can look one up without scrolling back through the examples.

Create a constant tensor

Create an operator

Create a variable tensor

Open a session

FAQs