Introduction to TensorFlow 2

Introduction

TensorFlow began its life in 2011 as DisBelief, an internal, closed source project at Google. DisBelief was a machine learning system that employed deep learning neural networks. This system morphed into TensorFlow, which was released to the developer community under an Apache 2.0 open source license, on November 9, 2015. Version 1.0.0 made its appearance on February 11, 2017.
At the time of writing this blog post, the most recent version is TensorFlow 2.1.0.

What is Tensor? 

A tensor is a generalization of vectors and matrices to possibly higher dimensions. The rank of a tensor is the number of indices it takes to uniquely specify each element of that tensor. A scalar (a simple number) is a tensor of rank 0, a vector is a tensor of rank 1, a matrix is a tensor of rank 2, and a 3-dimensional array is a tensor of rank 3. A tensor has a datatype and a shape (all of the data items in a tensor must have the same type).

 An example of a 3-dimensional tensor (that is, rank 3) is an image where the dimensions are an example within—height, width, and color channel.

For example: 

tensor_3d = tf.zeros([28,28,3])#  example-within height by width by color

The main area of TensorFlow is research and development applications of Deep Neural Networks (DNN), where it has been used in diverse areas such as voice and sound recognition, text-based applications such as language translators; image recognition, time series applications, recommendation systems, etc.

In this blog post, we will discuss the following:

• The modern TensorFlow ecosystem
• Installing TensorFlow
• Useful TensorFlow operations

The modern TensorFlow ecosystem

Eager execution: The first incarnation of TensorFlow involved constructing a computational graph made up of operations and tensors, which had to be subsequently evaluated in session. This is still a common way to write TensorFlow programs. However, eager execution, available from release 1.5 onward in research form and baked into TensorFlow proper from release 1.7, involves the immediate evaluation of operations, with the consequence that tensors can be treated like NumPy arrays (this is known as imperative programming).

Google says that eager execution is the preferred method for research and development but that computational graphs are to be preferred for serving TensorFlow production applications.

tf.data is an API that allows you to build complicated data input pipelines from simpler, reusable parts. The highest level abstraction is Dataset, which comprises both elements of nested structures of tensors and a plan of transformations that are to act on those elements. There are classes for the following:

• There's Dataset consisting of fixed length record sets from at least one binary file (FixedLengthRecordDataset)
• There's Dataset consisting of records from at least one TFRecord file (TFRecordDataset)
•  There's Dataset consisting of records that are lines from at least one text file(TFRecordDataset)
• There is also a class that represents the state of iterating through Dataset(tf.data.Iterator)

Estimator, which is a high-level API that allows you to build greatly simplified machine-learning programs. Estimators take care of training, evaluation, prediction, and exports for serving.

TensorFlow.js is a collection of APIs that allow you to build and train models using either the low-level JavaScript linear algebra library or the high-level layers API. Hence, models can be trained and run in a browser.

TensorFlow Lite is a lightweight version of TensorFlow for mobile and embedded devices. It consists of a runtime interpreter and a set of utilities. The idea is that you train a model on a higher-powered machine and then convert your model into the .tflite format using the utilities. You then load the model into your device of choice. At the time of writing, TensorFlow Lite is supported on Android and iOS with a C++ API and has a Java wrapper for Android. If The android device supports the Android Neural Networks (ANN) API for hardware acceleration, then the interpreter will use this, or else it will default to the CPU for execution.

TensorFlow Hub is a library designed to foster the publication, discovery, and use of reusable modules of machine learning models. In this context, a module is a self-contained piece of a TensorFlow graph together with its weights and other assets. The module can be reused in different tasks in a method known as transfer learning. The idea is that you train a model on a large dataset and then re-purpose the appropriate module for your different but related tasks. This approach brings a number of advantages—you can train a model with a smaller dataset, you can improve generalization, and you can significantly speed up training.

TensorFlow Extended (TFX) is a TensorFlow-based general-purpose machine learning platform. Libraries released to open source to date include TensorFlow Transform, TensorFlow Model Analysis, and TensorFlow Serving.

tf.keras is a high-level neural networks API, written in Python, that interfaces to TensorFlow(and various other tensor tools).tf.keras supports fast prototyping and is user-friendly, modular, and extensible. It supports both convolutional and recurrent networks and will run on CPUs and GPUs. Keras is the API of choice for developing in TensorFlow 2.

TensorBoard is a suite of visualization tools supporting the understanding, debugging and optimizing of TensorFlow programs. It is compatible with both eager and graph execution environments. You can use TensorBoard to visualize various metrics of your model during training.

Installing TensorFlow

 

The best programming support for TensorFlow is provided for Python.

There is a wealth of information on the web for installing TensorFlow for Python.

It is standard practice, also recommended by Google, to install TensorFlow in a virtual environment, that is, an environment that isolates a set of APIs and code from other APIs and code and from the system-wide environment.

There are two distinct versions of TensorFlow—one for execution on a CPU and another for execution on a GPU. This last requires that the numerical libraries CUDA and CuDNN are installed. Tensorflow will default to GPU execution where possible. See https://www.tensorflow.org/guide/gpu.

Let's write a simple Tensorflow 2.0 model

import tensorflow as tf

W = tf.Variable(tf.ones(shape=(3,3)), name="W")

b = tf.Variable(tf.zeros(shape=(3)), name="b")

@tf.function

def model(x):

return W * x + b

out_a = model([1,0,0])

print(out_a)

 Output:

tf.Tensor(
[[1. 0. 0.]
 [1. 0. 0.]
 [1. 0. 0.]], shape=(3, 3), dtype=float32)

Basic TensorFlow operations

Importing TensorFlow

 

import tensorflow as tf

print("TensorFlow version: {}".format(tf.__version__))

print("Eager execution is: {}".format(tf.executing_eagerly()))

print("Keras version: {}".format(tf.keras.__version__))

 Output:

TensorFlow version: 2.1.0
Eager execution is: True
Keras version: 2.2.4-tf

Declaring eager variables

The way to declare a TensorFlow eager variable is as follows

# Declaring eager variables

t0=34

t1=tf.Variable(35)

t2=tf.Variable([[1,2,3],[1,2,3]])

t0,t1,t2

Output:

(34,
 <tf.Variable 'Variable:0' shape=() dtype=int32, numpy=35>,
 <tf.Variable 'Variable:0' shape=(2, 3) dtype=int32, numpy=
 array([[1, 2, 3],
        [1, 2, 3]], dtype=int32)>)

TensorFlow will infer the datatype, defaulting to `tf.float32` for floats and `tf.int32` for integers.

Alternatively, the datatype can be explicitly specified, as here:

t64=tf.Variable(56,dtype=tf.int64)

t64

Output: 

<tf.Variable 'Variable:0' shape=() dtype=int64, numpy=56>

 To reassign a variable, use var.assign()

f1 = tf.Variable(59.)

f1

# <tf.Variable 'Variable:0' shape=() dtype=float32, numpy=59.0>

f1.assign(67.)

f1

# <tf.Variable 'Variable:0' shape=() dtype=float32, numpy=67.0>