Multi-layer Perceptron in TensorFlow

Multi-Layer perceptron defines the most complex architecture of artificial neural networks. It is substantially formed from multiple layers of the perceptron. TensorFlow is a very popular deep learning framework released by, and this notebook will guide to build a neural network with this library. If we want to understand what is a Multi-layer perceptron, we have to develop a multi-layer perceptron from scratch using Numpy.

The pictorial representation of multi-layer perceptron learning is as shown below-

MLP networks are used for supervised learning format. A typical learning algorithm for MLP networks is also called back propagation’s algorithm.

A multilayer perceptron (MLP) is a feed forward artificial neural network that generates a set of outputs from a set of inputs. An MLP is characterized by several layers of input nodes connected as a directed graph between the input nodes connected as a directed graph between the input and output layers. MLP uses backpropagation for training the network. MLP is a deep learning method.

Now, we are focusing on the implementation with MLP for an image classification problem.

  # Import MINST data   from tensorflow.examples.tutorials.mnist import input_data   mnist = input_data.read_data_sets(“/tmp/data/”, one_hot = True)     import tensorflow as tf   import matplotlib.pyplot as plt     # Parameters   learning_rate = 0.001   training_epochs = 20   batch_size = 100   display_step = 1     # Network Parameters   n_hidden_1 = 256     # 1st layer num features  n_hidden_2 = 256 # 2nd layer num features   n_input = 784 # MNIST data input (img shape: 28*28) n_classes = 10   # MNIST total classes (0-9 digits)     # tf Graph input   x = tf.placeholder(“float”, [None, n_input])   y = tf.placeholder(“float”, [None, n_classes])     # weights layer 1   h = tf.Variable(tf.random_normal([n_input, n_hidden_1])) # bias layer 1   bias_layer_1 = tf.Variable(tf.random_normal([n_hidden_1]))   # layer 1 layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, h), bias_layer_1))     # weights layer 2   w = tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2]))     # bias layer 2   bias_layer_2 = tf.Variable(tf.random_normal([n_hidden_2]))     # layer 2   layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, w), bias_layer_2))     # weights output layer   output = tf.Variable(tf.random_normal([n_hidden_2, n_classes]))     # biar output layer   bias_output = tf.Variable(tf.random_normal([n_classes])) # output layer   output_layer = tf.matmul(layer_2, output) + bias_output    # cost function   cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(     logits = output_layer, labels = y))     #cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(output_layer, y))   # optimizer   optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost)     # optimizer = tf.train.GradientDescentOptimizer(     learning_rate = learning_rate).minimize(cost)     # Plot settings   avg_set = []   epoch_set = []     # Initializing the variables   init = tf.global_variables_initializer()     # Launch the graph   with tf.Session() as sess:      sess.run(init)           # Training cycle     for epoch in range(training_epochs):         avg_cost = 0.         total_batch = int(mnist.train.num_examples / batch_size)                 # Loop over all batches         for i in range(total_batch):            batch_xs, batch_ys = mnist.train.next_batch(batch_size)            # Fit training using batch data sess.run(optimizer, feed_dict = {              x: batch_xs, y: batch_ys})            # Compute average loss            avg_cost += sess.run(cost, feed_dict = {x: batch_xs, y: batch_ys}) / total_batch        # Display logs per epoch step         if epoch % display_step == 0:            print            Epoch:”, ‘%04d’ % (epoch + 1), “cost=”, “{:.9f}”.format(avg_cost)        avg_set.append(avg_cost)         epoch_set.append(epoch + 1)     print      “Training phase finished”           plt.plot(epoch_set, avg_set, ‘o’, label = ‘MLP Training phase’)      plt.ylabel(‘cost’)      plt.xlabel(‘epoch’)      plt.legend()      plt.show()           # Test model      correct_prediction = tf.equal(tf.argmax(output_layer, 1), tf.argmax(y, 1))           # Calculate accuracy      accuracy = tf.reduce_mean(tf.cast(correct_prediction, “float”))      print      “Model Accuracy:”, accuracy.eval({x: mnist.test.images, y: mnist.test.labels})  

The above line of codes generating the following output-

Multi-layer Perceptron in TensorFlow

Creating an interactive section

We have two basic options when using TensorFlow to run our code:

Build graphs and run sessions [Do all the set-up and then execute a session to implement a session to evaluate tensors and run operations].
Create our coding and run on the fly.

For this first part, we will use the interactive session that is more suitable for an environment like Jupiter notebook.

Creating placeholders

It’s a best practice to create placeholder before variable assignments when using TensorFlow. Here we’ll create placeholders to inputs (“Xs”) and outputs (“Ys”).

Placeholder “X”: Represent the ‘space’ allocated input or the images.

Each input has 784 pixels distributed by a 28 width x 28 height matrix.
The ‘shape’ argument defines the tensor size by its dimensions.

Next TopicWhat is Machine Learning

advantages of tensorflow disadvantages of tensorflow installation of tensorflow through conda installation of tensorflow through pip pros and cons of tensorflow tensorflow tensorflow architecture tensorflow installation tensorflow introduction tensorflow overview tensorflow tutorial what is tensorflow

Multi-layer Perceptron in TensorFlow

Multi-layer Perceptron in TensorFlow

Creating an interactive section

Creating placeholders

Tableau Tutorial | Data Visualization Tool

Exclude Include Test Cases in TestNG

You may also like