Deep Learning- A way To Better Future
Deep learning is actually a sub-field of Machine Learning and further Artificial Intelligence. It imitates the workings of the human brain in processing data and creating patterns for use in decision making. Deep learning is a subset of Machine Learning in artificial intelligence that has networks capable of learning unsupervised from data that is unstructured or unlabeled. Also known as deep neural learning or deep neural network.
Key Points:
Deep learning AI is able to learn without human supervision, drawing from data that is both unstructured and unlabeled and provide a solution without the working of Human.
Deep learning is an AI function that mimics the workings of the human brain in processing data for use in detecting objects, recognizing speech, translating languages, and making decisions.
History Of Deep Learning
The first general working learning algorithm for supervised, deep, feed forward, Multilayer Perceptrons was published by Alexey Ivakhnenko and Lapa in 1967.
The term Deep Learning was introduced to the machine learning community by Rina Dechter in 1986, and to artificial neural networks by Igor Aizenberg and colleagues in 2000, in the context of Boolean threshold neurons.
In 1989, Yann LeCun applied the standard back-propagation algorithm, which had been around as the reverse mode of automatic differentiation since 1970, to a deep neural network with the purpose of recognizing handwritten ZIP codes on mail.
Scientist Who did Remarkable inventions in deep learning:
- Frank Rosenblatt-Perceptron
2.Yann LeCun- optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets.
3.Geoff Hinton- backpropagation algorithm for training multi-layer neural networks
Machine Learning vs Deep Learning
Now we will discuss what are the basic differences of Machine Learning and Deep Learning on the basis of following parameters
Data dependencies
The most important difference between deep learning and traditional machine learning is its performance as the scale of data increases. When the data is small, deep learning algorithms don’t perform that well. This is because deep learning algorithms need a large amount of data to understand it perfectly. On the other hand, traditional machine learning algorithms with their handcrafted rules prevail in this scenario. Below image summarizes this fact.
Hardware dependencies
Deep learning algorithms heavily depend on high-end machines, contrary to traditional machine learning algorithms, which can work on low-end machines. This is because the requirements of deep learning algorithm include GPUs which are an integral part of its working. Deep learning algorithms inherently do a large amount of matrix multiplication operations. These operations can be efficiently optimized using a GPU because GPU is built for this purpose.
Execution time
Usually, a deep learning algorithm takes a long time to train. This is because there are so many parameters in a deep learning algorithm that training them takes longer than usual. State of the art deep learning algorithm ResNet takes about two weeks to train completely from scratch. Whereas machine learning comparatively takes much less time to train, ranging from a few seconds to a few hours.
This is turn is completely reversed on testing time. At test time, deep learning algorithm takes much less time to run. Whereas, if you compare it with k-nearest neighbors (a type of machine learning algorithm), test time increases on increasing the size of data. Although this is not applicable on all machine learning algorithms, as some of them have small testing times too.
Interpret-ability
We have interpret-ability as a factor for comparison of machine learning and deep learning. This factor is the main reason deep learning is still thought 10 times before its use in industry.
Let’s take an example. Suppose we use deep learning to give automated scoring to essays. The performance it gives in scoring is quite excellent and is near human performance. But there’s is an issue. It does not reveal why it has given that score. Indeed mathematically you can find out which nodes of a deep neural network were activated, but we don’t know what there neurons were supposed to model and what these layers of neurons were doing collectively. So we fail to interpret the results.
On the other hand, machine learning algorithms like decision trees give us crisp rules as to why it chose what it chose, so it is particularly easy to interpret the reasoning behind it. Therefore, algorithms like decision trees and linear/logistic regression are primarily used in industry for interpret-ability.
Feature engineering
In Machine learning, most of the applied features need to be identified by an expert and then hand-coded as per the domain and data type.
For example, features can be pixel values, shape, textures, position and orientation. The performance of most of the Machine Learning algorithm depends on how accurately the features are identified and extracted.
Deep learning algorithms try to learn high-level features from data. This is a very distinctive part of Deep Learning and a major step ahead of traditional Machine Learning. Therefore, deep learning reduces the task of developing new feature extractor for every problem. Like, Convolutional NN will try to learn low-level features such as edges and lines in early layers then parts of faces of people and then high-level representation of a face.
Perceptron
A perceptron is a simple binary classification algorithm, proposed by Cornell scientist Frank Rosenblatt. It helps to divide a set of input signals into two parts — “yes” and “no”. But unlike many other classification algorithms, the perceptron was modeled after the essential unit of the human brain — the neuron and has an uncanny ability to learn and solve complex problems.
perceptron is a very simple learning machine. It can take in a few inputs, each of which has a weight to signify how important it is, and generate an output decision of “0” or “1”. However, when combined with many other perceptrons, it forms an artificial neural network. A neural network can, theoretically, answer any question, given enough training data and computing power.
Multi-layer Perceptron
A multilayer perceptron (MLP) is a perceptron that teams up with additional perceptrons, stacked in several layers, to solve complex problems. The diagram below shows an MLP with three layers. Each perceptron in the first layer on the left (the input layer), sends outputs to all the perceptrons in the second layer (the hidden layer), and all perceptrons in the second layer send outputs to the final layer on the right (the output layer)
sends multiple signals, one signal going to each perceptron in the next layer. For each signal, the perceptron uses different weights. In the diagram above, every line going from a perceptron in one layer to the next layer represents a different output. Each layer can have a large number of perceptrons, and there can be multiple layers, so the multilayer perceptron can quickly become a very complex system. The multilayer perceptron has another, more common name — a neural network. A three-layer MLP, like the diagram above, is called a Non-Deep or Shallow Neural Network. An MLP with four or more layers is called a Deep Neural Network. One difference between an MLP and a neural network is that in the classic perceptron, the decision function is a step function and the output is binary. In neural networks that evolved from MLPs, other activation functions can be used which result in outputs of real values, usually between 0 and 1 or between -1 and 1. This allows for probability-based predictions or classification of items into multiple labels.
Structure of a Perceptron
The Perceptron Learning Process:
- Takes the inputs, multiplies them by their weights, and computes their sum
- Adds a bias factor, the number 1 multiplied by a weight
- Feeds the sum through the activation function
- The result is the perceptron output
Types Of Artificial Neural Network
Convolutional Neural Network:
Convolutional neural networks are similar to feed forward neural networks, where the neurons have learnable weights and biases. Its application has been in signal and image processing which takes over OpenCV in the field of computer vision.
Below is a representation of a ConvNet, in this neural network, the input features are taken in batch-wise like a filter. This will help the network to remember the images in parts and can compute the operations. These computations involve the conversion of the image from RGB or HSI scale to the Gray-scale. Once we have this, the changes in the pixel value will help to detect the edges and images can be classified into different categories.
Recurrent Neural Network(RNN)
The Recurrent Neural Network works on the principle of saving the output of a layer and feeding this back to the input to help in predicting the outcome of the layer.
Here, the first layer is formed similar to the feed forward neural network with the product of the sum of the weights and the features. The recurrent neural network process starts once this is computed, this means that from one time step to the next each neuron will remember some information it had in the previous time-step.
This makes each neuron act like a memory cell in performing computations. In this process, we need to let the neural network to work on the front propagation and remember what information it needs for later use.
Auto Encoder:
An auto-encoder is a type of artificial neural network used to learn efficient data — coding in an unsupervised manner.The aim of an auto-encoder is to learn a representation(encoding) for a set of data, typically for dimension reduction, by training the network to ignore signal “noise”. Along with the reduction side, a reconstructing side is learnt, where the auto-encoder tries to generate from the reduced encoding a representation as close as possible to its original input, hence its name. Several variants exist to the basic model, with the aim of forcing the learned representations of the input to assume useful properties.
Generative Adversarial network:
A generative adversarial network (GAN) is a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in 2014.Two neural networks contest with each other in a game (in the sense of game theory, often but not always in the form of a zero-sum game). Given a training set, this technique learns to generate new data with the same statistics as the training set. For example, a GAN trained on photographs can generate new photographs that look at least superficially authentic to human observers, having many realistic characteristics. Though originally proposed as a form of generative model for unsupervised learning, GANs have also proven useful for semi-supervised learning, fully supervised learning, and reinforcement learning.
Application Of Deep learning:
Top Applications of Deep Learning Across Industries are:
Self Driving Cars
News Aggregation and Fraud News Detection
Natural Language Processing
Virtual Assistants
Entertainment
Visual Recognition
Fraud Detection
Healthcare
Personalisations
Detecting Developmental Delay in Children
Colourisation of Black and White images
Adding sounds to silent movies
Automatic Machine Translation
Automatic Handwriting Generation
Automatic Game Playing
Language Translations
Pixel Restoration
Photo Descriptions
Demographic and Election Predictions
Deep Dreaming
References:
https://en.wikipedia.org/wiki/Deep_learning
https://analyticsindiamag.com/6-types-of-artificial-neural-networks
