Deep Learning Tutorial for Beginners: Neural Network Basics
Evelyn Clarke
⚡ Smart Summary
Deep Learning is a branch of machine learning built on artificial neural networks that stack many hidden layers, so each layer learns a richer feature of the data than the layer before it.
What is Deep Learning?
Deep Learning is computer software that mimics the network of neurons in a brain. It is a subset of machine learning based on artificial neural networks with representation learning. It is called deep learning because it makes use of deep neural networks. This learning can be supervised, semi-supervised or unsupervised.
Deep learning algorithms are constructed with connected layers, as the diagram below shows.
- The first layer is called the Input Layer
- The last layer is called the Output Layer
- All layers in between are called Hidden Layers. The word deep means the network joins neurons across more than two layers.
Each hidden layer is composed of neurons. The neurons are connected to each other. A neuron processes the input signal it receives and then propagates the result to the layer above it. The strength of the signal passed to a neuron in the next layer depends on the weight, the bias and the activation function.
The network consumes large amounts of input data and operates on it through multiple layers, so the network can learn increasingly complex features of the data at each layer.
Deep Learning Process
A deep neural network provides state-of-the-art accuracy in many tasks, from object detection to speech recognition. Such networks can learn automatically, without predefined knowledge explicitly coded by the programmers. In practice, a deep learning project moves through five stages:
- Understand the problem
- Identify the data
- Select a deep learning algorithm
- Train the model
- Test the model
To grasp the idea of deep learning, imagine a family with an infant and parents. The toddler points at objects with his little finger and always says the word ‘cat.’ As his parents are concerned about his education, they keep telling him ‘Yes, that is a cat’ or ‘No, that is not a cat.’ The infant persists in pointing at objects but becomes more accurate with ‘cats.’ The little kid, deep down, does not know why he can say it is a cat or not. He has simply learned to organise the features that make up a cat into a hierarchy, looking at the pet overall and then focusing on details such as the tail or the nose before making up his mind.
A neural network works in much the same way. Each layer represents a deeper level of knowledge, that is, a hierarchy of knowledge. A neural network with four layers will learn more complex features than one with two layers.
The learning occurs in two phases:
- First phase: applying a nonlinear transformation of the input and creating a statistical model as output.
- Second phase: improving the model with a mathematical method based on derivatives, which is how backpropagation adjusts the weights.
The neural network repeats these two phases hundreds to thousands of times until it has reached a tolerable level of accuracy. Each repetition of the two phases is called an iteration.
To give a deep learning example, take a look at the animation below, in which the model is trying to learn how to dance. After 10 minutes of training, the model does not know how to dance, and its output looks like a scribble.
After 48 hours of learning, the computer masters the art of dancing, as the second animation shows.
Classification of Neural Networks
Networks are first grouped by hidden-layer count.
Shallow neural network: The shallow neural network has only one hidden layer between the input and the output.
Deep neural network: Deep neural networks have more than one hidden layer. For instance, the GoogLeNet model for image recognition counts 22 layers.
Today, deep learning appears in driverless cars, mobile phones, the Google search engine, fraud detection and television.
Types of Deep Learning Networks
Several architectures have become standard, each shaped by the kind of data it handles. The chart below, published by the Asimov Institute, maps the wider family.
Feed-forward neural networks
This is the simplest type of artificial neural network. With this type of architecture, information flows in only one direction, forward. It means the information flow starts at the input layer, goes to the “hidden” layers, and ends at the output layer. The network does not have a loop, and information stops at the output layer.
Recurrent neural networks (RNNs)
An RNN is a multi-layered neural network that can store information in context nodes, allowing it to learn data sequences and output a number or another sequence. In simple words, it is an artificial neural network whose connections between neurons include loops. RNNs are well suited to processing sequences of inputs, because the output is fed back into the network as memory.
For example, if the task is to predict the next word in the sentence “Do you want a …?”, the network proceeds like this:
- The RNN neurons receive a signal that points to the start of the sentence.
- The network receives the word “Do” as an input and produces a vector of numbers. This vector is fed back to the neuron to provide a memory to the network. This stage helps the network remember that it received “Do” and that it received it in the first position.
- The network proceeds similarly to the next words. It takes the word “you” and then “want.” The state of the neurons is updated on receiving each word.
- The final stage occurs after receiving the word “a.” The neural network provides a probability for each English word that could complete the sentence. A well-trained RNN probably assigns a high probability to “café,” “drink,” “burger,” and similar words.
Common uses of RNN
- Help securities traders to generate analytic reports
- Detect abnormalities in financial statements
- Detect fraudulent credit-card transactions
- Provide a caption for images
- Power chatbots
- The standard uses of RNN occur when practitioners are working with time-series data or sequences, for example audio recordings or text.
Convolutional neural networks (CNN)
A CNN is a multi-layered neural network with a unique architecture designed to extract increasingly complex features of the data at each layer to determine the output. CNNs are well suited to perceptual tasks.
CNN is mostly used when there is an unstructured data set, such as images, and practitioners need to extract information from it.
For instance, if the task is to predict an image caption:
- The CNN receives an image of, let us say, a cat. In computing terms this image is a collection of pixels, generally one layer for a greyscale picture and three layers for a colour picture.
- During the feature learning stage, that is, the hidden layers, the network identifies unique features, for instance the tail of the cat or the ear.
- Once the network has thoroughly learned how to recognise a picture, it can provide a probability for each image class it knows. The label with the highest probability becomes the prediction of the network.
Reinforcement Learning
Reinforcement learning is a subfield of machine learning in which systems are trained by receiving virtual “rewards” or “punishments,” essentially learning by trial and error. It is a learning paradigm rather than a network shape, but its modern algorithms are built on deep networks. Google’s DeepMind used reinforcement learning to beat a human champion at Go. Reinforcement learning is also used in video games to improve the gaming experience by providing smarter bots.
Some of the most famous algorithms are:
- Q-learning
- Deep Q network
- State-Action-Reward-State-Action (SARSA)
- Deep Deterministic Policy Gradient (DDPG)
The table below summarises when each architecture fits.
| Architecture | Data it suits | Distinguishing trait | Typical task |
|---|---|---|---|
| Feed-forward | Fixed-length tabular input | No loops; information moves forward only | Scoring and simple classification |
| Recurrent (RNN) | Sequences and time series | Context nodes give it memory | Text prediction, audio, forecasting |
| Convolutional (CNN) | Grid-like unstructured data | Convolution and pooling extract features | Image classification and captioning |
| Reinforcement learning | An environment, not a dataset | Learns from rewards and punishments | Game agents, robotics, control |
Examples of Deep Learning Applications
These architectures are already in production across very different industries:
AI in Finance
The financial technology sector has already started using AI to save time, reduce costs, and add value. Deep learning is changing the lending industry through more robust credit scoring. Credit decision-makers can use AI for credit lending applications to achieve faster, more accurate risk assessment, using machine intelligence to factor in the character and capacity of applicants.
Underwrite is a fintech company providing an AI solution for credit decision-makers. underwrite.ai uses AI to detect which applicant is more likely to pay back a loan, an approach the company says outperforms traditional scorecards.
AI in HR
Under Armour, a sportswear company, revolutionised hiring and modernised the candidate experience with the help of AI. Under Armour faced a surge of interest back in 2012 and was receiving, on average, more than 30,000 applications a month. Reading all of those applications and then starting the screening and interview process was taking too long. The lengthy process to get people hired and on-boarded affected Under Armour’s ability to have its retail stores fully staffed, ramped and ready to operate.
At that time, Under Armour had all of the ‘must have’ HR technology in place, such as transactional solutions for sourcing, applying, tracking and onboarding, but those tools were not enough on their own. Under Armour chose HireVue, an AI provider of HR solutions, for both on-demand and live interviews. The results were striking: the company reported cutting time to fill by 35% while raising the quality of the staff it hired.
AI in Marketing
AI is a valuable tool for customer service management and personalisation challenges. Improved speech recognition in call-centre management and call routing, as a result of applying AI techniques, allows a more seamless experience for customers.
For example, deep learning analysis of audio allows systems to assess a customer’s emotional tone. If the customer is responding poorly to the AI chatbot, the system can reroute the conversation to a real, human operator who takes over the issue.
Apart from the three deep learning examples above, AI is widely used in other sectors and industries.
Why is Deep Learning Important?
Deep learning is a powerful tool for turning predictions into actionable results. Deep learning excels at pattern discovery and knowledge-based prediction. Big data is the fuel for deep learning. When both are combined, an organisation can reap results in productivity, sales, management and innovation that were previously out of reach.
Deep learning can also outperform traditional methods. On perception problems in particular, deep networks have been the top performers for years: image classification, speech recognition and face recognition are all dominated by deep architectures, with face-recognition models reaching close to 99% accuracy on standard public benchmarks. The gain comes from the network learning its own features rather than relying on hand-engineered ones.
Limitations of Deep Learning
Those results come with real costs.
Data labeling
Most current AI models are trained through supervised learning. It means humans must label and categorise the underlying data, which can be a sizeable and error-prone chore. For example, companies developing self-driving-car technologies hire hundreds of people to manually annotate hours of video feeds from prototype vehicles in order to train these systems.
Obtain huge training datasets
It has been shown that deep learning techniques like CNN can, in some cases, imitate the knowledge of experts in medicine and other fields. That wave of machine learning, however, requires training data sets that are not only labeled but also sufficiently broad and universal.
Deep learning methods require thousands of observations for models to become relatively good at classification tasks and, in some cases, millions for them to perform at the level of humans. Unsurprisingly, deep learning is most common at giant technology companies, which use big data to accumulate petabytes of examples. That scale allows them to create impressive and highly accurate deep learning models.
Explain a problem
Large and complex models can be hard to explain in human terms, for instance why a particular decision was reached. It is one reason that acceptance of some artificial intelligence tools is slow in application areas where interpretability is useful or indeed required.
Furthermore, as the application of AI expands, regulatory requirements could also drive the need for more explainable AI models.
