Deep Learning: A Quick Overview
In order for neural networks and deep learning to work properly you need two things, one is you need Data and you need a lot of Data and other is you need a processing power you need a strong computer to process that Data and facilitate the neural network’s. So let's have a look at how data or storage of data has evolved over the years, here we got three years in the above image, how does storage look back in 1956 there is the hard drive and that is 5MB hard drive been transporting to some other place. A company had to pay 2500$ of those days’ dollars to just to rent it not buy it and just for one month.
After that in 1980, the situation improved a little bit, so you have 10MB for 3500$ and was still very expensive and has only 10MB and can store only one pic of nowadays.
And today in 2017 we have got 256GB for 150$ which can fit on your finger. If we compare these years across the board and before even taking price and size to consider, just take compare the capacity from 1956 to 1980 the capacity increased about double and then from the 1980s to 2017 it increased about 25600x times. And the length of the period is not that much but a huge jump in technological progress.
Here we have a chart of hard drive cost per gigabyte (USD).
Over the years this going to go even further right now scientist are even looking into using DNA for storage. From 10 to 20 years from now, everybody is going to use DNA storage if we go in this direction. Here are some stats.
So all of this brings us to what is deep learning?
here is the image of Geoffery Hinton and is known as the God Father of deep learning and he did research on deep learning in the 80s and he has done lots of work on Deep learning and published lots of research papers he now works in Google.
The idea behind deep learning is to look at the human brain, how the human brain operates. because human brain seems to be a very powerful tool in this planet for learning and adapting skills and applying it if the computer can copy that then we can just leverage what natural selection had already decided for us for all that algorithm.
So in the brain, we have neurons and us hundred of billions of neuron in the human brain and each neuron is connected with thousands of neurons with its neighbor. In below pic, this is what it looks like.
This is an actual section of the human brain this is called the cerebellum.and is responsible for balance so you can see there are billion and billion of neurons so that is what we are going to recreate.
So how do we create this on the computer?
In the above image: We will create an artificial structure called an artificial neural network. Where we have nodes or neurons, so we are going to have some neuron for input values. Which will take some input values in the input layer and we will have an output layer which will show the predicted result. And in between, we have hidden layers so before giving output it will go through billion and billions of neurons before it gives us output, so this is the whole concept behind it so we are going to model the brain. so we need this hidden layer. the input layer neurons are connected to hidden layer neurons and hidden layers neuron is then connected to output layers neurons.
So why is it called deep learning?