Exploring Neural Networks and Deep Learning

The average person doesn’t spend much time thinking about the structure of their brain unless it starts causing problems. They don’t wonder what the various folds and valleys do or what it feels like when their neurons fire. Most are content to understand that they have a functioning brain in their head. What happens when you compare your brain to what you might find on a computer? What are neural networks and deep learning and what is the connection between the two?

Defining Neural Networks

While technology is just beginning to catch up with the things we’ve imagined, humans have been dreaming about creating a ‘machine that thinks’ since Ancient Greece. Greek mathematicians even created what is widely considered the world’s first computer — the Antikythera Mechanism — that scientists didn’t manage to figure out until 2022. 

At its most basic, a neural network is a computer system modeled after the human brain and nervous system. It takes everything we’ve learned about how these biological systems function and replicates it in an artificial system. Instead of transmitting data as simple 1s and 0s, artificial neural networks transmit information in a more organic way, similar to how your brain’s neurons signal to one another. 

Neural network nodes each have a specific threshold. If that threshold is reached, it transmits the information to the next node. If not, the data stays on that network layer until something else triggers that threshold. Neural networks can be designed with as many layers as necessary to learn the requested information or complete the necessary tasks. 

Each node operates as an individual linear regression model. The formula includes input data, weights, biases or the threshold it needs to reach and the expected output. It looks complex on paper, but the ultimate goal is to create an artificial brain that works without understanding its coding. 

Neural Networks vs. The Human Brain

A neural network’s name is a bit of a misnomer. It doesn’t have anything to do with the design of human neurons. It would take an incredibly powerful neural network to replace a single human neuron. In a biological brain, basic (shallow) neural networks only have three layers — an input, a hidden layer and an output. 

These basic networks are a close approximation of biological neurons, though they process a fraction of the data. Biological neurons operate on an “integrate and fire” model. Essentially, it totals up the excitatory input and subtracts any inhibitory input. If the result exceeds the biological threshold, the neuron fires. Once you start including additional hidden layers — creating a deep neural network — the integrate and fire model stops functioning. Researchers found that looking at some of the brain’s more complex neurons, such as those in the cerebral cortex, an artificial equivalent would need seven layers — each with 128 units — to even come close to simulating their processing power. 

All of this brings us and the researchers studying neural networks, to a single conclusion — the human brain is a massive deep learning network. The average human brain has more than 10 billion neurons. If replicated in an artificial neural network, each of those individual neurons would require a deep learning system with at least seven full hidden layers to compete. 

Defining Deep Learning

Deep learning and neural networks are terms that are often used interchangeably. On paper, researchers define deep learning as “a subfield of machine learning concerned with algorithms inspired by the structure and function of the human brain.” You can probably see where this is going. While every deep learning system might be a neural network, not every neural network is a deep learning system. 

Think of deep learning as the central component of an artificial intelligence nesting doll. The biggest doll is artificial intelligence. Within that rests machine learning and within machine learning, you’ll find neural networks. Finally, hiding in the center, you’ll find deep learning. 

Exploring Neural Networks and Deep Learning

Neural networks and deep learning sound like technology that you might see in a sci-fi movie, but it’s already impacting our daily lives. Accessibility technology like computer vision and speech recognition uses neural networks and various types of machine learning algorithms to provide a better experience for those that might need them. Google searches and the natural language processing algorithms that fuel them rely heavily on neural networks. 

The downside of these applications is that they learn incredibly slowly and there’s a steep learning curve when introducing new data. That’s why it takes your phone’s voice to text so long to adapt to your speech patterns or dialect choices. Once it has a solid foundation to work with, it’s an incredibly useful tool, but if anything changes or needs a reset, you’re essentially starting from scratch. 

Shaping the Future

Neural networks and deep learning might seem interchangeable, but they’re just two pieces of the same puzzle. These complex algorithms form the foundation for artificial intelligence and related technologies shaping the future.