What Is a Neural Network? How It Works, Types, and Examples

A neural network is a machine learning model that turns numbers into predictions by passing them through layers of small connected units called artificial neurons. A photo classifier, for example, receives thousands of pixel values and returns a probability that the picture shows a cracked phone screen. Every neuron in between weighs its inputs, and training adjusts those weights until the predictions are right often enough to be useful.

Learning from examples instead of hand-written rules lets a network deal with photos, speech, text, and sensor readings. The same idea runs computer vision, natural language processing (NLP), speech recognition, fraud scoring, and modern chatbots. The trade-off is real: neural networks need more data and computing than simpler models, while their decisions are harder to explain to a customer or an auditor.

Answering what is a neural network properly means opening one up to look at its input layer, hidden layers, output layer, weights, biases, and activation functions. This guide calculates a single neuron by hand and follows one backpropagation update with real numbers. It then compares the main types of neural networks and shows when a simpler model is the better buy.

Key Takeaways

• A neural network is a stack of layers in which each artificial neuron multiplies its inputs by weights, adds a bias, then applies an activation function.
• Training repeats four steps (forward pass, error calculation, backward pass, weight update) until the loss stops falling.
• Hidden layers with nonlinear activations let a network learn curved decision boundaries that a single perceptron cannot draw.
• Feedforward, convolutional, recurrent, transformer, autoencoder, adversarial, and graph networks each suit a different shape of data.
• Neural networks usually need more data and computing than classic models, so a simpler algorithm is often the better first choice for a small spreadsheet of records.

What Is a Neural Network?

A neural network is a machine learning model made of artificial neurons arranged in layers, which learns to map inputs to outputs by adjusting the weights and biases on its connections. That neural network definition covers a broad family of models, from a single neuron to networks with billions of parameters. Each one takes data in, passes it through weighted connections, and returns a prediction such as a label, a number, or a probability.

Researchers call the same idea an artificial neural network (ANN), a neural net, or a connectionist model. Each name points to the same design: many simple computing units that are weak alone, yet able to recognize complex patterns once wired into layers and trained on enough examples.

Stripped to the basics, a network is a function with adjustable settings. It takes an input, makes a guess, and keeps the settings that make its guesses hold up on data it has never seen. That makes every neural network one kind of AI model, so the usual rules about training data, testing, and deployment apply to it.

What Does a Neural Network Mean in AI and Machine Learning?

In artificial intelligence, artificial neural networks are the main technique for learning patterns directly from raw data, and inside machine learning they form the foundation of deep learning. The term neural networks means a family of models rather than one product. A face unlock feature, a demand forecaster, and a large language model all count as neural networks while looking nothing alike from the outside.

Artificial intelligence is the broad goal of making computers handle tasks that need human judgment, and machine learning is the branch where systems learn from data. Neural networks are one family of machine learning models. Since the early 2010s they have taken over most image, speech, and language work.

Where the knowledge lives is what separates an AI neural network from rule-based AI. A rule-based system stores readable if-then rules, whereas a neural network stores what it learns as numbers spread across thousands or millions of connections. That split sits at the core of the long-running debate between symbolic and connectionist AI, which still shapes how engineers choose between the two approaches for a given project.

How Is an Artificial Neural Network Similar to the Human Brain?

An artificial neural network borrows three ideas from the brain: many simple units, weighted connections between them, and a threshold that decides whether a signal moves forward. The human brain holds about 86 billion neurons, according to a cell-counting study published in Frontiers in Human Neuroscience. Each biological neuron collects electrical signals through its dendrites, sums them in the cell body, then fires along its axon once the total crosses a threshold.

The artificial version keeps the outline of that process while dropping the biology. Inputs take the place of dendrites, while weights stand in for the strength of each synapse. The weighted sum plays the role of the cell body, and the activation function acts as the firing threshold. The output value then travels to the next layer, where it becomes one of many inputs for the neurons waiting there.

The resemblance stops at that outline once you look at how real neurons behave. Biological neurons communicate with timed electrical spikes while rewiring throughout a person's life, whereas most artificial neurons pass a single continuous number that stays fixed once training ends. A neural network is a mathematical model inspired by the brain rather than a copy of it. Claims that a network "thinks" the way a person does go well beyond what the math supports.

What Are the Parts of a Neural Network Model?

A neural network model has six parts: the input layer, the hidden layers, the output layer, the neurons, the weights and biases, and the activation functions. Each part has a specific job, and the way designers arrange these parts is what people mean by neural network architecture. Many networks follow this simple stack of layers, while modern designs can add skip connections, loops, or several branches that run side by side.

• Input layer: holds the raw features, such as the 784 pixel values of a 28 by 28 grayscale image or the columns of a customer record.
• Hidden layers: transform the inputs into new representations, with each layer working on the output of the one before it.
• Output layer: produces the final result, which can be one number for regression, one probability for a yes-or-no question, or a set of probabilities across several classes.
• Neurons: perform the actual computation inside every layer, which is why engineers call them nodes or units.
• Weights and biases: store everything the network has learned; finding good values for them is the entire purpose of training.
• Activation functions: add the nonlinearity that lets the network model curved relationships instead of straight lines.

What Is a Neuron in a Neural Network?

A neuron in a neural network is a small function that multiplies each input by a weight, adds a bias, then passes the total through an activation function. Written as math, one neuron computes z = w₁x₁ + w₂x₂ + … + wₙxₙ + b and then outputs a = f(z), where f is the activation function.

The first half of that calculation is a linear transformation, the same operation linear regression performs. The activation function turns a plain weighted sum into a flexible building block, letting the neuron respond strongly to some input combinations while staying quiet for others.

What Do Weights and Biases Do?

Weights decide how strongly each input influences a neuron, while biases shift the point at which the neuron becomes active. A large positive weight pushes the output up whenever its input rises, while a negative weight pushes it down. A weight near zero tells the neuron to mostly ignore that input.

The bias lets a neuron produce a meaningful output even when every input is zero, which gives the model room to fit data that does not pass through the origin. Together, weights and biases are called the model parameters. A network starts with small random values for all of them, and training is the search for a set of values that makes the predictions accurate.

How Many Parameters Does a Neural Network Have?

To count the parameters in a fully connected layer, multiply the number of inputs by the number of neurons, then add one bias per neuron. Take a small digit-recognition network with 784 inputs, hidden layers of 128 and 64 neurons, and 10 outputs. It holds 100,480 parameters in the first layer, 8,256 in the second, and 650 in the last, for a total of 109,386.

Large language models follow exactly the same arithmetic at a much bigger scale, with parameter counts in the billions. The number matters in practice, since every parameter has to be stored in memory, updated during training, and supported by enough examples to settle on a useful value.

Which Activation Functions Do Neural Networks Use?

Neural networks mainly use four activation functions: sigmoid, tanh, ReLU, and softmax. Each one bends the neuron's output in a different way, which affects how quickly a network trains as well as what kind of answer the output layer can give.

ReLU became the standard choice for hidden layers since it is cheap to compute and keeps gradients from shrinking toward zero in deep stacks. Vanishing gradients of that kind slow sigmoid and tanh networks. The Stanford CS231n course notes point out the main weakness of ReLU: a neuron pushed into large negative values can get stuck at zero and stop learning. Variants such as Leaky ReLU exist to prevent that. The output layer follows a different rule from the hidden layers. Sigmoid suits a single yes-or-no answer, while softmax suits a choice among several classes. Regression models usually leave the output neuron with no activation so it can return any number.

How Do Neural Networks Work?

Neural networks work in two phases: a forward pass that turns inputs into a prediction, and a training loop that corrects the weights whenever that prediction is wrong. Once training ends, only the forward pass runs, which is why a finished model can answer in milliseconds even when training it took weeks.

What Happens During the Forward Pass?

During the forward pass, data moves from the input layer through each hidden layer to the output layer, with every neuron computing its weighted sum and activation along the way. A network where data flows only in this one direction is called a feedforward neural network. It is one of the foundational designs, while convolutional, recurrent, graph, and transformer networks rely on their own connection patterns and operations.

The output of one layer becomes the input of the next, so the network gradually reshapes raw features into a form the output layer can act on. Engineers write each layer as a matrix multiplication, which lets graphics processors handle thousands of neurons in parallel.

A Neural Network Example Worked by Hand

The clearest neural network example is a single neuron calculated with real numbers. Suppose, as a made-up illustration, that a subscription business wants to predict whether a customer will renew. The model uses three inputs scaled between 0 and 1: login frequency (0.8), satisfaction score (0.9), and an annual plan flag (1.0).

1. Assign the weights: the neuron starts with weights of 0.6, 1.2, and 0.5, plus a bias of −1.0.
2. Compute the weighted sum: z = (0.8 × 0.6) + (0.9 × 1.2) + (1.0 × 0.5) − 1.0 = 0.48 + 1.08 + 0.50 − 1.00 = 1.06.
3. Apply the activation: the sigmoid function turns 1.06 into 0.743, which the model reports as a 74.3% renewal score.
4. Make the decision: with a threshold of 0.5, the neuron predicts that the customer renews.

That score is the model's estimate under its current weights, so it only becomes a trustworthy probability after the network has been trained and checked on real customer data. The calculation itself is the entire job of one neuron. A real network repeats it across hundreds of neurons per layer and passes each result forward, yet nothing more mysterious happens inside any of them.

How Do Artificial Neural Networks Work During Training?

Artificial neural networks learn by measuring each error with a loss function, tracing it back with backpropagation, then moving every weight a small step in the direction that lowers the error. The process repeats four steps for every batch of training examples.

1. Run the forward pass to produce a prediction.
2. Calculate the error with a loss function, such as binary cross-entropy for yes-or-no outputs.
3. Run the backward pass, where backpropagation applies the chain rule of calculus to find how much each weight and bias contributed to the error.
4. Update the weights with gradient descent, shifting each one against its gradient by an amount set by the learning rate.

The renewal example shows how neural networks work in practice, using real numbers. The customer did renew, so the correct answer is 1, which makes the loss −ln(0.743) = 0.297. For a sigmoid neuron trained with cross-entropy, the gradient of the loss with respect to z equals the prediction minus the target, here 0.743 − 1 = −0.257.

Multiplying that value by each input gives the weight gradients of −0.206, −0.232, and −0.257. With a learning rate of 0.5, gradient descent raises the weights to 0.703, 1.316, and 0.629 while moving the bias to −0.871. A second forward pass produces z = 1.504 and a prediction of 0.818, cutting the loss from 0.297 to 0.201 after a single update.

David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized backpropagation as a practical training method in a 1986 paper published in Nature. The method matters since it computes every gradient in one backward sweep through the network, which keeps training affordable even for models with millions of weights.

Which Training Settings Shape the Result?

Four training settings shape how well a network learns: the learning rate, the batch size, the number of epochs, and the weight initialization. Engineers call these hyperparameters, since they choose them before training instead of letting the network learn them.

• Learning rate: sets the size of each weight update, where values that are too large make the loss bounce around and values that are too small make training crawl.
• Batch size: sets how many examples the network sees before each update, with values between 32 and 512 being common.
• Epochs: count the full passes the network makes over the training set.
• Weight initialization: sets the random starting values, which affects whether gradients stay in a useful range during the first updates.

None of these settings can rescue weak examples. A network can only learn patterns that exist in its training data, so data quality usually decides the result more than any hyperparameter does.

Why Do Neural Networks Need Hidden Layers?

Neural networks need hidden layers with nonlinear activation functions since a single layer can only separate data with a straight line, while most real problems need curved boundaries. The classic proof is the XOR problem, where the output should be 1 when exactly one of two inputs is 1 and 0 otherwise.

Plot the four XOR cases on a grid and no straight line can split the 1s from the 0s, so a single perceptron cannot learn XOR however long it trains. Marvin Minsky and Seymour Papert drew attention to this limit in their 1969 book Perceptrons. Many researchers moved away from neural networks in the years that followed.

Adding one hidden layer with two neurons fixes the problem. Each hidden neuron draws its own line, and the output neuron combines the two lines into a region that captures exactly the XOR cases. Stacking linear layers without a nonlinear activation function would not help, since several linear transformations applied in a row collapse into one linear transformation.

The broader result arrived in 1989 from George Cybenko, writing in Mathematics of Control, Signals and Systems. He proved that one hidden layer of sigmoid neurons can approximate any continuous function to any accuracy, given enough neurons. That universal approximation theorem is why researchers describe neural networks as trainable function approximators. The theorem only proves that suitable weights exist. It says nothing about whether training will find them, how many neurons the job takes, or whether the result will generalize to new data. In practice, deeper networks often reach the same accuracy with far fewer neurons than one very wide layer.

What Are the Main Types of Neural Networks?

There are eight main types of neural networks: feedforward networks, CNNs, RNNs, LSTMs, transformers, autoencoders, GANs, and GNNs. Each type changes how neurons connect so the network fits a particular shape of data, and the full names appear in the table below.

These categories overlap rather than sitting side by side as equals. An LSTM is a special kind of RNN, while autoencoders and GANs describe training setups that can be built from feedforward or convolutional layers. Neural network architectures follow the same core principles regardless of type: learned weights and biases, stacked layers, nonlinear activations, and end-to-end training through backpropagation. What changes between them is the wiring that decides which neurons see which parts of the input, and production systems often combine two or more designs in one pipeline.

How Does a Transformer Neural Network Work?

A transformer neural network reads a whole sequence in parallel during training, using attention to decide how much each token should draw on the others. Researchers at Google introduced the design in the 2017 paper "Attention Is All You Need," posted on arXiv. Transformers replaced recurrent networks for most language tasks since attention can be computed in parallel on GPUs.

Four components around the attention mechanism make transformers work:

• Attention: scores how relevant each token is to every other token, then builds a weighted mix of their representations.
• Positional encodings: add information about word order, since attention on its own ignores position.
• Residual connections: let each layer's input skip around the layer and join its output, which keeps gradients flowing through deep stacks.
• Normalization: rescales the values inside each layer so training stays stable.

Each block passes every token through a small feedforward network after attention, and later research has produced many variants of this layout. The original transformer used an encoder-decoder model, where an encoder reads the input sentence and a decoder writes the output, as in translation. Most chat models keep only the decoder and generate text one token at a time, which makes them the engine behind most generative AI tools. These decoder models apply a causal mask, so each token can attend only to the tokens before it and never to words that have not been written yet. Under the new vocabulary, a transformer is still a neural network built from the same weights, biases, activations, and backpropagation as a small feedforward model.

Deep Learning vs Neural Network: What Is the Difference?

Deep learning is the practice of training neural networks with many hidden layers, so every deep learning model is a neural network, but not every neural network is deep. The word "deep" refers to the number of layers, and deep neural networks (DNNs) often stack dozens or even hundreds of them.

No official layer count separates the two terms, so the boundary stays fuzzy in everyday use. Our comparison of AI vs machine learning vs deep learning explains where each term sits in the wider family.

Where Are Neural Networks Used?

Neural networks are used in seven major areas: computer vision, natural language processing, speech recognition, recommendation systems, forecasting, healthcare imaging, and cyber security.

• Computer vision: convolutional neural networks (CNNs) read license plates, check products for defects, and run facial recognition on phones.
• Natural language processing: transformers translate text, summarize long documents, and answer customer questions in chat.
• Speech recognition: recurrent and transformer models turn spoken words into text for voice assistants and meeting transcripts.
• Recommendation systems: networks rank products, songs, or videos based on what similar users chose before.
• Forecasting: models predict energy demand and delivery times from historical records.
• Healthcare imaging: CNNs flag possible tumors or fractures for a radiologist to review.
• Cyber security: networks score files, emails, and traffic for signs of an attack.

Business teams rarely train these models from scratch. Most projects start from pre-trained neural network models that are adapted with company data, which is how our AI development services team adds vision or language features to existing products. Apps that must answer offline run smaller networks directly on the phone, a common request in AI mobile app development. Browser-based tools follow the same pattern in AI web development.

How Are Neural Networks Used in Cyber Security?

In cyber security, neural networks detect malware, flag intrusions, and catch phishing by learning what normal activity looks like, then scoring anything that departs from it. An autoencoder trained only on normal network traffic rebuilds familiar traffic with low error and unfamiliar traffic with high error. A spike in that reconstruction error becomes an alert for an analyst.

The same models create a new attack surface. The National Institute of Standards and Technology (NIST), in its March 2025 report on adversarial machine learning (NIST AI 100-2e2025), groups attacks on predictive machine learning systems into three classes. Evasion attacks alter inputs to fool a trained model, poisoning attacks corrupt training data, and privacy attacks try to extract information about the data or the model. The same report covers a separate set of attacks on generative AI systems, including prompt injection and supply chain attacks, so the three classes are not the full picture. Teams that deploy neural networks in security products have to test for these risks alongside ordinary software flaws.

What Are the Advantages and Limitations of Neural Networks?

Neural networks offer four main advantages and carry five main limitations. The advantages explain why they lead image, speech, and language tasks, while the limitations explain why simpler models still win many business problems.

The four main advantages of neural networks are:

• Feature learning: networks extract useful features from raw pixels, audio, or text without the manual feature engineering that classic models need.
• Nonlinear modeling: hidden layers capture curved, interacting relationships that linear models miss.
• Scaling: larger models trained on more data can improve results when the architecture, training method, and data quality support the extra capacity.
• Transfer: a network pretrained on one large dataset can be fine-tuned for a related task with far fewer examples.

The five main limitations of neural networks are:

• Data needs: large models trained from scratch can require very large datasets, while a pretrained network can often be adapted with a much smaller labeled set.
• Compute cost: training big models requires GPUs or specialized chips, along with the energy they consume.
• Opacity: the reasoning behind a prediction is spread across millions of weights, which makes decisions hard to explain to auditors or customers.
• Overfitting: a network with too many parameters can memorize its training set and fail on new data.
• Brittleness: small, deliberate changes to an input can flip a prediction, a weakness attackers exploit.

Engineers manage overfitting by watching the bias-variance tradeoff, where a model that is too simple misses real patterns and a model that is too complex learns noise. Common fixes include early stopping, weight regularization, more training data, and dropout. Dropout, described in the Journal of Machine Learning Research, randomly switches off neurons during training so no single neuron carries too much of the load. Our guide to overfitting in machine learning walks through each fix with examples.

When Should a Business Use a Neural Network?

A business should use a neural network when its data is unstructured, its patterns resist hand-built rules, and its budget covers the computing costs a model brings. Five checks settle the decision before any code is written.

• Data type: images, audio, free text, and sensor streams favor neural networks, while a tidy spreadsheet of records often works well with gradient boosting, a decision tree, or logistic regression.
• Data volume: a pretrained model can be adapted with a few thousand labeled examples, whereas training a large network from scratch needs far more.
• Speed and cost: every prediction costs computing time, so a model that must answer in milliseconds on a phone has to stay small.
• Explainability: lending, hiring, and medical decisions often need a clear reason for each outcome, which favors simpler models or added explanation tools.
• Evaluation: compare the network against a simple baseline on held-out data before launch, then keep monitoring accuracy once real users arrive.

Pick a decision tree or logistic regression instead, if the data is a small spreadsheet and every decision must be explained line by line.

How Did Neural Networks Develop?

Neural networks developed over eight decades, starting with a mathematical neuron in 1943 and reaching today's transformer models after several bursts of progress separated by long stalls.

Warren McCulloch and Walter Pitts described the first mathematical model of a neuron in 1943 in the Bulletin of Mathematical Biophysics. Their paper showed that networks of simple threshold units could compute logical functions. Their model had no way to learn, yet it set the template that every later network follows.

Frank Rosenblatt added learning in 1958 with the perceptron, an algorithm that adjusted its own weights from examples. The Cornell Chronicle recounts a 1958 demonstration in which a room-sized computer learned to tell cards marked on the left from cards marked on the right after 50 trials. Rosenblatt later built the Mark I Perceptron, one of the first neural network computers, which now sits in the Smithsonian Institution.

The 1969 critique by Minsky and Papert slowed the field until the 1986 backpropagation paper showed how to train networks with hidden layers. Sepp Hochreiter and Jürgen Schmidhuber introduced the LSTM in 1997. A deep convolutional network trained on GPUs won the ImageNet contest in 2012, and the 2017 transformer paper opened the path to today's large language models. Our history of artificial intelligence places these milestones within the wider story of AI research.

How Do Developers Build a Neural Network Model?

Developers build a neural network model with a deep learning framework such as PyTorch, TensorFlow, or Keras, which handles the matrix math, backpropagation, and GPU acceleration automatically. A developer describes the layers, picks a loss function and an optimizer, then calls a training function, leaving the framework to compute every gradient.

The short Keras example below trains a network on the XOR problem, where the output should be 1 only when exactly one of the two inputs is 1.

import numpy as np

from tensorflow import keras

keras.utils.set_random_seed(42) # makes the run repeatable

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype="float32")

y = np.array([0, 1, 1, 0], dtype="float32")

model = keras.Sequential([

keras.Input(shape=(2,)),

keras.layers.Dense(4, activation="tanh"), # hidden layer

keras.layers.Dense(1, activation="sigmoid"), # output layer

])

model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.05),

loss="binary_crossentropy", metrics=["accuracy"])

model.fit(X, y, epochs=500, verbose=0)

print(model.predict(X).round(2))

This model has 17 parameters: 12 in the hidden layer (2 inputs × 4 neurons, plus 4 biases) and 5 in the output layer (4 weights plus 1 bias). We ran this exact code with TensorFlow 2.21 on Keras 3.15 using a standard CPU. With the seed fixed at 42, it printed 0, 1, 1, and 0 on two separate runs. A single-layer perceptron can never reach that result, since it cannot separate the XOR cases however long it trains. Other hardware or library versions can shift the raw numbers slightly, as a fixed seed does not make every operation fully deterministic.

Production systems wrap the model in data pipelines, monitoring, and retraining jobs, which is where most of the engineering time goes. Teams that need that full stack can work with our custom software development group or plan an automated workflow with our AI agent development team. Each project follows the stages laid out in our process.

Conclusion

The answer to what is a neural network comes down to layers of artificial neurons that weigh inputs, add biases, and apply activation functions. Those neurons learn from their mistakes through backpropagation and gradient descent. The input layer, hidden layers, and output layer give every network the same basic shape, from a single neuron predicting renewals to a transformer with billions of parameters.

That design explains both the benefits and the limits. Neural networks learn patterns from raw data for computer vision, natural language processing, speech recognition, and cyber security. They pay for that ability with more data, more computing, and closer attention to overfitting than simpler models need. Start with the problem and the data you have, then choose the smallest network that solves it.

O2SOFT builds AI features into web and mobile products. You can book a free consultation to scope a neural network project with our engineers.

Frequently Asked Questions

How would you explain a neural network in simple terms?

A neural network is a computer program that learns from examples. It passes numbers through layers of small units that each weigh their inputs, compares its answer with the correct one, then adjusts those weights until its answers become accurate.

Is a neural network the same as AI?

No, a neural network is one technique within AI rather than AI itself. A neural network in AI sits alongside rule-based systems, search algorithms, and planning methods that use no neural networks at all, as our guide to how AI works explains.

What does a neural network look like mathematically?

Mathematically, a neural network is a chain of matrix multiplications, bias additions, and nonlinear functions. Each layer computes a = f(Wx + b), where W is the weight matrix, x is the input vector, b is the bias vector, and f is the activation function. The full network nests these layer functions inside one another.

Is ChatGPT a neural network?

Yes, ChatGPT runs on large language models that are transformer neural networks with billions of parameters. Those models learn by predicting the next token in huge amounts of text, using the same backpropagation and gradient descent as any smaller network.

How many layers does a neural network need?

A neural network needs at least an input layer plus an output layer, while one hidden layer is enough for problems such as XOR. Image and language models use dozens to hundreds of layers, since depth lets them build complex features from simpler ones.

Can neural networks learn without labeled data?

Yes, neural networks can learn without labeled data in three main ways. Unsupervised learning finds structure on its own, as when an autoencoder learns by rebuilding its inputs. Self-supervised learning creates its own targets from raw data, as when a language model predicts the next word in a sentence. Reinforcement learning is a separate approach, where an agent learns from rewards it collects by acting in an environment.

What are neural networks used for in everyday products?

Neural networks run face unlock and photo search on phones, voice assistants that turn speech into text, and product recommendations in online stores. Banks use them to score card payments for fraud, while hospitals use them to flag scans for a radiologist to review.

What is the difference between an ANN and a DNN?

An ANN is any artificial neural network, while a DNN is an ANN with many hidden layers. Every DNN is an ANN, but a small network with a single hidden layer is an ANN that few engineers would call deep.

Why do neural networks need so much data?

Neural networks often need large datasets since they have many parameters to set, and each parameter needs enough examples to settle on a useful value. The real requirement depends on model size, task difficulty, data quality, and whether you start from a pretrained model. A network with 100,000 parameters trained from scratch on 500 examples will usually memorize them instead of learning the pattern, which is overfitting.