01
THE TRAINING DATA
Before the perceptron can learn anything, we define what we want it to learn. This is the full truth table for AND, as two C arrays - one for inputs, one for expected outputs.
8float training_inputs[4][2] = {
9 {0, 0},
10 {0, 1},
11 {1, 0},
12 {1, 1}
13};
15float training_targets[4] = {0, 0, 0, 1};
training_inputs is a 4x2 array. Four examples, two inputs each. training_targets is what we want the perceptron to output for each example. Only the last case (1 AND 1) should produce 1.
training_inputs[4][2] -- click a row to highlight
[0]
[0][0]0
[0][1]0
→
target0
0 AND 0 = 0
[1]
[1][0]0
[1][1]1
→
target0
0 AND 1 = 0
[2]
[2][0]1
[2][1]0
→
target0
1 AND 0 = 0
[3]
[3][0]1
[3][1]1
→
target1
1 AND 1 = 1 ✓
02
THE FORWARD PASS
This is how the perceptron produces an output from inputs. Multiply each input by its weight, sum everything up, add the bias, pass through the step function. Every single inference is just this loop.
18int forward(float *inputs, float *weights, float bias) {
19 float sum = bias; // start at bias, not 0
20 for (int i = 0; i < INPUTS; i++) {
21 sum += inputs[i] * weights[i]; // accumulate
22 }
23 return step(sum);
24}
We initialize
sum = bias instead of 0 so the bias is just automatically included in the accumulation. Then each input gets multiplied by its corresponding weight. The bias lets the threshold shift - without it the decision boundary is always forced through the origin.
WATCH THE WEIGHTS FLOW
SELECT INPUT PAIR:
0.20
0.10
-0.20
03
THE LEARNING RULE
After each forward pass, we compare the output to the target. The difference is the error. We use that error to nudge the weights in the right direction. This is the entire learning algorithm.
38int output = forward(training_inputs[i], weights, bias);
39float error = training_targets[i] - output;
40for (int j = 0; j < INPUTS; j++) {
41 weights[j] += LEARNING_RATE * error * training_inputs[i][j];
42}
43bias += LEARNING_RATE * error;
w = w + ( lr × error × input )
w = the weight being updated
lr = learning rate (how big a step to take, 0.1)
error = target - output (can be -1, 0, or 1)
input = the input value for this weight
If error is 0, the weight doesn't change. If we output 0 but should have output 1, error = +1, so we increase the weights. If we output 1 but should have output 0, error = -1, so we decrease. The input term means: weights connected to inputs that were 0 don't change at all - they didn't contribute to the wrong answer, so they don't get blamed.
Step through an epoch -- watch weights update after each example
| EXAMPLE | A, B | TARGET | OUTPUT | ERROR | W0 BEFORE | W1 BEFORE | BIAS BEFORE | W0 AFTER | W1 AFTER | BIAS AFTER |
|---|
W0=0.00 | W1=0.00 | BIAS=0.00 | example 0/4