Artificial-Neural-Networks-The-Tutorial-With-MATLAB(2).pdf

Artificial Neural Networks

The Tutorial

With MATLAB

Contents

1. PERCEPTRON ........................................................................................................................... 3

1.1. C LASSIFICATION WITH A 2- INPUT PERCEPTRON ........................................................................ 3

1.2. C LASSIFICATION WITH A 3- INPUT PERCEPTRON ........................................................................ 5

1.3. C LASSIFICATION WITH A 2- NEURON PERCEPTRON .................................................................... 6

1.4. C LASSIFICATION WITH A 2- LAYER PERCEPTRON ...................................................................... 7

2. LINEAR NETWORKS ...............................................................................................................9

2.1. P ATTERN ASSOCIATION WITH A LINEAR NEURON ..................................................................... 9

2.2. T RAINING A LINEAR LAYER .................................................................................................... 11

2.3. A DAPTIVE LINEAR LAYER ...................................................................................................... 13

2.4. L INEAR PREDICTION ............................................................................................................... 14

2.5. A DAPTIVE LINEAR PREDICTION .............................................................................................. 15

3. BACKPROPAGATION NETWORKS ................................................................................... 17

3.1. P ATTERN ASSOCIATION WITH A LINEAR NEURON ................................................................... 17

1. Perceptron

1.1. Classification with a 2-input perceptron.

SIMUP - Simulates a perceptron layer.

TRAINP - Trains a perceptron layer with perceptron rule.

Using the above functions a 2-input hard limit neuron is trained to classify 4 input vectors into two

categories.

DEFINING A CLASSIFICATION PROBLEM

A row vector P defines four 2-element input vectors:

P = [-0.5 -0.5 +0.3 +0.0;

-0.5 +0.5 -0.5 +1.0];

A row vector T defines the vector's target categories.

T = [1 1 0 0];

PLOTTING THE VECTORS TO CLASSIFY

We can plot these vectors with PLOTPV:

plotpv(P,T);

The perceptron must properly classify the 4 input vectors in P into the two categories defined by T.

DEFINE THE PERCEPTRON

Perceptrons have HARDLIM neurons. These neurons are capable of separating an input pace with

a straight line into two categories (0 and 1).

INITP generates initial weights and biases for our neuron:

[W,b] = initp(P,T)

INITP - Initializes a perceptron layer.

[W,B] = INITP(P,T)

P - RxQ matrix of input vectors.

T - SxQ matrix of target outputs.

Returns weights and biases.

INITIAL PERCEPTRON CLASSIFICATION

The input vectors can be replotted...

plotpv(P,T)

...with the neuron's initial attempt at classification.

plotpc(W,b)

The neuron probably does not yet make a good classification! Fear not...we are going to train it.

TRAINING THE PERCEPTRON

TRAINP trains perceptrons to classify input vectors.

TRAINP returns new weights and biases that will form a better classifier. It also returns the number

of epochs the perceptron was trained and the perceptron's errors throughout training.

[W,b,epochs,errors] = trainp(W,b,P,T,-1);

TRAINP Train perceptron layer with perceptron rule.

[W,B,TE,TR] = TRAINP(W,B,P,T,TP)

W - SxR weight matrix.

P - RxQ matrix of input vectors.

T - SxQ matrix of target vectors.

TP - Training parameters (optional).

Returns:

W - New weight matrix.

B - New bias vector.

TE - Trained epochs.

TR - Training record: errors in row vector.

TP(1) - Epochs between updating display, default = 1.

TP(2) - Maximum number of epochs to train, default = 100.

Missing parameters and NaN's are replaced with defaults.

If TP(1) is negative, and a 1-input neuron is being trained

instead of the network error.

PLOTTING THE ERROR CURVE

Here the errors are plotted with respect to training epochs:

ploterr(errors);

USING THE CLASSIFIER

We can now classify any vector using SIMUP.

Lets try an input vector of [-0.5; 0.5]:

p = [-0.5; 0.5];

a = simup(p,W,b)

B - Sx1 bias vector.

Training parameters are:

the input vectors and classification line are plotted

SIMUP Simulate perceptron layer.

SIMUP(P,W,B)

P - RxQ matrix of input (column) vectors.

B - Sx1 bias (column) vector.

Returns outputs of the perceptron layer.

Now, use SIMUP yourself to test whether [0.3; -0.5] is correctly classified as 0.

1.2. Classification with a 3-input perceptron

Using the above functions a 3-input hard limit neuron is trained to classify 8 input vectors into two

categories.

DEFINING A CLASSIFICATION PROBLEM

A matrix P defines eight 3-element input (column) vectors:

P = [-1 +1 -1 +1 -1 +1 -1 +1;

-1 -1 +1 +1 -1 -1 +1 +1;

-1 -1 -1 -1 +1 +1 +1 +1];