DPS906 Lecture 8

Machine Learning

Teach a program to learn to do something.
Something==Pattern Recognition (usually)

Training Data (computer creates rules from this data)
Testing Data (test the computers rules with this data)

The way that it comes out with an answer is often not something that can be explained with a rule.

Neural Networks

In AI neural nets are made of nodes. Nodes have a bunch of input... the could come from anywhere depending on the situation. The inputs have a weight when they input data to the node. When the node outputs it does calculation based on the inputs... during the "training phase" the weights of the input are adjusted. See written notes for more info

Nodes typically have the same activation function.
3 common functions

1. Threshold function: (if IjWj > threshold return 1 else return 0) (have to adjust threshold)
   
2. Sign function: similar to threshold add in an extra weight and input to the threshold function (W0I0) W0 is threshold (allows you to adjust the threshold as a weight rather than the threshold) I0 = -1
3. Signoid function: a function that doesn't return 1 or 0 only... returns values inbetween (eg 0.234 or 0.453) Has a funky Sigma curve.
   

Perceptron

Feed forward, single layer, neural net.

• feed forward == input/output goes one way, never backward (see drawings). inputs for a node always come from a node in previous layers never from later layers. Nodes in same layer do not connect.
• Single layer means only one layer. Single layer + feed forward means the nodes never talk to each other.

Good things
Given time/data a perceptron will learn to separate anything that is linearly separable. (AND OR are both linear separate, XOR is not linear separate)

Actual learning of a perceptron
DEFINITION EPOCH: Each time you update the weights in the perceptron is called an epoch.
Perception begins its "life" as a bunch of nodes. Create your layer of nodes with inputs and outputs. Randomly assign weights to all inputs and go through the 1st "training" stage. You may get an error.
Input(I)
Error(E) = Training Answer(T) - Neural Net Output(O).
if( e > 0 ) t must be > o
if( e <> t

if (Ij) is -ve increasing weight will decrease output
if Ij is +ve increasing weight will increase output
weight = weight + learning rate(this is a constant) * Ij * E.