Skip to main content

Artificial Neural Networks - Intro for beginners

The perceptron

Single node perceptron

Perceptrons form the basis for ANNs. Perceptron takes input and produces output as below:

Input ➡️ Activation Function ➡️ Output

Input

If the weight is 0- input remains unaltered coming into the perceptron. Below is what happens to the input

∑ wizi  ≽ t then y = 1. Else y = 0.
 i

t is the threshold which is set by the outgoing part. So the key to output is based on the weighted sum and the threshold.

Activation function

This is the processing part of the neuron and this determines output. So most commonly the ones used are the Sigmoid function and the hyperbolic tangent.

QUESTION: Which activation function should I use?

I am going to talk of three key activation functions

a) Sigmoid

The Sigmoid returns 0 or 1 and in code can be written as

return 1.0/(1.0+Math.exp(-x)

A Sigmoid is a mathematical curve having the characteristic S shape.

DISADVANTAGE: Descending Gradient

b) Tanh




DISADVANTAGE: Descending Gradient

c) RELU - This is the one you should use (especially in the hidden layer)

#Look up Leaky Relu and Maxout

Output

This looks at the Error and this introduces the concept of BackPropagation. This is basically going back, adjusting the weight getting the output and ultimately reducing the error as much as possible.

Backpropagate to update weights!

 


Sample ANN 

aj = wx +b
aj = sigmoid(hj)          -> Or whichever AF you desire
E = y - ลท (Actual - Prediction)

#Back Propagate as necessary
Labels:

Comments

Post a Comment

Popular posts from this blog

PRG, PRF, PRP in Cryptography - What are they?

So I have been reading up on my cryptography and I figured I should give out a brief lesson on these three amazing concepts What are they ? a) PRG (Pseudo Random Generator) You probably know the difference between stream and block cipher. One of the main differences between them is key size. Stream ciphers require the key to be of equal length of greater than the plaintext ,   whereas Block Ciphers take a key smaller than the PT and is then expanded. This is the PRG The PRG expands the seed Considerations: Stream Ciphers base on Perfect Secrecy whereas Block Ciphers base on Semantic Security b) PRF (Pseudo Random Function) Lets share a secret- imagine something- you want to authenticate yourself with me by proving that you know a secret that we both share. Here's a possible option i) Possible Option 1:  PRNGs We both seed a PRNG with the shared secret, I pick and then send you some random number i.  You   then have to prove that you know the secret by respon
Post a Comment
Read more

Django & Firebase - A marriage of awesomeness

Requirements 1) Django (obviously) 2) Pyrebase (pip install pyrebase, you know the drill) So to give a better appreciation- I will first show you the HTML way then I'll proceed to show you how its done in Python. METHOD 1 : The HTML way Then you need to go to the firebase console. To setup a new project. After that you will see a web setup and you select that. Once selected you will see the config. It should be similar to this : Now that you have configured firebase, we now need to select the components which will be needed. depending on what you want for your app- you can now select the modules that you need. For us today, we will do the login authentication. Make sure you include firebase app first, so this is my screen: METHOD 2: Enter Python Open your dev environment and create a file named  pyrebase_settings within your django app folder. In it, you will have the following: Now, lets go to views.py!
1 comment
Read more