Artificial Neural Networks - Intro for beginners

The perceptron

Single node perceptron

Perceptrons form the basis for ANNs. Perceptron takes input and produces output as below:

Input ➡️ Activation Function ➡️ Output

Input

If the weight is 0- input remains unaltered coming into the perceptron. Below is what happens to the input

∑ wizi ≽ t then y = 1. Else y = 0.

t is the threshold which is set by the outgoing part. So the key to output is based on the weighted sum and the threshold.

Activation function

This is the processing part of the neuron and this determines output. So most commonly the ones used are the Sigmoid function and the hyperbolic tangent.

QUESTION: Which activation function should I use?

I am going to talk of three key activation functions

a) Sigmoid

The Sigmoid returns 0 or 1 and in code can be written as

return 1.0/(1.0+Math.exp(-x)

A Sigmoid is a mathematical curve having the characteristic S shape.

DISADVANTAGE: Descending Gradient

b) Tanh

DISADVANTAGE: Descending Gradient

c) RELU - This is the one you should use (especially in the hidden layer)

#Look up Leaky Relu and Maxout

Output

This looks at the Error and this introduces the concept of BackPropagation. This is basically going back, adjusting the weight getting the output and ultimately reducing the error as much as possible.

Backpropagate to update weights!

Sample ANN

aj = wx +b
aj = sigmoid(hj) -> Or whichever AF you desire
E = y - ŷ (Actual - Prediction)

#Back Propagate as necessary

Comments

Django & Firebase - A marriage of awesomeness

Requirements 1) Django (obviously) 2) Pyrebase (pip install pyrebase, you know the drill) So to give a better appreciation- I will first show you the HTML way then I'll proceed to show you how its done in Python. METHOD 1 : The HTML way Then you need to go to the firebase console. To setup a new project. After that you will see a web setup and you select that. Once selected you will see the config. It should be similar to this : Now that you have configured firebase, we now need to select the components which will be needed. depending on what you want for your app- you can now select the modules that you need. For us today, we will do the login authentication. Make sure you include firebase app first, so this is my screen: METHOD 2: Enter Python Open your dev environment and create a file named pyrebase_settings within your django app folder. In it, you will have the following: Now, lets go to views.py!

PRG, PRF, PRP in Cryptography - What are they?

So I have been reading up on my cryptography and I figured I should give out a brief lesson on these three amazing concepts What are they ? a) PRG (Pseudo Random Generator) You probably know the difference between stream and block cipher. One of the main differences between them is key size. Stream ciphers require the key to be of equal length of greater than the plaintext , whereas Block Ciphers take a key smaller than the PT and is then expanded. This is the PRG The PRG expands the seed Considerations: Stream Ciphers base on Perfect Secrecy whereas Block Ciphers base on Semantic Security b) PRF (Pseudo Random Function) Lets share a secret- imagine something- you want to authenticate yourself with me by proving that you know a secret that we both share. Here's a possible option i) Possible Option 1: PRNGs We both seed a PRNG with the shared secret, I pick and then send you some random number i. You then have to prove that you know the secret by respon

Modern Learning : Information Age

Times have changed. We moved from making fire with sticks to amazing things like 3d printing, biotechnology, block-chain and artificial intelligence. No doubt- we are only starting. But one thing is of worry- how we learn. Learning can be subdivided into 4 parts(in the particular order) namely: Fundamentals Information Skills Innovation These can be applied to the Technology, Business and Finance and the Leadership and Management branches. The main goal nowadays is not to be an expert if you want to quickly build your career and make some money in the process. The main aim for now is to gain competence. Why I say that is because it is cheaper for an organisation to get someone with competence as opposed to getting an expert. The most efficient way is to aim for the skill level as opposed to the expert level(which tends to come by itself as you sharpen the skills level) We want to maximize financial benefit from the most efficient way possible. 1. Fundamentals Fundamental

Shingirayi Mandebvu's Blog

Search This Blog