How to Create Simple Neural Networks in JavaScript

  Programming

Let’s face it, if you are an ordinary web or mobile application developer, learning how to use machine learning libraries such as TensorFlow or Caffe isn’t easy. Their APIs are probably intuitive for people who are already quite familiar with neural networks, matrix operations, and advanced mathematics. For the layman, however, they simply don’t make much sense. Fortunately, last week, I found a JavaScript library called Synaptic. I must say, I had a lot of fun using it. In this tutorial, I’m going to show you how to use it create simple neural networks.

Installation

You really don’t need to install anything. Synaptic is available on CDNJS, which means, you can simply add the following code to your HTML page:

<script src="https://cdnjs.cloudflare.com/ajax/libs/synaptic/1.0.8/synaptic.min.js"></script>

Additionally, let’s add Underscore.js, because we will be needing some of its very handy array-related methods.

<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>

Creating a Simple Neural Network

Before you create a neural network, you must decide what you want it to do. Now, I won’t be showing you how to perform complex tasks such image classification or sentiment analysis. Instead, let’s do something trivially easy.

Imagine you are creating a simple neural network that’s going to be put inside your really dumb hamster. The neural network should make him do the following things:

  • If you clap, he must sit down.
  • If you whistle, he must run.
  • If you clap AND whistle, he must jump.

To model this behavior, we’re going to need two input neurons (CLAP, WHISTLE), and three output neurons(SIT, RUN, JUMP). That means, we’re going to have two layers of neurons.

Let us create the two layers using the synaptic.Layer class.

var input = new synaptic.Layer(2); // Two inputs
var output = new synaptic.Layer(3); // Three outputs

Right now, there are no connections between the two layers. To connect them, you must use the project() method.

input.project(output); // Connect input to output

At this point, both the layers are fully connected. In other words, every neuron of the input layer is connected to every neuron of the output layer.

Training the Neural Network

Our neural network is ready to be trained. So, let’s start creating our training data. It’s going to be two arrays: one for the input, and one for the output. As you might have guessed, input will have two values, and output will have three values. The values must always range from 0 to 1. In this tutorial, we shall stick with exactly 0 and 1.

The first element of the input array will be 1 if you are clapping, and 0 otherwise. Similarly, the second element will be 1 if you are whistling, and 0 otherwise.

And, the first element of the output will be 1 if the hamster must sit, and 0 otherwise. The second element will be 1 if the hamster must run, and 0 otherwise. Lastly, the third element will be 1 if the hamster must jump, and 0 otherwise.

Accordingly, here’s what the training data will look like:

var trainingData = [
    {input: [1, 0], output: [1, 0, 0]}, // Clap -> Sit
    {input: [0, 1], output: [0, 1, 0]}, // Whistle -> Run
    {input: [1, 1], output: [0, 0, 1]}, // Clap+Whistle -> Jump
];

It looks simple, but spend some time to fully understand the training data’s arrays. Almost every machine learning framework needs training data, and understanding how to create and format that data is extremely important.

Before we proceed any further, we should decide how impressionable our neural network should be. In other words, we must give it a learning rate. A very low learning rate will usually make the neural network a slow, but very good learner. On the other hand, a very high learning rate will make it learn quickly, but also very erratically. For now, let’s say the learning rate is 0.4.

We can now send the input data to the input layer of our network using the activate() method. And then, we should call the propagate() method on the output layer to help the neural network adjust its configuration. The propagate() method needs the output data as its input.

var learningRate = 0.4;

function train() {
    for(var i = 0; i < trainingData.length; i++) {
        input.activate(trainingData[i]["input"]);
        output.activate();
        output.propagate(learningRate, trainingData[i]["output"]);
    }
}

Just like humans, neural networks don’t learn to do things the right way after training just once. Well, they kinda do, but their accuracy will be extremely low. To improve their accuracy levels, they need to be trained repeatedly. Therefore, let’s train our neural network about 1000 times, and also shuffle the order of the trainingData each time.

function retrain() {
    for(var i = 0; i < 1000; i++) {
        trainingData = _.shuffle(trainingData);
        train();
    }
}

retrain(); // Start the training

Using the Neural Network

The neural network is now trained and ready to be used. To use it, you must pass an array to the input layer, and then check the neurons of the output layer. The activate() method can be used for doing both.

Let’s say you just whistle. Here’s what you would do:

input.activate([0,1]); // Whistle
var result = output.activate();

The result will now contain three numbers, all ranging from 0 to 1. Here’s how you can print all of them out as percentages:

console.log("Sit Neuron: " + result[0] * 100 + "%");
console.log("Run Neuron: " + result[1] * 100 + "%");
console.log("Jump Neuron: " + result[2] * 100 + "%");

I got these values on the console:

Sit Neuron: 0.014995911436294706%
Run Neuron: 99.39201708556335%
Jump Neuron: 1.2615268528823826%

As you can see, the Run neuron’s value is really high, which means, the neural network has learned everything correctly. Pass a different input array, say [1,1], and you should see the Jump neuron’s value very high.

Sit Neuron: 0.04761738707168859%
Run Neuron: 0.4768082300857423%
Jump Neuron: 98.24123037285819%

Conclusion

You now know how to create, train, and use a simple neural network using JavaScript and Synaptic. We created something very simple, and trivial, but if you are a beginner, I’m sure you learned to do something new. In my next post, I’ll be showing you how to create more complex neural networks, which contain more layers.

If you found this article useful, please share it with your friends and colleagues!