How to Create Deep Neural Networks in JavaScript

  Programming

In my previous post, I showed you how to create a simple neural network that has only an input layer, and an output layer. By adding a few hidden layers between the input and output layers, you can create a deep neural network. Deep neural networks are usually far more powerful, and are capable of both supervised and unsupervised learning. In this tutorial, I’ll show you how to create a deep neural network with two hidden layers, and use it to perform yet another rather trivial task: decide what I should wear based on the temperature.

Creating the Deep Neural Network

So, this time, I want to build a neural network that can take the temperature in Celsius as an input, and tell me what I should be wearing. Here’s the algorithm it should follow:

if TEMPERATURE <= 5°C
    Suggest THICK JACKET

if TEMPERATURE > 5°C and TEMPERATURE <= 15°C
    Suggest SWEATER

if TEMPERATURE > 15°C and TEMPERATURE <= 40°C
    Suggest T-SHIRT

if TEMPERATURE > 40°C
    Suggest NOTHING 

Now, you might be thinking we can simply use one neuron in the input layer for the temperature. Well, you can, however, your neural network’s accuracy would then be rather low. In this tutorial, we’re going to convert the temperature into a 7-digit binary form, and use seven neurons for the input layer.

You can use the following function to do the conversion:

function convertToBinaryArray(temperature) {
    var tempInBinary = temperature.toString(2); // Convert to binary

    // If it is more than 7 digits long, truncate
    if(tempInBinary.length > 7)
        return [1,1,1,1,1,1,1];

    // If it less than 7 digits long, add zeroes
    while(tempInBinary.length < 7) { 
        tempInBinary = "0" + tempInBinary;
    }

    // Convert string to array
    return tempInBinary.split("").map(function(i) {
        return parseInt(i); }
    );
}

Feel free to modify the function if you think you can make it shorter or more readable!

As for the output layer, four neurons will be enough(THICK JACKET, SWEATER, T-SHIRT, NOTHING). Additionally, we shall have 2 hidden layers with three neurons each.

Although you can create the layers individually, and connect them, Synaptic offers a class called synaptic.Architect.Perceptron that lets you build complex networks in just a few seconds. Here’s how you can use synaptic.Architect.Perceptron to create the deep neural network:

var myDeepNetwork = new synaptic.Architect.Perceptron(
    7, // Input layer with 7 neurons
    3, // First hidden layer with 3 neurons
    3, // Second hidden layer with 3 neurons
    4 // Output layer with 4 neurons
);

Our neural network is now ready!

If you want to, you can also call this deep neural network a Multi-Layer Perceptron(MLP).

Creating the Training Data

We can now create a loop to generate our training data.

var trainingData = [];

for(var i = 1;i < 75; i++) {
    var input = convertToBinaryArray(i); // Input layer
    var output = [0,0,0,0]; // Undecided state of output layer

    if(i <= 5)
        output = [1,0,0,0]; // THICK JACKET
    else if(i > 5 && i <= 15)
        output = [0,1,0,0]; // SWEATER
    else if(i > 15 && i <= 40)
        output = [0,0,1,0]; // T-SHIRT
    else
        output = [0,0,0,1]; // NOTHING

    trainingData.push({
        input: input,
        output: output
    });
}

Note that in the real world, you would not have to write any code to generate the data. You would simply pick the data up from somewhere. You would, however, still have to write code to convert it into input and output arrays.

Training the Neural Network

Synaptic has a class called synaptic.Trainer that allows you to train your neural network using just one method called train(). Let’s use that now.

var myTrainer = new synaptic.Trainer(myDeepNetwork); // Create trainer
myTrainer.train(trainingData, {
    rate: 0.1,
    iterations: 10000,
    shuffle: true
}); // Train with training data

As you can see, you can pass the learning rate, number of iterations and other configurations details directly to the train() method.

Using the Neural Network

Our neural network is trained and ready to be used. Let’s call its activate() method for the temperature 11°C, and see what it recommends.

var cTemp = convertToBinaryArray(11);
var recommendations = myDeepNetwork.activate(cTemp);

// Log neuron outputs
console.log("THICK JACKET neuron: " + (recommendations[0] * 100) + "%");
console.log("SWEATER neuron: " + (recommendations[1] * 100) + "%");
console.log("T-SHIRT neuron: " + (recommendations[2] * 100) + "%");
console.log("NOTHING neuron: " + (recommendations[3] * 100) + "%");

Here’s the output I got:

THICK JACKET neuron: 1.187852121842318%
SWEATER neuron: 99.04285103672623%
T-SHIRT neuron: 0.19623392618093957%
NOTHING neuron: 0.2812308783464911%

As you can see, for 11°C, the SWEATER neuron is firing with a value of >99%, while all other neurons have a value of less than 2%.

Let us now see what it says for 32°C.

THICK JACKET neuron: 0.5111861782203928%
SWEATER neuron: 0.00020342986269563567%
T-SHIRT neuron: 87.58129843432297%
NOTHING neuron: 0.17906434966864235%

The neural network has the right answer again. This time, the T-SHIRT neuron is firing with the highest value.

What about a border condition, like 5°C.

THICK JACKET neuron: 71.57886740905687%
SWEATER neuron: 19.97632896204967%
T-SHIRT neuron: 0.5732055868587239%
NOTHING neuron: 0.044105602954815616%

The neural network now shows a value of 71% for THICK JACKET and ~20% for SWEATER. Nevertheless, because THICK JACKET has the highest value, it is still correct.

Conclusion

You now know how to construct and use deep neural networks. Feel free to experiment with different values for the learning rates, number of hidden layers, and other such configurations details of the neural network.

If you found this article useful, please share it with your friends and colleagues!