Explore machine learning's essentials for software engineers, delve into supervised learning, neural networks, and deep learning, and gain skills to tackle real-world data challenges effectively.

pml_final.tar.gz

just_run

chapter06

SPA_07

SPA_gdb_frontEnd

J12-1

J12-2

SPA-server_image

Handson-10

Machine learning is the future for the next generation of software professionals.

This course serves as a guide to machine learning for software engineers. You’ll be introduced to three of the most relevant components of the AI/ML discipline; supervised learning, neural networks, and deep learning. You’ll grasp the differences between traditional programming and machine learning by hands-on development in supervised learning before building out complex distributed applications with neural networks. You’ll go even further by layering networks to create deep learning systems. You’ll work with complex real-world datasets to explore machine behavior from scratch at each phase.

By the end of this course, you’ll have a working knowledge of modern machine learning techniques. Using software engineering, you’ll be prepared to build complex neural networks and wrangle real-world data challenges.

Fundamentals of Machine Learning for Software Engineers

# CNN for CIFAR-10
 A convolutional neural network for CIFAR-10 is:

In the code below, our Cifar-10 classifier undergoes training for 3 `epochs`, but to obtain better `accuracy` we must increase the number of epochs. The results of training the classifier for 20 `epochs` are available below this code widget.

A convolutional neural network for CIFAR-10 is given below:


# A convolutional neural network that trains on CIFAR-10 images.

import numpy as np
from keras.models import Sequential
from keras.layers import Conv2D, Dropout, Dense
from keras.layers import BatchNormalization, Flatten
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical
from keras.datasets import cifar10

# Preparing dataset
(X_train_raw, Y_train_raw), (X_test_raw, Y_test_raw) = cifar10.load_data()

# Using a portion of the larger dataset 
X_train_raw = X_train_raw[0:10000]
Y_train_raw = Y_train_raw[0:10000]
X_test_raw = X_test_raw[0:1000]
Y_test_raw = Y_test_raw[0:1000]

X_train = X_train_raw / 255
X_test_all = X_test_raw / 255
X_validation, X_test = np.split(X_test_all, 2)
Y_train = to_categorical(Y_train_raw)
Y_validation, Y_test = np.split(to_categorical(Y_test_raw), 2)

# Building the CNN
model = Sequential()

model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Flatten())

model.add(Dense(1000, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(10, activation='softmax'))


# Compiling the model
model.compile(loss='categorical_crossentropy',
              optimizer=Adam(),
              metrics=['accuracy'])

# Fitting the model to test its performance(training)
history = model.fit(X_train, Y_train,
                    validation_data=(X_validation, Y_validation),
                    epochs=3, batch_size=32)


import React from 'react';
import { shallow, configure } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';

configure({ adapter: new Adapter() });

import HelloWorld from './app';

var TestResult = function() {
    this.succeeded = false;
    this.reason = "";
    this.input = "";
    this.expected_output = "";
    this.actual_output = "";
}

export const executeTests = function() {

  var results = [];

  result = new TestResult();
  result.input = 'HelloWorld Component';
  result.expected_output = "span containing text 'Hello World'"

  let wrapper = shallow(<HelloWorld />);

  // Call your Challenge function here.

  let type = wrapper.type();
  let testSuccessful = true;
  let failureReason;

  if (type !== 'span') {
    testSuccessful = false;
    failureReason = "You need to render exactly one span HTML element";
  } else if (wrapper.props().children != "Hello World") {
     testSuccessful = false;
     failureReason = "You have rendered wrong message in your span element";
  }

  result.actual_output = wrapper.html();

  if (testSuccessful) {
    result.succeeded = true;
    result.reason = "Succeeded"
  } else {
    result.succeeded = false;
    result.reason = failureReason;
  }

  results.push(result);

  return results;
}


React

Dart

GoJS React

Typescript React

Vue.js

 The code above involves many arbitrary decisions. For example, we decide to have two convolutional layers to use ReLUs in every layer. We also use techniques from the previous chapter for _batch normalization to improve accuracy, and dropout to reduce overfitting_. We didn’t check whether those decisions result in a better neural network than the alternatives. That exploration will be done in the final exercise of this chapter.

However, some of the choices are part of the default design of a CNN and are proven useful by many deep learning practitioners. One is the idea of building a CNN as _a sequence of convolutional layers, followed by a handful of fully connected layers_. Nothing prevents us from using convolutional layers throughout, but it’s customary to have those fully connected layers at the end.

Let’s traverse the network from start to end, and look at each layer in detail.

First, the two convolutional layers:

```python
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
```

We have to decide how many filters to use, and their size. For both layers, we use 3 by 3 filters. As for their number, we follow another common rule that _the number of filters tends to grow as you traverse the network_. We use 16 filters in the first layer and 32 in the second layer. If this network had a third convolutional layer, we would probably have used 64 filters in it.

It’s always insightful to calculate the dimensions of the tensors inside the network. If we wish, we can do that ourselves, applying the rules from  [Convolutions](https://www.educative.io/courses/fundamentals-of-machine-learning-for-software-engineers/the-building-blocks-of-cnns#Convolutions). Otherwise, let’s do it together. A CIFAR-10 image is a (32, 32, 3) tensor. After the first layer, its width and height become $32 - 3 + 1 = 30$, and its depth becomes equal to the number of filters. So the output of the first layer is a (30, 30, 16) tensor. By the same rules, the output of the second layer is a (28, 28, 32) tensor.

So far we have said that convolutional layers take images as their input. To be more precise, that’s only true of the input layer. The image has become a stack of convolved data from the second layer onwards. 

In the next chapter, we’ll answer what a convolutional neural network sees in that convolved data and how it understands images like the ones in CIFAR-10. For now, let’s move forward in our examination of the neural network’s code.


The next line solves the problem of passing data from the last convolutional
layer to the first fully connected layer. At that point, the data must be flattened
from a four-dimensional tensor of images to a bi-dimensional matrix of flat
data. Keras’ `Flatten` layer takes care of that:
```python
model.add(Flatten())
```
When it reaches the `Flatten` layer, we calculate that each example has become
a (28, 28, 32) tensor. After the flattening, it becomes a row of 28 * 28 * 32 = 25088 bytes. Those bytes pass through a couple of fully connected layers, and finally the output layer:
```python3
model.add(Dense(1000, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
```
And that’s it! The rest of the code comes straight from the fully connected
network we had before.

Here’s the result of 20 epochs of training:
>Train on 50000 samples, validate on 5000 samples<br>
Epoch 1 - loss: 1.7176 - acc: 0.4219 - val_loss: 1.2997 - val_acc: 0.5442<br>
…<br>
Epoch 20 - loss: 0.4808 - acc: 0.8305 - val_loss: 0.8315 - val_acc: 0.7334

It takes just one epoch for CNN to beat the fully connected network. After 20 epochs, it ruins the fully connected network.

Although it might not sound like a lot, 73% accuracy is impressive considering how hard CIFAR-10 is, and the fact that we trained this network on a simpler computer. All things considered, we can grab that 73% and run with it!


# Summary
* Before this chapter, we only built and trained fully connected neural networks,
where each node is connected with all the nodes in the layers before and after
it. In this chapter, we moved beyond that generic architecture to see an
example of a more specialized one, called  **a convolutional neural network**.
* CNNS are different from fully connected networks and preserve the multidimensional shape of data such as images. In a CNN, the weights are organized into structures called **filters** that detect the geometric features in the data. Each layer applies an operation called a **convolution** between its input and the filters. CNNs are better than fully connected networks at dealing with images because they respect the spatial information in the images instead of squashing them to mostly meaningless traffic of bytes.
* Finally, we built our own convolutional neural network. We applied a few common practices, like the idea of having a few convolutional layers followed by fully connected layers. When we trained the CNN, we obtained good accuracy on the tough CIFAR-10 dataset.

* Besides CNNs, deep learning spans many other architectures, each suited to
different problems and different types of data. Speaking of which, we still
did not come to a conclusion on _what deep learning is, how it works, and what
its boundaries are_. 

How Machine Learning Works

Our First Learning Program

Walking the Gradient

Hyperspace

A Discern Machine

Get Real

The Final Challenge

The Perceptron

Designing the Network

Building the Network

Training the Network

How Classifiers Work

Batchin’ Up

The Zen of Testing

Let’s Do Development

A Deeper Kind of Network

Diabetes Prediction Using Keras

Defeating Overfitting

Taming Deep Networks

Beyond Vanilla Networks

Into the Deep

Recognize Handwritten Digits Using a Deep Neural Network

Run on Convolutions

CNN for CIFAR-10