Making Predictions with Multilinear Regression in PyTorch

The multilinear regression model is a supervised learning algorithm that can be used to predict the target variable y given multiple input variables x. It is a linear regression problem where more than one input variables x or features are used to predict the target variable y. A typical use case of this algorithm is predicting the price of a house given its size, number of rooms, and age.

In previous tutorials, we focused on simple linear regression where we used only a single variable x to predict the target variable y. From here on we’ll be working with multiple input variables for prediction. While this tutorial only focuses on a single output prediction y from multiple input variables x, in subsequent ones we’ll introduce you to multiple input-multiple output regression problems. Usually, same practice is opted in real world scenarios to build more sophisticated neural network architectures.

This tutorial will show how you can implement a multi

linear regression model in PyTorch. Particularly, you’ll learn:

• How to review linear regression in multiple dimensions.
• How to make predictions with multilinear regression model using Pytroch.
• How to use Linear class for multilinear regression in PyTorch.
• How to build custom modules using nn.Module in PyTorch.

Overview

This tutorial is in three parts; they are

• Preparing Data for Prediction
• Using Linear Class for Multilinear Regression
• Visualize the Results

Preparing Data for Prediction

As in the case of simple linear regression model, let’s initialize the weights and bias for our model. Note that we have used multi-dimensional tensors for our weights and bias as we’ll be working with more than one input variables.

1 2 3 4 5 6 7

import torch torch.manual_seed(42)   # Setting weights and bias w = torch.tensor([[3.0],                   [4.0]], requires_grad=True) b = torch.tensor([[1.0]], requires_grad=True)

Next, we’ll define our forward function for prediction. Previously we used scalar multiplications but here we use the mm function from PyTorch for matrix multiplication. This function implements a linear equation with more than one input variables. Note that multi-dimensional tensors are matrices and require a few rules to be followed, like matrix multiplication. We’ll discuss more on this later.

1 2 3 4 5

# Defining our forward function for prediction def forward(x):     # using mm module for matrix multiplication     y_pred = torch.mm(x, w) + b     return y_pred

Now that we have initialized the weights and bias, and built a forward function for prediction, let’s define a tensor x for input variables.

1 2 3 4 5 6

# define a tensor 'x' x = torch.tensor([[2.0, 4.0]]) # predict the value with forward function y_pred = forward(x) # show the result print("Printing Prediction: ", y_pred)

This prints

1	Printing Prediction:  tensor([[23.]], grad_fn=<AddBackward0>)

Note that in matrix multiplication torch.mm(x, w), the number of columns in the matrix x must be equal to the number of rows in w. In this case, we have a $1\times 2$ tensor for x and $2\times 1$ tensor for w, resulting in a $1\times 1$ tensor after matrix multiplication.

Similarly, we can apply the linear equation for multiple samples. For instance, let’s create a tensor X where each row represents a sample.

1 2 3 4

# define a tensor 'X' with multiple rows X = torch.tensor([[1.0, 2.0],                   [3.0, 4.0],                   [5.0, 6.0]])

For prediction, we’ll use the same function as above.

1 2 3

# Making predictions for Multi-Dimensional tensor "X" y_pred = forward(X) print("Predictions for 'X': ", y_pred)

which prints

1 2 3

Predictions for 'X':  tensor([[12.],         [26.],         [40.]], grad_fn=<AddBackward0>)

As you can see, we have obtained the result for multiple input variables.

Using Linear Class for Multilinear Regression

Instead of writing the functions from scratch, we can use PyTorch’s own built-in class Linear for making predictions. This is more useful while building the complex and powerful model architectures.

Let’s create a Linear model and make predictions for the same tensor X defined above. Here we’ll define two parameters:

• in_features: represents the number of input variables X and number of model weights, which in this case is 2.
• out_features: represents number of output/predicted values, which in this case is 1.

1 2	# using Pytorch's own built-in fuction to define the LR model lr_model = torch.nn.Linear(in_features=2, out_features=1)

Now, let’s make predictions for X using our lr_model object, with randomly initialized weights and bias.

1 2 3

# Making predictions for X y_pred = lr_model(X) print("Predictions for 'X': ", y_pred)

The output in this case is as follows:

1 2 3

Predictions for 'X':  tensor([[-0.5754],         [-1.2430],         [-1.9106]], grad_fn=<AddmmBackward0>)

Note not the value but the shape of the output. This is same as the previous case when we used the matrix multiplication.

Creating Custom Modules with nn.Module

Alternatively, we can also create custom modules for our linear models. While this may seem redundant for the time being, this can be the requirement when we build state-of-the-art neural networks.

Note that custom modules are objects and classes. In this case, we’ll define a linear regression class LR and make it a subclass of the package nn.Module. Consequently, all the methods and attributes inside the nn.Module package will be inherited.

We’ll define the size of the input and output, namely input_features and output_features, in the arguments of the constructor. Plus, we’ll call super() in the object constructor which enables us to use methods and attributes from the parent class nn.Module. Now we can use the torch.nn.Linear object and define the arguments input_features and output_features inside.

Lastly, for making predictions, we’ll define the forward function.

1 2 3 4 5 6 7 8 9 10 11

... # creating custom modules with package 'nn.Module' class LR(torch.nn.Module):     # Object Constructor     def __init__(self, input_features, output_features):         super().__init__()         self.linear = torch.nn.Linear(input_features, output_features)     # define the forward function for prediction     def forward(self, x):         y_pred = self.linear(x)         return y_pred

We’ll build our linear regression model with two inputs and one output as follows.

1 2	# build the model object LR_model = LR(2, 1)

Now let’s make predictions again using our custom module for the tensor X having multiple input samples.

1 2 3

# make predictions for multiple input samples of 'X' y_pred  = LR_model(X) print("Predictions for 'X': ", y_pred)

which prints

1 2 3

Predictions for 'X':  tensor([[0.3405],         [0.5596],         [0.7787]], grad_fn=<AddmmBackward0>)

Using the parameters() method, we can obtain the list of randomly initialized parameters.

1	print(list(LR_model.parameters()))

which prints

1 2 3

[Parameter containing: tensor([[ 0.6496, -0.1549]], requires_grad=True), Parameter containing: tensor([0.1427], requires_grad=True)]

Alternatively, we can also use state_dict() method to check the parameters of the model.

Putting everything together, the following is the complete code to create multilinear regression models in different ways:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

import torch   # Setting weights and bias w = torch.tensor([[3.0],                   [4.0]], requires_grad=True) b = torch.tensor([[1.0]], requires_grad=True)   # Defining our forward function for prediction def forward(x):     # using .mm module for matrix multiplication     y_pred = torch.mm(x, w) + b     return y_pred   # define a tensor 'x' x = torch.tensor([[2.0, 4.0]]) # predict the value with forward function y_pred = forward(x) # show the result print("Printing Prediction: ", y_pred)   # define a tensor 'X' with multiple rows X = torch.tensor([[1.0, 2.0],                   [3.0, 4.0],                   [5.0, 6.0]])   # Making predictions for Multi-Dimensional tensor "X" y_pred = forward(X) print("Predictions for 'X': ", y_pred)   # using Pytorch's own built-in fuction to define the LR model lr_model = torch.nn.Linear(in_features=2, out_features=1)   # Making predictions for X y_pred = lr_model(X) print("Predictions for 'X': ", y_pred)   # creating custom modules with package 'nn.Module' class LR(torch.nn.Module):     # Object Constructor     def __init__(self, input_features, output_features):         super().__init__()         self.linear = torch.nn.Linear(input_features, output_features)     # define the forward function for prediction     def forward(self, x):         y_pred = self.linear(x)         return y_pred   # build the model object LR_model = LR(2, 1)   # make predictions for multiple input samples of 'X' y_pred  = LR_model(X) print("Predictions for 'X': ", y_pred)   print(list(LR_model.parameters()))

Summary

In this tutorial, you learned how you can make predictions using multilinear regression models. Particularly, you learned:

• How to review linear regression in multiple dimensions.
• How to make predictions with multilinear regression model using PyTorch.
• How to use class Linear for multilinear regression in PyTorch.
• How to build custom modules using nn.Module in PyTorch.