Lights on - PyTorch

Bite-sized articles to get you started with PyTorch

Photo by engin akyurt on Unsplash

What is PyTorch?

If you are new to Machine Learning, you would have found a lot of tutorials using Scikit-learn, SciPy, PyTorch, Keras, TensorFlow in their programs. These are some python libraries that contain some efficient tools for machine learning. PyTorch is one such open-source ML framework that is optimized for Deep Learning. It was initially developed by Facebook’s AI Research Lab (FAIR) in 2016. Unlike other frameworks, PyTorch can work with both Python and C++.

Training ML/DL models can take a lot of time depending on the size of data and model involved. PyTorch harnesses the power of GPUs for faster computation. It can distribute tasks among CPU and GPU to provide a better parallel processing experience.

Now, what’s a GPU?

GPU (Graphics Processing Unit) is a hardware component that is embedded with the CPU or can be added to your PC separately. This component is designed for parallel processing and is mostly used to render games and run software that is used for creative production (like video editing, 3D graphics rendering). It is also used to train ML models efficiently and mine cryptocurrencies for blockchains.

Check Intel’s description of GPUs to know more about what they do.

How do I install PyTorch on my PC?

The easiest way is through PyTorch’s official site.

Select your configuration and PyTorch generates a command to install components as per your selection. Always go for the latest stable version.

Installation options on PyTorch

I have got some GPU, how do I use it for my computation?

NVIDIA GPUs are popular for parallel computing and PyTorch can use your GPU to run computations. The framework has recently started to support AMD’s GPUs (ROCm) for computation. Check here for more information if you want to use your ROCm for computing. Since NVIDIA’s CUDA is the most common one in the market, I’ll continue with that throughout this series.

You might have to install a CUDA Toolkit if you already don’t have one by following the instructions from CUDA Zone.

CUDA Zone

Let’s first check your GPU specifications with this command on your PC’s command prompt,

nvidia-smi

Image by author

Now, let’s choose the latest CUDA on PyTorch’s download page and copy the command for installation.

Installation options on PyTorch

Most of the time, the issue with installation is that it takes a lot of time and freezes often before completion.

To avoid that, let’s create an environment to download PyTorch.

Open anaconda Powershell prompt and create an environment. I have given the name of the environment as pytorch , choose whatever name you like.

conda create -n pytorch

Type y to proceed to create an environment

Collecting package metadata (current_repodata.json): done
Solving environment: done
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate pytorch
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Now let’s activate the environment using the above-given command

conda activate pytorch

Paste the command that you copied from PyTorch’s page

conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch

You’ll get a list of all the packages that will be installed. Type y to proceed with the download.

Once it’s done, check if the installation was successful with the following command,

python

Try importing torch , if it was installed properly it shouldn’t throw any error. Then check if cuda is available.

Python 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

Congratulations! you have successfully installed PyTorch and CUDA on your PC. Now, let’s dive into some coding with PyTorch’s Tensors.

Tensors

You might have heard the word Tensor in your Physics class. Tensor is defined as a physical quantity that describes a multidimensional data structure (such as vectors, arrays, matrices) to represent mechanical concepts like stress and elasticity.

Tensor in PyTorch is also used to represent a data structure and can act in the same way as NumPy arrays. We already have an established library, NumPy to work with arrays and matrices. Then why do we need another one called Tensor? The biggest advantage to use a PyTorch Tensor is that it can run on both CPU and GPU, whereas NumPy can only run on a CPU.

Open your Jupiter Notebook with the environment you installed PyTorch

Image by author

Import torchon your Jupiter notebook to work with tensors. A tensor can be created in many ways.

• We can use torch.rand(row_count, column_count) it to create a simple tensor with random numbers.
• Like NumPy, we can create tensors with all zeros and ones with torch.zeros() , torch.ones() . You can also create a tensor by passing a list to it.
• You would have noticed by now that the default data type of tensor elements is float. You can change the type while creating a tensor like torch.ones(2, dtype = torch.int) .
• Like NumPy and lists, we can perform arithmetic operations on tensors by using operators + ,— ,*,/and their functions add(), sub(), mul(), div().
• You can also view a tensor in different dimensions using existing_tensor.view(dimensions) .

• You can also create tensors from NumPy from_numpy() and vice-versa numpy().

#Tensor from NumPy array
np_array = np.array([1,1])
tensor_np = torch.from_numpy(np_array)
#NumPy array from Tensor
tensor_arr = torch.rand(2,2)
num_arr = tensor_arr.numpy()

We can store our tensor into our GPU if it's available:

a = torch.rand(2,2)
if torch.cuda.is_available():
  a = a.to('cuda')
  print(f"Device tensor is stored on: {a.device}")
'''
Device tensor is stored on: cuda:0
'''

Hope you got a good idea on PyTorch and Tensors. Stay tuned to learn about Datasets and DataLoaders.