PyTorch provides a way to build neural networks simply and train them efficiently, which has led to PyTorch becoming the most popular framework used in research. It’s industry user base continues to grow in popularity, too.
Meta developed Pytorch in 2016 on top of the Lua-based Torch package, and since then it has received many improvements from Meta as well as the huge community built around PyTorch.
6 Reasons to Use PyTorch
- Widely accessible
- Rapid model development
- Fast training times
- Major corporate investment
- High quality GPU Training Support
- Strong ecosystem
How Does PyTorch Work?
PyTorch’s basic building block is the tensor, which is like an array or matrix. These tensors encode the inputs and outputs of models, as well as their parameters. PyTorch provides many convenient building blocks for defining neural networks to prototype models quickly. The training process uses the autograd library for automatic differentiation, which is implemented in C and makes it fast to execute.
Here’s an example of the simple model definition in PyTorch; you can find many more in their tutorials:
class NeuralNetwork(nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__()
self.flatten = nn.Flatten()
self.simple_net = nn.Sequential(
nn.Linear(32*32, 256),
nn.ReLU(),
nn.Linear(256, 256),
nn.ReLU(),
nn.Linear(256, 8),
)
def forward(self, x):
x = self.flatten(x)
logits = self.simple_net(x)
return logits
PyTorch Benefits
PyTorch has become one of the leading machine learning frameworks for many reasons.
- Widely Accessible: PyTorch’s primary access is via a Python interface, which makes it widely accessible to most machine learning practitioners. PyTorch is also built with Python best practices in mind, meaning that it feels like a native Python library thereby making it intuitive for Python users.
- Rapid Model Development: Due to the many convenient functions and classes to define neural networks, one can rapidly develop machine learning models.
- Fast Training Times: The autograd differentiation package that PyTorch uses for training means it has some of the fastest training times among machine learning frameworks.
- Major Corporate Investment: PyTorch has investment from two major corporations, Meta and Microsoft, which both maintain and develop the framework. There’s also a helpful community of researchers and developers who help with PyTorch usage.
- High Quality GPU Training Support: PyTorch has quality graphics processing unit (GPU) training support that allows practitioners to train their models on better hardware, which can decrease training time and costs.
- Strong Ecosystem: There is a strong ecosystem of tools built around PyTorch including PyTorch Lightning, ONNX and Skorch. Among other things, these tools decrease development costs and make state-of-the-art models more accessible.
PyTorch Examples and Applications
Due to its strong offering, PyTorch is the go-to framework in research and has many applications in industry.
Tesla uses PyTorch for Autopilot, their self-driving technology. The company uses PyTorch to train networks to complete tasks for their computer vision applications, including object detection and depth modeling.
Microsoft is also a major PyTorch user. It’s the primary framework the company uses to develop machine learning solutions to improve Bing search and Xbox, among other products. What’s more, Microsoft also owns and develops the PyTorch for Windows applications and runs a cloud option of PyTorch through Microsoft Azure.
Another application of PyTorch is in building natural language processing (NLP) models. Airbnb uses PyTorch in their customer service department to create smart replies and recommend agent responses to customer requests.
What Are the Alternatives to PyTorch?
PyTorch is just one example of an open-source machine learning framework; there are many alternatives. Some of the most popular projects still in active development are TensorFlow, Keras, Flux, Theano and Caffe.
While PyTorch has many perks, there are downsides to using PyTorch so alternative frameworks may be better. For example, TensorFlow has built-in visualization tools to better understand the progress of the training process. Many companies also have legacy code from the pre-PyTorch launch days, which means redevelopment costs would be quite high so it’s more convenient to use existing libraries.
