TensorFlow, PyTorch, and JAX: Choosing a Deep Learning Framework

Deep learning changes our lives more or less every day. Whether it’s Siri or Alexa following our voice commands, real-time translation apps on our phones, or computer vision technology enabling smart tractors, warehouse robots and self-driving cars, every month seems to bring new advancements. . And almost all of these deep learning applications are written in one of three frameworks: TensorFlow, PyTorch, and JAX.

Which of these deep learning frameworks should you use? In this article, we’ll take a high-level comparative look at TensorFlow, PyTorch, and JAX. We’ll do our best to give you an idea of ​​the types of apps that play to their strengths, as well as consider factors like community support and ease of use.

Should you use TensorFlow?

“No one was ever fired for buying IBM” was the rallying cry of computer science in the 1970s and 1980s, and the same could be said of the use of TensorFlow in the 2010s for learning in depth. But as we all know, IBM fell by the wayside as the 1990s dawned. Is TensorFlow still competitive in this new decade, seven years after its initial release in 2015?

Well, certainly. It’s not like TensorFlow has stood still this whole time. TensorFlow 1.x was all about building static graphs in a very non-Python way, but with the TensorFlow 2.x line you can also build models using the “eager” mode for immediate evaluation of operations, which makes things much more enjoyable. like PyTorch. At the top level, TensorFlow gives you Keras for easier development, and at the low level, it gives you the Accelerated Linear Algebra (XLA) optimization compiler for speed. XLA works wonders for increasing performance on GPUs, and it’s the primary method for harnessing the power of Google’s TPUs (Tensor Processing Units), which deliver unparalleled performance for large-scale model training.

Then there are all the things that TensorFlow has been doing well for years. Do you need to serve models in a well-defined and repeatable way on a mature platform? TensorFlow Serving is here for you. Do you need to retarget your model deployments for the web, or for low-power computing such as smartphones, or for resource-constrained devices like IoT devices? Both TensorFlow.js and TensorFlow Lite are very mature at this point. And of course, since Google still runs 100% of its production deployments using TensorFlow, you can be sure that TensorFlow can handle your scale.

But…well, there was a certain lack of energy around the project that’s kind of hard to ignore these days. The upgrade from TensorFlow 1.x to TensorFlow 2.x was, in a word, brutal. Some companies have looked at the effort required to update their code to work properly on the new major release and decided to port their code to PyTorch instead. TensorFlow has also lost steam in the research community, which began to prefer the flexibility offered by PyTorch a few years ago, leading to a decline in the use of TensorFlow in research papers.

Copyright © 2022 IDG Communications, Inc.

Leave a Reply

%d bloggers like this: