The Dawn Of The TPU Era: Can India’s AI Ecosystem End GPU Dependency?

Remember the bike racing game back in your school days? Life was offline then, toggling between Road Rash and Solitaire on your mammoth CRT monitor. 

The computer perhaps changed more than it changed life for us since then. 

And, in sync changed its brain, perhaps in an inverse dynamics. It shrank to create more room for virtual data. The quintessential Central Processing Unit – the CPU, as we call the brain of our good-old computers – was taken over by the Graphics Processing Unit (GPU) as games evolved and content diversified, until we turned more artificially intelligent, calling for the more advanced Tensor Processing Unit (TPU).  

But, what is this all about? Nothing much – just a smarter brain for a wiser computing that deals with super snazzy gaming, high resolution videography, and a huge pool of data. In a nutshell, it accelerates machine learning (ML) and Artificial Intelligence (AI) workloads. 

TPUs recently hit the headlines when OpenAI turned to Google to use its AI chips to power ChatGPT. OpenAI started renting Google’s TPUs, which could potentially help it lower the inference cost while also diversifying its dependence beyond leading chip maker NVIDIA and tech major Microsoft.

And, what is this inference cost? It is the cost an AI company bears for compute time, memory, and data transfer to help trained models arrive at a conclusion or make a prediction from analysing the existing  set of data. 

Although OpenAI later said that it was only “conducting early testing” with Google’s TPUs and had no plans to use them at scale, one cannot help asking: what are the implications of TPUs? Or, to start with, what are TPUs, how do they stand out from GPUs, and how are they going to change the AI model development process?

Before OpenAI, Anthropic announced its partnership with Google Cloud in 2023 to access its GPU and TPU clusters to train, scale, and deploy its AI systems. Even Apple used Google’s TPUs to train two of its AI models recently.

As AI gets deeper into our lives every passing day, there’s an increasing demand for compute systems across the world. Can Google’s specialised chips accelerate the AI journey?

 

Let’s dive deep to get into the multilinear link across data sets in the world of tensor processing. 

Understanding The TPU 

Google created a kind of computer chip a few years ago to help power its giant AI systems. These were designed for the complex processes that could be a key to the future of the computer industry. The internet giant said it would allow others to buy access to those chips through its cloud-computing service as it hopes to build a new business around the tensor processing units.

The tech giant started deploying these application-specific integrated circuits (ASICs) or TPUs back in 2015. Its first chip was TPU v1. Google kept upgrading the technology in tandem with the evolution in AI and ML. 

The TPU, which can be referred to as an AI accelerator, is used in building agents, recommendation engines, and personalisation models in code generation, media content creation, synthetic speech, vision services, and more. Google used TPUs in Search, Photos, Maps, and even in YouTube, and went on to build its large language model (LLM) Gemini and other AI-based models like Imagen and Gemma using the TPUs.

Google rolled out its sixth-generation TPU, Trillium, with a 4.7-fold increase in peak compute performance from TPU v5e, designed to transform AI development economics and advance machine learning and boost energy efficiency by 67%. Earlier this year, it introduced Ironwood, which can compute 5 times faster than Trillium. 

Besides top AI companies, Deloitte and some American startups like Essential AI and Deep Genomics have tied up with Google to use its Cloud TPUs. German healthcare major Bayer also uses TPUs for quantum-scale drug discovery. 

But, what turns the tide from GPUs to TPUs? 

“While GPUs were primarily used for graphical displays, some of the paradigms that are used in rendering the graphics are also used very heavily in AI, such as mathematical computations like matrix multiplication. At the core of all these chips, whether the CPUs, GPUs, or ASICs like TPUs, is how efficiently and fast they can run large matrix multiplication and move data,” explained Neeraj Poddar, cofounder and CTO of NimbleEdge.

Why GPU To TPU Conversion Matters

In this age of artificial intelligence, what is more precious than gold is data. Around 181 zettabytes of data is generated across the world every day. One zettabyte is equal to one followed by 21 zeroes of bytes. And, it’s growing at 22.8% every year. 

This huge pool of data needs an increasingly faster computing system for analysis. TPU is the present-day milestone to the sustained evolution of computing systems from CPUs to GPUs and onwards. This exponential spike in data generation has propelled the growth of Data Centres (DCs) across the world. 

Data centres that are equipped to handle AI processing loads would require $5.2 Tn of capital expenditures by 2030. And, about 60% of it, or $3.1 Tn, will go to technology developers and designers who produce chips and computing hardware for DCs. Another 25% of the remaining will be spent on energy systems to run these power-guzzlers.

At a time when AI models and application developers look out for efficient hardware and servers that can help cut costs while improving the time for training and inference, the chips supporting them should not only be efficient, but also be able to cut costs, while lowering energy consumption.

Google’s TPUs frame the answer to these needs. While the comparative cost of TPUs are still slightly higher than NVIDIA GPUs (differs based on the variants), they have managed to lower the per-watt power consumption significantly, setting its market on course to reach $22.57 Bn in 2029 at a 34% CAGR, outpacing GPUs.

NimbleEdge’s Poddar said that while GPUs are more like general accelerators that have the capability to do multiple operations in parallel, which helps in faster matrix multiplication, TPUs are built with some specialised circuitry that are more oriented towards AI-ML development, but they can’t be used for graphic processing. “TPUs undoubtedly are more beneficial because they have been written specifically for AI and ML training.”

Can TPUs completely outdo GPUs? Perhaps, no, and the reason is more than just the chips.

Karan Kirpalani, chief product officer at Neysa.ai, said GPUs, particularly from NVIDIA, gained market traction because of their libraries that developers have access to. “A GPU is a silicon, with which we can’t directly interact. We need a library of tools and frameworks to be able to interact with that silicon. That’s where frameworks like CUDA by NVIDIA help developers write software applications and models, giving most of the advantages of NVIDIA’s silicon.”

He believes that this access to its advanced tools and architectures is what made NVIDIA’s market, which is hard for anyone – whether other companies like AMD or Intel, or Google’s TPUs – to crack.

Google’s TPUs also have a similar library of tools that developers can get access to, such as TensorFlow, JAX, and PyTorch/XLA. But, these are open source, and developers can avail them even using GPUs.

Neysa, which offers AI acceleration cloud systems to large and medium enterprises, has more than 95% of its GPU inventory comprising NVIDIA chips. “It’s very easy to fine-tune models using all of the libraries and frameworks and the native integrations that are present within the NVIDIA CUDA ecosystem. That’s what our clients want.”

How India Stands To Benefit?

Artificial intelligence is not merely an advancement, but a challenge for India to stay the course to emerge as a $1-Tn digital economy and a global technology powerhouse. 

The government’s INR 10,300 Cr IndiaAI Mission has been designed to strengthen the country’s AI capabilities with 18,693 GPUs in operation that will make it one of the most extensive AI compute infrastructures globally. 

The emergence of TPUs as a smarter replacement for GPUs throws open a huge possibility for the world’s third-largest startup ecosystem. But AI startups in India are often found grappling with poor access to the best compute systems. The country’s nascent-yet-fast-evolving AI industry believes that it will take years before smaller and newer companies can leverage TPUs. 

Shunya Labs cofounder and CTO Sourav Banerjee pointed out that it is not only easier to get access to NVIDIA’s CUDA framework of GPUs, compared to the TPUs that require a lot of learning.

“In some of the most advanced matrix manipulations, TPUs outperform GPUs hands down. They can go up to 30 times faster. But, can a startup implement TPUs from day one? The answer is no, because you don’t get enough resources, you have less budget to train your current resources on TPU,” he pointed out. 

“There is also a strong community support in GPUs because they are a decade old. Most of the models launched in open domain are also GPU compatible, and not TPU compatible.” 

In fact, India and the entire Southeast Asia have access only to Cloud TPU v5e, which was its fourth iteration. The Google website shows that even its sixth generation TPU, Trillium, is generally available in North America, Europe, and parts of Asia.

This is more about dependence on the global ecosystem that has been created by NVIDIA alone, and today, even as other tech giants aim to capture the market for GPUs, it could take decades before the dominance is broken. Industry leaders believe that it will be broken by bigger companies only. 

NVIDIA stands in the GPU domain like nothing short of a monopoly. As of June 2025, the US major held a massive 94% share of the global PC-based graphics add-in board market, which reached 11.6 Mn units in Q2. The remaining 6% slice of the pie rests with AMD. 

In data centres, however, NVIDIA has around 25% of market share. The company is now betting on a data centre GPU market that’s expected to average 21% annual growth rate to reach $81.07 Bn by 2030, and also developing better GPUs to cut the reliance on data centres altogether.

Soket AI’s founder Abhishek Upperwal, however, said the use of ASICs like TPU or Language Processing Units (LPUs) developed by Groq will mostly find their initial adoption for inference rather than model training. 

In fact, he believes that bigger companies like OpenAI could be opting for Google’s TPUs as they are trying to do inferencing in a more distributed manner. Or, its internal team could be experimenting with the TPUs with small scale training runs.

As a builder, it’s difficult for an Indian startup like Soket AI to rely on a single cloud provider, as it increases risks and constraints model training scalability.

So, even as the demand for more compute rises, it might still be a few years before TPUs become mainstream. Even as a few Indian companies are also trying to build ASICs for AI development, they will largely depend on GPUs for training the models.

India, on its part, gears up to take a chip shot as it looks at an AI landscape that would add $1–1.7 Tn to the economy by 2035, while driving productivity in banking, manufacturing, pharma, and IT innovation. 

The rise of a parallel competitive market is being cheered by the AI enthusiasts, and many believe that in most cases where TPUs are being adopted, they will also be used alongside GPUs, which would help companies diversify and break the hardware limitations. Amid all this din, the big question remains: How far does Google want to open up its TPU infrastructure, especially when its competition becomes its customer? 

OpenAI-led ChatGPT has a 700-word answer, split into four sections, delivered in less than a second. 

[Edited By Kumar Chatterjee]