Here’s Everything You Need To Know About GPT

What Is A GPT?

Generative pre-trained transformers (GPTs) are a type of large language model (LLM) that are used in generative artificial intelligence (GenAI). GPTs can also be defined as artificial neural networks (ANNs) being used in natural language processing (NLP).

GPTs are built on a deep learning architecture called a transformer, pre-trained on large data sets of unlabelled text and able to generate human-like content.

What Is OpenAI’s GPT-3, 4 And Other GPT Variants?

Launched in 2018, OpenAI’s GPT-1 was the first-ever GPT model. Since then, the Microsoft-backed startup valued at $86 Bn has launched four more foundational GPT models, aimed at generating content. 

GPT-1 had 117 Mn parameters (internal variables that the model uses to make predictions or decisions) and was trained on 4.5 GB of data, at a training cost (amount of computing resources needed to train an LLM) of 1 petaFLOP/s-day (1020 operations a day). 

GPT-4, in contrast, has 1.7 Tn parameters, a million times more than GPT-1, and OpenAI has not publicly stated the corpus of data it was trained on. While the decacorn also did not specify the training cost for GPT-4, it has been estimated to be around 21,000 petaFLOP/s-day (around 2.1*1025 operations a day).

Coupled with its capabilities, including learning from human preferences (LFHP) and multimodality (ability to generate output across multiple media forms), GPT-4 is one of the most advanced GPT models to date.

Other GPT models include Google’s PaLM and Gemini and Meta’s LLaMA, among others. For comparison, Google’s PaLM has 540 Bn parameters, landing it somewhere between GPT-3.5 and GPT-4 in terms of sheer computing power.

What Are The Applications Of GPTs?

GPTs are designed in a way that they enable a wide range of downstream applications. This involves fine-tuning the GPT models to adapt to certain applications, such as chatbots. As such, this flexibility gives current GPTs the capability to adapt to applications such as:

• Content Creation: One of the most popular and important applications of GPTs has been to generate content. GPTs are being used to generate text, code, music, images, videos and more. They can also translate languages with increasing accuracy and fluency, as GPT-4 is being touted by OpenAI as its most accurate model to date in this regard.
• Customer Service & Communication: GPT-powered chatbots can handle customer inquiries and resolve issues more efficiently, freeing up human agents for more complex tasks. GPTs can also be used to generate content for social media platforms, manage online communities and respond to customer comments. Dozens of Indian startups, including Swiggy and Zomato, have automated their chat support via GPT-based tools.
• Education & Learning: GPTs can generate personalised learning plans for individuals based on their needs and learning pace. Further, GPTs can also act as interactive tutors, something that Indian and global edtech companies are doing. For instance, Indian edtech startups like BYJU’S have included GPT-based features in their products and interfaces.
• Healthcare: GPTs are being explored for their potential to analyse medical data and assist doctors in diagnosis and treatment planning. Further, they are also being used to generate new drug candidates and predict their potential effectiveness, accelerating the drug discovery process. Indian healthtech startups are employing GPT-based solutions for use cases such as customer interaction and analysing medical data, among others.

What Are Some Of The Concerns Regarding GPTs?

• Bias and Discrimination: GPTs are trained on large datasets of text, which can reflect societal biases and prejudices. This can lead to outputs that are discriminatory or offensive.
• Factual Accuracy and Misinformation: GPTs can generate text that sounds convincing but could be factually incorrect. This can be especially dangerous in areas like news reporting, healthcare, and scientific research, where inaccurate information can have serious consequences.
• Misuse: Bad actors are using GPTs to generate deepfakes, create fake news, or launch phishing attacks. The recent wave of deepfakes of popular figures across India and globally has become a cause of concern for governments across the world.
• Economic Impact: As GPTs become more sophisticated, they could automate jobs currently done by humans, leading to unemployment and economic disruption. Several companies are already replacing humans with GPT-based AI models in functions such as online customer support. In December 2023, Paytm sacked hundreds of employees citing AI-based automation across operations and marketing.
• Lack of Transparency and Explainability: GPT models are poorly understood in the way they arrive at their outputs, making it hard to assess reliability and address potential biases. This lack of transparency can hinder trust and accountability in their use.

Which Indian Startups Are Currently Working With GPTs?

Ever since the launch of GPT-3.5 in March 2022 and the subsequent launch of ChatGPT in November 2022, Indian startups have started developing AI-based use cases. Startups across segments such as edtech, fintech, ecommerce, enterprise tech and consumer services have developed AI-based use cases and features within their products.

However, many Indian startups are also developing in-house GPT models to develop highly specific use cases across content generation and no-code platforms. Some examples include Kombai and GlazeGPT.