How Observe.AI Is Redefining The Future Of Contact Centres

If there is one area that GenAI has prominently impacted, it is customer experience and service. Harnessing the power of the generative capability of the large language models (LLMs), enterprises globally have onboarded AI-powered chatbots and virtual agents in droves.

And it is easy to see why. Well, AI chatbots can resolve tickets raised by customers 18% faster with a success rate of 71%. This simply means increased customer satisfaction and better conversion rates. 

However, while many companies are upgrading their customer care centres (contact centres), fears of job losses loom large, along with sub-par customer satisfaction. 

Understandably, there are several things that AI still cannot do. Besides, in a country like India, multiple languages remain a major hurdle for LLMs. As a result, a large number of tickets (complaints) still end up at the employee desk.

Many companies are now realising that they need to equip humans with smart AI agents to make their respective customer support and sales generation processes more efficient.

Bengaluru-based Observe.AI, too, is on a mission to automate complex tasks for human agents and make a difference in the way companies handle customer support.  

Founded in 2017, Observe.AI has enabled more than 350 enterprises to make their contact centres more efficient through its AI stack.

The startup was incorporated by Swapnil Jain, Akash Singh and Sharath Keshava Narayana. In 2018, Jithendra Vepa joined as the CTO and later became a cofounder. Singh and Narayana exited the company in 2022.

The Genesis Of Observe.AI

Speaking with Inc42, Jain, a former technical team lead at Twitter (now X), stated that when AI was still a grey area for many in 2016, he realised that speech technology and deep learning were at an inflexion point.  

“The machines’ ability to understand human speech was advancing rapidly, and I knew there was an opportunity to build something transformative in this space.” 

During this time, voice assistants were in their early days, but a big wave of this technology had already started. Google Assistant, Siri, Alexa, and a few other assistants were launched between 2010 and 2016. These tools use natural language processing (NLP) and mostly operate on simple command-based interactions.

To understand the application of the technology better, Jain spent six months in a contact centre based in the Philippines with hundreds of agents. Here, he realised that dedicated agents and QA analysts worked hard but were constrained by manual and time-consuming processes.

However, Vepa was involved in the research and development (R&D) of speech technology after his PhD at the University of Edinburgh. He was building text-to-speech systems at Samsung. 

Jain met Vepa about a year after starting the company. Back then, he was looking for an expert who could help him build the core technology and market for Observe.AI’s quality assurance (QA) products for contact centres. 

Initially onboarded as the chief scientist at Observe.AI, Vepa soon became the cofounder and CTO of the company.

With a gamut of challenges to be tackled across communication channels, including chat and email, the founders picked the hard problem first – voice. This led to the development of a conversational intelligence platform with a range of applications.

Since launching its first product, which focussed on analytics and manual QA support for contact centres, in 2019, the company has expanded its suite to build multiple products to improve contact centre performance with real-time insights, analysis, summarisation, and more. Its tech stack comprises tools such as Knowledge AI, Summarization AI, Auto Coaching, and Auto QA, among others.

Helped by the demand boom for speech and voice technologies during the pandemic, Observe.AI claims that its business grew more than 2X. It also acquired its first large customer, National Debt Relief, in 2020.

By then, Observe AI had already raised more than $30 Mn from Nexus Venture Partners, Y Combinator, and Scale Venture Partners. In 2020, it raised $54 Mn from Menlo Ventures. So far, the startup has raised $214 Mn in total funding. It is also backed by SoftBank.

While the founders did not reveal the revenue or growth multiples, the startup counts the likes of Pearson, Cox Automotive, Accolade, and DailyPay among its top customers. Largely catering to US customers across healthcare, BFSI, travel, and homecare sectors, Observe.AI has two customers in India. Besides, a majority of its tech team is based in the country.

What’s At The Core Of Observe.AI’s Tech

Today, Observe.AI’s technology supports enterprises with quality assurance, coaching and agent assistance. Besides, it provides valuable raw data to help businesses enhance their overall processes.

The startup claims to have delivered its enterprise clients with up to 60% efficiency gains and more than 75% reduction in quality evaluation time.

“We built our technology on genuine observation, seeing the human experience firsthand, then creating AI solutions that address real needs rather than theoretical problems,” said Jain.

When Observe.AI started building products for contact centres, especially for customer service, it recognised three areas that could be targeted. 

First, there was a necessity to analyse the conversations between agents and customers for compliance and quality. This analysis was devised to improve the agent performance for a better customer satisfaction score (CSAT) and to see more sales. 

The second area of focus was to help agents fetch information more effectively from large documents so that customer and agent conversations could be more efficient and shorter. With these in place, Observe.AI decided to bring in automation with voice agents, which it recently launched by leveraging the capabilities of GenAI and LLMs.

When Observe.AI began its journey, GenAI had yet to create waves. In its initial days, the startup began working with transformer models and building its robust data pipeline. 

“AI is in our DNA. From day one, Observe.AI started building strong foundations, whether it was model training or building the core technology or data pipeline. With the advent of GenAI, we have been quick to adapt,” the CTO said. 

There are two aspects to its core technology — speech-to-text, which involves automatic speech recognition, and LLMs that can then do reasoning and analyse the text. The company claims that its LLM is built on 40 Bn parameters.

Both of its core models, including the LLM, are built in-house. However, these models are not foundational. They are built on top of open-source foundational models and trained on domain-specific data.

“While we built these technologies, the only missing piece was a text-to-speech model, which communicates back again in a human-like speech quality. That’s where the recent acquisition of Dubdub.ai came into the picture. Now we have all these three technologies to build a strong voice AI platform,” said Vepa.

Observe.AI acquired Dubdub.ai for an undisclosed amount in March to tap into its text-to-speech technology, which works in over 50 languages, including a few Indic languages and enables users to produce audio output in over 20 voices, capturing diverse age groups, genders, along with a spectrum of emotions, such as angry, excited, and sad.

Observe.AI’s Next Leap

Although India makes up a small part of its current customer base, Vepa says the company plans to develop more products for this market. However, the major barrier to this is India’s language diversity.

Therefore, in spite of spending heavily on building such capabilities, Observe.AI is more interested in acquiring companies with expertise in Indic languages. The acquisition of Dubdub was one such step.

Meanwhile, competition is growing in the conversational AI space. In India, startups like CoRover and Gnani.ai have built their in-house models to bolster speech-to-text and text-to-speech models. Startups like NoBroker, Zomato, and OfBusiness, too, have also launched their AI agents to unclog customer-end bottlenecks.

Globally, Amazon and many others have also built such capabilities. However, Vepa believes that Observe.AI will continue to stand out because a majority in the industry are struggling with high latency and hallucinations.

According to Vepa, the biggest issue with today’s voicebots is endpoint detection. When two humans speak, one person knows when to start speaking after the other person stops. “But machines do not understand this. AI must learn when to speak after a human stops speaking — neither can it pause for too long nor should it barge in during a conversation

“Besides, we still need to figure out which conversations can be handled by the machine. Therefore, co-existence is the only way forward for now,” Vepa added.

He also believes that most emerging startups in the conversational AI space will need time to achieve the level of predictability, reliability, and consistency demanded by enterprise-grade services.

As a result, the founders see a huge market. In 2025, Observe.AI aims to double down on its voice AI agents so that enterprises can see better sales conversion and CSAT with human and AI agent collaborations. 

Besides, it also wants to acquire some large US businesses with high-volume contact centre needs. From here, it would be interesting to see how Observe.AI’s capabilities give it an edge over industry giants in the conversational AI space.

[Edited By Shishir Parasher]