Table of Contents Show
In this interview, we conversed with Rishabh Mehrotra, Director, Machine Learning at ShareChat.
He has over 12 years of experience in machine learning (ML), leading projects from basic research to production with over 10 product launches for 350 million+ users.
He takes us through the exciting world of AI in eCommerce and personalized recommendations, which eases the purchase process for customers.
Companies excelling at personalization generate 40% more revenue than their counterparts. Let’s look at AI’s role in personalization to boost customer experience.
What is personalization?
To Rishabh, personalization in its simplest form is:
- Understanding user needs
- Developing user-sophisticated understanding
- Sharing content and services tailored to user needs
eCommerce growth leaders need to look into buyers’ interests and behavioral profiles and create offerings that engage effectively.
For instance, Spotify can leverage data gathered from a user ever since they joined the platform, predict the customer journey in the next six to twelve months, and recommend music preemptively.
The motivation behind getting into personalization
Rishabh began his journey with ML in 2013 when he undertook a Ph.D. in search and recommendation systems. He fell in love with personalization owing to its “omnipresence” on online platforms.
“It’s 2022…if you’re not employing data to your advantage then you’re missing out.”
— Rishabh Mehrotra, Director, Machine Learning at ShareChat
AI in eCommerce for personalization is already widespread on social media and food delivery platforms. But in the future, its scope could widen to finance and insurance companies. Audiences will see tailored recommendations based on credit risk profiles.
Another area is healthcare, where tech experts are trialing personalized medicine. This will provide opportunities to offer dedicated treatment plans based on medical history.
This magic of AI in eCommerce motivates Rishabh daily to develop models for use across industries.
Scope of personalization
Taking ed-tech as an example, leading platforms like Byju’s and Unacademy are deploying advanced machine learning models for crafting tailored curriculums relative to each student’s progress and capabilities. They can predict where a student may face difficulties and adapt the course to suit their pace of learning.
Moreover, where school and college syllabuses haven’t changed in twenty years, online personalized learning platforms instantaneously customize their curriculums with new and relevant topics.
Personalization can propel growth for every stakeholder
Not just for users, personalized product recommendations are crucial for stakeholder growth and should be designed as such.
Consider this. Zomato, Swiggy, and Uber Eats benefit restaurants and delivery partners, boosting economic growth. Users follow popular recommendations to select restaurants, pushing more footfalls to restaurants and orders for delivery personnel relevant to each user.
Platforms must also ensure that machine learning eCommerce systems shift consumption towards stakeholders in a niche space.
For instance, a user may love collecting miniature cars, an activity with niche demand. It’s the platform’s responsibility to expose buyers to the relevant seller.
But how can platforms personalize offerings with no information on the buyer? They can use ML models for intentional, intelligent exploration to improve product discovery.
The inner workings of personalized recommendations
Personalization models develop a user profile based on their history and interests. They assign vectors to build a user representation and map them to matching topics in the product catalog. This is how traditional recommendation systems work.
Today, AI in eCommerce uses models built on advanced neural networks, deep learning, and millions of parameters to develop user representations. This predicts the probability of a user’s response to a given content piece for more advanced personalization.
Overcoming recommendation fatigue
Now when the model has learned a user representation, it’s based on the user’s interaction with, let’s say, topics A and B. The model has information only on A and B and skews its recommendations to these.
On the other hand are contextual bandit models like Rishabh developed at Spotify. If the platform recommends 10 items, 8 will be based on user representation, while 2 will be random. The model conducts exploration for diverse recommendations to find new user interests and constantly updates the user representation.
How can small businesses leverage the benefits of personalization
Small businesses may not have the resources to deploy ML extensively, but they have more control over their website and conversions. It gives them opportunities to adapt in real-time to topical events or festivals.
For example, when Diwali is around the corner, brands can collect product data in their catalog and promote higher-selling products. Businesses may not personalize this approach for individual users but aggregately customize it to user groups.
Alternatively, businesses can offer subscription bundles on websites like Amazon or Alibaba to push more products at a discount.
Another option is to use personalization plugins that take the business data, analyze it and set up customer product recommendations.
Dealing with data challenges
Clean data is an ongoing challenge for most businesses.
Models may be trained on insufficient data, and when they’re queried in the real world, user recommendations are way off because the data used is old. Subsequently, there’s a model drift, and companies will notice a dip in consumer engagement and retention.
Here, monitoring data quality is crucial for success. Businesses shouldn’t underestimate the value of clean data and ensure they have the proper monitoring dashboards for regular, consistent checks.
They could also have operations or category managers oversee each category’s data. Operations or category managers can dig out relevant and rich unstructured data, which they can use for intelligent product recommendations.
Measuring ROI on personalization
Rishabh suggests three ways by which businesses can measure ROI for AI in eCommerce:
AB testing
Companies divide a scenario into control and treatment. Control is where they don’t introduce any change. Whereas in treatment, the intervention is introduced. The delta jump can be measured and attributed to the cause.
AB testing may not always be practical, especially for large brands like Amazon, which can’t stop product promotions. That’s where Counterfactual Estimation helps.
Counterfactual estimation
Businesses can plot product data they’ve seen previously, and when they introduce an intervention (offer, discount, price change), compare the delta to past outcomes. This delta could be conversions, engagement, or other customer attributes.
For instance, in healthcare, data scientists can run counterfactual estimation to predict patient progress if they’re introduced to treatment now versus later.
Time series
Data scientists conduct trend analysis wherein an intervention is introduced into the time series during a given period. And its effect on the time series is studied.
Businesses must be cognizant of seasonal effects (like festivals) on the time series and account for them.
Diving into the Spotify personalization engine
Rishabh and his team figured that Spotify wasn’t balancing familiarity with discovery among users and playlists. If a user liked a particular song, the platform only recommended other songs from the same artist (familiarity) and not similar music or genres (discovery).
Not all users had the same consumption propensity for familiarity or discovery. The team found a balance by deploying user-level prediction through decoupled, advanced machine learning models that identified where to introduce familiarity and discovery.
For example, users looking for the Top Hits in India playlist need familiarity, whereas those looking for 80s Hits want to discover.
The results:
- 7% short-term gains
- 9% 7-day return rate
- Increase in user-artist connections
Timeframe for personalization deployment
While it’s hard to estimate timelines for AI in eCommerce projects because multiple factors are involved, businesses can still gather the evidence before deployment.
A quick analysis of a small data set can show expected changes on a larger scale and if it’s worth moving forward with the model.
There are other methods for quick deployments, such as pre-trained models created by large companies and available for ready use. At the same time, different models can be trained on low code and are less expensive.
Personalization is shaping digital experiences for the next billion internet users
Internet users in the past decade come from a technological background or are tech-savvy. However, the internet revolution in India expanded its reach into tier 2 and tier 3 cities, where new users are coming online quickly.
The behavior of these new users is very different from previous ones in content consumption (formats and languages). They’re looking for more local, regional social content distribution platforms.
But Facebook isn’t spending time understanding regional nuances because it’s challenging to have multiple recommendation engines for various languages and source content for each. Regional creators have different ways of working, too.
As such, it’s imperative to develop tools for the next billion internet users by simplifying user experiences, understanding cultural nuances, and being inclusive.
The bottom line
Personalization as a marketing tool is quickly gaining ground. Nearly 25% of marketers achieved revenue growth of more than 20% due to personalization.
What’s important to remember is that recommendation engines must consider all stakeholders, including users and sellers. Their impact goes beyond business revenue, translating into economic and societal growth.
AI in eCommerce is opening up multiple avenues in personalization via machine learning, neural networks, deep learning, and the like. Organizations deploying personalization and recommendation engines can effectively boost customer experience and retention for tremendous success.
With ewiz commerce’s built-in AI-powered marketing tools, you can create personalized experiences across multiple channels.
