The Master Plan
AI is the future, Everyone knows that but few have the courage to invest in R&D, have the patience to endure the setbacks and trust in the team’s work. Our aim was to give the best AI’s to our clients, exceeding their expectations, ease their life, and push the boundary of social media AI. Like all great plans have humble beginnings, our journey of AI at Locobuzz started with Sentiment Analysis.
Data is the New Gold
To train an AI, immense amounts of data is required. We call it Dataset. We initially started with a 7,000 dataset, trained the sentiment AI and our accuracy was 63.2%. It was quite apparent that more data was required. So we search externally and internally for quality data. Locobuzz has a well managed Terabyte scaled database. Using this we were able to accumulate a large dataset but that was not enough. We needed variety hence we went to the internet, salvaged every bit of data we can get our hands on. Through all the hardships, we were able to accumulate a 9.3 Million dataset, spread across 38 Global Languages.
From Crude Oil to Petrol
Raw data is akin to Crude oil, very precious but of no significant value. Some major impurities with raw sentiment data are mentioned below
1.ORM executives have contrasting perceptions of sentiment ex. “Nice” can be tagged as positive by one agent and Neutral by another.
2.#LoveMyCountry is not understood by AI unless you split the word into “Love my country”.
3.Thai language has no space between its words.
So we required an extremely efficient refinement process hence we created Generic Preprocessor. It took 3 months of effort by our data scientists to create and refine. Everyday we add something new or redesign some old algorithm. At last our efforts paid off. The Generic Preprocessor can process any kind of NLP related data in more than 30 languages. It can resolve more than 50 issues in the data. This gives our NLP AI a significant boost in performance.
Making The Smart Kid
AI is akin to a child. At birth, It's pretty much a simpleton but as it grows it gains experience and knowledge. Some gain experience faster compared to the rest, hence we call them Smart. For Sentiment we were looking for such a smart algorithm. After months of experimentation, refinement and retry we were able to make it and not just one but two algorithms. We started using these two algorithms in hybrid mode. Locobuzz now has an bleeding edge Hybrid Sentiment AI that is fast to train and is very accurate. Compared to before, new AI gives 76.3% accuracy on the same 7,000 dataset.
A Good Teacher and A Good School
Smart AI requires a customised training procedure aka good teacher. Our Sentiment Algorithm has 2.4 Million parameters. It's equivalent to a maths equation with 2.4 Million parameters. When all the 2.4 Million parameters are set to correct values then and then only the sentiment gives the correct result. This requires a statically correct training procedure and experience of the data scientist. To perfect the Training procedure we required a cluster of High computationally intensive Graphic Processors (aka good school) which can handle the computations of 2.4 Million parameters. These processors are extremely costly and are paid by minutes of usage. So we had to find the best training procedure with the least amount of trials. It's a daunting task and our team did it very well.
Standing Strong in the Tsunami
After Months of hard work, we were able to pull the entire system together and created a highly accurate AI which gives an accuracy of 94.3%. Now our challenge was the volume of sentiment requests. On a Tsunami day, there are above 1500 sentiment requests per sec in about 30 different languages. So we needed to create an API engine that can support such storms. So our team of data engineers created SentimentAPIEngine. This system supports multithreaded request handling and no-downtime updates. This allows up to function 24 hours/365 days without any downtimes. Helping us cater our clients without any glitch.
Measure of Success
For every AI product, we follow Zero tolerance policy. We have a log of all the complaints we receive for every client we cater to. The goal of the data science team is Zero complaints and this is also the measure of our success. As we receive a complaint it goes through the verification and correction process. The model is updated every week and there are continuous revisions. We started realizing the value of what we were able to deliver for our clients and that was all the motivation. Today, we have state-of-art sentiment analysis with great accuracy, and we are still pushing our limits to achieve Zero tolerance.
A basket full of AI
Locobuzz has heavily invested in Computer Vision, Clustering technologies, and recommendation engines. Our Key projects are Object Detection, Obscure Optical Character Detection systems, Smart Reply system, Smart Compose System, contextual word cloud … etc. Similar to our sentiment AI, we have carefully worked on each of these systems and tried our very best to make them zero complaints. These AI required a lot of effort to make them generalized for every client but our team's ingenuity and creative ideas made it possible. We are very proud to say that each of these technologies is setting new industry standards.
Locobuzz is a SaaS platform that converges with technologies such as AI, ML, Big data and Analytics to provide brands with a 360-degree customer experience management suite. Locobuzz’s powerful analytics algorithms have helped seasoned brands establish a strong foothold in the digital hemisphere through the COVID19 crisis period. Visit our website www.locobuzz.com for more information on our CX management services that are catered toward businesses like yours!
Lead Data Scientist
Locobuzz Solutions Pvt. Ltd.