Rasbert-bot | An API that improves the customer experience

Hello again!
This time I will be telling you a little of the adventure it was for me and my team to develop this API.

First of all, my team, ZenCode 🚀 :

• Julián David Alejandro Álvarez Caro | LinkedIn / GitHub
• Jiseth Peña Arias | LinkedIn / GitHub
• Santiago Pinzón Domínguez | LinkedIn / GitHub
• Juan Camilo Villa Hernández | LinkedIn / GitHub
• Oscar Javier de León Giraldo | LinkedIn / GitHub

And our website: Rasbert-bot

We worked on this with mentoring from Vozy. A company dedicated to helping companies improve their customer service with a conversational artificial intelligence called Lili.

The idea behind this project was to improve how Lili answers questions and handles situations from a specific domain since sometimes it is difficult for her to understand some of the jargon and therefore, her next actions may not have the outcome that the customer needs.
We wanted to improve this by integrating two main components: Rasa and BERT (you can tell now where the name came from, huh?). BERT is an AI algorithm from Google.

Well, this meant a lot of challenges for us. We didn’t have a background in AI, machine learning, or anything like that. So first we had to learn the basic concepts and the technologies that would be used.

Here is the stack:

From this stack the only one we had a close experience with was Django!

Learning the “basic” concepts of machine learning took us some time. After that Rasa was another bump in the road; The installation seemed easy but actually, it caused a lot of troubles and conflicts in our computers, and the documentation about those particular issues was not that abundant. We were only able to solve them on one computer, which would be the chosen one to run Rasa.

I’ve mentioned Rasa, but what does that do? Why is it so important?
Well, this is the platform that is going to perform the training of the models, that is going to test them, and where we are going to define the data as we need it and considered it better.

The data… Ohhh, the data. We had almost 17.000 rows of data in our dataset. First, taking that from a spreadsheet to Rasa took us some time but that wasn’t the whole process, we had to make sure that all (or most) of the data was actually useful. Some random words that the Speech To Text had mistakenly understood had no use in this domain we were working with (the insurance domain) and had to be deleted. Another important task here was to enrich the dataset with more examples that we considered appropriate for the intention detected from the user.

Yeah, as you may be thinking, this part was not that fun and it was constant work.

Then comes the pipeline, which dictates the configuration of the model, and how the training process is going to be. We had to understand the components of a basic pipeline to be sure if it was useful or not, and how could we improve it. Understanding this part, learning how to use it, and actually finding a pipeline that would fit our purpose took most of the time.

As for the training, it was a simple process, just one command, and the model would start the training with the data and pipeline we had. The problem here was that the training could take from 30 minutes to 4 hours. Luckily we didn’t need a model all the time. As mentioned before, our work was mostly focused on finding a pipeline that would suit our needs, so what we needed was the results of testing the data and pipeline as we had it using K-fold cross-validation. This would not take between 30 minutes and 4 hours…. this would take up to 12 hours! (At least in the computer we had destined for this).
This made the trial and error process a lot harder. We couldn’t test every option we could think of, we had to make choices on what to test and how to test it. And if the results were better or worse we had to have the certainty of what caused it.

At this point you might be asking what we managed to achieve at the end, well, this is the result of the cross-validation of the model that we had at the end:

BERT model: BETO cased base

The Intent classification was not our main concern, it was the Entity recognition and we manage to get quite good results here (the closer the number is to 1, the better).

And if for some of the readers comes helpful, I’ll leave here the pipeline we used even though I won’t go into details in this post

pipeline:
  - name: HFTransformersNLP
    model_weights: "dccuchile/bert-base-spanish-wwm-cased"
    model_name: "bert"
    cache_dir: null
  - name: LanguageModelTokenizer
  - name: LanguageModelFeaturizer
  - name: RegexFeaturizer
  - name: DIETClassifier
    random_seed: 42
    use_masked_language_model: True
    epochs: 100
    number_of_transformer_layers: 4
    transformer_size: 256
    drop_rate: 0.2
    weight_sparsity: 0.7
    batch_size: [64, 256]
    embedding_dimension: 30
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: True
    model_confidence: linear_norm
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    constrain_similarities: true
    model_confidence: linear_norm
  - name: RulePolicy
    core_fallback_threshold: 0.2
    enable_fallback_prediction: True
    core_fallback_action_name: "utter_DEFAULT_FALLBACK"

This project was challenging for us in so many ways, but in the end, we accomplished great work, and great results to be a 6 weeks project.
All of this was thanks to the mentoring we got from Vozy, more specifically:

• Cristina Gómez | LinkedIn
• Juan Daniel López | LinkedIn

Thank you guys for your trust, patience, and the opportunity!

Here you can find the repository of our project:

zencode / Rasbert-bot

And a quick demo of our project:

Demo Rasbert-bot.mp4

Thanks for reading this!
Have a good one!