How Your Company Can Benefit from Machine Learning and NLP

The terms machine learning, artificial intelligence, neural networks, and natural language processing (NLP) are popular buzzwords nowadays because of the results these technologies can obtain in certain complex problems. For that reason, many tech companies want to include these capabilities in their products. However, deciding which of these technologies to include (and how) is not simple, mainly because the information out there is not always accurate. The purpose of this blog post is to clarify the capabilities of these technologies (focusing mostly on NLP) and to show some concrete examples of how your company can get the most out of them.

An Intro to the Terminology

We will start by briefly defining some of these concepts. These definitions are only a short introduction, but you can find more information in our AI & Machine Learning Glossary.

AI vs Machine Learning

Sometimes the terms machine learning and artificial intelligence are used interchangeably. However, these terms actually refer to different concepts.

Let’s start with artificial intelligence. There is no official consensus on what artificial intelligence means. The main idea behind the term can be found in Wikipedia’s definition: "In computer science, artificial intelligence (AI), sometimes called machine intelligence, is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and animals."

On the other hand, academia has defined machine learning (ML) in a much more specific way, and we can easily find multiple definitions. One of the definitions that best encapsulates the idea behind the term is the given by Dr. Tom Mitchell from Carnegie Mellon University. He explains that "a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E." In other words, machine learning is at work when a program learns how to do a task without human intervention.

When we analyze both definitions, we can see right away that each term is different. First of all, AI is often a colloquial term rather than a technical one. Machine learning, however, has several accepted academic definitions. Artificial Intelligence refers to the behavior of a program while machine learning refers to a way of "teaching" programs how to resolve a specific problem. The main reason that these terms are often confused is that the best approaches to implementing artificial intelligence use machine learning techniques.

Natural Language Processing (NLP)

The term Natural Language Process is a subfield of artificial intelligence, and it covers all the technologies and methods that focus on human language. This field includes a range of applications such as language understanding (NLU), language generation (NLG), and classification tasks like sentiment analysis and automatic labeling. Natural language processing can use machine learning algorithms. The following diagram is a graphic way to show the dependency between these terms.

If we look into other subfields of artificial intelligence, we can find other interesting topics like computer vision, robotics, game-solving, etc. However, in this post we'll focus on NLP and the following ways in which it can help your business.

• NLP is useful for most products.

  • All digital products can benefit from NLP, as applying it can improve the user experience. Users are humans, so if your product understands natural language, you can better understand users' reactions and provide better support to the problems that users encounter.

• NLP is a truly innovative technology.

  • Most of the technology behind the state-of-the-art in machine learning is actually pretty old; it was discovered in the 80s and 90s. The current popularity is a result of the large amount of data and higher quality machines we can access today. NLP, however, is still an innovative field where new discoveries emerge every day.

• NLP improves performance and reduces costs.

  • Activities like support and text classification involve NLP. These tasks are usually completed by humans because implementing a program seems too complex. However, the application of NLP has shown that it’s possible to obtain similar (and sometimes even better) results with a machine.

• NLP is not an alternative to humans.

  • People often assume that an ML algorithm would replace human workers; this is a myth. An ML algorithm can work alongside humans, helping them improve their efficiency by removing the trivial stuff and allowing them to focus on more important matters.

We have defined the main concepts and explained why we're focusing on NLP. Now we will explain the specific scenarios in which you can apply NLP. There is a misconception that ML and NLP are technologies that only big companies are able to implement. Ten years ago, that may have been correct, but today most companies can apply it. Today we'll focus on easy examples of NLP that every company can invest in and apply.

Automatic Text Classification

The problem of classifying a piece of text with a single label (or many labels) is one of the most common examples of NLP usage. Moreover, the algorithms and strategies designed to deal with this problem are very mature, giving us the possibility of obtaining accurate results.

One of the most common applications in this area is sentiment analysis. NLP researchers have been tackling this problem for several years, obtaining results that sometimes outperform human results. Sentiment analysis itself is possibly not a useful feature for your company, but the technology behind it can be applied to other problems such as automatically setting the urgency of a ticket when it arrives at a support ticket system.

The approaches to this problem depend on the information available. The best scenario is a system that already has a lot of manually labeled information. If the amount of information is big enough, you can easily create an algorithm that tags the incoming information with high accuracy.

Maybe the scenario is that the labeled information exists but you don’t have enough data to train an ML algorithm from scratch. In that situation, there are alternative ways to apply ML techniques to deal with the problem. You can search for public labeled data for a similar problem and then train a generic solution for that problem. After that, you can use your company's data to improve the previous algorithm and make it fit your specific problem. That strategy is called transfer learning, and it's one of the most widely used strategies to apply machine learning without large sets of labeled data.

Automating these kinds of tasks has several advantages:

• You can save time because the labeling is done immediately when the information arrives.

• You can increase the processing capacity right away. Sometimes, the task of labeling a text requires training people in the subject. So, if you want to increase the processing capacity you need to both hire new people and train them, and that takes time. Using ML, you can increase processing capacity immediately by increasing the number of processing units.

• You can avoid opinionated classifications. When you have humans doing a classification task, the personalities, emotions or even the experience of a person can affect the result.

Analyzing Textual Information

Nowadays, there are several products that manage huge amounts of textual information. Processing and analyzing that information is important, and it's not a trivial task. NLP can help you process this information and obtain metrics for it. For example, it can help you create a word cloud, cluster information, detect anomalies, etc.

This process cannot be completed by humans for several reasons:

• Usually, the amount of data to process is huge. It is impossible for a single person to process it manually. A team of people is also complex, because they must agree on the criteria to analyze the data. Defining the criteria is not easy and can generate discrepancies. These discrepancies can affect the performance and make the entire process slower. An automatic algorithm can efficiently process more data. Moreover, the discrepancies in the criteria only happen during the training phase. The day-to-day execution of the process is simple.

• Detecting discrepancies and anomalies in the data is a task that NLP algorithms can do more efficiently than humans. There are techniques (word2vec for example) that allow mapping a text into a numeric vector, taking the semantics of the text into account. Finding clusters and anomalies in a numeric vector space is easier than doing so manually.

• Taking an NLP approach allows you to transfer the learning between products more easily than with human teams. Today, the amount of public data in several languages is huge, and you can use that information to feed your algorithm and improve the accuracy or precision. Implementing that task with different human teams involves meetings, discussion, etc.

• An NLP approach is easier to apply to a different language. A single algorithm is an easier transfer to a different language without any complexity. As we mentioned earlier, we can transform any text into a numeric vector in an easy way, so changing the language only changes the mapping function. Implementing that task with humans is not doable unless you have several teams or multilingual teams.

• Finally, it is not required to totally remove humans. The algorithm can help your team focus on the important things instead of spending time on trivial stuff. We can create an NLP algorithm that suggests clusters that humans then analyze further.

Information is a valuable resource, and being able to understand and analyze it using automatic algorithms is a great advantage for any business.

A Chatbot to Interact with Users

When we talk about chatbots, we often think of crazy algorithms that replicate the human brain. There are some complex chatbots that look like real humans, but this kind of bot is difficult to implement and is usually expensive. However, a simple bot is not a complex task, and it can be a useful tool for your business.

Let’s talk about the advantages of having a bot:

• People already communicate using natural language. Interacting with a bot is easy for users, and hardly any training is required.

• There are some ideas that are easy to express in natural language that are not mapped directly to a different UI, for example: "Show me information about houses in California excluding LA". Defining a UI to support filters and exclusions that works for everyone is not easy.

• With a bot, you can reduce or specialize your support team. The bot can deal with most of the questions, letting your team focus only on the exceptions. You can also have specialist supporters and use the bot to find the best person to answer a question.

• A bot utilizes a visual and an audio interface. Enabling different ways to interact with users makes your product more accessible.

So, while adding a smart bot to your product might not be not possible, adding a simple bot is an easy task that can contribute value to your business and help your team work more efficiently.

These three examples are only a few of many possible ways to include NLP in your product or business. As we have shown, the size of your business should not be an impediment to using this technology; it's just a matter of finding the best way to apply and implement it considering your available data and resources.