Question 1

What is Deepopinion Studio

Accepted Answer

Deepopinion Studio is a web app giving you access to world-class AI at your fingertips. Test industry-specific models and contact us to arrange a demo to integrate our solutions into your business.

Question 2

Is there a free trial?

Accepted Answer

You have the chance to convince yourself for free. Just sign up in DeepOpinion Studio, try it out, and play around. After that, we offer various plans with additional customization options to provide solutions that work for every organization.

Question 3

How do I create a document information extraction model on DeepOpinion Studio

Accepted Answer

Create a document information extraction as a cognitive skill on DeepOpinion Studio then upload and label your documents to be able to train a model.

Question 4

Which performance metrics do we use in the Studio?

Accepted Answer

Summary

Having a look at our model's performance metrics, we can understand our models deeper. We can compare them and decide if they are state-of-the-art models or improve their performance. This section shows the analytical descriptions of all the performance metrics that we use in Studio.

Performance Metrics

The first thing that a data scientist does to check its model performance is to look at the confusion matrix! The confusion metric shows us the model labels’ predictions, e.g., for the sentiment, compared to the actual labels (extracted from the annotated data). We can calculate the below performance metrics based on the confusion matrix to understand our model’s performance.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/mGPNnsaodw/Screenshot%20from%202021-05-20%2016-13-35.png (Screenshot from 2021-05-20 16-13-35.png)

Before we proceed with the performance metrics definitions, let’s clarify here what the True Positives, the True Negatives, False Positives & False Negatives are so we understand more about the role of the None labels in our models. We do not confuse any of these concepts with the sentiment labels.

Keep in mind that our models try to be “smart” and their strategy is to avoid making mistakes rather than randomly guessing an answer when they are not confident about a prediction. Hence, if it is not “clear” for a trained model which label it should predict, the model does not anticipate any label resulting in assigning the None label to the specific piece of text. For example, if an ABSA model trained on finance data “sees” the following text:

“The weather is nice.”

It will return a None label, which means that it could not detect any aspects of the given text. Now, this is called a “negative,” and when it is predicted correctly (there is no aspect in the text), it is more precisely called a True Negative.

On the other hand, when the model was wrong, and it should have predicted some labels for the given, but it predicted a Negative (None), we have a False Negative.

So, the True & False Negatives are nothing more than the correct & false predictions of the None case (no label was assigned to the text/review), respectively.

It is a much easier task to understand the True and False Positives as they are the correct & false predictions of the different labels, respectively. So, for an ABSA model, we define the Positives as the cases where: we have at least one, predicted by the model, aspect, and there was at least one aspect assigned to the exact text, based on the annotated data (actual). To define if this Positive prediction was True or False, we compare the predicted & actual aspects, and we call our Positive as True Positive when we have the same aspects. Similarly, we call a Positive a False Positive when the predicted aspect is not the same as the actual aspect.

In a nutshell:

Having a look at the accuracy formula, we understand that the accuracy of a model shows the analogy between the correctly predicted by the model labels and the total predicted by the model labels. Hence, for an Aspect Based Sentiment Analysis model, the accuracy score shows the ratio of the correctly predicted by the model aspect-sentiment pairs to the total predicted pairs. Note here that the predicted as None labels (no aspect was detected in a piece of text) are also included in this calculation.

Accuracy = (TP+TN)/(TP+FP+FN+TN)

Sometimes, we need to investigate if the model returned many None labels (Negatives) and how many were predicted correctly as None (True Positives). Sometimes we want to see how our model predicted many correctly predicted labels (True Positives). And sometimes, we want to see if there were no predictions for a particular case, e.g., sentiment Neutral. These investigations can help us understand our model’s performance in-depth and discover what we need to improve it. For this purpose, we have a look at the following two metrics.

Precision and Recall are two metrics that focus on how well our model learned to predict labels from our data. They show that the True Negatives score (how many None labels were predicted correctly) by leaving outside of their formulas.

Here are their formulas:

Precision = TP/(TP+FP)

More specifically, precision represents the ratio of the correctly predicted labels to the sum of the correctly predicted labels and the falsely predicted labels. Recall represents the ratio of the correctly predicted labels to the sum of the correctly predicted labels and the falsely predicted as None.

In other words, precision is the number of correct results divided by all returned results. At the same time, recall is the number of accurate results divided by the number of the results that should have been returned.

Lastly, let’s look at the F1 score, which is a combination of the Precision and the Recall score. You can see its formula below:

F1 = (½)(PrecisionRecall)/(Precision + Recall)

We understand that F1 is a balance between the other two metrics, and we use it when we have many True Negatives (actual Nones).

The above are the four standard performance metrics used to describe a model’s performance. You can see all these metrics for your custom Studio models in the training section, in the Training Overview, and after selecting the expert mode, as shown in the picture below.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/4psYlyDM3w/Screenshot%20from%202021-05-20%2016-14-51.png (Screenshot from 2021-05-20 16-14-51.png)

But we did not stop there! We wanted to compare your model’s “IQ” with a human’s ability to annotate correctly, as a model can only be as “smart” as its training dataset allows it to be. For this purpose, we compare the F1 score from your model to the F1 scores from many of our high-performing industry models (you can find these models in our Model Library). This score is called the Overall Score or Standardized Score, and its color indicates how strong your model is. You can see in the picture below a scale that represents your model’s performance going from the lowest one (red) to the highest one (dark green).

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/4KUUaXPtK5/Screenshot%20from%202021-05-20%2016-15-32.png (Screenshot from 2021-05-20 16-15-32.png)

In the case of having a MultiClassLabel model like an ABSA model, we calculate the Model’s overall score (ABSA score) as a combination of the F1 Standardized Aspect Score and the Sentiment Task Weighted Accuracy Standardized Score.

Based on research conducted by the University of Innsbruck, we know that humans put significantly more weight on the correctness of the sentiment label while they annotate data. For this reason, we weigh the contribution of the Sentiment Task Weighted Accuracy Standardized Score to the calculation of the ABSA score.

Lastly, for calculating the Sentiment Task Weighted Accuracy Score, we calculate the accuracy (as discussed above) of the examples where the aspect was already correctly detected. Then, we compare this score with similar scores in high-performing industry models to standardize it.

Question 5

Why would I use ABSA / Opinion Mining?

Accepted Answer

Unstructured text data, like customer reviews, employee feedback, or answers to open-ended questions in questionnaires, exist in huge quantities today. Digging through this treasure chest of data has long been nearly impossible. Through text analysis solutions such as sentiment analysis, you can get a good impression of the data.

Using aspect-based sentiment analysis (ABSA), you can get an even better picture and derive actionable insights from your data. With DeepOpinion Studio, you can access the leading AI solution for Opinion Mining and the highest number of pre-trained industry models. Start making sense of your text data and gain a competitive advantage.

Question 6

How do I start with DeepOpinion Studio?

Accepted Answer

To get actionable insights from text data, you need to get familiar with DeepOpinion Studio.

Once you're logged in, you can see your models and all the available pre-trained models, which change to "My Models" when starring them. First, select a model you wish to use.

After choosing a model, you're in the demo menu of this particular model, where you can see a description and all pre-defined and pre-trained aspects (click on "classification categories"). You can select a demo document or insert your text example to convince yourself of the model’s capabilities in the section below.

In the next section of the menu, called "batch" you can start analyzing more extensive data sets to derive actionable insights. Simply upload your data set(s) in the "upload data" section and follow the instructions. The bell icon in the upper right corner will let you know once the analysis is completed and the results are ready to download.

Please keep in mind that the amount of data you can analyze depends on your subscribed plan (for more info, click "update" in the menu).

Question 7

How Do I Connect PowerBI to DeepOpinion Studio Automation

Accepted Answer

Why Connect PowerBI to DeepOpinion Studio

As a DeepOpinion Studio user you are automating your text based processes. Depending on your work this can be intelligent routing of customer support tickets or customer review analysis or even insurance claim processing. In some cases, you might be interested in building reports and live dashboards out of this data. Many users prefer to customize their own dashboard or perhaps feed this data to an existing dashboard to have a single source of truth. PowerBI happens to be the BI tool of choice for many of our customers. Therefore, this integration makes it super easy to pull data to PowerBI to make you able to tell your data story.

How to Connect PowerBI to DeepOpinion Studio

There are two main steps to complete the PowerBI integration:

Create an automation on DeepOpinion Studio with PowerBI as the output step
Use PowerBI Query to get the data from DeepOpinion Studio

Step 1 Create an Automation

The first step is to create an automation. An automation consists of an input that has the raw data, a DeepOpinion AI model to analyze the data, and an output where the analysis results will be automatically sent. For example, if you would like to monitor your customer support, an input might be your customer support tickets from Zendesk, your model can be opinion mining, and your output can be sent to PowerBI. Alternatively, you can choose to trigger this automation manually which allows you to upload data on-demand and it will be directly reflected on PowerBI.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/S7wmr8nggS/automation_workflow.png (automation_workflow.png)

Once you select PowerBI as an output a screen will slide in explaining the instructions. Here you just have to copy the query that you will later add to your PowerBI account to get the results of this specific model.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/wIIdjm8Dcg/PowerBI%20Automation.png (PowerBI Automation.png)

Step 2 Create a PowerBI Customer Query

Now move to PowerBI, click on "Get Data" on the top menu and choose "Blank Query" from the drop down list.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/lHCPJ6VyNG/PBI_Get_Data.png (PBI_Get_Data.png)

An empty window will pop up, click on "Advanced Editor" at the centre of the top menu and another small window will pop up. Here on the advanced editor is where you will paste the query you copied from DeepOpinion Studio and then click on "Done" on the bottom of this tiny window.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/5pnOU9Yeu2/PBI_query.png (PBI_query.png)

Now by this, all the analysis that has been performed by this specific model will be pulled from DeepOpinion Studio. Then click on "Close & Apply" on the top right. All future analysis will be updated here as well when you open PowerBI again.

https://malcolm-en-gb.s3.eu-west-1.amazonaws.com/instances/oiJS2eQZyE/resources/LRj6MFKJCq/PBI_Data.png (PBI_Data.png)

Now you are ready to do your magic with data visualization on PowerBI. We look forward to hearing about it!

Question 8

How Do I Create A Freshdesk Ticket Automation on DeepOpinion Studio?

Accepted Answer

Automate your Freshdesk tickets routing, tagging, and prioritization DeepOpinion Studio in 3 simple steps.

The 3 steps are:

Select your input trigger, which in this case is receiving a new ticket on freshdesk
Choose the model you want to perform the ticket categorization. You can also train a model on your freshdesk tickets historical data
Select your output, which in this case would be updating a field on the freshdesk ticket

Follow this step-by-step guide to learn how:

Leave us a comment you would like more details.