Sentiment Analysis 2.0: Migration guide

We have released a new version of one of our more popular APIs: Sentiment Analysis. In Sentiment Analysis 2.0:

  • The rules used for defining polarity terms have been greatly improved, adding new operators and making the models used much more flexible, which in turn leads to better results.
  • Sentiment analysis is now done at more levels, allowing to identify more complex syntactic structures and to obtain more detailed information about how the polarity is expressed.
  • More configuration options have been added related to the morphosyntactic analysis over which the sentiment analysis is carried out.
  • The architecture of the service has changed, leading to a tenfold improvement in the response time.
  • An integration with the Lemmatization, PoS and Parsing API has been added in order to ease the way of creating applications that use the information provided by both APIs.
  • Dictionary customization has been fully integrated in order to get out the most out of its functionality.

All these improvements mean the migration process is not as fast as it would be with a minor version. These are the things you need to know to migrate your applications from Sentiment Analysis 1.2 to Sentiment Analysis 2.0.

Request

The most relevant changes in the request are the following:

Sentiment Analysis 1.2 Sentiment Analysis 2.0
Endpoint http://api.meaningcloud.com/sentiment-1.2 http://api.meaningcloud.com/sentiment-2.0
Parameter model Accepted values:

  • es-general
  • en-general
  • fr-general
  • empty
 Accepted values:

  • general_es
  • general_en
  • general_fr
  • auto
Parameter entities Form|Alias_1|Alias_2|…|Alias_N Integrated with dictionary customization.
Parameter concepts Form|Alias_1|Alias_2|…|Alias_N Integrated with dictionary customization.

All the other parameters from Sentiment Analysis 1.2 not explicitly mentioned (key, of, txt and ud) behave exactly the same in Sentiment Analysis 2.0.

In order to use dictionary customization to add concepts and entities to the sentiment analysis, you just have to follow these steps:

  1. Copy the values you sent in the entities parameter and save them in a file with the following format in which ‘\t‘ is a tabulation: 
    Form\tAlias_1|Alias_2|...|Alias_N\t\t
  2. Do the same thing with the values in the concepts parameter and save them in a different file using the same format as in the previous point.
  3. Go to dictionary customization and create a dictionary.
  4. Once the dictionary is created, you are going to import the two files you have saved in the steps 1 and 2.
  5. Use the name of your brand new dictionary with the imported entities and concepts to the ud parameter

Sentiment Analysis 2.0 includes a number of new parameters that give more flexibility to what you can customize in your sentiment analysis. None of these new parameters are required, so you don’t need to add them to your request if you don’t want to. You can find out more about them in the documentation.

Response

The change in the response is more significant than the one in the request. The complete response is documented in detail in the documentation, so here we’ll just comment on the most important points:

  • The sentiment analysis is shown at a sentence level before dividing it into segments, which means more levels in the response.
  • There are no longer numeric polarity values (that is, score does not exist), polarity is given using score_tag.
  • The standard deviation value, sd, is now called agreement, and the numeric value no longer exists (sd_tag).
  • keywords now are called polarity_terms.
  • irony and subjectivity are now detected only at a global level.
  • A confidence value has been added to the polarity scores to give an idea of how reliable the changes applied to the original polarity of a term is.
  • Concepts and entities elements have been standardized to be more coherent within the Sentiment Analysis response and with other APIs, and are now shown as sentimented_entity and sentimented_concept elements.
  • Offset values (inip and endp) have been added at different levels to ease integration and visualization.
  • A new mode has been added, tv, that allows to integrate the sentiment analysis with an output such as the one obtained in the Lemmatization, PoS and Parsing API.

In the following table you can see side by side how the analysis of the sentence “The restaurant was great even though it’s not near Madrid.” changes between versions.

Sentiment 1.2
Sentiment 2.0
 
{
  "status": {
    "code": "0",
    "msg": "OK",
    "credits": "1"
  },
  "model": "en-general",
  "score": "0.80",
  "score_tag": "P+",
  "sd": "0.00",
  "sd_tag": "AGREEMENT",
  "subjectivity": "OBJECTIVE",
  "irony": "NONIRONIC",
  "segment_list": [
    {
      "inip": "0",
      "endp": "57",
      "text": "The restaurant was great even though it's not near Madrid.",
      "score": "0.80",
      "score_tag": "P+",
      "sd": "0.00",
      "sd_tag": "AGREEMENT",
      "subjectivity": "OBJECTIVE",
      "irony": "NONIRONIC",
      "keyword_list": [
        {
          "text": "great",
          "score": "0.80",
          "score_tag": "P+",
          "entity_list": [
            {
              "text": "Madrid",
              "id": "3d0a16c68d",
              "type": "Top>Location>GeoPoliticalEntity>City"
            }
          ],
          "concept_list": [
            {
              "text": "restaurant",
              "id": "4d5e117421",
              "type": "Top>Location>Facility"
            }
          ]
        }
      ]
    }
  ],
  "entity_list": [
    {
      "text": "Madrid",
      "id": "3d0a16c68d",
      "type": "Top>Location>GeoPoliticalEntity>City",
      "score": "0.80",
      "score_tag": "P+",
      "sd": "0.00",
      "sd_tag": "AGREEMENT"
    }
  ],
  "concept_list": [
    {
      "text": "restaurant",
      "id": "4d5e117421",
      "type": "Top>Location>Facility",
      "score": "0.80",
      "score_tag": "P+",
      "sd": "0.00",
      "sd_tag": "AGREEMENT"
    }
  ]
}
{
  "status": {
    "code": "0",
    "msg": "OK",
    "credits": "1"
  },
  "model": "general_en",
  "score_tag": "P+",
  "agreement": "AGREEMENT",
  "subjectivity": "SUBJECTIVE",
  "confidence": "100",
  "irony": "NONIRONIC",
  "sentence_list": [
    {
      "text": "The restaurant was great even though it's not near Madrid.",
      "inip": "0",
      "endp": "57",
      "bop": "y",
      "confidence": "100",
      "score_tag": "P+",
      "agreement": "AGREEMENT",
      "segment_list": [
        {
          "text": "The restaurant was great even though it's not near Madrid",
          "segment_type": "main",
          "inip": "0",
          "endp": "56",
          "confidence": "100",
          "score_tag": "P+",
          "agreement": "AGREEMENT",
          "polarity_term_list": [
            {
              "text": "great",
              "inip": "19",
              "endp": "23",
              "confidence": "100",
              "score_tag": "P+",
              "sentimented_concept_list": [
                {
                  "form": "restaurant",
                  "variant": "restaurant",
                  "inip": "4",
                  "endp": "13",
                  "type": "Top>Location>Facility",
                  "score_tag": "P+"
                }
              ]
            }
          ],
          "sentimented_entity_list": [
            {
              "form": "Madrid",
              "variant": "Madrid",
              "inip": "51",
              "endp": "56",
              "type": "Top>Location>GeoPoliticalEntity>City",
              "score_tag": "NONE"
            }
          ]
        }
      ],
      "sentimented_entity_list": [
        {
          "form": "Madrid",
          "type": "Top>Location>GeoPoliticalEntity>City",
          "score_tag": "NONE"
        }
      ],
      "sentimented_concept_list": [
        {
          "form": "restaurant",
          "type": "Top>Location>Facility",
          "score_tag": "P+"
        }
      ]
    }
  ],
  "sentimented_entity_list": [
    {
      "form": "Madrid",
      "type": "Top>Location>GeoPoliticalEntity>City",
      "score_tag": "NONE"
    }
  ],
  "sentimented_concept_list": [
    {
      "form": "restaurant",
      "type": "Top>Location>Facility",
      "score_tag": "P+"
    }
  ]
}

Again, all the details can be found in the Sentiment Analysis 2.0 documentation. And of course, if you have any questions or issues during the migration, we are always available either through our support form, or just by writing us to support@meaningcloud.com.

2 thoughts on “Sentiment Analysis 2.0: Migration guide”

This migration is for custom integrations with the API. RapidMiner is already integrated with the latest versions of the APIs.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*