Hi,
All (from what I understand) transformer-based sentiment models output class probabilities (e.g., positive, neutral, negative), each in the range [0, 1].
I would like to transform these probabilities into a single polarity score ranging from -1 (negative) to 1 (positive). Is there a standard or recommended way to do this?
Best,
Malkolm
It seems that a rather primitive method is still the de facto standard…
The easiest good default is:
negative, neutral, positive to -1, 0, +1polarity = P(positive) - P(negative). (Statsmodels)That is usually the best starting point because it is:
[-1, 1]0 when the model is unsure or mostly neutral. (Statsmodels)Transformer sentiment models do not always directly output probabilities. In the standard Hugging Face setup, the model returns logits, which the docs describe as classification scores before SoftMax. The pipeline then turns those into postprocessed scores. (Hugging Face)
So in practice, the usual flow is:
For a 3-class sentiment model:
P(neg)P(neu)P(pos)use:
polarity = P(pos) - P(neg)
That is the same as taking the expected value of the scale [-1, 0, +1]. Neutral does not disappear conceptually. It just has value 0, so it automatically shrinks the score toward the center. (Statsmodels)
neg=0.80, neu=0.10, pos=0.10 → polarity -0.70neg=0.20, neu=0.60, pos=0.20 → polarity 0.00neg=0.05, neu=0.15, pos=0.80 → polarity 0.75That matches the intuition most people want. More positive mass pushes right. More negative mass pushes left. More neutral mass pulls inward. This follows directly from the ordered-label interpretation. (Statsmodels)
This is just an expected-value calculation.
If the classes are ordered, you can assign each class a location on a sentiment line and take the weighted average using the class probabilities. For the common 3-class case, the simplest anchors are -1, 0, +1. (Statsmodels)
That said, there is an important warning: ordered labels do not automatically come with a built-in numeric distance. The statsmodels ordinal-model docs state that ordinal labels are ordered, but the labels have no numeric interpretation besides the ordering. So the move from labels to -1, 0, +1 is a modeling choice. It is usually a sensible one, but it is still a choice. (Statsmodels)
Not in the sense of one official universal rule. There is no general transformer standard that says “all sentiment probabilities must be collapsed this exact way.” Hugging Face documents logits, probabilities, and task setup, but not one canonical polarity-collapse formula. (Hugging Face)
In practice, though, the probability-weighted ordered scale is the cleanest and most defensible approach. For 3-way sentiment, that means P(pos) - P(neg). (Statsmodels)
If your model only has negative and positive, then:
P(pos) - P(neg) works2 * P(pos) - 1So the binary case is simpler. Most of the ambiguity comes from handling neutral, or more than three sentiment levels. (Hugging Face)
If your classes are ordered, like:
then use the same idea with more anchors, for example:
-1.0-0.50.00.51.0and take the weighted average. (Statsmodels)
This is usually the right extension. Just document your anchors, because equal spacing is an assumption, not something the labels guarantee by themselves. (Statsmodels)
This part matters more than most people expect.
Even if P(pos) - P(neg) is the right formula, the final scalar is only as trustworthy as the underlying probabilities. Scikit-learn’s calibration guide explains that models can produce poor probability estimates, and that calibration methods are used to improve them. The current sklearn calibration tools support isotonic, sigmoid, and temperature scaling. (scikit-learn)
This means:
For multi-class sentiment, temperature scaling is especially relevant because sklearn describes it as a natural way to obtain better calibrated multi-class probabilities, and its CalibratedClassifierCV docs explain how it applies softmax(logits / T). (scikit-learn)
Use:
polarity = P(pos) - P(neg)
This is the best default for a normal 3-class sentiment model. It is easy to read, easy to explain, and usually matches intuition. (Statsmodels)
If you will use the score for:
then calibrate the probabilities on a held-out validation set first. Scikit-learn’s calibration docs explicitly recommend fitting calibration on data independent of the classifier training data. (scikit-learn)
If the real task is inherently scalar, Hugging Face supports regression-style sequence-classification settings as well. In other words, if you truly want one continuous polarity output, it can be cleaner to train a regression model instead of forcing a classifier into post-hoc scalar conversion. The model-output docs note that sequence-classification outputs can also be used for regression when num_labels == 1. (Hugging Face)
Sometimes people use:
(P(pos) - P(neg)) / (P(pos) + P(neg))
This measures sentiment direction conditional on being non-neutral.
It can be useful, but it is not the best default. It tends to exaggerate tiny scraps of positive or negative evidence when most probability mass is neutral. That is why the simple P(pos) - P(neg) score is usually safer and easier to interpret. This is an inference from the formulas and the ordered-scale setup, rather than a separate official rule. (Statsmodels)
def polarity_from_probs(p_neg: float, p_neu: float, p_pos: float) -> float:
return p_pos - p_neg
General ordered-class version:
import numpy as np
def ordered_polarity(probs, anchors):
probs = np.asarray(probs, dtype=float)
anchors = np.asarray(anchors, dtype=float)
return float(np.dot(probs, anchors))
These implementations are just the expected-value idea applied directly to ordered sentiment classes. (Statsmodels)
Use this unless you have a strong reason not to:
polarity = P(positive) - P(negative)
That is the clearest recommended mapping for negative / neutral / positive probabilities into one score in [-1, 1]. It is not a universal official standard, but it is the most natural and defensible default. The two main warnings are: