<p>I am currently trying to find 'Price Outliers' in a set of data. Given a set of product prices (within a product category, e.g. milk) I want to filter for prices that are suspicious - for example I have a set of milk prices $\{1,1.5,10,0.1\}$ then I want to filter for $10$ and $0.1$. The problem is that within different product categories price ranges can vary a lot - for example fish: a tin of tuna and a tin of caviar, converting the prices to 1kg has a very big difference in pricing.</p>

<p>So the question is, if anyone of you knows a (mathematical) model, to describe pricing within 'similar' products - I am more interested in relative conclusions about prices rather than absolute, for example: I am not interested in 1.5 Euro per liter milk against 1 Euro per liter milk but I am interested in the 'range prices can have' with the aim to find suspicious prices. </p>

<p>So far I worked with an empirical expected value and standard deviation, assumed prices are log-normal distributed and calculated the 2-sigma intervals and filtered all prices not within the 2-sigma interval. This gives more or less good results but is not really satisfying.</p>

CC BY-SA 3.0
2017-08-14T12:05:14.973Z
[This paper](http://www.mit.edu/~afc/papers/Cavallo-Rigobon-Distribution.pdf) seems to be a good entry point to enter the 'more mathematical' approach to the problem you are describing.
sen_saven
CC BY-SA 3.0
2017-08-14T17:00:52.550Z
Do you have any additional information about how "similar" products are? If you have something quantifiable, perhaps, you could build a stochastic frontier. See this survey for example http://pages.stern.nyu.edu/~wgreene/FrontierModeling/SurveyPapers/LIMDEP-Chapter33.pdf
kitsune
CC BY-SA 3.0
2017-08-15T07:20:32.543Z
@sen_saven So I've read the paper and as far as I can tell, they aggregate the data for a retailer and test for noticeable price changes. As you said, it seems like a good entry point but I am not quite sure whether this is applicable in my case, as I am interested in the price ranges in a given product group and not price changes within an aggregated group of products. Please, correct me if I am wrong. But thanks anyway, I will try to track the mentioned papers down and see if I can find anything.
Abbraxas
CC BY-SA 3.0
2017-08-15T07:21:41.717Z
@kitsune Unfortunately no, I do not have a quantifiable characteristic. I'll look into the FrontierModeling and give feedback.
Abbraxas
CC BY-SA 3.0
2017-08-15T08:23:23.680Z
@kitsune So I looked into it and I am not quite sure where to start? You suggested the stochastic frontier, could you help me how that would be applicable in my case?
Abbraxas
CC BY-SA 3.0
2017-08-16T04:26:00.313Z
seems like it can be filtered using simple conditioning. do you want straight up mathematical equations or do you want some code which will do it for you?
EconJohn
CC BY-SA 3.0
2017-08-23T13:54:02.027Z
@EconJohn Hi, sorry for answering this late, had a lot to do at work... I am a mathematician so equations would always be nice for me. If you have a model and could share it or hint me to a source I would be very thankful.
Abbraxas
<price>
Modelling prices to identify price outliers
Abbraxas
# Modelling prices to identify price outliers

(Asked by: username_0 on 2017-08-14)

I am currently trying to find 'Price Outliers' in a set of data. Given a set of product prices (within a product category, e.g. milk) I want to filter for prices that are suspicious - for example I have a set of milk prices $\{1,1.5,10,0.1\}$ then I want to filter for $10$ and $0.1$. The problem is that within different product categories price ranges can vary a lot - for example fish: a tin of tuna and a tin of caviar, converting the prices to 1kg has a very big difference in pricing.

So the question is, if anyone of you knows a (mathematical) model, to describe pricing within 'similar' products - I am more interested in relative conclusions about prices rather than absolute, for example: I am not interested in 1.5 Euro per liter milk against 1 Euro per liter milk but I am interested in the 'range prices can have' with the aim to find suspicious prices.

So far I worked with an empirical expected value and standard deviation, assumed prices are log-normal distributed and calculated the 2-sigma intervals and filtered all prices not within the 2-sigma interval. This gives more or less good results but is not really satisfying.

**Tags:** price

> **Comments:**
> **username_1**: [This paper](http://www.mit.edu/~afc/papers/Cavallo-Rigobon-Distribution.pdf) seems to be a good entry point to enter the 'more mathematical' approach to the problem you are describing.  
> **username_2**: Do you have any additional information about how "similar" products are? If you have something quantifiable, perhaps, you could build a stochastic frontier. See this survey for example http://pages.stern.nyu.edu/~wgreene/FrontierModeling/SurveyPapers/LIMDEP-Chapter33.pdf  
> **username_0**: @username_1 So I've read the paper and as far as I can tell, they aggregate the data for a retailer and test for noticeable price changes. As you said, it seems like a good entry point but I am not quite sure whether this is applicable in my case, as I am interested in the price ranges in a given product group and not price changes within an aggregated group of products. Please, correct me if I am wrong. But thanks anyway, I will try to track the mentioned papers down and see if I can find anything.  
> **username_0**: @username_2 Unfortunately no, I do not have a quantifiable characteristic. I'll look into the FrontierModeling and give feedback.  
> **username_0**: @username_2 So I looked into it and I am not quite sure where to start? You suggested the stochastic frontier, could you help me how that would be applicable in my case?  
> **username_3**: seems like it can be filtered using simple conditioning. do you want straight up mathematical equations or do you want some code which will do it for you?  
> **username_0**: @username_3 Hi, sorry for answering this late, had a lot to do at work... I am a mathematician so equations would always be nice for me. If you have a model and could share it or hint me to a source I would be very thankful.  

---

## 0 answers

