<p>You're heading along the right lines, but there are a few things you might want to make sure you've thought about when deciding on your method.</p>

<p><strong>1. Notation</strong></p>

<p>A long time ago there was an Expert Team on Climate Change Detection, Monitoring and Indices (ETCCDMI) that came up with some naming conventions for various climate indices, because it gets a bit cumbersome to talk about things like "the annual minimum of daily maxima" all the time.  They were first described in this paper:</p>

<p>Alexander et al. (2006), Global observed changes in daily climate extremes of temperature and precipitation, J. Geophys. Res., 111, D05109, doi:10.1029/2005JD006290</p>

<p>In your case, the 95th percentile of daily maximum temperature (TX) would be called TX95p.</p>

<p><strong>2. Think about your science question</strong></p>

<p>Most temperature time series for the last few decades will contain a warming trend, which has the effect that hot days will occur more frequently in the later part of the data than the earlier part.  You need to think about what aspect of temperature causes the impact in the system you're studying to decide if you need to detrend the data or not.  If the problem occurs when the absolute temperature exceeds some limit, then you probably don't want to detrend.  If the the system can adapt to the mean change but the problem occurs if the hot days occur in clusters, then you may want to detrend the data.</p>

<p>You need to make similar decisions about de-seasonalising the data, which you're currently doing by considering each calendar month separately when estimating TX95p.  In most cases it does make sense to de-seasonalise the data, particularly in places with a large annual cycle.  While you're at it, think about whether you need to analyse the whole year.  If the impact you're worried about only occurs in a particular season, then target your analysis to that season: e.g., a warm extreme in summer typically has a greater impact than a warm extreme in winter.</p>

<p><strong>3. Estimating TX95p</strong></p>

<p>Unless you have some specific hypothesis or protocol that needs to separate the periods, I'd use all years of data (1980-2015) to estimate TX95p.  If you're assuming that TX95p is stationary between the periods, then using more years will give you a more accurate estimate of the threshold.</p>

<blockquote>
  <p>Recently, I have come across a method to calculate daywise percentiles, high daily temperature for a specific day in a year with the high temperature for that same day across all years. Days with above 95th percentile are selected as events.</p>
</blockquote>

<p>What you've found here it that there's no standard way of estimating TX95p, the climatological threshold for a hot day.  A quick glance through my notes throws up various methods:</p>

<ul>
<li><p>Daily climatology using 15-day window centred on day-of-year (Della-Marta et al, 2007; Fischer and Schaer, 2010; Perkins et al, 2012; Teng et al, 2013)</p></li>
<li><p>Daily climatology using 10-day window centred on day-of-year after detrending the data (Kreuger et al, 2015)</p></li>
<li><p>Daily climatology using 5-day window centred on day-of-year (Lorenz et al, 2010; Mueller and Seneviratne, 2012)</p></li>
<li><p>Daily climatology using 1-day window and then smoothed with a Fast Fourier Transform (Eade et al, 2012).</p></li>
<li><p>Single threshold value using all daily TX in summers (Vautard et al, 2013).</p></li>
</ul>

<blockquote>
  <p>I have tried to compare both the methods for a farm location. I found that previous method (monthwise percentiles) identify less number of events in comparison to the recent method.</p>
</blockquote>

<p>I suspect that trying to estimate daily TX95p from just 31 values for each day of year gives very inaccurate results, whereas estimating monthly TX95p from about 930 (31*30) values is more realistic.  The daily estimates probably jump around from one day of year to the next, which would clearly be a sampling artefact, and yield values that are quite close to TX100p, so you diagnose fewer hot days.  This is why the papers listed above use various smoothing methods.</p>

CC BY-SA 4.0
2019-02-14T17:10:23.277Z
2019-02-14T17:10:23.277Z
Deditos
<p>I am analyzing the effect of extreme weather on milk production (Panel data). I am looking at direct effects and indirect effects of weather on animal (milk production). </p>

<p>Regarding extreme weather, I am looking at the following variables - Tmin, Tmax and Precipitation. The dependent variable (milk litres) is a monthly cumulative value. Therefore, I have to generate more meaningful extreme weather variables at the monthly level. For e.g. No of days of Tmax > 95th percentile, No of days of Temperature Humidity Index (THI) at certain threshold, etc. To determine the no of days of Tmax >95th percentile, I have used the following method. </p>

<p>I have calculated the Tmax value of 95th percentile for each month and for each farm  using the base period data 1980-2010. Then I have applied the determined value on daywise data (2011-2015) to determine whether the day value is exceeding the 95th percentile value. If it exceeds, then given 1 else 0. Then aggregated the no of days (with Tmax >95th percentile) by month. Do you think that this method will identify the extremes? </p>

<p>Recently, I have come across a method to calculate daywise percentiles, high daily temperature for a specific day in a year with the high temperature for that same day across all years. Days with above 95th percentile are selected as events.</p>

<p>I have tried to compare both the methods for a farm location. I found that previous method (monthwise percentiles) identify less number of events in comparison to the recent method. </p>

<p>Suggestions are welcome to use the correct method.</p>

CC BY-SA 4.0
2019-02-13T22:42:59.233Z
I think that the 95th (or 99th) percentile Max and Min should be calculated for each (or all) years, no per month. One day is not extremely hot if is hotter than the 95th percentile hotter mid-winter day, and is not extremely cold if colder than the 95th percentile colder mid-summer day.
Camilo Rada
CC BY-SA 4.0
2019-02-14T06:31:17.507Z
I can't give any references anymore but I remember that people working on health effect of heat waves would identify them via night-time temperatures (i.e. Tmin)
Christoph
<climate><extreme-weather>
Method to determine Tmax 95th percentile in a climate data
Mohana
# Method to determine Tmax 95th percentile in a climate data

(Asked by: username_0 on 2019-02-13)

I am analyzing the effect of extreme weather on milk production (Panel data). I am looking at direct effects and indirect effects of weather on animal (milk production).

Regarding extreme weather, I am looking at the following variables - Tmin, Tmax and Precipitation. The dependent variable (milk litres) is a monthly cumulative value. Therefore, I have to generate more meaningful extreme weather variables at the monthly level. For e.g. No of days of Tmax > 95th percentile, No of days of Temperature Humidity Index (THI) at certain threshold, etc. To determine the no of days of Tmax >95th percentile, I have used the following method.

I have calculated the Tmax value of 95th percentile for each month and for each farm using the base period data 1980-2010. Then I have applied the determined value on daywise data (2011-2015) to determine whether the day value is exceeding the 95th percentile value. If it exceeds, then given 1 else 0. Then aggregated the no of days (with Tmax >95th percentile) by month. Do you think that this method will identify the extremes?

Recently, I have come across a method to calculate daywise percentiles, high daily temperature for a specific day in a year with the high temperature for that same day across all years. Days with above 95th percentile are selected as events.

I have tried to compare both the methods for a farm location. I found that previous method (monthwise percentiles) identify less number of events in comparison to the recent method.

Suggestions are welcome to use the correct method.

**Tags:** climate, extreme-weather

> **Comments:**
> **username_1**: I think that the 95th (or 99th) percentile Max and Min should be calculated for each (or all) years, no per month. One day is not extremely hot if is hotter than the 95th percentile hotter mid-winter day, and is not extremely cold if colder than the 95th percentile colder mid-summer day.  
> **username_2**: I can't give any references anymore but I remember that people working on health effect of heat waves would identify them via night-time temperatures (i.e. Tmin)  

---

## 1 answers

### Answer by username_3 on 2019-02-14. Score: 2

You're heading along the right lines, but there are a few things you might want to make sure you've thought about when deciding on your method.

**1. Notation**

A long time ago there was an Expert Team on Climate Change Detection, Monitoring and Indices (ETCCDMI) that came up with some naming conventions for various climate indices, because it gets a bit cumbersome to talk about things like "the annual minimum of daily maxima" all the time. They were first described in this paper:

Alexander et al. (2006), Global observed changes in daily climate extremes of temperature and precipitation, J. Geophys. Res., 111, D05109, doi:10.1029/2005JD006290

In your case, the 95th percentile of daily maximum temperature (TX) would be called TX95p.

**2. Think about your science question**

Most temperature time series for the last few decades will contain a warming trend, which has the effect that hot days will occur more frequently in the later part of the data than the earlier part. You need to think about what aspect of temperature causes the impact in the system you're studying to decide if you need to detrend the data or not. If the problem occurs when the absolute temperature exceeds some limit, then you probably don't want to detrend. If the the system can adapt to the mean change but the problem occurs if the hot days occur in clusters, then you may want to detrend the data.

You need to make similar decisions about de-seasonalising the data, which you're currently doing by considering each calendar month separately when estimating TX95p. In most cases it does make sense to de-seasonalise the data, particularly in places with a large annual cycle. While you're at it, think about whether you need to analyse the whole year. If the impact you're worried about only occurs in a particular season, then target your analysis to that season: e.g., a warm extreme in summer typically has a greater impact than a warm extreme in winter.

**3. Estimating TX95p**

Unless you have some specific hypothesis or protocol that needs to separate the periods, I'd use all years of data (1980-2015) to estimate TX95p. If you're assuming that TX95p is stationary between the periods, then using more years will give you a more accurate estimate of the threshold.

> Recently, I have come across a method to calculate daywise percentiles, high daily temperature for a specific day in a year with the high temperature for that same day across all years. Days with above 95th percentile are selected as events.

What you've found here it that there's no standard way of estimating TX95p, the climatological threshold for a hot day. A quick glance through my notes throws up various methods:

* Daily climatology using 15-day window centred on day-of-year (Della-Marta et al, 2007; Fischer and Schaer, 2010; Perkins et al, 2012; Teng et al, 2013)
* Daily climatology using 10-day window centred on day-of-year after detrending the data (Kreuger et al, 2015)
* Daily climatology using 5-day window centred on day-of-year (Lorenz et al, 2010; Mueller and Seneviratne, 2012)
* Daily climatology using 1-day window and then smoothed with a Fast Fourier Transform (Eade et al, 2012).
* Single threshold value using all daily TX in summers (Vautard et al, 2013).

> I have tried to compare both the methods for a farm location. I found that previous method (monthwise percentiles) identify less number of events in comparison to the recent method.

I suspect that trying to estimate daily TX95p from just 31 values for each day of year gives very inaccurate results, whereas estimating monthly TX95p from about 930 (31\*30) values is more realistic. The daily estimates probably jump around from one day of year to the next, which would clearly be a sampling artefact, and yield values that are quite close to TX100p, so you diagnose fewer hot days. This is why the papers listed above use various smoothing methods.


---

