This is a continuation of two previous posts on the exponential distribution (an introduction and a post on the connection with the Poisson process). This post presents more properties that are not discussed in the two previous posts. Of course, a serious and in-depth discussion of the exponential distribution can fill volumes. The goal here is quite modest – to present a few more properties related to the memoryless property of the exponential distribution.
The Failure Rate
A previous post discusses the Poisson process and its relation to the exponential distribution. Now we present another way of looking at both notions. Suppose a counting process counts the occurrences of a type of random events. Suppose that an event means a termination of a system, be it biological or manufactured. Furthermore suppose that the terminations occur according to a Poisson process at a constant rate per unit time. Then what is the meaning of the rate ? It is the rate of termination (dying). It is usually called the failure rate (or hazard rate or force of mortality). The meaning of the constant rate is that the rate of dying is the same regardless of the location of the time scale (i.e. regardless how long a life has lived). This means that the lifetime (the time until death) of such a system has no memory. Since the exponential distribution is the only continuous distribution with the memoryless property, the time until the next termination inherent in the Poisson process in question must be an exponential random variable with rate or mean . So the notion of failure rate function (or hazard rate function) runs through the notions of exponential and Poisson process and further illustrates the memoryless property.
Consider a continuous random variable that only takes on the positive real numbers. Suppose and are the CDF and density function of , respectively. The survival function is . The failure rate (or hazard rate) is defined as:
The function can be interpreted as the rate of failure at the next instant given that the life has survived to time . Suppose that the lifetime distribution is exponential. Because of the memoryless proeprty, the remaining lifetime of a -year old is the same as the lifetime distribution for a new item. It is then intuitively clear that the failure rate must be constant. Indeed, it is straightforward to show that if the lifetime is an exponential random variable with rate parameter , i.e. the density is , then the failure rate is . This is why the parameter in the density function is called the rate parameter.
On the other hand, the failure rate or hazard rate is not constant for other lifetime distributions. However, the hazard rate function uniquely determines the distributional quantities such as the CDF and the survival function. The definition shows that the failure rate is derived using the CDF or survival function. We show that the failure rate is enough information to derive the CDF or the survival function. From the definition, we have:
Integrate both sides produces the following:
Exponentiate on each side produces the following:
Once the survival function is obtained from , the CDF and the density function can be obtained. Interestingly, the derivation in can give another proof of the fact that the exponential distribution is the only one with the memoryless property. If is memoryless, then the failure rate must be constant. If is constant, then by , is the exponential survival function. Of course, the other way is clear: if is exponential, then is memoryless.
The preceding discussion shows that having a constant failure rate is another way to characterize the exponential distribution, in particular the memoryless property of the exponential distribution. Before moving onto the next topics, another example of a failure rate function is where both and are positive constants. This is the Weibull hazard rate function. The derived survival function is , which is the survival function for the Weibull distribution. This distribution is an excellent model choice for describing the life of manufactured objects. See here for an introduction to the Weibull distribution.
The Minimum Statistic
The following result is about the minimum of independent exponential distributions.
Suppose that are independent exponential random variables with rates , respectively. Then the minimum of the sample, denoted by , is also an exponential random variable with rate
For the minimum to be , all sample items must be , Thus is:
This means that has an exponential distribution with rate . As a result, the smallest item of a sample of independent exponential observations also follows an exponential distribution with rate being the sum of all the individual exponential rates.
Suppose that are independent exponential random variables with rates for . Calculate and .
Let and . Then is an exponential distribution with rate . As a result, . Finding the expected value of the maximum requires calculation. First calculate .
Differentiate to obtain the density function .
Each term in the density function is by itself an exponential density. Thus the mean of the maximum is:
To make sense of the numbers, let . Then and . In this case, the expected value of the maximum is 5.5 times larger than the expected value of the minimum. For , and , and . In the second case, the expected value of the maximum is 7.3 times larger than the expected value of the minimum.
Ranking Independent Exponential Distributions
In this section, are independent exponential random variables where the rate of is for . What is the probability of ? Here the subscripts are distinct integers from . For example, for sample size of 2, what are the probabilities and ? For sample size of 3, what are and ? First, the case of ranking two independent exponential random variables.
Ranking and .
where is the rate of and is the rate of . Note that this probability is the ratio of the rate of the smaller exponential random variable over the total rate. The probability can be computed by evaluating the following integral:
The natural next step is to rank three or more exponential random variables. Ranking three variables would require a triple integral and ranking more variables would require a larger multiple integral. Instead, we rely on a useful fact about the minimum statistic. First, another basic result.
When one of the variables is the minimum:
The above says that the probability that the th random variable is the smallest is simply the ratio of the rate of the th variable over the total rate. This follows from since we are ranking the two exponential variables and .
We now consider the following theorem.
Let be independent exponential random variables. Then the minimum statistic and the rank ordering of are independent.
The theorem basically says that the probability of a ranking is not dependent on the location of the minimum statistic. For example, if we know that the minimum is more than 3, what is the probability of ? The theorem is saying that conditioning on makes no difference on the probability of the ranking. Let . The following establishes the theorem.
The key to the proof is the step to go from the second line to the third line. Assume that each is the lifetime of a machine. When , all the lifetimes . By the memoryless property, the remaining lifetimes are independent and exponential with the original rates. In other words, each has the same exponential distribution as . Consequently, the second line equals to the third line. To make it easier to see the step from the second line to the third line, think of the two dimensional case with and .
The following is a consequence of Theorem 1.
Let be independent exponential random variables. Then the event and the rank ordering of are independent.
Another way to state the corollary is that the knowledge that is the smallest in the sample has no effect on the ranking of the variables other than . This is the consequence of Theorem 1. To see why, let .
We now present examples demonstrating how these ideas are used.
Suppose that a bank has three tellers for serving its customers. The random variable are independent exponential random variables where is the time spent by teller serving a customer. The rate parameter of is where . If all three tellers are busy serving customers, what is ? If the bank has 4 tellers instead, then what is ?
The answer is given by the following:
The derivation uses Corollary 2 and the idea in . The same idea can be used for .
The above applies the independence result twice, the first time on , the second time on . This approach is much preferred over direction calculation, which would involve integral calculation that is tedious and error prone.
As in Example 2, a bank has three tellers for serving its customers. The service times of the three tellers are independent exponential random variables. The mean service time for teller is minutes where . You walk into the bank and find that all three tellers are busy serving customers. You are the only customer waiting for an available teller. Calculate the expected amount of time you spend at the bank.
Let be the total time you spend in the bank, which is where is the waiting time for a teller to become free and is the service time of the teller helping you. When you walk into the bank, the tellers are already busy. Let be the remaining service time for teller , . By the memoryless property, is exponential with the original mean . As a result, the rate parameter of is .
The waiting time is simply . Thus . To find , we need to consider three cases, depending on which teller finishes serving the current customer first.
Finishing the calculation,