Empirical Bayes Analysis on the Power Law Process with Natural Conjugate Priors

: The power law process has been used extensively in software reliability models, reliability growth models and more generally reliable systems. In this paper we work on the Power Law Process via empirical Bayes (EB) approach. Based on a two-hyperparameter natural conjugate prior and a more generalized three-hyperparameter natural conjugate prior, which was stated in Huang and Bier (1998), we work out an empirical Bayes (EB) procedure and provide statistical inferences based on the natural conjugate priors. Given past experience about the parameters of the model, the empirical Bayes (EB) approach uses the observed data to estimate the hyperparamters of priors and then proceeds as though the prior were known.


Introduction
Bayesian inference on the power law process was also studied during the past two decades.Bayesian point and interval estimates were obtained by Guida, M. (1989); Kyparisis and Singpurwalla (1985) and Calabria and Pulcini (1997).Informative and noninformative priors were both employed on failure truncated data case.Bar-Lev et al. (1991) discussed both time and failure truncated data by using noninformative priors.They derived prediction distributions of future failure times and the number of failures in some future time interval.Calabria (1990) also derived prediction distribution by using noninformative and informative priors.These references are given on a single system and usually assume parameters are independent.Crow (1974) and Bain (1978) analyzed independent equivalent multi-system by employing power law process.Power bounds for a test of equality of trends in several independent power law processes were discussed by Calabria, R., Guida, M. and Pulcini, G.(1992).Huang and Bier (1998) developed a natural conjugate prior for the PLP.The prior they proposed has relatively simple closed-form expression for its moments.
When we only consider a single system, Bayesian framework can be utilized appropriately.Now suppose we have k similar but non-identical systems and each is modeled by PLP.This assumption is reasonable if k systems are made from the same manufacturing process.Parametric empirical Bayes model is proposed for the PLP with estimation of prior hyperparameters.We assume that the parameters (in our case the (η i , β i )s) of multiple systems are drawn from certain prior distribution π(η, β).In section 2, we work on the EB procedure with employing two-hyperparameter natural conjugate prior and in section 3 three-hyperparameter natural conjugate prior.In our paper, we applied Huang's (1998) prior to obtain Bayesian maximum likelihood estimate of a hyperparameter a, posterior means of the shape parameter β and the scale parameter η in closed forms.In section 4, several contour plots and three-dimension plots for the congugate prior illustrate some properties of the hyperparameters in the natural conjugate prior by changing hyperparameters' value.Finally in section 5, we make some remarks and conclusions.

Parametric Empirical Bayes on the PLP-Two Hyperparameters (a, m)
Nonhomogeneous Poisson Process (NHPP) is a Poisson process in which intensity function is not a constant.Let N (t) denote the cumulative number of failures from time 0 to time t and suppose we have a sequence of successive failure times.The power law process (PLP) can be described as a nonhomogeneous Poisson process {N (t), t ≥ 0} with intensity function where α and β are the scale parameter and shape parameter respectively.Hence, the mean value function λ(t) of the process is It can be shown that the number of failures in any interval (t 1 , t 2 ] has a Poisson distribution with mean When β < 1, the intensity function ν(t) is decreasing and the system is improving.Under this situation, the power law process can be applied as a reliability growth model.
Let η = α −β .Parametric empirical Bayes procedures on the Power Law Process are easier to work with if the intensity function is parametrized as The power law process has been widely used in reliability growth (Crow (1982)), and software reliability models (Kyparisis and Singpurwalla (1985)), and in repairable systems (Ascher and Feingold (1984); Engelhardt and Bain (1986); Rigdon and Basu (1989)).
Assume there are k similar systems from the same manufacturing process.Let t i denote the vector of failure times for system i, and let denote the two-dimensional array of failure times.Suppose that the ith system has been operated from time 0 until the n i th failure and the n i th failure occurs at time t in i .Clearly t i1 < t i2 < . . .< t in i .This is the failure truncated case, where n i is a constant.The likelihood function of η, β associated with the first n i failure times for system i is (2.1) A natural conjugate prior distribution for the parameter η, β is given by here (m, a) are positive hyperparameters.c is a constant.
Then the marginal distribution of failure time t i from system i given (m, a) is Suppose k systems are independent, we have the marginal distribution of T given (m, a) is In order to obtain Maximum likelihood estimates (MLEs) of m and a, we need to take a logarithm of (2.4) (2.5) Then after taking the derivative with respect to a in (2.5) and setting it equal to zero, we have Similarly, after taking the derivative with respect to m in (5) and setting it equal to zero, we have (2.7)According to the following well known property of Gamma function: MLEs of a and m can be obtained by solving equations (2.6) and (2.8) numerically.In general, the likelihood equations do not admit a closed form solution and a numerical method must be employed to approximate the MLEs of the hyperparameters (a, m).
In the special case that we consider one system, which is k = 1, with observations (t 1 , t 2 , . . ., t n ), we are able to obtain the estimate of a in a closed form from Bayesian maximum likelihood approach.The following results shall be considered as Bayesian inference.
Then we can use Newton-Raphson method to obtain MLE for m.The posterior distribution is .
(2.9) From the form of equation (2.9), the prior distribution and posterior distribution are from the same family, therefore, priors are natural conjugate priors.The posterior mean for η has a closed form.The posterior mean is considered as a Bayesian estimation of η which is given by ηB .
The posterior mean for β also has a closed form and is given by .

Parametric Empirical Bayes on the PLP-Three Hyperparameters
(m, a, y m ) Now we consider a more general conjugate prior with three hyperparameters (m, a, y m ), which adds a hyperparameter y m .Recall the likelihood function: The natural conjugate prior distribution for the power law failure model is given by The parameters y 1 . . .y m can be interpreted as a pseudo-data set, where m is the number of failures and y i is the time of the ith failure.For simplicity and without loss of generality, we can choose ∏ m i=1 y i = exp(−a)y m m .Thus the natural conjugate prior becomes here (m, a, y m ) are positive hyperparameters.c is a constant and Since k systems are independent, the likelihood distribution of T given (m, a, y m ) is Again the likelihood equations do not admit a closed form solution and a numerical method need to be employed to approximate the maximum likelihood estimates of (a, m, y m ).However, suppose we only have observations (t 1 , t 2 , . . ., t n ) from one system, which means we only have a random sample of size one (η, β) from the prior π(η, β|a, m, y m ), our inference shall be regarded as Bayesian maximum likelihood approach.For this special case, we have the posterior distribution of (η, β) is The posterior mean for η is The posterior mean for β is It should be addressed that the problem with parametric Emperical Bayes (PEB) is that we assume that the estimates of the prior parameters are the prior parameters themselves.The PEB approach does not account for uncertainty in the estimates of these hyperparameters.Variation in these estimates would lead to more variation in the estimates of function of parameters, such as intensity and reliability etc.

Prior Plots
The joint prior density is given by (3.2).By taking integral with respect to η, π 0 (β|m, a, y m ) = a m β m−1 exp(−aβ) Γ(m) . (4.1) The marginal prior distribution of β has a Gamma distribution with mean m/a and variance m/a 2 .The conditional prior distribution of η given β is which is Gamma distribution with mean m/y β m and variance m/y 2β m .Figure 1 and 2 are the prior density plots given different value of hyperparameters.

Conclusions and Remarks
An empirical Bayes (EB) approach on the Power Law Process with natural conjugate priors has been developed in this paper.We obtained a closed form of marginal distribution of observed data in the two-hyperparameter case.For a special case of single system, we derived Bayesian maximum likelihood estimates of hyperparameters and also found a closed form of the posterior means of model parameters (η, β).Similar empirical Bayes approach based on the threehyperparameter prior has been shown and the marginal distribution of observed data and estimates of (η, β) have been derived in a simpler form.
It is fundamental basis of Bayesian decision theory that statistical inference should start with the determination of three factors: the distribution family for the observations, the prior distribution for the parameters and the loss associated with decisions.Further work can be done to check the robustness of the priors we used, such as noninformative priors, natural conjugate EB priors and nonparametric Empirical Bayes priors.We can slightly change the prior and see what happens to the decision.Two commonly used measures are the range of posterior decision and comparing Bayes risk criteria.

Figure 1 :
Figure 1: Prior Contour Plots and Three-Dimension Plots -Part I

Figure 2 :
Figure 2: Prior Contour Plots and Three-Dimension Plots -Part II