Suppose one observes a fractional Brownian motion process (fBm) with linear drift and unknown drift coefficient. We are interested in sequentially testing the hypotheses that the drift coefficient is positive or negative. By a sequential test we call a procedure which continues to observe the process until a certain time (which generally depends on a path of the process, so it is a stopping time), and then decides which of the two hypotheses should be accepted. We consider a Bayesian setting where the drift coefficient has a prior normal distribution, and we use an optimality criterion of a test which consists of a linear penalty for the duration of observation and a penalty for a wrong decision proportional to the true value of the drift coefficient. The goal of this paper is to describe the structure of the exact optimal test in this problem, i.e. specify a stopping time and a rule to choose between the two hypotheses.
The main novelty of our work compared to the large body of literature related to sequential tests (for an overview of the field, see e.g. [13, 29]) is that we work with fBm. To the best of our knowledge, this is the first non-asymptotic solution of a continuous-time sequential testing problem for this process. Recall that the fBm is a Gaussian process, which generalizes the standard Brownian motion (sBm) and allows for dependent increments; see the definition in Section 2. It was first introduced by Kolmogorov  and gained much attention after the work of Mandelbrot and van Ness . Recently, this process has been used in various models in applied areas, including, for example, modeling of traffic in computer networks and modeling of stock market prices and their volatility; a comprehensive review can be found in the preface to the monograph .
It is well-known that a fBm is not a Markov process, neither a semimartingale except the case when it is a sBm. As a consequence, many standard tools of stochastic calculus and stochastic control (Itô’s formula, the HJB equation, etc.) cannot be applied in models based on fBm. In particular, recall that a general method to construct exact sequential tests, especially in Bayesian problems, consists in reduction to optimal stopping problems for processes of sufficient statistics; see e.g. Chapter VI in the book . In the majority of problems considered in the literature sufficient statistics are Markov processes, and so the well-developed general theory of Markov optimal stopping problems can be applied. On the other hand, optimal stopping problems for non-Markov and non-semimartingale processes, like fBm, often cannot be solved even by numerical methods.
Fortunately, in the problem we consider it turns out to be possible to change the original problem for fBm so that it becomes tractable. One of the key steps is a general transformation outlined in the note , which allows to reduce sequential testing problems for fBm to problems for diffusion processes. It is achieved by integration of a certain kernel with respect to an observable process and using the known fact that a fBm can be expressed as an integral with respect to a sBm, and vice versa.
In the literature, the result which is most closely related to ours is the sequential test proposed by H. Chernoff , which has exactly the same setting and uses the same optimality criterion, but considers only sBm. For a prior normal distribution of the drift coefficient, Chernoff and Breakwell [1, 5]
found asymptotically optimal sequential tests when the variance of the drift goes to zero or infinity. In the paper, we extended their result and constructed an exact optimal test. An important step was a transformation of the problem that reduced the optimal stopping problem for the sufficient statistic process, as studied by Chernoff and Breakwell, to an optimal stopping problem for a standard Brownian motion with nonlinear observation cost. A similar transformation is used in the present paper as well, see Section 3.
Let us mention two other recent results in the sequential analysis of fBm, related to estimation of its drift coefficient. Cetin, Novikov, and Shiryaev considered a sequential estimation problem assuming a normal prior distribution of the drift with a quadratic or a -function penalty for a wrong estimate and a linear penalty for observation time. They proved that in their setting the optimal stopping time is non-random. Gapeev and Stoev  studied sequential testing and changepoint detection problems for Gaussian processes, including fBm. They showed how those problems can be reduced to optimal stopping problems and found asymptotics of optimal stopping boundaries. There are many more results related to fixed-sample (i.e. non-sequential) statistical analysis of fBm. See, for example, Part II of the recent monograph , which discusses statistical methods for fBm in details.
Our paper is organized as follows. Section 2 formulates the problem. Section 3 describes a transformation of the original problem to an optimal stopping problem for a sBm and introduces auxiliary processes which are needed to construct the optimal sequential test. The main result of the paper – the theorem which describes the structure of the optimal sequential test – is presented in Section 4, together with a numerical solution. Section 5 contains the proof. Some technical details are in the appendix.
2 Decision rules and their optimality
Suppose one observes the stochastic process
where , , is a fractional Brownian motion (fBm) with known Hurst parameter and unknown drift coefficient . Recall that a fBm is a continuous zero-mean Gaussian process with the covariance function
In the particular case this process is a standard Brownian motion (sBm) and has independent increments; its increments are positively correlated in the case and negatively correlated in the case . Except the case , a fBm is not a Markov process, neither a semimartingale.
We will consider a Bayesian setting and assume that, independent of it and having a normal distribution with known mean and known variance .
It is assumed that neither the value of , nor the value of can be observed directly, but the observer wants to determine whether the value of is positive or negative based on the information conveyed by the combined process . We will look for a sequential test for the hypothesis versus the alternative . By a sequential test we call a pair which consists of a stopping time of the filtration , generated by , and an -measurable function assuming values
. The stopping time is the moment of time when observation is terminated and a decision about the hypotheses is made; the value ofshows which of them is accepted.
We will use the criterion of optimality of a decision rule consisting in minimizing the linear penalty for observation time and the penalty for a wrong decision proportional to the absolute value of . Namely, with each decision rule we associate the risk
where if and if . The problem consists in finding that minimizes over all decision rules. Note that one can consider a more general setting when the penalty for observation time is equal to with some constant (or the penalty for a wrong decision is ), but this case can be reduced to the one we consider by a change of the parameters (see ), and so we’ll focus only on .
This problem was proposed by H. Chernoff in  for sBm, and we refer the reader to that paper and the subsequent papers [1, 5, 6] for a rationale for this setting. Those papers contain results about the asymptotics of the optimal test and other its properties, including a comparison with Wald’s sequential probability ratio test. Our paper  contains a result which allows to find an exact (non-asymptotic) optimal test by a relatively simple numerical procedure.
3 Reduction to an optimal stopping problem
We will transform the problem of finding a decision rule minimizing (1) by eliminating the function from it and reducing it to an optimal stopping problem. On the first step, we’ll obtain an optimal stopping problem for a fBm. Then by changing time and space coordinates it will be reduced to an optimal stopping problem for a sBm, which will allow to apply well-developed methods to solve it.
From the relation , where , , and the fact that is -measurable, one can see that the optimal decision rule should be looked for among rules with . Hence, it will be enough to solve the optimal stopping problem which consists in finding a stopping time such that , where
(for brevity, we’ll use the same notation for the functional associated with a decision rule, and the functional associated with a stopping time). Then the optimal decision rule will be with if and otherwise.
Next we are going to transform the expression inside the expectation in to the value of some process constructed from a sBm. It is known (see e.g.  and the earlier results [20, 17]) that the following process , , is a sBm, and the filtrations generated by and coincide:
with the kernel
where is the Gauss hypergeometric function, and the constant is defined by
Using the above connection between and and computing the corresponding integral with respect to we obtain that
and the filtrations of the processes and coincide. The constant is defined by
where is the beta function (see the appendix for computational details).
From the Cameron–Martin theorem or the Girsanov formula and the general Bayes theorem (see e.g. Chapters 6, 7 in, and the appendix) one can find that the conditional distribution of is normal:
with the processes
Then can be written as
with the function for a normal random variable . In the explicit form, , where are the standard normal distribution and density functions.
Observe also that (3) implies that for the optimal stopping time , the corresponding optimal function is equal to 1 if and if .
Define the process , ,
The innovation representation (see Chapter 7.4 in ) implies that is a standard Brownian motion. Then the process satisfies the SDE
Next we’ll apply the Itô formula to . In order to avoid problems caused by that is not smooth at , consider the function . It can be easily verified that , the derivative is continuous at for any , and for all , . From this identity, applying the Itô formula we obtain
Using that , one can see that is finite, so is a square-integrable martingale. Therefore, for any stopping time we have , which transforms (4) to
(Explicitly, .) Observe that satisfies the equation
For brevity, denote
Then under the following monotone change of time (see the appendix for details)
where runs through the half-interval when runs through , the process
is a sBm in , and the filtrations and coincide. Therefore, for any stopping time of the filtration we have
where is a stopping time of the filtration , and the constant
Thus, the optimal stopping problem for in -time is equivalent to the following optimal stopping problem for in -time:
Namely, if is an optimal stopping time in (6), then an optimal decision rule is given by
4 The main result
In this section we formulate the main theorem about the solution of problem (6), which provides an optimal sequential test via (7). Throughout we will assume that the parameters of the problem , , remain fixed and will denote the function
It is well-known that under general conditions the solution of an optimal stopping problem for a Markov process can be represented as the first time when the process enters some set – a stopping set. Namely, let us first rewrite our problem in the Markov setting by allowing the process to start from any point :
where the infimum is over all stopping times of the Brownian motion such that a.s. In particular, for the quantity from (6) we have . We subtract in the definition of to make the function bounded. For we define .
The following theorem describes the structure of the optimal stopping time in problem (8). In its statement, we set
Obviously, for and for . We find the solution of problem (8) by describing the boundary of the stopping set as a function of time : in the case this will be done only for , while in the case for all . Unfortunately, our method of proof does not work for when , although the boundary can still be formally found from the equation we obtain in the theorem; see the discussion below.
1) There exists a function defined on , which is continuous, non-increasing, and strictly positive for with , such that for any and the optimal stopping time in problem (8) is given by
Moreover, for any the function satisfies the inequality
2) The function is the unique continuous non-negative solution of the integral equation
with the functions and for a standard normal random variable .
The main reason why we characterize the boundary only for in the case is that the method of proof we use to show that satisfies the integral equation requires it to be of bounded variation (at least, locally). This condition is needed as a sufficient condition to apply the Itô formula with local time on curves , on which the proof is based on. In the case and for in the case by a direct probabilistic argument we can prove that is monotone and therefore has bounded variation; this argument however doesn’t work for in the case , and, as a formal numerical solution shows, the boundary seems to be indeed not monotone in that case. Of course, the assumption of bounded variation can be relaxed while the Itô formula can still be applied (see e.g. [22, 7, 8]), however verification of weaker sufficient conditions is problematic. Although the general scheme to obtain integral equations of type (10) and prove uniqueness of their solutions was discovered quite a while ago (the first full result was obtained by Peskir  for the optimal stopping problem for American options), and has been used many times in the literature for various optimal stopping problems (a large number of examples can be found in ), we are unaware of any of its non-trivial applications in the case when stopping boundaries are not monotone. Nevertheless, a formal numerical solution of the integral equation shows that the stopping boundaries “look smooth”, but we admit that a rigor proof of this fact in the above-mentioned cases remains an open question.
Note also, that in the case the space-time transformation we apply to pass from the optimal stopping problem for the process to the problem for is essential from this point of view, because the boundaries in the problem for are not monotone. Moreover, they are not monotone even in the case , when is obtained by simply shifting in time and space, see [4, 19].
The second remark we would like to make is that in the case we do not know whether is finite. In the case the finiteness of follows from inequality (9), which is proved by a direct argument based on comparison with a simpler optimal stopping problem (one can easily see from the proof that (9) extends to for ). It seems that a deeper analysis is required for the case , which is beyond this paper.
Figure 1 shows the stopping boundary for different values computed by solving equation (10) numerically. The solution can be obtained by backward induction on a discrete set of points of starting with and going towards using that the expression under the integral depends only on the values of for ; the method is described in more details, for example, in .
5 The proof of the theorem
The proof will be conducted in several steps: (i) prove the monotonicity and continuity of the function ; (ii) analyze the stopping set and its boundaries; (iii) formulate a free-boundary problem for ; (iv) derive the integral equation for ; (v) prove the uniqueness of its solution.
(i) It is clear that
One can easily verify that is increasing in , which implies that doesn’t decrease in for each fixed ( is the point of minimum of on ). Also, for any the function is non-decreasing in on and non-increasing in . This follows from the inequality for any and any stopping time .
The monotonicity of both in and implies that in order to prove the continuity of this function for , it is enough to prove the continuity in each argument (see e.g. ). The continuity in follows from that for any , we have
and also the continuity of is obvious. In order to prove the continuity in , fix and consider arbitrary , . Then for any stopping time
where the first inequality follows from straightforward algebraic transformations, and to prove the second one the Markov property of Brownian motion was used. We have
where the process , , is a Brownian motion. In the right inequality we used the well-known bound , which easily follows from the Wald identity and Jensen’s inequality. Taking in (12) the infimum over all stopping times we obtain . According to what was proved above, . Therefore, , which proves the continuity of in for . The continuity at is obvious since as .
(ii) Define the stopping set and the set . The continuity of implies that is closed.
It is clear that is symmetric in , and from the monotonicity of it follows that can be represented in the form
where is some non-increasing function on . Obviously, and one can easily see that for any , since for for any it is possible to find a sufficiently small non-random such that , hence .
Let us prove inequality (9) for . By using the inequality for any , one can see that for any and any stopping time
Denote the expression in the brackets by . Observe that if , then and hence is a stopping time, and also . Then
where . For the optimal stopping problem in the RHS, the solution is well-known (see e.g. Section 16 in ): the optimal stopping time is . Hence, if , then and , which proves (9). In particular, is finite-valued for all .
In order to show that is finite-valued for when , one can use that , since is the point of minimum of the function when . Again, all the points with should be in the stopping set, so is bounded by .
Next, for any point , , define the candidate optimal stopping time – the first entry into the stopping set:
In the general theory of optimal stopping for Markov processes, it is well-known that the first entry into the stopping set is an optimal stopping time under mild conditions. In our problem this fact can be proved similarly to  (one subtlety here is that general conditions for the optimality of typically require some boundedness of the payoff function, see for example Chapter 1 in ; since the payoff in our problem is unbounded a finer argument is needed).
Now we prove that is continuous on . Since it doesn’t increase and the set is closed, it is clear that is right-continuous. Let us prove that it is left-continuous. Using that we can write
Suppose for some . Consider points with and sufficiently small . Let denote the random event that exists the rectangle through the right boundary. Denote for brevity . Then from (13) we have
where in the first inequality we used that on and in the second one we applied the Cauchy–Schwarz inequality to .
According to Doob’s martingale inequality, . Since when , there exists a sufficiently small such that , which contradicts the definition of . This proves the continuity of on . The continuity at follows from inequality (9).
(iii) As follows from the general theory of optimal stopping for Markov processes, inside the continuation set the value function is
and satisfies the partial differential equation (see Section 7 in)
Together with the condition in the set , this constitutes a free boundary problem for the value function with the unknown free boundary .
The continuity of implies the so-called condition of continuous fit: , i.e. is continuous at the stopping boundary. Let us now prove the smooth-fit condition, which states that the -derivative of is continuous at the stopping boundary:
The function in concave in since it is the infimum (over ) of concave functions. Therefore, there exist the left and right derivatives . Clearly, , since for . Moreover, for any sufficiently small we have
since and . Therefore, . Let us prove the opposite inequality.
Fix . Set and let be sufficiently small. Then for the optimal time we have
where we used that and . Transform the obtained expression:
The second term can be bounded in absolute value by . Then
which proves the inequality .
(iv) So far we have established the following properties: (a) is continuous on and is in and in the interior of ; (b) is continuous and non-increasing on ; (c) is locally bounded in and in the interior of , which follows from (14); (d) the function is concave and the function is continuous.
where is the local time processes of on the curves (see ), and .
Smooth-fit condition (15) implies that the two last terms in (16) are equal to zero. Also, the derivative is uniformly bounded according to (11), and therefore the expectation of the stochastic integral in (16) is also zero.
From (14) and the fact that for we obtain
By passing to the limit , we have