Differentiable reservoir computing
share
Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so called echo state and fading memory properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this paper where the differentiability of reservoir filters is fully characterized for very general classes of discrete-time deterministic inputs. The local nature of the differential allows the formulation of conditions that ensure both the local and global existence of differentiable and, in passing, fading memory solutions, which links to existing research on the input-dependent nature of the echo state property. A Volterra-type series representation for reservoir filters with semi-infinite discrete-time inputs is constructed in the analytic case using Taylor's theorem and corresponding approximation bounds are provided. Finally, it is shown as a corollary of these results that any fading memory filter can be uniformly approximated by a finite Volterra series with finite memory.
READ FULL TEXT