. We present a programmable system capable of generating Gaussian random variates by extracting the noise properties of a MEMS sensor. We demonstrate its principle and application. Sampling a random physical process with a given distribution provides a continuous random variable with a theoretically unlimited sample rate. In a hardware implementation, the sample rate of the analog to digital convertor limits random number generation rate.
I-a Generating Uniform Random Variates Is Easy
Uniform random numbers are generated for cryptography . Gaussian random variate generation is typically an order of magnitude slower and less efficient than uniform random variate generation . Table I
compares several state-of-the-art methods of generating non-uniform random variates with a Gaussian distribution. We propose a method superior to all of the state-of-the-art methods in terms of sample rate and efficiency, consisting of a physical noise source and an analog to digital convertor.
|Device||Sample rate (Gsample/s)||Efficiency (Msample/J)|
I-B Generating Non-Uniform Random Variates Is Harder
The inversion method and accept reject method are used in software for generating samples from non-uniform random variates . Let and be uniform and non-uniform random variates respectively and
the analytical closed-form solution for the inverse cumulative distribution function. Algorithm 1 shows the inversion method. The inversion method requires that the inverse cumulative distribution function has an analytical closed-form solution. The Gaussian distribution has no analytical closed-form solution for its inverse cumulative distribution function so cannot be used with the inversion method . The accept reject method must be used instead. Let and be independent and the density on of . Where is -dimensional Euclidean space. Algorithm 2 shows the accept reject method. The accept reject method requires more math operations than the inversion method to transform a sample . This causes it to take more clock cycles to compute and therefore, more time to transform each sample. When using the accept reject method, samples deviating from the desired distribution are rejected .
I-C Uses Of Non-Uniform Random Variates
Non-uniform random variate generators are fundamental to applications employing Monte Carlo Methods ,
population balance modelling of the crystallisation process ,
generating phylogenetic trees , drug discovery ,
ray tracing , communication channel emulation , financial computing , local area networks ,
modelling of manufacture systems  and measuring healthcare strategy effectiveness .
We calculate the marginal likelihood by integrating the joint density 
. In practice, the analytical calculation of the marginal likelihood is impossible for all but the simplest joint distributions. We instead sample from the joint distribution to obtain summary statistics that we can use to describe it 
. These random samples from bespoke probability distributions must be produced using a non-uniform random variate generator.
Ii Motivating Example
We performed Monte Carlo integration of a Gaussian with a mean of 0 and standard deviation of 1 using samples from a Gaussian with the same parameters. We ran the experiment with a Gaussian generated by the C++ random library and repeated it with the random number generation rate adjusted to match that of our proposed non-uniform random variate generator based on physical noise sources. We performed the same integration using samples from a uniform distribution with a minimum of -3 and a maximum of 3. Letbe the error of the integration, be the time taken by the integration, be the number of random samples and be the distribution (either uniform or Gaussian). Let samps be the array of random numbers, be the area, be the rectangle base length, be the rectangle height and
the probability density function of the Gaussian for integration. Algorithm3 shows the integration scheme that we used. We repeated each integration process 1000 times and calculated the average. Figure 1 shows that the proposed hardware random number generator outperforms the C++ uniform random number generator after 1000 samples and reduces the error by after 1 million samples. The proposed hardware random number generator always performs the task at least twice as fast as the C++ Gaussian random number generator.
A non-uniform random variate generator based on physical noise sources must have negligible drift of the mean and standard deviation over time. Drift would cause errors in calculations using the output of the random number generator. Any environmental parameter that causes non-negligible drift must be physically controlled, for example using a voltage regulator to control voltage and a Peltier device to control temperature. We investigated the dependence of 100,000 sample distributions on voltage and temperature. We sampled the z-axis of a MEMS accelerometer (we used the accelerometer in the Bosch BMX055) to obtain the distributions.
Iii-a Temperature-Controlled Experiments
Figure 2 shows the experimental setup. We placed the microcontroller, accelerometer, tilt and rotate stage and vibration isolation platform inside a Binder MK56 thermal chamber. We connected a microcontroller to the sensor via I2C for a 1154 Hz sample rate. We used a Keithly 2450 source measurement unit to power the sensor and measure the current drawn. We set the chamber temperature to 25 C and allowed 30 minutes for the temperature of the sensor to equilibrate whilst constantly sampling z-axis acceleration values from it. We then sampled 100,000 values from the BMX055 sensor at a 3.6 V supply voltage. We repeated this for all the voltages in the range of 3.6 to 1.4 V with a 0.2 V decrement. We then repeated this process for the temperature from 25 down to -5 C with a decrement of 5 C.
Iii-B Quantisation Investigation
We investigated the effect of quantisation on the Kullback–Leibler (KL) divergence between a discrete distribution and its ideal fitted curve. We used the MATLAB normrnd function to generate 100,000 values from a Gaussian distribution with the same mean and standard deviation as the BMX055 z-axis at 2.6 V and 10 C. We then discretized the values into various numbers of bins, fitted a Gaussian distribution to them and calculated the KL divergence between the fitted distribution and the actual distribution.
Iv Results and Discussion
We calculated the KL divergence between two discrete distributions using the following equation where and are discrete probability distributions, is a given sample value and the sample space :
Figure 3 shows the BMX055 z-axis acceleration distribution at 2.6 V and 10 C. We found that the KL divergence between the distribution from the BMX055 z-axis and its fitted Gaussian (0.00263) was more than an order of magnitude smaller than the equivalent result for a MATLAB generated distribution (0.0392). We rounded the MATLAB generated floats for comparison with the BMX055 generated integers. The KL divergence between a MATLAB generated uniform distribution and its fitted Gaussian is 0.116 for reference. We averaged the KL divergence calculations over 100 distributions to account for the random variations in the measurement. The numbers generated by the sensor are closer to an ideal Gaussian distribution than those generated by MATLAB.
Figures 4 and 5 show how voltage and temperature effect the mean and standard deviation of the BMX055 z-axis acceleration measurement. Temperature has a greater effect on mean and standard deviation than voltage. Both voltage and temperature have a greater effect upon the standard deviation than the mean. A random number generator based on this phenomenon should control both the temperature and the supply voltage of the sensor.
Figure 6 shows the effect of increasing the bin size on the divergence between a distribution of 100,000 values and its ideal fitted distribution. This shows that increased quantization increases the difference between a distribution and its fitted Gaussian.
V Gaussian To Gaussian Transform
A univariate Gaussian can be transformed to any other univariate Gaussian with one multiplication and one addition. This is significantly less computation than the accept reject method which requires at least 10 operations per each accept reject test: an exponential, square, square root, subtraction, comparison and five divisions / multiplications. The accept reject method may furthermore need to repeat this set of operations numerous times for each random variate. Two uniform random numbers must be generated for one accept reject method sample. Rapid addition and multiplication can be achieved using fast adders and multipliers implemented on an FPGA, Figure 7 shows how this could be achieved. The CPU requests a distribution by specifying parameters to the transform circuitry which proceeds to transform the input Gaussian to fit the requested output distribution. The transform circuitry then stores the values in a small high-speed cache that the CPU can read from. The asynchronous transform circuitry on the FPGA will be free to execute transformations at a rate not bound by the global clock frequency. Offloading the transformation to dedicated hardware leaves the processor free to execute other instructions which will improve performance.
Sensors are a feasible source of non-uniform random variates at a higher sample rate and with greater efficiency than all of the state-of-the-art methods used to generate them. The mean and standard deviation of the noise produced by the z-axis of a commercial accelerometer depend upon the temperature of the environment and the supply voltage. The parameters of the distribution drift with temperature and supply voltage so both must be kept constant to avoid this. Quantizing a Gaussian distribution increases the KL divergence between it and a fitted Gaussian. This type of non-uniform random number generator can reduce the error of Monte Carlo integration of a univariate Gaussian by a factor of whilst doubling the speed.
Combined quadrature method of moments and method of characteristics approach for efficient solution of population balance models for dynamic modeling and crystal size distribution control of crystallization processes. Industrial & Engineering Chemistry Research 48 (18), pp. 8575–8584. Cited by: §I-C.
-  (2010) Monte carlo simulation for the pharmaceutical industry: concepts, algorithms, and case studies. CRC Press. External Links: Cited by: §I-C.
-  (1986-01) Stochastic sampling in computer graphics. ACM Trans. Graph. 5 (1), pp. 51–72. External Links: Cited by: §I-C.
-  (2016-08) Building bayesian inference graphs for healthcare statistic evidence. In 2016 45th International Conference on Parallel Processing Workshops (ICPPW), Vol. , pp. 415–420. External Links: Cited by: §I-C.
-  (2000-12) Efficient fpga implementation of gaussian noise generator for communication channel emulation. In ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445), Vol. 1, pp. 366–369 vol.1. External Links: Cited by: §I-C.
-  (1986) Non-Uniform Random Variate Generation. Springer-Verlag, McGill University Montreal H3A 2K6 Canada. Cited by: §I-B.
-  KLDIV. Nima Razavi. Note: [Online]. Available: https://uk.mathworks.com/matlabcentral/fileexchange/13089-kldiv?s_tid=FX_rc1_behav, Accessed: 27/03/2019 Cited by: §IV.
-  (2014) Why the monte carlo method is so important today. Wiley Interdisciplinary Reviews: Computational Statistics 6 (6), pp. 386–392. Cited by: §I-C.
A student’s guide to bayesian statistics. SAGE. Cited by: §I-C, §I-C.
-  (2006) Stochastic modeling of manufacturing systems. Springer. External Links: Cited by: §I-C.
-  (2005) Probability and computing: randomized algorithms and probabilistic analysis. Cambridge University Press, New York, NY, USA. External Links: Cited by: §I-C.
-  (2019-01) Marginal Likelihoods in Phylogenetics: A Review of Methods and Applications. Systematic Biology 68 (5), pp. 681–697. Cited by: §I-C.
-  Randomness requirements for security. Network Working Group. Cited by: §I-A.
-  (1995) Simulation of local area networks. CRC Press, Inc., Boca Raton, FL, USA. External Links: Cited by: §I-C.
-  (2007-07) Non-uniform random number generation through piece-wise linear approximations. IET Computers Digital Techniques 1 (4), pp. 312–321. External Links: Cited by: §I.
Resource efficient generators for the floating-point uniform and exponential distributions. In 2008 International Conference on Application-Specific Systems, Architectures and Processors, Vol. , pp. 102–107. External Links: Cited by: §I-C.
-  (2006-04) Efficient hardware generation of random variates with arbitrary distributions. In 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, Vol. , pp. 57–66. External Links: Cited by: §I-B, §I.
-  (2009) A comparison of cpus, gpus, fpgas, and massively parallel processor arrays for random number generation. In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, FPGA ’09, New York, NY, USA, pp. 63–72. External Links: Cited by: §I-A, TABLE I, §I.
-  (2015-Sep.) Nanoscale resonance energy transfer-based devices for probabilistic computing. IEEE Micro 35 (5), pp. 72–84. External Links: Cited by: §I.
-  (2005-12) Reconfigurable acceleration for monte carlo based financial simulation. In Proceedings. 2005 IEEE International Conference on Field-Programmable Technology, 2005., Vol. , pp. 215–222. External Links: Cited by: §I-C.