Functional magnetic resonance imaging (fMRI) offers a unique means of observing the functional brain architecture and its variation during development, aging, or disease. Since its invention in the early 1990s, fMRI has revolutionized neuroscience by characterizing functional connectivity (FC) as the correlation between regional brain activity over time. Despite the insights into network formation and functional growth of the brain, in utero fMRI of living human fetuses, however, remains challenging. Unconstrained and potentially large movements of the fetuses during fMRI acquisition result in artifacts such as in-plane image blurring, slice cross-talk and spin-history artifacts that can considerably affect the image quality and bias any subsequent conclusions on the FC of the developing brain.
Some of the early works in this area established tools from preprocessing of adult fMRI data (eg. MCFLIRT ) and censored parts of the data with excessive motion (scrubbing). The reported rejection rate in these studies is very high and many subjects were removed since there was not enough data left after scrubbing. Further attempts to correct for fetal fMRI motion-related artifacts mainly relied on improving the estimate of the realignment parameters for potentially large movement of the fetuses. Ferrazzi et al.  proposed a comprehensive processing framework that incorporates bias field and spin-history correction, combined with slice to volume registration and scattered data interpolation, which allowed the data to be analyzed within a single consistent space . Furthermore, to avoid that surrounding maternal tissues dominate the similarity metric of registration algorithms, 
trained a convolutional neural network for rapid and accurate isolation of the fetal brain and then utilized rigid registration for motion correction. You et al. also in present an optimized design based on the MCFLIRT registration to identify independently moving objects (i.e., the placenta and fetal brain) and estimate the local motion for each. All these studies, however, used 3D volume interpolation of each time frame independently to reconstruct the whole FC data. More recently, 
extended super resolution technique (will be discussed in the next section) with separate reconstruction for each 3D volume within the time sequence. Since in-utero motion is unconstrained and complex, the rate of scattered points versus the points to infer is less than 1 to 1 in each 3D volume. Therefore, taking the advantage of temporal structure in fMRI imaging can potentially improve the reconstruction of image in this challenging problem. Here we propose a new method that, rather than treating each volume independently, takes both spatial- and the temporal domain into account and iteratively reconstruct 4D fetal fMRI time series.
Our method directly reconstructs fetal fMRI time series on a 4D regular grid from motion scattered slices. We formulate our problem as a maximum likelihood (ML) estimation in which the conditional probability density function (pdf) of the acquired slices given the current estimate of the reconstruction is maximized:
Here, denotes an estimate of the final 4D reconstructed image i.e. , and is the k-th acquired slice. is a prior on and is a constant with respect to
. Using the assumption of Gaussian noise with zero mean and standard deviation of, and independent acquisition of each slice, the above conditional pdf can be written as
where are the samples from the acquired slices , and are the samples from the estimated slices by considering a slice acquisition model as . is the subsampling matrix, is the blur matrix representing the point spread function (psf), and is the observation noise. The motion matrix
is defined for each slice through slice-to-volume registration as a 6-DOF 3-D rigid transformation (including three rotations and three translations). When the noise residuals are presumed to be drawn from a Gaussian distribution, and using the logarithmic function, the above problem is equivalent to obtaining the Maximum A posteriori Probability (MAP) estimate by minimizing the log likelihood function:
The ML formulation is applied in structural fetal T2-weighted MRI where a high-resolution volume is reconstructed using several orthogonal series of thick slices, called as super-resolution technique [6, 7, 8, 9]. The point spread function used in that case is derived directly from the image acquisition process. In fMRI analysis, however, low-frequency fluctuations of the signal over time is desired and post-processing commonly includes low-pass filtering to derive temporally smooth signal. In this work we propose, rather than using a separate post-processing filter, to integrate this step within the reconstruction framework by using a four dimensional point spread function. This also provides further stability of the reconstruction by incorporating information from the neighboring slices in time in case of a large motion and missing data. The 4D psf can be formulated as:
where the values represent the point spread in x, y, z, and t, which are the neighborhood of a voxel is defined on both spatial and temporal domain. To create a distance metric in space–time, we scaled the time dimension, as suggested in , by a factor of , where res(z) is the through-plane resolution of the image and TR is the repetition time.
The algorithm involves iterative backprojection of the motion scattered slices onto a regular 4D grid using a 4D psf, estimating the slice intensities given the current estimate of the 4D image (forward projection using 4D psf), and computing the residual error between these two. This error is backprojected and added to the regularization term which was computed using a first order Tikhonov function. Optimization terminates when the residual error between estimated and measured data converges.
2.1 Implementation details
We implemented our algorithm of 4D reconstruction based on the publicly available toolkit of Numerical Solver Library (NSol) which is a part of NiftyMIC package  for volumetric reconstruction of the structural 3D images. A binary brain mask was manually delineated on the average volume of each fetus and dilated to ensure it covers the fetal brain through all ranges of the motion. A four dimensional estimate of the bias field for spatiotemporal signal non-uniformity correction in fMRI series was obtained using N4ITK algorithm  as suggested previously . The motion correction step involved globally co-registering all time points, creating a target volume as proposed in  by automatically finding a set of consecutive volumes of fetal quiescence and averaging over them, and finally performing hierarchical slice-to-volume registration based on the interleaved factor of acquisition as implemented in NiftyMIC package.
3 Experimental Results
We evaluated the proposed algorithm on fetal fMRI scans of 15 subjects between 22 and 39 weeks of gestation (Table.1). None of the cases showed any neurological pathologies. Pregnant women were scanned on a 1.5T clinical scanner (Philips Medical Systems, Best, Netherlands) using single-shot echo-planar imaging (EPI), and a sensitivity encoding (SENSE) cardiac coil with five elements. Image matrix size was 144144, with 1.741.74 in-plane resolution, 3 slice thickness, flip angle of 90, and 96 volumes per acquisition. We used a hierarchical approach based on a symmetric block matching registration algorithm to estimate the rigid realignment parameters of each slice. Table.1 summarizes the estimated motional parameters for all subjects, showing especially free excessive rotation in fMRI images of the fetal population. Figure 1 shows the slice-wise realignment parameters for a subject with strong rotation (27.9°in left-right direction). When such big motion occurs, gaps might be open up in the reconstructed image between slices that are spatially adjacent. Therefore, interpolation methods using only spatial information, although faster in implementation, may not correct it as there would be no data contributing to the reconstruction of a given regular grid. As fMRI data is temporally correlated, adjacent slices in time can potentially add information to the reconstruction. Here we used a four dimensional Gaussian kernel as psf to take both spatial and temporal structure of the data into account, and iteratively optimized our reconstruction through inverse problem framework.
|GA||Max translation (mm)||Max rotation (°)||Sharpness||Standard deviation|
3.1 Reconstruction accuracy
We compared the proposed 4D reconstruction with standard 3D linear reconstruction. We evaluated the sharpness  of the averaged volume after motion correction and reconstruction, and the standard deviation of BOLD signal fluctuations at each voxel over time. Figure 2 illustrates the average volume (top row), and standard deviation of intensity over time (bottom row) of the original data without motion correction, data after single step 3D linear interpolation, and data reconstructed using the proposed algorithm for two subjects with different motion characteristics. Severe blurring is observed in the image obtained from 3D linear interpolation. In contrast, the proposed method yields qualitatively sharper images as the effect of slice motion is appropriately decreased. Quantitative results are reported in Table 1 for all 15 subjects. Figure 3 shows the boxplot for the relative changes of the sharpness (left) and standard deviation (right) in 3D interpolation vs 4D Gaussian reconsruction normalized to the corresponding value in raw data. In 12 out of 14 subjects the proposed method yields higher sharpness than 3D interpolation, with an overall average sharpness higher than for 3D interpolation, in 11 it exhibits lower standard deviation, with an average standard deviation reduction of compared to 3D interpolation. For several subjects with lower gestational age the highest sharpness and lowest standard deviation occurs with un-corrected images. One possible cause is that the advanced gyrification at later gestational ages results in a pronounced sharpness advantage with correction, while for younger fetuses this is not the case and might reflect image noise rather than anatomical structure.
In this work we proposed a novel spatio-temporal reconstruction approach for the fetal brain acquired while there is unconstrained motion of the head. Our technique integrates the temporal continuity of fMRI time series with the spatial coherency of the neighboring brain voxels by using a four dimensional Gaussian point spread function for reconstructing motion scattered slices. Qualitative results of the resulting reconstruction of clinical fetal fMRI data shows improvement by the method. Quantitative evaluation confirms this improvement predominantly for higher gestational age. As the developed formulation for fMRI time series reconstruction is general and can be effectively used in adult studies as well, it would be of interest to investigate its performance on the adult population in the future.
This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 765148, and partial funding from the Austrian Science Fund (FWF, P35189). This work was also supported by the Swiss National Science Foundation (project 205321-182602). We acknowledge access to the expertise of the CIBM Center for Biomedical Imaging, a Swiss research center of excellence founded and supported by Lausanne University Hospital (CHUV), University of Lausanne (UNIL), Ecole polytechnique fédérale de Lausanne (EPFL), University of Geneva (UNIGE) and Geneva University Hospitals (HUG).
-  Mark Jenkinson, Peter Bannister, Michael Brady, and Stephen Smith, “Improved optimization for the robust and accurate linear registration and motion correction of brain images,” Neuroimage, vol. 17, no. 2, pp. 825–841, 2002.
-  Giulio Ferrazzi, Maria Kuklisova Murgasova, Tomoki Arichi, Christina Malamateniou, Matthew J Fox, Antonios Makropoulos, Joanna Allsop, Mary Rutherford, Shaihan Malik, Paul Aljabar, et al., “Resting state fmri in the moving fetus: a robust framework for motion, bias field and spin history correction,” Neuroimage, vol. 101, pp. 555–568, 2014.
-  Saige Rutherford, Pascal Sturmfels, Mike Angstadt, Jasmine Hect, Jenna Wiens, Marion I van den Heuval, Dustin Scheinost, Moriah Thomason, and Chandra Sripada, “Observing the origins of human brain development: Automated processing of fetal fmri,” bioRxiv, p. 525386, 2019.
-  Wonsang You, Iordanis E Evangelou, Zungho Zun, Nickie Andescavage, and Catherine Limperopoulos, “Robust preprocessing for stimulus-based functional mri of the moving fetus,” Journal of Medical Imaging, vol. 3, no. 2, pp. 026001, 2016.
-  Daniel Sobotka, Roxane Licandro, Michael Ebner, Ernst Schwartz, Tom Vercauteren, Sebastien Ourselin, Gregor Kasprian, Daniela Prayer, and Georg Langs, “Reproducibility of functional connectivity estimates in motion corrected fetal fmri,” in Smart Ultrasound Imaging and Perinatal, Preterm and Paediatric Image Analysis, pp. 123–132. Springer, 2019.
-  Ali Gholipour, Judy A Estroff, and Simon K Warfield, “Robust super-resolution volume reconstruction from slice acquisitions: application to fetal brain mri,” IEEE transactions on medical imaging, vol. 29, no. 10, pp. 1739–1758, 2010.
Maria Kuklisova-Murgasova, Gerardine Quaghebeur, Mary A Rutherford, Joseph V
Hajnal, and Julia A Schnabel,
“Reconstruction of fetal brain mri with intensity matching and complete outlier removal,”Medical image analysis, vol. 16, no. 8, pp. 1550–1564, 2012.
-  Sébastien Tourbier, Xavier Bresson, Patric Hagmann, Jean-Philippe Thiran, Reto Meuli, and Meritxell Bach Cuadra, “An efficient total variation algorithm for super-resolution in fetal brain mri with adaptive regularization,” NeuroImage, vol. 118, pp. 584–597, 2015.
-  Michael Ebner, Guotai Wang, Wenqi Li, Michael Aertsen, Premal A Patel, Rosalind Aughwane, Andrew Melbourne, Tom Doel, Steven Dymarkowski, Paolo De Coppi, et al., “An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain mri,” NeuroImage, vol. 206, pp. 116324, 2020.
-  Nicholas J Tustison, Brian B Avants, Philip A Cook, Yuanjie Zheng, Alexander Egan, Paul A Yushkevich, and James C Gee, “N4itk: improved n3 bias correction,” IEEE transactions on medical imaging, vol. 29, no. 6, pp. 1310–1320, 2010.
-  Sharmishtaa Seshamani, Xi Cheng, Mads Fogtmann, Moriah E Thomason, and Colin Studholme, “A method for handling intensity inhomogenieties in fmri sequences of moving anatomy of the early developing brain,” Medical image analysis, vol. 18, no. 2, pp. 285–300, 2014.
-  Sharmishtaa Seshamani, Mads Fogtmann, Xiaoyin Cheng, M Thomason, Chris Gatenby, and Colin Studholme, “Cascaded slice to volume registration for moving fetal fmri,” in 2013 IEEE 10th International Symposium on Biomedical Imaging. IEEE, 2013, pp. 796–799.
José Luis Pech-Pacheco, Gabriel Cristóbal, Jesús Chamorro-Martinez,
and Joaquín Fernández-Valdivia,
“Diatom autofocusing in brightfield microscopy: a comparative
Proceedings 15th International Conference on Pattern Recognition. ICPR-2000. IEEE, 2000, vol. 3, pp. 314–317.