Towards a Computer Vision Particle Flow

03/19/2020 ∙ by Francesco Armando Di Bello, et al. ∙ Weizmann Institute of Science INFN 8

In high energy physics experiments Particle Flow (PFlow) algorithms are designed to reach optimal calorimeter reconstruction and jet energy resolution. A computer vision approach to PFlow reconstruction using deep Neural Network techniques based on Convolutional layers (cPFlow) is proposed. The algorithm is trained to learn, from calorimeter and charged particle track images, to distinguish the calorimeter energy deposits from neutral and charged particles in a non-trivial context, where the energy originated by a π^+ and a π^0 is overlapping within calorimeter clusters. The performance of the cPFlow and a traditional parametrized PFlow (pPFlow) algorithm are compared. The cPFlow provides a precise reconstruction of the neutral and charged energy in the calorimeter and therefore outperform more traditional pPFlow algorithm both, in energy response and position resolution.



There are no comments yet.


page 5

page 6

page 7

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

General-purpose high energy collider experiments are designed to measure both charged particle trajectories and
calorimeter clustered energy deposits. The charged particle tracks in a magnetic field and the topology of energy deposits in calorimeters provide most of the information necessary to reconstruct, identify and measure the energy of the particles that constitute the event, which for the most part are charged and neutral hadrons, photons, electrons, muons and neutrinos. The latter escaping detection and are measured by the imbalance in momentum in electron-positron collision events or transverse momentum in hadron collision events. Other particles created during a high energy collision, typically have too short lifetimes to be directly detected in the experiment and need to be reconstructed from the aforementioned particles that do traverse the detector.

The goal of Particle Flow (PFlow) algorithms is to make optimal use of the complementarity of the measurements of the experiment’s detectors to reconstruct the particle content and its energy response for the entire event. A precise reconstruction of the entire event is essential for the measurement of those particles, such as neutrinos, escaping detection, as well as the reconstruction of jets of particles originating from the fragmentation and hadronization of hard scattering partons. One of the most challenging aspects of PFlow algorithms is to disentangle particles of different nature when they are close to one another and possibly overlap. The reconstruction performance in general and the performance of PFlow algorithms, in particular, will critically depend on the detector design specifications, for instance, the size and field of the tracking volume, the granularity of the calorimeters and their energy resolution. The first PFlow algorithm was designed by the CELLO collaboration at PETRA Behrend and others (1982), where an optimal reconstruction of the event "Energy Flow "

was measured by subtracting the expected energy loss from charged particles in the calorimeter, to estimate the

"neutral energy" and its spatial distribution. This algorithm, developed in collisions, was aimed at a precise measurement of the hadronic activity for measurement of . Since then, PFlow algorithms relying on the parametrization of the expected charged energy deposits in the calorimeter have been further developed at  Buskulic and others (1995) and  ATLAS Collaboration (2017); CMS Collaboration (2017) collider experiments. The success of these algorithms has been such that future collider experiment projects are taking PFlow algorithms into account in the design of the projected detectors Brient (2004); Thomson (2009); Ruan and Videau (2013); Bicer and others (2014); Ahmad and others (2015).

In this paper, we explore the capabilities of computer vision algorithms to provide solutions to this complex question in a context of two overlapping particles, a charged and a neutral pion and , in which traditional PFlow algorithms have been developed and tuned ATLAS Collaboration (2017). That is the most common scenario in high energy hadron collisions where most of the event particle content originate from parton fragmentation and are typically charged and neutral pions.

This work aims to provide a proof-of-concept of the use of a computer vision PFlow (cPFlow) algorithm based on a convolutional neural network to optimally infer the magnitude and direction of the neutral energy in cases where both charged and neutral particles are present and possibly overlapping.

For these studies, a detector that captures the main characteristics of a multipurpose detector such as ATLAS Collaboration (2008), with a simplified geometry is used. As discussed above, a critically important aspect to consider for the performance of PFlow algorithms is the granularity of the calorimeter both in the longitudinal direction of the development of hadronic and electromagnetic showers and in the plane transverse to it. The specificities of granularity and calorimeter energy response chosen in the studies presented herein, correspond to a generic multipurpose particle detector. Events are then generated with a and a in various ranges of energies corresponding to the typical single-particle energies of processes at the LHC. The two particles are emitted very close to one another to simulate the most difficult scenario where the charged and neutral pions overlap. The electromagnetic showers of the photons from the subsequent decay of the neutral pion and the hadronic shower of the charged pion are simulated using GEANT 4 et al (2016). Electronic noise in the calorimeter is also taken into account.

Image-based deep learning techniques have drawn significant attention in high energy Physics experiments. In particular, the study of calorimeter shower and hadronic jet tagging using a convolutional neural network has been studied in detail in Ref.

Cogan et al. (2015); de Oliveira et al. (2016); De Oliveira et al. (2020); Belayneh and others (arXiv : 1912.06794). For the sake of simplicity, these approaches assume uniform two or three dimensional (2D or 3D) images. The more intricate case of varying granularity layered images has been recently addressed using graph neural networks in Ref. Qasim et al. (2019). In this paper, while building deep learning-based PFlow algorithm, cPFlow, we also propose a different approach to address the issue of variable granularity.

The detector description is discussed in section 2, followed by a discussion of the simulated data in section 3. The details on the structure of the a deep neural network, the cPFlow model, is described in section 4. The description and implementation of a parametrized PFlow algorithm (pPFlow) is given in section 5. The performance of the
pPFlow and the cPFlow are compared in section 6.

2 Detector Model and Simulation

The experimental setup is composed of three sub-detectors. The electromagnetic calorimeter (ECAL) and hadronic
calorimeter (HCAL) which are fully simulated, and a parameterized simulation of a charged particle tracker. Each calorimetric sub-detector consists of three layers, with varying granularity, placed subsequently one after another. The overall detector layout is schematically represented in Figure 1. Both, the ECAL and the HCAL are sampling calorimeters where for simplicity and to produce a realistic shower development each layer is modeled as a homogeneous calorimeter using an equivalent molecule corresponding to a mixture of the absorber and scintillator materials.

The simulated energy deposits in each layer are then smeared to reproduce the corresponding sampling energy resolution. For the ECAL, the absorber material is lead and the active material is liquid argon, mixed with a mass proportion of , yielding an equivalent radiation length of . For the HCAL, the absorber material used is iron with polyvinyl toluene plastic material used as a scintillating active material. These are combined with a mass proportion of 4.7:1.0 yielding an equivalent interaction length of . The choices of the detector geometry, material and smearing parameters are tuned to replicate single-particle energy responses similar to the one obtained with the ATLAS detector.

Figure 1: Illustration of the detector design. Each layer is separated from one another to illustrate each layers’ granularity, while in the simulation all layers are adjacent. The source of the charged and neutral pions is placed at 150 cm from the calorimeters, facing the first ECAL detector layer.

Overall the total length of the detector is 227.5 , including a gap between the ECAL and HCAL blocks. The lateral profile of the calorimeters are squares of  . The main characteristics of the calorimeters are summarized in Table 1.

Detector Absorber Scintillator Subdetector (Legth)
ECAL Lead Liquid Argon ECAL1 (   3 )
ECAL2 ( 16 )
ECAL3 (   6 )
HCAL Iron Plastic organic HCAL1 ( 1.5 )
HCAL2 ( 4.1 )
HCAL3 ( 1.8 )
Table 1: Material budget of the ECAL and HCAL calorimeters as well as the corresponding equivalent radiation and interaction lengths.

The transverse granularities of each layer are indicated in Table 2.

3 Simulated data

To ensure significant overlap between the charged and neutral hadron, the polar angle of the / momenta varies randomly between to radians, whereas the azimuthal angle varies uniformly between and radians with a relative separation of radians. The and are generated using the GEANT particle gun functionality. The source of the gun is located away w.r.t. the first ECAL layer (Figure 1). To populate different parts of the detector, the initial location of the neutral and charged hadron in the event is randomly chosen from the corner of a square at the source location with a length size equal to . Four sets of independent simulations are run with different energy ranges of: GeV, GeV, GeV and

GeV. The energy of the generated charged and neutral pions are randomly sampled from a uniform distribution bounded within these ranges, without any correlation among the pions energies. The generation parameters of the particle gun ensure that a large proportion of detector cells have a significant amount of energy overlap, originating from the individual showers. The average fraction of neutral energy within groups of clustered cells, referred to as topoclusters (see Section


), is around 60%. The effect of electronic noise is emulated using gaussian distributions centered at zero with variables widths for different layers. The per cell levels of noise in each layer are given in Table

2. For each cell in the event an energy is sampled from these distributions and added to the total energy.

Finally, a track is formed by smearing the momentum by a resolution , given by , and keeping the original momentum direction unchanged. The chosen momentum resolution of emulates the track resolution of the ATLAS tracking system and track reconstruction algorithms Collaboration (2008). The smearing of the track direction is neglected as it is expected to have sub-dominant effects to the results presented in this document.

Detector Layer Resolution Noise ()
ECAL1 13
ECAL2 34
ECAL3 17
HCAL1 14
HCAL3 14
Table 2: Transverse segmentations of the ECAL and HCAL individual layers and the corresponding simulated electronic noise per cell.

4 Deep neural network models

The target of the Neural Network is to regress the per-cell neutral energy fraction using deep learning methods to yield an accurate image of the neutral energy deposits. The chosen approach is to apply computer vision techniques to perform this task.

Calorimeter images from the total energy deposit in the calorimeter cells in each layer are formed. The layers serve as image channels, (similar to RGB in regular images), each, with a different resolution due to varying granularity of the calorimeter layers. To perform an image recognition analysis, each of the image channels (with a granularity given in Table 2) is first mapped to a uniform resolution of pixels. This mapping is performed by an individual NN block referred to as "UpConv" block. The track is represented by the track layer, a single channel, pixel image. The image, therefore, has only one non-zero pixel which contains the value of the track energy. The location of this pixel in the image is determined from the transverse position where the hits the first ECAL layer surface. The track information is crucial in the design and performance of a particle flow algorithm as it sets the scale of the expected charged energy deposits as well as the expected position of the shower.

The track layer is combined with the six Up-Convoluted calorimeter image channels to form a seven-layer image that is fed to a neural network block consisting of convolutional layers, termed "ConvNet" blocks. The output of the convnet block is a six-channel image with a uniform resolution of pixels. Each layer is then mapped to a lower resolution image, of single-channel, through a down convolutional learnable neural network block, called DownConv block. More details about UpConv and DownConv blocks can be found in A. The cPFlow output images of the six calorimeter layers preserve the same resolutions of the input images as quoted in Table 2. The overall flow of the model is described in Figure 2.

Figure 2: An illustration of the cPFlow model. The model takes a seven layer image (six calorimeter layers with varying granularity and one track image) as input and regresses the neutral energy fraction per cell of the calorimeter. Each calorimeter layer image is upscaled through an UpConv block. One such example block is described in Equation 1.

The loss function is designed to regress the neutral energy fraction of each cell in the event, with a larger weight assigned to more energetic cells, to reduce the effect of noise and simultaneously enrich the performance of the cells originating from the pions. The loss function is defined on an event-basis as follows:


where is the total energy collected by the six calorimeter layers, is the total energy of a given cell indexed by , and represent the target and predicted energy fractions.

The cPFlow model is trained with the Adam stochastic optimizer Kingma and Ba (arXiv : 1412.6980) and a learning rate of . The training dataset consists of 80,000 images while the validation dataset has 20,000 images. The performance of the trained model is evaluated on a test dataset consisting of 6000 images. The relative difference between the predicted neutral energy and truth neutral energy in the test sample serves as the baseline metric.

5 Implementation of a parametric Particle Flow algorithm

To quantify the performance of the designed cPFlow computer vision algorithm, a traditional pPFlow algorithm is also implemented.

The algorithm is divided into two separate steps: (i) the topocluster formation and (ii) the expected charged energy subtraction. The implementation of both steps is inspired by the PFlow algorithm currently used by the ATLAS experiment ATLAS Collaboration (2017).

The Topological clustering algorithm groups cells based on their energies and topological location. The algorithm is designed to cluster cells originated by a single neutral or charged particle as well as to suppress noise. The algorithm starts by ranking cells based on their significance over the corresponding nominal noise values. Cells with a significance larger than five are considered as seeds and a topological search is performed on their adjacent cells in the longitudinal direction and on their adjacent and next-to adjacent cells in the transverse plane. If one of the clustered cells has a significance 2 , an additional clustering step is performed. If a seed cell is found to be adjacent, within two cells to another topocluster, the two topoclusters are merged. The closest topocluster to the extrapolated track position is considered to be matched to the track.

The expected charged energy is estimated using a parametrisation of the energy deposited by a within the matched topocluster, referred to as . This parametrisation is computed from template distributions obtained using a pure sample of without contamination of and it is dependent on the track momentum and the estimated Layer of First Interaction (LFI, where the first nuclear interaction takes place). The track position is extrapolated to the calorimeter layers and rings of radius equal to a single cell pitch are built. The rings are then ordered according to their expected energy density. The ring energies are subsequently progressively subtracted from the topocluster in decreasing order of energy density. The algorithm proceeds until the total amount of removed energy exceeds . If the energy in the ring is larger than the required energy to reach , the energy in that ring is not fully subtracted but scaled to the fraction needed to reach the expected energy released by the . The remaining energy in the topolcluster is considered as originating from neutral particles.

6 Results

An event display showing the truth energies and the predictions of the topocluster based pPFlow and cPFlow algorithms is shown in Figure 3. It illustrates how the cPFlow produces a more accurate image of the neutral energy deposits.

Figure 3: An event display showing the individual components of charged and neutral energy per layer along with the noise distribution for the first four calorimeter layers. The the extrapolated track position is shown by the red cross for the first three columns. The energy of the initial ranges between 2-5 . The total energy also includes noise contribution and topoclusters boundaries are outlined in white. The fourth and fifth columns shows the predicted neutral energy distribution from for the cPFlow and pPFlow algorithm, respectively. In this example, it is seen that the electromagnetic shower from decay has been correctly predicted by the trained model. The trained model also learns to suppress the noise pattern.

To quantify the overall performance of the cPFlow versus the pPFlow algorithm, two figures of merit related to the reconstruction of the neutral energy deposits are considered. The first is the energy resolution of the residual neutral energy after the charged energy subtraction. The second is the overall direction assigned to the charged energy deposit as a probe of the topological reconstruction of the neutral energy deposits.

Figure 4 shows the relative residual distribution for the cPFlow and the pPFlow algorithm. Both, the predicted and the truth neutral energies used in the computation of the residuals are computed as the sum of cells energy belonging to the topocluster matched to the track. The distributions are fitted with a sum of two Gaussian distributions, to quantify the main gaussian resolution in energy and the non-gaussian tails. The estimated energy resolution of the cPFlow shows an improvement by more than a factor of 3 compared to the pPflow in the lowest energy range corresponding to 2-5 . The relative improvement progressively decreases to reach approximately 40% in the highest energy range of 15-20 .

For both algorithms, a bias in the fitted mean is observed. The bias in the pPFlow algorithm, ranges from 20% in the low energy range, to 5 % in the highest energy range. It was found to originate from the parametrization of the charged energy predictions which is derived from a pure sample of . The size and energy of the topocluster in isolated are systematically lower compared to the evaluation sample which features a large and overlap. As the energy of the increases, the size of the topoclusters in both, isolated and overlapped topologies becomes comparable thus reducing the bias in the high energy ranges.

For the cPFlow results, the bias is much smaller and approximately constant at negative values of 5-6%. As the pions energies increase, the bias is only slightly reduced. This effect was found to originate from the training strategy of the cPFlow which is carried out without prior knowledge of the topocluster boundaries. It therefore predicts, as expected, a small fraction of the neutral energy outside the topocluster, deposited in cells with energy typically comparable to the injected noise values, resulting in a slight underestimate within the topocluster boundaries. To mitigate this effect, dedicated training is performed only for cells belonging to the topoclusters. Figure 5 shows the relative residual distribution of the cPFlow when the training is run over the full image and for training using only cells belonging to the topoclusters. For the latter, the vast majority of low energetic cells are effectively cut out by the clustering algorithm thus reducing the noise contribution during the training. Similarly to the previous case, the distributions were fit with a sum of two gaussian distributions. An overall shift of the distributions towards a lower bias value is seen when the training is run only for the cells belonging to the topoclusters, especially for the lowest energy range. The non-gaussian tails of the distributions are also slightly reduced when training over a cell of the topoclusters by approximately 10% for the lowest energy range. These features are expected given that the loss function was chosen to favor high energetic cells and thus predictions for the low energetic cells are expected to be less accurate by construction. It can also be noted, that the cPFlow predictions provide an excellent noise suppression per-cell, which is further improved when the great majority of low energetic cells are removed during training using topocluster cells only.

Figure 4: The relative residuals distributions of the pPflow and cPFlow algorithms for different energy ranges described in the text. The distributions are fit with a sum of two gaussians to catch non-gaussian tails. The values of the central gaussians are shown in the plots.
Figure 5: Relative energy residuals distribution of the cPFlow predictions when the training is performed over the full calorimeter image (red) and when only cells belonging to the topoclusters are used (green). The distributions are fit with a sum of two gaussians to catch non-gaussian tails. The values of the central gaussians are shown in the plots.

The spatial resolution of the neutral component was also studied. Figure 6 shows the distance, in number of cells, between the barycenter of the truth neutral energy and the predicted neutral energy within the topocluster for both the the pPFlow algorithm and the cPFlow predictions in the ECAL2 layer. The cPFlow predictions provide an accurate estimate of the barycenter, outperforming the pPflow results for all the considered energy ranges. This is because the pPFlow energy subtraction is performed within rings around the extrapolated track position and therefore it gives a very approximate estimate of the precise topology of the neutral energy deposit, in contrast with the image recognition algorithm. Improvements of a factor of 5 or more in the spatial resolution are observed for the cPFlow versus the pPFlow algorithm in all energy ranges.

Figure 6: The distance computed in number of cells between the barycenter of the predicted and truth neutral energy in the ECAL2 layer when using the pPflow or cPFlow algorithms for different energy ranges described in the text. The equivalent distributions in different calorimeter layers show a similar behaviour.

7 Conclusion and outlook

Particle Flow reconstruction has an important role in high energy particle collider experiments and is being considered for the design of future experiments. In this paper a Computer Vision approach to particle flow reconstruction, based on deep Neural Network techniques is proposed. Its performance is quantified using a simplified layout of electromagnetic and hadron calorimeters, focussing on the challenging case of overlapping showers of charged and neutral hadrons. For charged hadrons with low to intermediate momenta, where the momentum resolution from the measurement of its track in an inner tracking detector is more precise than the energy measurements in the calorimeters, the role of the particle flow algorithm is to optimally infer the expected charged hadron energy deposits to then subtract it from the calorimeter and replace it with the measured track, which will not only be superior in energy resolution, but also in the reconstruction of its direction. The particle flow algorithm, therefore, aims at the most precise reconstruction of the neutral energy in the calorimeters. While standard particle flow algorithms use parametrizations of the expected charged energy deposits in the calorimeters (pPFlow), the cPFlow algorithm is trained to learn the image of the neutral energy deposits.

Comparing the cPFlow and pPFlow estimates of the neutral energy deposits in the calorimeter, an improvement of the resolution of the neutral energy response of approximately a factor of two is obtained. The precision in the neutral object direction reconstruction is also improved significantly by a factor of approximately 5 or more.

Combining the cPFlow algorithm with standard topological calorimeter clustering algorithms provides both an optimal performance in the challenging case of overlapping showers, and a simple path to generalize the algorithm from the two-particle case to a global event particle flow.

These studies provide a proof of concept for the construction of a computer vision-based PFlow algorithm, cPFlow, which, via the judicial implementation of UpConvolution and DownConvolution layers, successfully estimates the neutral energy in the event using track and calorimeter images of varying granularity as input.

8 Acknowledgements

SG, EG and JS were supported by the NSF-BSF Grant
2017600 and the ISF Grant 2871/19.


  • [1] M. Ahmad et al. (2015) CEPC-SPPC Preliminary Conceptual Design Report. 1 Physics and Detector. IHEP-CEPC-DR-2015-01. Cited by: §1.
  • [2] ATLAS Collaboration (2017) Jet reconstruction and performance using particle flow with the ATLAS Detector. Eur. Phys. J. C77 (7), pp. 466. External Links: Document, 1703.10485 Cited by: §1, §1, §5.
  • [3] H. J. Behrend et al. (1982) An Analysis of the Charged and Neutral Energy Flow in Hadronic Annihilation at 34-GeV, and a Determination of the QCD Effective Coupling Constant. Phys. Lett. 113B, pp. 427–432. External Links: Document Cited by: §1.
  • [4] D. Belayneh et al. (arXiv : 1912.06794) Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics. External Links: 1912.06794 Cited by: §1.
  • [5] M. Bicer et al. (2014) First Look at the Physics Case of TLEP. JHEP 01, pp. 164. External Links: Document, 1308.6176 Cited by: §1.
  • [6] J. C. Brient (2004) Particle flow and detector geometry. In Linear colliders. Proceedings, International Conference, LCWS 2004, Paris, France, April 19-23, 2004, pp. 995–998. Cited by: §1.
  • [7] D. Buskulic et al. (1995) Performance of the ALEPH detector at LEP. Nucl. Instrum. Meth. A360, pp. 481–506. External Links: Document Cited by: §1.
  • [8] CMS Collaboration (2017) Particle-flow reconstruction and global event description with the CMS detector. JINST 12 (10), pp. P10003. External Links: Document, 1706.04965 Cited by: §1.
  • [9] J. Cogan, M. Kagan, E. Strauss, and A. Schwarztman (2015) Jet-Images: Computer Vision Inspired Techniques for Jet Tagging. JHEP 02, pp. 118. External Links: Document, 1407.5675 Cited by: §1.
  • [10] T. A. Collaboration (2008-08) The ATLAS experiment at the CERN large hadron collider. Journal of Instrumentation 3 (08), pp. S08003–S08003. External Links: Document, Link Cited by: §1, §3.
  • [11] L. de Oliveira, M. Kagan, L. Mackey, B. Nachman, and A. Schwartzman (2016) Jet-images — deep learning edition. JHEP 07, pp. 069. External Links: Document, 1511.05190 Cited by: §1.
  • [12] L. De Oliveira, B. Nachman, and M. Paganini (2020) Electromagnetic Showers Beyond Shower Shapes. Nucl. Instrum. Meth. A951, pp. 162879. External Links: Document, 1806.05667 Cited by: §1.
  • [13] J. A. et al (2016) Recent developments in geant4. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 835, pp. 186 – 225. External Links: ISSN 0168-9002, Document, Link Cited by: §1.
  • [14] D. P. Kingma and J. Ba (arXiv : 1412.6980) Adam: a method for stochastic optimization. External Links: 1412.6980 Cited by: §4.
  • [15] S. R. Qasim, J. Kieseler, Y. Iiyama, and M. Pierini (2019) Learning representations of irregular particle-detector geometry with distance-weighted graph networks. Eur. Phys. J. C79 (7), pp. 608. External Links: Document, 1902.07987 Cited by: §1.
  • [16] M. Ruan and H. Videau (2013) Arbor, a new approach of the Particle Flow Algorithm. In Proceedings, International Conference on Calorimetry for the High Energy Frontier (CHEF 2013): Paris, France, April 22-25, 2013, pp. 316–324. External Links: 1403.4784 Cited by: §1.
  • [17] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang (arXiv : 1609.05158)

    Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

    External Links: 1609.05158 Cited by: Appendix A.
  • [18] M. A. Thomson (2009) Particle Flow Calorimetry and the PandoraPFA Algorithm. Nucl. Instrum. Meth. A611, pp. 25–40. External Links: Document, 0907.3577 Cited by: §1.
  • [19] B. Xu, N. Wang, T. Chen, and M. Li (arXiv : 1505.00853) Empirical evaluation of rectified activations in convolutional network. External Links: 1505.00853 Cited by: Appendix A.

Appendix A Up and Down Convolution description

The UpConv block is built out of two-dimensional convolutional layers (Conv2D) with

kernel size, followed by batch-normalization (BN) layers and Leaky ReLU activation function

[19], with slope parameter . A single UpConv block consists of five sequences of Conv2D, BN and Leaky ReLU activation function followed by pixel shuffle upsampling [17] operation. The pixel shuffle process converts a multi-channel low-resolution image to a higher resolution image with a lower number of channels such that the total number of pixels in both the images are equal. One such NN block is presented in Equation 1. The in the same equation represents a pixel shuffle layer with upscale factor .

The DownConv blocks are made out of Conv2D layers whose kernel size and stride depend on the down-scaling factor. For example if we want to reduce the resolution of an image from size

to , i.e. a reduction by a scale factor 4, we use a layer Conv2D[1, 1, K=(4,4), stride=(4,4)].