The on-going COVID-19 pandemic, especially with the recent rise of the Omicron variants, has resulted in significant negative impacts on the health and well-being of the global population. A major aspect of the COVID-19 clinical workflow is imaging-based assessment, and there has been a significant increase in the use of point-of-care ultrasound (POCUS) imaging alongside chest radiography due to its low cost, low maintenance, disinfection ease, portability, and lack of ionizing radiation. However, a major barrier with widespread adoption of POCUS in the COVID-19 clinical workflow is the scarcity of expert clinicians that can interpret POCUS examinations for COVID-19 assessment . Therefore, there has been a considerable interest in deep learning-driven clinical decision support systems .
Despite the promise, a major challenge to building deep neural networks for COVID-19 screening is the heterogeneity in the types of probes used to capture ultrasound images. For example, POCUS images captured using convex probes have very different visual appearances than images captured using linear probes. Limiting data to a single type increases consistency at the cost of significantly reduced dataset size.
Motivated to tackle this challenge, in this study we explore the impact of leveraging extended linear-convex ultrasound augmentation learning on producing enhanced deep neural networks for COVID-19 assessment. More specifically, we leverage projective transformations as part of the data augmentation pipeline to transform linear probe data to better resemble convex data, as well as increase the diversity of viewing windows to improve generalization. Finally, we will explore its efficacy using an efficient deep columnar anti-aliased convolutional neural network that provides balance between efficiency and representation capacity.
This paper is organized as follows. Section 2 describes the methodology behind the proposed extended linear-convex ultrasound augmentation learning strategy for producing enhanced deep neural networks for COVID-19 assessment, as well as the COVID-Net US-X efficient deep columnar anti-aliased convolutional neural network architecture design for exploring the impact of extended linear-convex ultrasound augmentation learning in a clinical resource-constrained scenario. Section 3 describes the experimental setup used in this study, as well as a discussion of the experimental results studying the impact of the proposed learning strategy. Finally, conclusions are drawn and future work is discussed in Section 4.
In this study, the concept of extended linear-convex ultrasound augmentation learning is premised on the fact that we wish to leverage as much POCUS data possible to mitigate data scarcity challenges, while at the same time accounting for the heterogeneity in the types of probes used to capture POCUS images that can lead to very significant visual appearance differences.
For example, POCUS images captured using a convex probe have a greater field-of-view with a cone shaped viewing window. This window differs in its viewing angle from device to device, and can also have a variety of forms as shown in Figure 1. In contrast, POCUS images captured using a linear probe has a more restricted field-of-view, but have a linear viewing window with visual content appearing distorted compared to that of images captured using a convex probe.
2.1 Extended linear-convex ultrasound augmentation learning
To deal with these various challenges, we leverage random projective augmentations to not only increase the diversity of viewing windows but also to transform linear probe data in such a way that they better resemble the visual appearance of convex probe data. This will allow for augmented learning with both convex probe data and linear probe data in a way that better promotes generalization since the deep neural network is being exposed to a greater diversity of data but also in a form that is more consistent from visual appearance perspective.  applies a similar idea through rectifying convex images to appear more similar to linear ones using its polar coordinates, but this process is unable to be used as an on-the-fly data augmentation process
If the bounds of the viewing window is known, then the POCUS image can be transformed to have various different angles by remapping the corners of the viewing window via a random projective transform. This process is shown in Figure 2. Furthermore, we transform the POCUS images captured using linear probes using random projective transforms so that they bare a much stronger visual resemblance to POCUS images captured using a convex probe to enable the deep neural network to better leverage this additional data source. An example of this POCUS augmentation is shown in Figure 3a and Figure 3b.
2.2 COVID-Net US-X Architecture Design
To explore the impact of extended linear-convex ultrasound augmentation learning in a clinical resource-constrained scenario, we introduce COVID-Net US-X, a convolutional neural network for COVID-19 assessment using POCUS that balances efficiency and representation capacity. The architecture design is a deep columnar antialiased residual architecture with highly customized macroarchitecture and microarchitecture designs that were automatically determined using a machine-driven design exploration strategy called Generative Synthesis  for high computational and representational efficiency. The architecture design has 636M FLOPs and 4.69M parameters.
The advantage of ultrasound imaging systems lies in being a low-cost and low-power solution. As such, using generative synthesis allows us to maintain that power and computational efficiency through automatically striking a balance between detection accuracy and network performance.
The COVID-Net US-X network architecture design features progressively increasing inter-column connectivity as we go deeper into the network. Such variations in inter-column connectivity enables greater efficiency in the lower degrees of abstraction while achieving greater representational complexity at higher degrees of abstraction. Futhermore, the COVID-Net US-X network architecture exhibits antialiasing architectural characteristics, which enables greater representational stability to allow the network to be able to achieve greater robustness and greater generalization , which is especially important for reliable clinical decision support applications.
3 Experiments and Results
To evaluate the impact of the proposed extended linear-convex ultrasound augmentation learning strategy, we leveraged a newer version of the COVIDx-US benchmark dataset  in this study. To evaluate the impact of the proposed extended linear-convex ultrasound augmentation learning strategy, we leveraged a newer version of the COVIDx-US benchmark dataset  in this study. This version of the benchmark dataset consists of 16,649 POCUS images, with 12,260 captured using convex probes and 4389 captured using linear probes, which we leverage fully in this study and is larger and more diverse than what was utilized in . In terms of disease breakdown, 6764 of the POCUS images were from COVID-19 positive patients, 9985 were from COVID-19 negative patients. The train, validation, and testing splits are 72%, 14%, and 14%, respectively.
The models are trained with no additional data augmentation techniques. Projective image transformations are set with a slope sampled from a normal distribution centered around either the original slope for convex images or 2.5 for linear images, and a standard deviation of 0.15.
The test accuracy and area-under-the-curve (AUC) for the tested deep neural network under different forms of data augmentation is shown in Fig. . It can be observed that the proposed extended linear-convex ultrasound augmentation learning strategy significantly increases performance, with a gain of 5.1% in test accuracy and 13.6% in AUC. Therefore, these results clearly illustrate that the proposed augmented learning strategy can have significant benefits in leveraging additional data from different probe types to improve the performance and generalization of deep neural networks for COVID-19 assessment using POCUS images.
In this study, we introduced an extended linear-convex ultrasound augmentation learning strategy for creating enhanced deep neural networks for COVID-19 assessment using POCUS images. More specifically, we performed data augmentation on convex probe data alongside linear probe data randomly transformed in a way that better resembles convex probe data. We conducted experiments using COVID-Net US-X, an efficient deep columnar anti-aliased convolutional neural network designed via a machined-driven design exploration strategy that we introduced here to evaluate the impact of the proposed augmentation learning strategy. Experimental results demonstrated the efficacy of the proposed extended linear-convex ultrasound augmentation learning in significantly increasing performance, with a gain of 5.1% in test accuracy and 13.6% in AUC.
Future work involves investigating the efficacy of the proposed extended linear-convex ultrasound augmentation learning strategy on a larger, more diverse set of POCUS images. Furthermore, we plan on exploring additional ultraound-specific augmentation types as well as different augmentation ranges to determine the optimal augmentation learning policy for building enhanced deep neural networks for COVID-19 assessment via POCUS images.
-  Laura J Brattain, Brian A Telfer, Manish Dhyani, Joseph R Grajo, and Anthony E Samir. Machine learning for medical ultrasound: status, methods, and future opportunities. Abdominal radiology, 43(4):786–799, 2018.
-  Ashkan Ebadi, Pengcheng Xi, Alexander MacLean, Stéphane Tremblay, Sonny Kohli, and Alexander Wong. Covidx-us–an open-access benchmark dataset of ultrasound imaging data for ai-driven covid-19 analytics. arXiv preprint arXiv:2103.10003, 2021.
-  Alexander MacLean, Saad Abbasi, Ashkan Ebadi, Andy Zhao, Maya Pavlova, Hayden Gunraj, Pengcheng Xi, Sonny Kohli, and Alexander Wong. Covid-net us: A tailored, highly efficient, self-attention deep convolutional neural network design for detection of covid-19 patient cases from point-of-care ultrasound imaging. In Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health, pages 191–202. Springer, 2021.
-  Alexander Wong, Mohammad Javad Shafiee, Brendan Chwyl, and Francis Li. Ferminets: Learning generative machines to generate efficient neural networks via generative synthesis. arXiv preprint arXiv:1809.05989, 2018.
-  Daniel Yaron, Daphna Keidar, Elisha Goldstein, Yair Shachar, Ayelet Blass, Oz Frank, Nir Schipper, Nogah Shabshin, Ahuva Grubstein, Dror Suhami, et al. Point of care image analysis for covid-19. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8153–8157. IEEE, 2021.
-  Richard Zhang. Making convolutional networks shift-invariant again. In International conference on machine learning, pages 7324–7334. PMLR, 2019.