Machine learning (ML), especially deep learning is gaining greater attention due to its success in various cognitive applications. Recently, machine learning approaches have been applied for electromagnetic simulation and modeling, which has shown great promise . Deep neural networks can be trained to build accurate models for nonlinear electromagnetic responses with different design parameters based on the simulated and experimental data. However, such data-driven models based on deep neural networks or other machine learning methods may suffer the violation of some important physical properties, as they may not be able to provide a physical insight into the prediction of electromagnetic parameters. For instance, trained model based on the given data may fail to consider physics constraints, such as initial, boundary, and passive conditions. Therefore, it is important to address these issues for learning-based electromagnetic modeling to improve the physical prediction ability .
Recently, artificial neural networks (ANN) have been used to predict the S-parameter of high-speed interconnects based on an open SI/PI-Database [1, 3]. However, the data-driven neural networks  can lead to negative insertion loss, which violates the passive characteristics that were demonstrated in this paper. Insertion loss is a positive number that represents signal loss from the input power to the output power . Therefore, positive quantity is a physics constraint of insertion loss for the passive high-speed interconnects.
Based on this observation, in this paper, we try to mitigate this problem by exploring advanced deep learning strategies. We propose two methods to enforce a positive quantity of insertion loss for ML-based parameterized transmission line models. In the first method, we propose to add a regularization item, which represents physics constraints of insertion loss, into the existing loss function to enforce positive quantity. This method is similar to the idea of physics-informed neural network concept (PINNs) . But here we just enforce the one specific physic law or constraint. In the second method, a third-order polynomial expression is applied to approximate the insertion loss first. The polynomial expression is designed so that it can ensure the positiveness constraints on the insertion loss. Then, the neural network is applied to predict the coefficients of the polynomial expression instead, which shares a similar structure to recently proposed DeepONet structure for general function or dynamic system approximation . The experimental results on an open-sourced SI/PI database of a PCB design show that both methods can ensure the positiveness for the insertion loss. Furthermore, the physics-enforced DNN method and polynomial-based DeepONet method can achieve similar accuracy in both training and inferences, while the polynomial-based DeepONet method is faster than DeepONet-based method in training time.
Ii Background and Motivation
It is well known that the insertion loss () is defined as 
where represents the transmission S-parameter of high-speed interconnects from port 1 to port 2. We perform our study on an open SI/PI dataset, which has transmission lines on 11 cavity PCB with two 10x10 via-arrays . Fig. 1 shows the top view of the first signal layer on the PCB. There are six striplines with 12 ports, which provide a high-speed link between two 10x10 via-arrays. The side view of the PCB is shown in Fig. 1. The PCB consists of dielectric, vias, power planes, ground planes and signal layers. A cavity is formed between two adjacent power and ground planes. Fig. 1 shows the insertion loss of the six interconnects on the PCB with specific design parameters.
We then try to develop the parameterized models of the insertion loss with respect to some design parameters. One straightforward solution is to apply fully connected neural networks. For instance, we can model the insertion loss of the interconnect from port 1 to port 7. The inputs and outputs of the neural networks are listed in Table I.
The dataset with 7030 samples is randomly split into a training set with 5624 samples (80%) and test set with 1406 samples (20%). To train the neural networks easily, we rescale all input and output features to [-1, 1] using the min-max normalization method. The predictions on the test set are shown in Fig. 2. The mean-square-error (MSE) is 0.0145.
Fig. 2 indicates that there are lots of samples which have negative insertion loss predicted by neural networks (NN). However, the results from ground truth are greater than 0. Therefore, NN fails to consider the positive quantity of insertion loss. The violation usually occurs at low frequency, as shown in Fig. 2. This issue should be addressed since it violates the passive characteristics.
Iii Physics-enforced Modeling of Insertion Loss
To enforce positiveness of insertion loss, we propose to use two ML-based methods on the open SI/PI-Database.
Iii-a Physics-enforced DNN
In the physics informed neural networks, the initial conditions, boundary conditions and governing equations are considered by adding several loss functions. Based on the idea, an additional loss function representing the positive quantity of insertion loss is added to the total loss function of neural networks, which is expressed as
where is the mean-squared-error (MSE), which is represented by
where and are the prediction and ground truth of the insertion loss, respectively. is the number of sample points. is a penalty coefficient. is the physics constraints of insertion loss, which is given by
which represents that insertion loss is not less than 0. The auxiliary information can guide neural networks to train the model so that it satisfies the physics constraints.
Iii-B Polynomial-based DeepONet
The neural networks can be capable of fitting any functions and it can easily find several solutions of the problem. Some solutions may not meet the physics constraints. To avoid this situation, we can first define the form of the solution. In this method, we use a third-order polynomial expression to approximate the insertion loss , which is written as
where , , and are the coefficients, and is the frequency. Then, we apply the neural networks to find the coefficients of the expression. The new architecture of neural networks is shown in Fig. 3.
The input and output of neural networks are dimensional parameters and coefficients, respectively. Power series module generates frequency to the power of 1, 2, and 3. After that, the insertion loss is obtained by the multiplication of the frequency power series and corresponding coefficients. The new networks in Fig. 3 can be represented by
where NN is the th variable of the neural networks’ outputs. Note in our implementation, we first use the fitting method in Matlab to obtain the coefficients , , and , which becomes functions of the input design parameters. Then, we train the neural networks for the three functions , , and .
Iv Experimental Results
In this section, we demonstrate the two proposed methods to enforce the positiveness of insertion loss on the same test set, which is illustrated in Section II. Fig. 4 shows the results predicted by Physics-enforced DNN, called PDNN. The minimum values of predictions is almost the same as ground truth. There is no negative value in the predictions and thus this method satisfies the physics constraints of insertion loss. Fig. 4 shows that the predicted insertion loss by the polynomial-based DeepONet (PDeepONet) method for different samples. As we can see, no negative values are predicted. This is because the defined polynomial expression gives the constraint on insertion loss. In summary, the two ML-based methods ensure the positiveness of insertion loss on test set.
To further compare the two methods, we test them on one sample in test cases. The results are shown in Fig. 5. The negative insertion loss usually occurs at low frequency for the NN method. It can be observed that the proposed two methods can fix the problem.
Although the two methods can predict positive insertion loss, their performances are different, which are illustrated in Table II. The training and test mean squared errors (MSEs) of PDNN are 0.015 and 0.0145, respectively, which are the same as those of NN. The errors of PDeepONet are slightly larger than those of NN and PDNN. The inference of PDeepONet costs the maximum time even though its training time is less than those of NN and PDNN. Because PDeepONet needs to consider the computation time in Matlab. It can be seen from Table II that time for PDeepONet consists of two parts which are time in neural networks and time in Matlab. The PDNN spends more training time than NN since it takes extra time to satisfy the physics constraints. One caveat we want to point out is that different implementations of the two methods may change the time differences in both training and inferences and the observed trend also need to be further investigated for more datasets in the future.
|Training set||Test set|
|MSE||Time (s)||MSE||Time (s)|
In this paper, we have proposed two methods to enforce the positive quantity of insertion loss predicted by machine learning-based methods. The first one is physics-enforced DNN method. We add an additional loss function, which represents the passive condition, to the final loss function to enforce negative quantity of insertion loss. In the second method, a third-order polynomial expression is defined to approximate the insertion loss. Then, the neural network is applied to predict the coefficients of the polynomial expression. The experimental results on an open-sourced SI/PI database of transmission line design showed that both methods can ensure the positiveness for the insertion loss. Furthermore, both methods can achieve similar prediction results, while the polynomial-based DeepONet method is faster in training time.
-  M. Schierholz, A. Sánchez-Masís, A. Carmona-Cruz, X. Duan, K. Roy, C. Yang, R. Rimolo-Donadio, and C. Schuster, “Si/pi-database of pcb-based interconnects for machine learning applications,” IEEE Access, vol. 9, pp. 34 423–34 432, 2021.
M. Raissi, P. Perdikaris, and G. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,”Journal of Computational Physics, vol. 378, pp. 686–707, 2019.
-  K. Scharff, C. M. Schierholz, C. Yang, and C. Schuster, “Ann performance for the prediction of high-speed digital interconnects over multiple pcbs,” in 2020 IEEE 29th Conference on Electrical Performance of Electronic Packaging and Systems (EPEPS), 2020, pp. 1–3.
-  D. M. Pozar, Microwave Engineering, 4th ed. New York, NY, USA: Wiley, 2012.
-  L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, “Learning nonlinear operators via deeponet based on the universal approximation theorem of operators,” Nature Machine Intelligence, vol. 3, no. 3, pp. 218–229, 2021.
-  M. Y. Koledintseva, A. V. Rakov, A. I. Koledintsev, J. L. Drewniak, and S. Hinaga, “Improved experiment-based technique to characterize dielectric properties of printed circuit boards,” IEEE Transactions on Electromagnetic Compatibility, vol. 56, no. 6, pp. 1559–1566, 2014.