## I Introduction

Data centers centralize compute and storage requirements of an enterprise or a service provider. A task in a data center is typically performed in a distributed manner using a set of compute-and-storage (CSN) nodes. The interconnection network, called the data center network (DCN), that connects these nodes has a significant impact on task completion times. Further, low power consumption and high scalability (in terms of number of CSN nodes) are crucial system design requirements.

Several network architectures have been proposed [2, 3, 4] to satisfy high-performance, high-scalability and low power objectives. A formal approach to design a high-performance and low power data center network was proposed in [1]. This was modelled as a constraint optimization problem (CoP). This used a combinatorial approach and explored all possible component sequences to identify the best possible sequence. It involved evaluation of several thousand component sequences for feasibility. It is not feasible to study the individual sequences using detailed simulations or using experiments. Thus, owing to large volume of inputs, the evaluation must be largely theoretical.

The combinatorial solver uses a library of components to create the component sequences [5]. These components include power splitters, combiners, wavelength multiplexers, demultiplexers, reconfigurable add-drop multiplexers, wavelength routers, optical amplifiers and transceivers. When dynamically constructing a component sequence from these components, it must meet the stringent optical domain requirements.

One of the critical decisions to be made by the combinatorial solver is to decide whether the chosen network can operate with tolerable bit error rates (BER). This decision is known as BER satisfiability decision (BSD). BSD must be made for all networks that are be created by combining a set of components. A typical data center network is a (relatively) short distance multi-fiber network. Though theoretical models are available for long distance single-fiber spans with amplifiers, the specific class of short distance multi-fiber networks are not experimented widely to the best of our knowledge. Thus, associated theoretical models are not readily available.

This paper attempts to address this gap by conducting experiments and thereby derives an empirical model. A small set of networks are experimentally created. Optical power is measured for these networks at all points and the received signal is recorded. The BER is computed by analyzing this recorded signal. The decision tree algorithm, which is popular in analytics, is used to arrive at BSD with optical power levels as its input. Networks can be designed using passive optical components alone or using passive components along with an amplifier. Both these network designs are experimentally studied in this paper.

From the analysis it is observed that the optical power at the receiver and before the amplifier influence the BSD decision. Interestingly, these factors are also part of the single-fiber long distance model. Finally, BSD can be made by considering two optical power level thresholds.

## Ii Solver description

Current data center networks must satisfy many requirements simultaneously. These requirements include power consumption, throughput and latency. Researchers have proposed data center networks that satisfy one requirement at a time. If a proposal outperforms others on latency, it is often outperformed on another requirement. Thus, it is difficult to build a one-size-fits-all network that satisfies a wide range of requirements. Hence, custom-designed networks that satisfy the given set of requirements are needed.

When these requirements are encoded as constraints, a custom-built constraint optimization problem (CoP) solver [5] finds the best possible solution. This solver explores a large N-dimensional search space for solutions that satisfy the constraints. Then, based on the objective the best solution is identified. A solution identified by the solver is a component sequence. This sequence is built from a library of discrete optical components and their characteristics. The solver attempts to find the optimal sequence made up of these optical components. Many component sequences must be explored to find the best possible component sequence. Being combinatorial, several thousand component sequences must be explored to find the best possible component sequence for large networks.

Evaluating these component sequences is a tough task. There are three approaches widely used for evaluation namely: theoretical, simulation based and experimental evaluation. Solver adopts theoretical evaluation. This approach is well suited for exploring large search spaces in minimum time. However, a suitable theoretical model must be available to the CoP solver [5] to take critical decisions.

The DCN is a relatively short distance network that typically spans a few Km at most. Thus, the corresponding component sequence is also a short-distance network. Optical domain characteristics must be modelled for this network and this must be encoded as constraints. This ensures that the optimal sequence identified satisfies critical optical domain constraints such as bit-error-rate (BER) requirements. The library of components contains one-to-many (e.g. splitters) and many-to-one (e.g. combiners). These components work with multiple fibers. While, long distance single-fiber networks are studied widely including in [6], a similar model for short-distance multi-fiber network is not readily available. This paper explores this aspect and attempts to model this specific class of networks.

## Iii Experiment description

The objective of the experiment is to empirically decide on whether a network passes the BSD test or not. In other words, networks that have a BER of less than are accepted and the rest are rejected by the BSD test. A set of experimental networks were created using Lightrunner kit [7] and their BER was computed. The signal was modulated with data at 2.5 MHz.

The power level before and after every component was measured. This was measured in addition to measuring the power levels at the transmitter and the receiver. The difference in power levels before and after a passive component is equal to the loss inserted by the component. For an amplifier, the difference in power levels is the amplifier gain.

Different components were used to create a component sequence. It includes a power splitter, a power combiner, a wavelength multiplexer, a wavelength demultiplexer and an Erbium doped fiber amplifier (EDFA) amplifier.

### Iii-a BER estimation

The signal received for every network is recorded in persistence mode. This signal is then fed to MATLAB curve fitting tool to fit the raw signal data to a double Gaussian curve. Let and

be the estimated mean for bit one and bit zero. Similarly, let

andbe the estimated standard deviation for bit one and bit zero. The Q factor of the signal is given by

. Then, the corresponding BER is given by .## Iv Experimental Data Analytics

A decision tree algorithm, available in the R [8] package, was used for data analytics. The inputs for this algorithm are the measured values and the output is the classification of the network based on its BER.

Using optical components, two types of DCN designs are possible. The first one uses passive optical components between a pair of transceivers, as shown in Fig. 1(a). The second one additionally also an amplifier Fig. 1(b). Both these scenarios are experimentally studied. All experimental networks have a transmitter and receiver on either sides. One or more black-boxes are presented in the schematic diagram. During the study, these black-boxes are replaced by actual optical components.

### Iv-a Without amplifier

A set of scenarios were created without an amplifier between the transmitter and the receiver. The corresponding schematic is presented in Fig. 1(a). In this case, two wavelengths (1510 and 1550 nm) were launched and measured separately. Signal power was recorded at all points as described before. The received signal was recorded. This was used to estimate the corresponding BER.

S.No | BB | in nm | Rx Power (in dBm) | BER |
---|---|---|---|---|

1 | SS | 1510 | -16.25 | 4.08E-21 |

1550 | -16.65 | 1.9E-08 | ||

2 | MM | 1510 | -11.45 | 1.22E-21 |

1550 | -10.55 | 2.24E-15 | ||

3 | SMMS | 1510 | -17.15 | 2.43E-06 |

1550 | -18.95 | 3.57E-09 | ||

4 | SMSSMS | 1510 | -28.15 | 4.20E-04 |

1550 | -25.65 | 1.48E-05 | ||

5 | SMSMMSMS | 1510 | -29.65 | 4.23E-03 |

1550 | -32.25 | 1.81E-03 |

Table I presents the scenarios that were experimented. The contents of the black box in Fig. 1 (a) is presented in the second column. Subsequent columns present the transmission wavelength used for the experiment, received power measured and estimated BER values respectively. Every row presents an experimental scenario. Though many other power values were recorded, they are not shown here for brevity. For instance, the black box of the first row contains SS. It has two optical splitter or combiners connected back-to-back. When 1510 nm is transmitted from the left side of the black-box, the received power on the right side is observed to be -16.25 dBm. The signal received is recorded and BER is estimated. The corresponding BER is . This error rate is less than the tolerable BER. Considering third row of the table, the black box contains an optical combiner, a wavelength multiplexer, a wavelength demultiplexer and then an optical splitter from the left to the right side. When 1510 nm is launched at the transmitter, the received power is observed to be -17.15 dBm and the corresponding estimated BER is . The error rate is more than the tolerable BER and hence, given its received power, this black box is not a valid combination. Similarly, other entries in the table are examined. Examining all the entries in the table, it is observed that when the received power is at least -16.25 dBm, the corresponding BER is less than tolerable BER.

### Iv-B With amplifier

These networks have an amplifier sandwiched between two black-boxes. The other end of the left and right black-boxes is connected to a transmitter and a receiver respectively as shown in Fig. 1(b).

In the experimental network, the black-boxes are replaced with one or more passive optical components. The left black box cannot be empty but the right black-box can be empty. When this black-box is empty, the output of the amplifier is directly connected to the receiver.

Initially power levels observed at all stages and the corresponding BER were subjected to visual correlation. Then a subset of power levels that had good correlation were fed as input to the decision-tree algorithm. This decision tree is shown in Fig. 2. From this decision tree it can be observed that BSD can be made by considering whether power level before amplification is less than -26.38 dBm. This empirical model or the decision tree can be directly embedded into the combinatorial solver to evaluate other component interconnections.

Data pertaining to different experimental scenarios is presented in Table II. The table presents the power level before amplification and the calculated BER for every scenario along with the composition of its left and right black boxes. An empty right black box can be seen in some scenarios. This indicates that the amplifier’s output was connected directly to the receiver’s input using a fiber.

Let us consider row 4 from Table II. In this scenario, two wavelength demultiplexers (denoted by MM)were connected between the transmitter and the amplifier’s input. On the right side an optical splitter (denoted by S) was connected between the amplifier’s output and receiver. In this case, when launch power was -1.63 dB (not shown in the table), the power before amplification is -3.97 dB. In this case, estimated BER at the receiver was . This BER value is good compared to the tolerable BER value of . However, when the launch power was reduced to -22.2 dB for the same setup (row 5), the power before amplification was observed as -26.62 dB. The corresponding estimated BER was and this does not meet the tolerable BER. Thus, it can be observed that when the power level before amplification is below a certain threshold level, signal at the receiver does not meet the tolerable BER. It can also be observed that launch power also has a small but significant role in the network performance.

To illustrate this, let us consider row 16. It has an optical splitter followed by a wavelength multiplexer (denoted by SM) between the transmitter and the amplifier’s input. On the other side between the amplifier’s output and the receiver, it had a wavelength demultiplexer (denoted by M). In this scenario, the launch power was -19.28 dBm. The observed power level before amplification is -26.64 dBm. This is less than the power level observed with row 5. However, the estimated BER is . This network is able to achieve a good BER.

It can be seen that when the power before amplification is more than -26.38 dBm, the tolerable BER is achieved in all scenarios except row 16. Scenario 11 was conducted to confirm the impact of noise figure. In this scenario, the received power level was -12.54 dBm. This is less than -12.25 dBm power level expected at the receiver. It can be seen that in this case, tolerable BER is not achieved. Thresholds for power level before amplification and for received power level were able to handle almost all scenarios but for an outlier.

S.No | LBB | RBB | Power (in dBm) | BER |
---|---|---|---|---|

1 | M | - | -20.38 | 3.01E-15 |

2 | MM | - | -12.37 | 4.76E-38 |

3 | MM | - | -24.24 | 8.47E-21 |

4 | MM | S | -3.97 | 9.35E-15 |

5 | MM | S | -26.62 | 3.57E-03 |

6 | MS | - | -29.99 | 9.13E-08 |

7 | MS | - | -21.53 | 2.38E-74 |

8 | MS | M | -8.53 | 1.51E-14 |

9 | MS | S | -6.58 | 1.44E-22 |

10 | MS | S | -29.93 | 6.47E-03 |

11 | MS | S | -29.99 | 1.67E-04 |

12 | S | - | -18.66 | 2.11E-17 |

13 | SM | M | -7.23 | 1.05E-133 |

14 | SM | M | -23.94 | 4.41E-22 |

15 | SM | M | -24.05 | 8.50E-30 |

16 | SM | M | -26.64 | 4.89E-17 |

17* | SM | S | -11.24 | 9.38E-07 |

18 | SM | S | -11.25 | 4.42E-35 |

19 | SM | S | -21.53 | 2.63E-29 |

20 | SM | S | -26.38 | 6.18E-26 |

21 | SS | - | -12.13 | 2.94E-59 |

22 | SS | - | -31.76 | 3.48E-10 |

23 | SS | M | -10.26 | 5.24E-28 |

## V Conclusions

This paper presented an approach to empirically decide whether an data center network switch design can operate within tolerable bit error rates. These designs are multi-fiber short distance networks that are not widely reported in literature. This paper provides important experimental data that is required for evaluation of these networks. It was observed that the BER decision can be made by considering the optical power at the receiver for networks with passive optical components. When an amplifier is added to the network, the BER decision must additionally consider the optical power before amplification. This model provides a simple way of using thresholds to evaluate a large number of candidate architectures. This simple model also eliminates the need for researchers to perform detailed simulations or experimentation in a large scale.

## References

- [1] G. C. Sankaran and K. M. Sivalingam, “Combinatorial Approach for Network Switch Design in Data Center Networks,” in IEEE INFOCOM, p. 9999, 2017.
- [2] S. Yoo, Y. Yin, and R. Proietti, “Elastic Optical networking and low-latency high-radix optical switches for Future Cloud Computing,” in International Conference on Computing, Networking and Communications (ICNC), 2013, pp. 1097–1101, IEEE, 2013.
- [3] M. Fiorani, S. Aleksic, M. Casoni, L. Wosinska, and J. Chen, “Energy-Efficient Elastic Optical Interconnect Architecture for Data Centers,” IEEE Communications Letters, vol. 18, pp. 1531–1534, Sept 2014.
- [4] G. C. Sankaran and K. M. Sivalingam, “Optical traffic grooming-based data center networks: Node architecture and comparison,” IEEE Journal on Selected Areas in Communications, vol. 34, pp. 1618–1630, May 2016.
- [5] G. C. Sankaran and K. M. Sivalingam, “Grammar based Combinatorial Solver.” https://sourceforge.net/projects/arch-solver/, Dec. 2015.
- [6] A. Gumaste and T. Antony, DWDM network designs and engineering solutions. Cisco Press, 2003.
- [7] Fiber Optika Technologies, “Light Runner - Fiber Optic Kit.” http://www.fiberoptika.com/fiber-optics-kit.php, Jan. 2011.
- [8] R Core Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2014.