VEER: Disagreement-Free Multi-objective Configuration
share
Software comes with many configuration options, satisfying varying needs from users. Exploring those options for non-functional requirements can be tedious, time-consuming, and even error-prone (if done manually). Worse, many software systems can be tuned to multiple objectives (e.g., faster response time, fewer memory requirements, decreased network traffic, decreased energy consumption, etc.). Learning how to adjust the system among these multiple objectives is complicated due to the trade-off among objectives; i.e., things that seem useful to achieve one objective could be detrimental to another objective. Consequentially, the optimizer built for one objective may have different (or even opposite) insights on how to locate good solutions from the optimizer built from another objective. In this paper, we define this scenario as the model disagreement problem. One possible solution to this problem is to find a one-dimensional approximation to the N-objective space. In this way, the case is converted to a single-objective optimization, which is naturally confusion-free. This paper demonstrates VEER, a tool that builds such an approximation by combining our dimensionality-reduction heuristic on top of one of the state-of-the-art optimizers, FLASH. VEER can explore very large configuration spaces by evaluating just a small fraction of the total number of configurations (e.g., a space of 81,000 configurations can be explored by 70 samples). The experimental result in this paper demonstrates the feasibility of our approach in terms of the on-par quality of the solution set generated by the optimizer and the resolved model disagreement within the optimizer. Moreover, we demonstrate that VEER has an improved computational complexity compared to the original optimizer (up to 1,000 times faster while maintaining on-par performance).
READ FULL TEXT