DeepAI AI Chat
Log In Sign Up

Simpler Hyperparameter Optimization for Software Analytics: Why, How, When?

by   Amritanshu Agrawal, et al.

How to make software analytics simpler and faster? One method is to match the complexity of analysis to the intrinsic complexity of the data being explored. For example, hyperparameter optimizers find the control settings for data miners that improve for improving the predictions generated via software analytics. Sometimes, very fast hyperparameter optimization can be achieved by just DODGE-ing away from things tried before. But when is it wise to use DODGE and when must we use more complex (and much slower) optimizers? To answer this, we applied hyperparameter optimization to 120 SE data sets that explored bad smell detection, predicting Github ssue close time, bug report analysis, defect prediction, and dozens of other non-SE problems. We find that DODGE works best for data sets with low "intrinsic dimensionality" (D = 3) and very poorly for higher-dimensional data (D over 8). Nearly all the SE data seen here was intrinsically low-dimensional, indicating that DODGE is applicable for many SE analytics tasks.


Is AI different for SE?

What AI tools are needed for SE? Ideally, we should have simple rules th...

Optimizing Predictions for Very Small Data Sets: a case study on Open-Source Project Health Prediction

When learning from very small data sets, the resulting models can make m...

How to "DODGE" Complex Software Analytics?

AI software is still software. Software engineers need better tools to m...

Can You Explain That, Better? Comprehensible Text Analytics for SE Applications

Text mining methods are used for a wide range of Software Engineering (S...

Why is Differential Evolution Better than Grid Search for Tuning Defect Predictors?

Context: One of the black arts of data mining is learning the magic para...

Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

In many domains, there are many examples and far fewer labels for those ...

When SIMPLE is better than complex: A case study on deep learning for predicting Bugzilla issue close time

Is deep learning over-hyped? Where are the case studies that compare sta...