DeepAI AI Chat
Log In Sign Up

HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning

by   Oguzhan Gencoglu, et al.
HUAWEI Technologies Co., Ltd.
Association for Computing Machinery
Morpho, Inc.
Tampere Universities

Recent advancements in machine learning research, i.e., deep learning, introduced methods that excel conventional algorithms as well as humans in several complex tasks, ranging from detection of objects in images and speech recognition to playing difficult strategic games. However, the current methodology of machine learning research and consequently, implementations of the real-world applications of such algorithms, seems to have a recurring HARKing (Hypothesizing After the Results are Known) issue. In this work, we elaborate on the algorithmic, economic and social reasons and consequences of this phenomenon. We present examples from current common practices of conducting machine learning research (e.g. avoidance of reporting negative results) and failure of generalization ability of the proposed algorithms and datasets in actual real-life usage. Furthermore, a potential future trajectory of machine learning research and development from the perspective of accountable, unbiased, ethical and privacy-aware algorithmic decision making is discussed. We would like to emphasize that with this discussion we neither claim to provide an exhaustive argumentation nor blame any specific institution or individual on the raised issues. This is simply a discussion put forth by us, insiders of the machine learning field, reflecting on us.


page 1

page 2

page 3

page 4


Recent Advances in the Foundations and Applications of Unbiased Learning to Rank

Since its inception, the field of unbiased learning to rank (ULTR) has r...

Deep Learning in the Wild

Deep learning with neural networks is applied by an increasing number of...

Towards Ecologically Valid Research on Language User Interfaces

Language User Interfaces (LUIs) could improve human-machine interaction ...

Data and its (dis)contents: A survey of dataset development and use in machine learning research

Datasets have played a foundational role in the advancement of machine l...

Neural Algorithmic Reasoning

Algorithms have been fundamental to recent global technological advances...

Some Ethical Issues in the Review Process of Machine Learning Conferences

Recent successes in the Machine Learning community have led to a steep i...

Differentiating Approach and Avoidance from Traditional Notions of Sentiment in Economic Contexts

There is growing interest in the role of sentiment in economic decision-...