An Empirical Study of Software Exceptions in the Field using Search Logs

05/30/2020
by   Foyzul Hassan, et al.
0

Software engineers spend a substantial amount of time using Web search to accomplish software engineering tasks. Such search tasks include finding code snippets, API documentation, seeking help with debugging, etc. While debugging a bug or crash, one of the common practices of software engineers is to search for information about the associated error or exception traces on the internet. In this paper, we analyze query logs from a leading commercial general-purpose search engine (GPSE) such as Google, Yahoo! or Bing to carry out a large scale study of software exceptions. To the best of our knowledge, this is the first large scale study to analyze how Web search is used to find information about exceptions. We analyzed about 1 million exception related search queries from a random sample of 5 billion web search queries. To extract exceptions from unstructured query text, we built a novel and high-performance machine learning model with a F1-score of 0.82. Using the machine learning model, we extracted exceptions from raw queries and performed popularity, effort, success, query characteristic and web domain analysis. We also performed programming language-specific analysis to give a better view of the exception search behavior. These techniques can help improve existing methods, documentation and tools for exception analysis and prediction. Further, similar techniques can be applied for APIs, frameworks, etc.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/19/2019

The Usage of Web Search for Software Engineering

Internet plays a key role in accomplishing many tasks. For many such tas...
05/01/2020

Studying Ransomware Attacks Using Web Search Logs

Cyber attacks are increasingly becoming prevalent and causing significan...
03/22/2018

Evaluating How Developers Use General-Purpose Web-Search for Code Retrieval

Search is an integral part of a software development process. Developers...
03/22/2021

Exploring Web Search Engines to Find Architectural Knowledge

Software engineers need relevant and up-to-date architectural knowledge ...
07/26/2022

Using clarification questions to improve software developers' Web search

Context: Recent research indicates that Web queries written by software ...
06/28/2021

Revelio: ML-Generated Debugging Queries for Distributed Systems

A major difficulty in debugging distributed systems lies in manually det...
02/04/2018

Software Engineers vs. Machine Learning Algorithms: An Empirical Study Assessing Performance and Reuse Tasks

Several papers have recently contained reports on applying machine learn...