Human-Centered Automation in Software Engineering

Devjeet Raj Roy

The rapid expansion in scale and complexity of software systems over the past two decades has necessitated the development of advanced software engineering (SE) technologies aimed at enhancing developer productivity and reducing maintenance costs. A central challenge in this evolution is ensuring that these automated systems align with human factors—developer perceptions, needs, and workflows—to be effective and widely adopted. Misalignment can lead to tools that, despite technical sophistication, fail to deliver practical benefits to their intended users. This work focuses on bridging the gap between automated SE technologies and human factors, emphasizing the importance of a human-centric approach in the development and evaluation of these tools. We begin by examining how reliance on automatic evaluation metrics as proxies for human assessment can create a disconnect between SE technologies and developers. Specifically, we analyze two contexts: (a) code summarization, where automatic metrics are pivotal benchmarks yet may not reflect developer preferences, and (b) code readability assessment, where metrics intended for direct use by developers may not align with their perceptions of readability. Our findings reveal that commonly used automatic metrics often fail to accurately represent human assessments, underscoring the need for more human-aligned evaluation methods. Next, we explore how integrating insights from human factors research, such as program comprehension studies, can enhance the effectiveness of SE technologies. First, we propose human-centered interventions in automated testing: an underutilized domains with a wealth of literature that has not seen large scale adoption. By incorporating developer feedback and cognitive principles, the intervention is designed to align more closely with real-world workflows and have been preferred by practitioners over traditional methods, demonstrating the value of a user-guided development approach. Second, we extend our human-centric focus to applications of Large Language Models (LLMs) in software development tasks, which are increasingly being used across a diverse set of subdomains in SE. This work underscores the necessity of aligning automated SE technologies with human factors at every stage—from evaluation to system design. By placing human comprehension at the forefront of computation, this work contributes to the development of more effective, adoptable, and trustworthy tools in software engineering, ultimately bridging the divide between automated systems and their human users.

Human-Centered Automation in Software Engineering

Files and links (1)

Abstract

Metrics

Details