Wissenschaftliche Keynote: Chunyang Chen

Keynote: “Towards Human-like Software Testing”

Software testing has long benefited from automation such as program analysis, GUI testing, fuzzing, yet much of today’s automated testing remains misaligned with how software is actually used and assessed by humans. In practice, teams still face “so many bugs” alongside “so many false positives”, where automated approaches can generate infeasible or unrealistic interaction sequences which can rarely be triggered by real-world users. Such generated test cases or traces rarely represent human behavior and often yield false positives and low-value defects resulting in great workload for developers to reproduce and repair those unnecessary issues, which reduce developers’ trust in testing tools. Moreover, metric-driven testing — optimizing for coverage, defect metrics, or execution throughput — can incentivize over-testing and provide an inflated sense of progress without proportional gains in actionable quality. This keynote motivates a shift from tool-centric automation towards on-demand and human-centric testing, where the goal is to explore software using strategies, constraints, and intentions closer to those of end users.

The keynote shares a line of studies that carry out human-like testing using large language model(LLM)-driven testing agents, especially in GUI testing. First, we present LLM-powered automated mobile GUI testing that frames interaction between LLM and mobile apps as a functionality-aware exploration process, demonstrating strong activity coverage and real-world bug discovery at scale. Second, to emulate how human testers accumulate experience, we introduce dynamic memory for GUI testing agents, including interaction-level episodic memory, function-level reflective memory, and app-level strategic memory, with on-demand invocation. This “experience layer” can be integrated as a plugin to improve both coverage and bug yield across different GUI testing tools and settings. Then, we extend human-like testing beyond single-user interactions by addressing multi-user interactive features, which are common in real apps and difficult to test with record-and-replay scripts due to device independence and action coordination requirements.

Finally, we will also introduce how to inject persona into testing agents to mimic different human testers as a kind of automated crowd testing. In addition, I will key challenges and research directions in this direction such as scalability, efficiency, perception, oracle missing, and testing thoroughness.

Programm

•  Wissenschaftliches Programm, Freitag, 09:00 – 10:00 (siehe  Programm)

Bio

 Chunyang Chen ist ordentlicher Professor an der School of Computation, Information and Technology der Technischen Universität München (TUM), Deutschland. Sein Hauptforschungsinteresse liegt im Bereich der automatisierten Softwaretechnik, insbesondere in der datengestützten Entwicklung mobiler Anwendungen. Darüber hinaus interessiert er sich auch für Mensch-Computer-Interaktion und Softwaresicherheit.

Er hat zahlreiche Forschungsarbeiten auf führenden Konferenzen wie ICSE, FSE, ASE, CHI und CSCW veröffentlicht und arbeitet eng mit der Industrie zusammen, unter anderem mit Google, Microsoft und Meta. Für seine Forschung wurde er mehrfach ausgezeichnet, darunter mit dem ACM SIGSOFT Early Career Researcher Award, einem Facebook Research Award, vier ACM SIGSOFT Distinguished Paper Awards (ICSE'23/21’/20’, ASE'18) sowie mehreren Best Paper/Demo Awards.