One λ at a time

Lambda
Our full paper, One λ at a time: What do we know about presenting human-friendly output from program analysis tools?, has been accepted to the 8th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU) at SPLASH 2017, Vancouver, British Columbia. The paper is useful for human-computer interaction researchers who want to understand the design space of error messages from program analysis tools.

The abstract of the paper follows:

Program analysis tools perform sophisticated analysis on source code to help programmers resolve compiler errors, apply optimizations, and identify security vulnerabilities. Despite the utility of these tools, research suggests that programmers do not frequently adopt them in practice—a primary reason being that the output of these tools is difficult to understand. Towards providing a synthesis of what researchers know about the presentation of program analysis output to programmers, we conducted a scoping review of the PLDI conference proceedings from 1988-2017. The scoping review serves as interim guidance for advancing collaborations between research disciplines. We discuss how cross-disciplinary communities, such as PLATEAU, are critical to improving the usability of program analysis tools.

Expressions on the Nature and Significance of Programming and Play

All is play.

My single-author full paper, Expressions on the Nature and Significance of Programming and Play, has been accepted to the IEEE Symposium on Visual Languages and Human-Centric Computing. I presented the paper in Raleigh, North Carolina.

The abstract of the paper follows:

Play is all around us, an essential and innate phenomenon that serves as an important mediator in creativity, interest, learning, and drive. Though play is thought to be universal, the way in which it materializes is situationally-dependent and not well-understood, particularly in software engineering. To understand how programmers express the concept of play, we conducted a qualitative study on the online social news website, Hacker News—a venue for software practitioners. From Hacker News, we qualitatively analyzed nearly 1,000 user-submitted comments containing the terms “programming” and “play.” The contribution of this work is a contemporary synthesis of how software practitioners interpret programming and play in experiential terms. Our findings suggest how programming and play can be understood through rich metaphors, among them, play as: art, playgrounds, spontaneity, and tinkering. Hacker News authors reflect about childhood experiences as a catalyst for learning programming, and contrast play against work.

Do Developers Read Compiler Error Messages?


Our full paper, Do Developers Read Compiler Error Messages?, has been accepted to the International Conference on Software Engineering. I will be presenting the paper at the conference in Buenos Aires, Argentina.

The abstract of the paper follows:

In integrated development environments, developers receive compiler error messages through a variety of textual and visual mechanisms, such as popups and wavy red underlines. Although error messages are the primary means of communicating defects to developers, researchers have a limited understanding on how developers actually use these messages to resolve defects. To understand how developers use error messages, we conducted an eye tracking study with 56 participants from undergraduate and graduate software engineering courses at our university. The participants attempted to resolve common, yet problematic defects in a Java code base within the Eclipse development environment. We found that: 1) participants read error messages and the difficulty of reading these messages is comparable to the difficulty of reading source code, 2) difficulty reading error messages significantly predicts participants’ task performance, and 3) participants allocate a substantial portion of their total task to reading error messages (13%-25%). The results of our study offer empirical justification for the need to improve compiler error messages for developers.

From Quick Fixes to Slow Fixes

Our full paper, From Quick Fixes to Slow Fixes: Reimagining Static Analysis Resolutions to Enable Design Space Exploration, has been accepted to the International Conference on Software Maintenance and Evolution (ICSME). In addition, I also served as Web Chair of the Organizing Committee. ICSME is hosted this year in Raleigh, North Carolina.

The abstract of the paper follows:

Quick Fixes as implemented by IDEs today prioritize the speed of applying the fix as a primary criteria for success. In this paper, we argue that when tools over-optimize this criteria, such tools neglect other dimensions that are important to successfully applying a fix, such as being able to explore the design space of multiple fixes. This is especially true in cases where a fix only partially implements the intention of the developer.

In this paper, we implement an extension to the FindBugs defect finding tool, called FixBugs, an interactive resolution approach within the Eclipse development environment that prioritizes other design criteria to the successful application of suggested fixes. Our empirical evaluation method of 12 developers suggests that FixBugs enables developers to explore alternative designs and balances the benefits of manual fixing with automated fixing, without having to compromise in either effectiveness or efficiency. Our analytic evaluation method with six usability experts identified trade-offs between FixBugs and Quick Fix, and suggests ways in which FixBugs and Quick Fix can offer complementary capabilities to better support developers.

Supplemental materials for the FixBugs study have also been made available.

Blending Programming Environments and Games

Titus presents. Photo by Austin Henley.

I presented our full paper, A Perspective on Blending Programming Environments and Games: Beyond Points, Badges, and Leaderboards, at the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2016) in Cambridge, UK.

The abstract of the paper follows:

Programming environments and game environments share many of the same characteristics, such as requiring their users to understand strategies and solve difficult challenges. Yet, only game designers have been able to capitalize on methods that are consistently able to keep their users engaged. Consequently, software engineers have been increasingly interested in understanding how these game experiences can be transferred to programming experiences, a process termed gamification.

In this perspective paper, we offer a formal argument that gamification as applied today is predominately narrow, placing emphasis on the reward aspects of game mechanics at the expense of other important game elements, such as framing. We argue that more authentic game experiences are possible when programming environments are re-conceptualized and assessed as holistic, serious games. This broad gamification enables us to more effectively apply and leverage the breadth of game elements to the construction and understanding of programming environments.

European Conference on Object-Oriented Programming Retrospective

Through my research in HCI and software engineering tools, I typically make the annual rounds in conferences such as ICSE, FSE, and VL/HCC. Thus, having the opportunity to attend ECOOP 2016 was definitely outside of my comfort area, but it was worthwhile in that it provided me with exposure to an otherwise enigmatic area of Computer Science.

As a first-timer to ECOOP, I expected all of the conference to basically be about programming language (PL) theory (or as we call it in HCI, the land of Greek letters). I was surprised, however, to find workshops on the usability aspects of programming languages. I attended two such workshops:

  • LIVE, a workshop on live programming systems that “abandon the traditional edit-compile-run cycle in favor of fluid user experiences.”
  • Grace, a workshop on the emerging Grace programming language. Originating at ECOOP in 2010, the Grace programming language is designed to allow novices to discover object-oriented programming in simpler ways.

To gain exposure to new ideas, I also attended ICOOOLPS, a workshop on compiler optimization and performance for object-oriented programming.

Although my own research community and ECOOP have relatively little intersection, through these smaller workshops I quickly met new colleagues, including James Noble, Michael Kölling, and my own PhD advisor’s advisor, Andrew Black.

An aspect of ECOOP that I particularly appreciated was the morning breakfast sessions, where students like myself were paired with faculty members to learn more about ECOOP research. I took full advantage of these sessions and introduced myself to a new faculty member each day for breakfast: Matthias Felleisen, Tobias Wrigstad, Laurence Tratt, and Jan Vitek.

Another highlight of the conference was the ECOOP Summer School. The lecturers for these talks made a significant effort to provide a gentle introduction to programming language theory and to explain the types of problems researchers in PL study. One of the more memorable lectures was a hands-on session by Laurence Tratt and Carl Friedrich Bolz, where we worked on our laptops to implement a JIT in Python.

ECOOP Summer School

Thanks again to the NSF for providing this amazing opportunity.

The Bones of the System: A Case Study of Logging and Telemetry at Microsoft

Our full paper, The Bones of the System: A Case Study of Logging and Telemetry at Microsoft, has been accepted to the International Conference on Software Engineering, Software Engineering in Practice Track (ICSE SEIP 2016). ICSE is hosted this year in Austin, Texas.

The abstract of the paper follows:

Large software organizations are transitioning to event data platforms as they culturally shift to better support data-driven decision making. This paper offers a case study at Microsoft during such a transition. Through qualitative interviews of 28 participants, and a quantitative survey of 1,823 respondents, we catalog a diverse set of activities that leverage event data sources, identify challenges in conducting these activities, and describe tensions that emerge in data-driven cultures as event data flow through these activities within the organization. We find that the use of event data span every job role in our interviews and survey, that different perspectives on event data create tensions between roles or teams, and that professionals report social and technical challenges across activities.

I am delighted to have been able to collaborate with Microsoft Research for this study. Thanks to Robert DeLine, Steven Drucker, and Danyel Fisher, the co-authors of the paper.

Challenges in Using Event Data

Challenges in Using Event Data

I ♥ Hacker News

I Heart HN

Our short paper, I ♥ Hacker News: Expanding Qualitative Research Findings by Analyzing Social News Websites, has been accepted to the joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2015, New Ideas). ESEC/FSE is hosted this year in Bergamo, Italy.

The abstract of the paper follows:

Grounded theory is an important research method in empirical software engineering, but it is also time consuming, tedious, and complex. This makes it difficult for researchers to assess if threats, such as missing themes or sample bias, have inadvertently materialized. To better assess such threats, our new idea is that we can automatically extract knowledge from social news websites, such as Hacker News, to easily replicate existing grounded theory research — and then compare the results. We conduct a replication study on static analysis tool adoption using Hacker News. We confirm that even a basic replication and analysis using social news websites can offer additional insights to existing themes in studies, while also identifying new themes. For example, we identified that security was not a theme discovered in the original study on tool adoption. As a long-term vision, we consider techniques from the discipline of knowledge discovery to make this replication process more automatic.

Fuse: A Reproducible, Extendable, Internet-scale Corpus of Spreadsheets

Fuse Distribution

Our paper, Fuse: A Reproducible, Extendable, Internet-scale Corpus of Spreadsheets, has been accepted to the 12th Working Conference on Mining Software Repositories.

Fuse is obtained by filtering through 1.9 petabytes of raw data from Common Crawl, using Amazon Web Services. See our Fuse Spreadsheet Corpus project page for details on obtaining the spreadsheets and using the spreadsheet metadata.

The abstract of the paper follows:

Spreadsheets are perhaps the most ubiquitous form of end-user programming software. This paper describes a corpus, called Fuse, containing 2,127,284 URLs that return spreadsheets (and their HTTP server responses), and 249,376 unique spreadsheets, contained within a public web archive of over 26.83 billion pages. Obtained using nearly 60,000 hours of computation, the resulting corpus exhibits several useful properties over prior spreadsheet corpora, including reproducibility and extendability. Our corpus is unencumbered by any license agreements, available to all, and intended for wide usage by end-user software engineering researchers. In this paper, we detail the data and the spreadsheet extraction process, describe the data schema, and discuss the trade-offs of Fuse with other corpora.

Can Social Screencasting Help Developers Learn New Tools?

Screencasting Tool Usages
Our short paper, Can Social Screencasting Help Developers Learn New Tools?, has been accepted to the 8th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE 2015). The workshop is colocated with ICSE, and will be hosted in Florence, Italy.

The lead author of the paper is Kubick Lubick. The abstract of the paper follows:

An effective way to learn about software development tools is by
directly observing peers’ workflows. However, these tool knowledge transfer events happen infrequently because developers must be both colocated and available. We explore an online social screencasting system that removes the dependencies of colocation and availability while maintaining the beneficial tool knowledge transfer of peer observation. Our results from a formative study indicate these online observations happen more frequently than in-person observations, but their effects are only temporary. We conclude that while peer observation facilitates online knowledge transfer, it is not the only component — other social factors may be involved.