From Quick Fixes to Slow Fixes

Our full paper, From Quick Fixes to Slow Fixes: Reimagining Static Analysis Resolutions to Enable Design Space Exploration, has been accepted to the International Conference on Software Maintenance and Evolution (ICSME). In addition, I also served as Web Chair of the Organizing Committee. ICSME is hosted this year in Raleigh, North Carolina.

The abstract of the paper follows:

Quick Fixes as implemented by IDEs today prioritize the speed of applying the fix as a primary criteria for success. In this paper, we argue that when tools over-optimize this criteria, such tools neglect other dimensions that are important to successfully applying a fix, such as being able to explore the design space of multiple fixes. This is especially true in cases where a fix only partially implements the intention of the developer.

In this paper, we implement an extension to the FindBugs defect finding tool, called FixBugs, an interactive resolution approach within the Eclipse development environment that prioritizes other design criteria to the successful application of suggested fixes. Our empirical evaluation method of 12 developers suggests that FixBugs enables developers to explore alternative designs and balances the benefits of manual fixing with automated fixing, without having to compromise in either effectiveness or efficiency. Our analytic evaluation method with six usability experts identified trade-offs between FixBugs and Quick Fix, and suggests ways in which FixBugs and Quick Fix can offer complementary capabilities to better support developers.

Supplemental materials for the FixBugs study have also been made available.

Blending Programming Environments and Games

Titus presents. Photo by Austin Henley.

I presented our full paper, A Perspective on Blending Programming Environments and Games: Beyond Points, Badges, and Leaderboards, at the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2016) in Cambridge, UK.

The abstract of the paper follows:

Programming environments and game environments share many of the same characteristics, such as requiring their users to understand strategies and solve difficult challenges. Yet, only game designers have been able to capitalize on methods that are consistently able to keep their users engaged. Consequently, software engineers have been increasingly interested in understanding how these game experiences can be transferred to programming experiences, a process termed gamification.

In this perspective paper, we offer a formal argument that gamification as applied today is predominately narrow, placing emphasis on the reward aspects of game mechanics at the expense of other important game elements, such as framing. We argue that more authentic game experiences are possible when programming environments are re-conceptualized and assessed as holistic, serious games. This broad gamification enables us to more effectively apply and leverage the breadth of game elements to the construction and understanding of programming environments.

ECOOP Retrospective

Through my research in HCI and software engineering tools, I typically make the annual rounds in conferences such as ICSE, FSE, and VL/HCC. Thus, having the opportunity to attend ECOOP 2016 was definitely outside of my comfort area, but it was worthwhile in that it provided me with exposure to an otherwise enigmatic area of Computer Science.

As a first-timer to ECOOP, I expected all of the conference to basically be about programming language (PL) theory (or as we call it in HCI, the land of Greek letters). I was surprised, however, to find workshops on the usability aspects of programming languages. I attended two such workshops:

  • LIVE, a workshop on live programming systems that “abandon the traditional edit-compile-run cycle in favor of fluid user experiences.”
  • Grace, a workshop on the emerging Grace programming language. Originating at ECOOP in 2010, the Grace programming language is designed to allow novices to discover object-oriented programming in simpler ways.

To gain exposure to new ideas, I also attended ICOOOLPS, a workshop on compiler optimization and performance for object-oriented programming.

Although my own research community and ECOOP have relatively little intersection, through these smaller workshops I quickly met new colleagues, including James Noble, Michael Kölling, and my own PhD advisor’s advisor, Andrew Black.

An aspect of ECOOP that I particularly appreciated was the morning breakfast sessions, where students like myself were paired with faculty members to learn more about ECOOP research. I took full advantage of these sessions and introduced myself to a new faculty member each day for breakfast: Matthias Felleisen, Tobias Wrigstad, Laurence Tratt, and Jan Vitek.

Another highlight of the conference was the ECOOP Summer School. The lecturers for these talks made a significant effort to provide a gentle introduction to programming language theory and to explain the types of problems researchers in PL study. One of the more memorable lectures was a hands-on session by Laurence Tratt and Carl Friedrich Bolz, where we worked on our laptops to implement a JIT in Python.

ECOOP Summer School

Thanks again to the NSF for providing this amazing opportunity.

A process for surviving survey design and sailing through survey deployment

Perspectives on Data Science for Software Engineering I’m happy to announce that our book chapter, A process for surviving survey design and sailing through survey deployment, contained within Perspectives on Data Science for Software Engineering (eds. Tim Menzies, Laurie Williams and Thomas Zimmermann), is now available in press. The book chapter is intended for software engineering researchers and practitioners who are interested in deploying surveys within their organization.

As an author, an interesting aspect of the writing process for this book is that the materials were developed and peer-reviewed collaboratively on the GitHub ds4se repository. For example, here are the reviews for our own book chapter from the drafting process.

The abstract of the chapter follows:

Odysseus and the Sirens by Herbert James Draper, c. 1909.

In the Odyssey, Odysseus and his crew must confront Persephone’s Sirens, creatures that sang in irresistible voices and lured sailors to their deaths. Still, adventurers risked the journey for the promise of knowledge, for through the Sirens’ song, “[o]ver all the generous earth [the adventurers] know everything that happens.” With guidance from Circe, the goddess of magic, Odysseus instructed his crew to plug their ears to deafen them from the song. He then had his crew tie him to the mast and commanded them not to release him under any circumstances. And so Odysseus and his crew navigated safely past the Sirens. Odysseus, having heard the Sirens’ song, sailed away a wiser man.

Surveys, as with Persephone’s Sirens, are an attractive instrument that offer a similar lure of knowledge. Like Circe, we offer guidance, grounded in our own experiences, on a successful process for understanding the practice of software engineering through surveys.

European Conference on Object-Oriented Programming

This year I’m able to attend the European Conference on Object-Oriented Programming (ECOOP) in Rome, Italy, thanks to generous travel support from the National Science Foundation. This will be my first time at ECOOP.

ECOOP

I’m looking forward to meeting other researchers, building collaborations, discovering emerging research areas, and attending the ECOOP Summer School lectures to learn more about the formal and technical aspects of programming languages and tools.

Thesis proposal: How should static analysis tools explain anomalies to developers?

Eclipse Explanations

On April 26, 2016, I presented my thesis proposal to a committee of five members: Dr. Emerson Murphy-Hill (Chair), Dr. Jing Feng (Graduate School Representative), Dr. Shriram Krishnamurthi (External Member), Dr. James Lester, and Dr. Christopher Parnin.

I received a conditional pass. A conditional pass means that a formal re-examination is not required, but that the committee expects additional revisions before approving the proposal.

I suspect that there are some students who do not even realize that they have received a conditional pass, since the event does not seem to be recorded anywhere that is student-accessible.

In the weeks that followed, I made several revisions to the thesis proposal document, incorporating feedback from the presentation:

  1. The committee reduced the scope of required experiments from five to three.
  2. The committee added a new requirement that I conduct a systematic literature review on static analysis notification techniques.
  3. I added a thesis contract to explicitly state the dissertation deliverables.

On May 11, 2016, I submitted the revised proposal to the committee.

On May 20, 2016, I was notified that the committee had approved the revisions.

Although some students prefer to keep their thesis proposal secret until graduation, I have made the proposal and presentation materials available so that they may help other students in structuring their own proposals:

Abstract

Despite the advanced static analysis tools available within modern integrated development environments (IDEs) for detecting anomalies, the error messages these tools produce to describe these anomalies remain perplexing for developers to comprehend. This thesis postulates that tools can computationally expose their internal reasoning processes to generate assistive error explanations in a way that approximates how developers explain errors to other developers and to themselves. Compared with baseline error messages, these error explanations significantly enhance developers’ comprehension of the underlying static analysis anomaly. The contributions of this dissertation are: 1) a theoretical framework that formalizes explanation theory in the context of static analysis anomalies, 2) a set of experiments that evaluate the extent to which evidence supports the theoretical framework, and 3) a proof-of-concept IDE extension, called Radiance, that applies my identified explanation-based design principles and operationalizes these principles into a usable artifact. My work demonstrates that tools stand to significantly benefit if they incorporate explanation principles in their design.

The Bones of the System: A Case Study of Logging and Telemetry at Microsoft

Our full paper, The Bones of the System: A Case Study of Logging and Telemetry at Microsoft, has been accepted to the International Conference on Software Engineering, Software Engineering in Practice Track (ICSE SEIP 2016). ICSE is hosted this year in Austin, Texas.

The abstract of the paper follows:

Large software organizations are transitioning to event data platforms as they culturally shift to better support data-driven decision making. This paper offers a case study at Microsoft during such a transition. Through qualitative interviews of 28 participants, and a quantitative survey of 1,823 respondents, we catalog a diverse set of activities that leverage event data sources, identify challenges in conducting these activities, and describe tensions that emerge in data-driven cultures as event data flow through these activities within the organization. We find that the use of event data span every job role in our interviews and survey, that different perspectives on event data create tensions between roles or teams, and that professionals report social and technical challenges across activities.

I am delighted to have been able to collaborate with Microsoft Research for this study. Thanks to Robert DeLine, Steven Drucker, and Danyel Fisher, the co-authors of the paper.

Challenges in Using Event Data

Challenges in Using Event Data

Timeful: My E-mail Policy

Due to increased demands on my time and the need to minimize distractions, I am posting an official policy on my use of e-mail.

This policy is effective 11/27/2015, and is available at:

http://go.barik.net/timeful

E-mails sent to my personal account or University account are now batched. This means that your e-mail will be intentionally delayed against one of the following time boundaries (all listed times are in Eastern time):

  • Monday through Friday: 9 AM, 1 PM, 3 PM, 7 PM. If you send an e-mail after 7 PM, I will receive your e-mail at 9 AM the following day.

  • Saturday and Sunday: 9 PM only. Weekends are reserved for time with my family.

If you need to contact me urgently, you may send me a text message at 251-454-1579.

EmailProductivityCurve-600px

Social Media Addendum

  • I check Facebook once a week, usually on Friday night.

I ♥ Hacker News

I Heart HN

Our short paper, I ♥ Hacker News: Expanding Qualitative Research Findings by Analyzing Social News Websites, has been accepted to the joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2015, New Ideas). ESEC/FSE is hosted this year in Bergamo, Italy.

The abstract of the paper follows:

Grounded theory is an important research method in empirical software engineering, but it is also time consuming, tedious, and complex. This makes it difficult for researchers to assess if threats, such as missing themes or sample bias, have inadvertently materialized. To better assess such threats, our new idea is that we can automatically extract knowledge from social news websites, such as Hacker News, to easily replicate existing grounded theory research — and then compare the results. We conduct a replication study on static analysis tool adoption using Hacker News. We confirm that even a basic replication and analysis using social news websites can offer additional insights to existing themes in studies, while also identifying new themes. For example, we identified that security was not a theme discovered in the original study on tool adoption. As a long-term vision, we consider techniques from the discipline of knowledge discovery to make this replication process more automatic.

Improving Error Notification Comprehension in IDEs by Supporting Developer Self-Explanations

Explanatory interface mockup, conceptualized in the Eclipse IDE.

My second graduate consortium submission, Improving Error Notification Comprehension in IDEs by Supporting Developer Self-Explanations, has been accepted to the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) in Atlanta, Georgia. The abstract of the paper follows:

Despite the advanced static analysis techniques available to compilers, error notifications as presented by modern IDEs remain perplexing for developers to resolve. My thesis postulates that tools fail to adequately support self-explanation, a core metacognitive process necessary to comprehend notifications. The contribution of my work will bridge the gap between the presentation of tools and interpretation by developers by enabling IDEs to present the information they compute in a way that supports developer self-explanation.

You can compare this submission with my prior VL/HCC Graduate Consortium.