On Proper Bug Reporting

A post for those of you readers who work in QA, especially those of you at my former place of employment who know my usual complaints about improperly logged bugs.

A recent bug report against the Chrome browser has been getting some press on the Internet. In case you’ve missed it, I’ll reproduce it below. You can see the original report and the resulting discussion in the Chromium bug tracker. The comments below the report were my first reactions on seeing it.


Issue 224182: Chrome wakes me up in the middle of the night, with monsters.

Chrome Version: 25.0.1364.172 m

What steps will reproduce the problem?

1. Before going to bed, enjoy the wonderful goodness that is watching an episode of Supernatural on Netflix. Some people might not like creepy TV shows before bed, but what can I say, I can't get enough of that show!

2. After the episode ends, turn off your monitor, crawl directly into bed, close your eyes, relax, think happy thoughts, and drift off into a nice peaceful sleep.

3. Around 3 AM when you're just sinking into the depths of your second full sleep cycle, perhaps Windows would like to install some Windows Updates and reboot your computer. This is of course no concern to you as you're fast asleep.

4. WAKE UP IN A PANIC TO THE SOUND OF MONSTERS IN YOUR ROOM.

5. After getting out of bed and changing your pants, realize that after your computer restarted, Chrome helpfully re-opened all of your tabs, including Netflix, and so it restarted playing the episode of Supernatural that you watched before bed.

What is the expected result?
Ideally when Chrome restores your browsing state, it would also restore the state of plugins (e.g. my netflix player wasn't actually playing when Chrome shut down, so it shouldn't start playing when Chrome starts back up). This might require API changes to Silverlight / Flash so that Chrome can trigger them to save / restore state, so I don't know if that's feasible, but it would be nice!

Barring that, it would be nice if Chrome didn't run any plugins when restoring your tabs and instead displayed a bar at the top of the window, saying something like "Plugins were prevented from running while restoring tabs. [Start Plugins Now]"

I’d like to address a couple of problems with this report. Note: I won’t go into detail on items that may be specific to certain locations bug standards, such as the failure to provide a link to relevant documentation or attach error logs, screenshots, or (most relevant in this case), webcam captures.

First, remember that the steps to reproduce should be as concise as possible and include only the minimum number of actions necessary to allow the bug to be recreated. This report contains several unnecessary steps and does not correctly indicate all required actions:

  • Turning off the monitor and thinking happy thoughts are not required to arrive at a nice, peaceful sleep.
  • Windows restarting is a required step, yet the report does not make this clear. It suggests that “perhaps” Windows could install updates and restart. Eliminate the “perhaps” and make it clear that other causes of Windows restarting will also trigger this issue.
  • Changing your pants is not required to reproduce the bug. Also, keep in mind that not everyone wears pants when sleeping, and in fact, those who don’t wear pants will rate the severity of the bug higher, as they will need to change the sheets and perhaps the mattress as well, depending on how often the bug occurs.

Second, Step 5 is not actually a step to reproduce, it is the actual result. This should be more clearly delineated.

Third, the expected result is incorrect. The report states the reporter’s desired behaviour, rather than the behaviour defined in the appropriate requirements document.

Finally, the most important point is that this is not actually a bug and should be closed as “As Designed”. Had the user properly considered the development of the Internet as a communication tool, the history of Windows as an OS expected to restart frequently, and the intent of Chrome’s design, he would have realized that the sole purpose of the last 50 years of technical innovation was to scare the shit out of him.

3 thoughts on “On Proper Bug Reporting

  1. *applause*
    Twitter version:

    Issue 1: Anxiety disorder
    Version: All
    What steps will reproduce the problem? Use any technology of the last 50 years

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s