Bug Reports

[vakkotaur]

This morning Slashdot has an article asking about getting better (useful) bug reports from users. I don't have an answer other than there ought to be more than simply "file a report" and a single acknowledgement. That is, if I do file a bug report (and it is a decent report) then please let me know more, even if it's some time late. I'm much more likely to file another good report (assuming I trip over another bug...) if a previous one was handled well. A simple "this issue has been assigned to a maintainer/developer" helps. If I see the problem corrected, that's ideal. If I get a work-around sent to me, hey that's at least something.

But that does assume I send a good bug report. A good bug report is not "No bugs found." as wonderful as that would be. It's "I'm using this hardware, in this configuration, connected this way, to that thing, and when I do *THIS ACTION* on this program, *THIS* happens rather than *THAT* which was supposed to happen." It's more than "It's broken." It tells someone what sort of hardware is involved (Does Intel have a weird bug? Does AMD? Is that new video card causing trouble?) and what I do to trigger the bug. Did I type something into a certain text box? Did I click something? Is it a matter of which server I was trying to use? It's like going a mechanic with a car problem. You tell the mechanic the symptoms such as "There's a squeak on right turns, especially over about 30 mph." rather than "It's broken. Please fix it."

I used to deal with bug reports and most of the time what I got was the useless, "It's broke, fix it!" crap. Fix what, exactly? There was one customer of the company I worked for that did really good bug reporting. They also got very fast turn-around fixes. When I got a report from them, it was "We've got this model device, running this version software, and when we go into this particular menu and change this particular setting..." And that was often enough to aim me right at the issue they had. It wasn't quite a big red blinking [BUG HERE!] sign, but it was as close as one is ever likely to get. I didn't have to waste time guessing what was going on, I could get right to the issue. It wasn't always easy, but it often shaved hours if not days, weeks, or even months off of a real solution. As much as I disliked having a bug creep in or pop up, if I heard of one that made it past internal testing, I really wanted to hear it from those guys.

A different customer had a complaint that was reported as pretty much "Version 1 works, version 2 doesn't." and wanted changes made to the obsolete (and painful to work on...) version 1. They eventually got that, because I was unable to fix version 2 (which already had the feature they wanted...) since they didn't say exactly what was going on. I'm sure that they thought they did, but I could never duplicate the problem. Months later another customer hit the same problem, but gave detailed information about how to trigger the problem. Then I could duplicate the problem - and only then I could start the actual fixing of the real problem. It was a real, "Damn, why didn't those other guys tell us that bit?!" moment.

Even if you think something isn't important, don't omit it. It might just be the one critical piece that solves everything - or at least pinpoints the actual problem. Until it's checked, all data is valuable. Don't throw it away before it can even be used. Will some be irrelevant? Almost certainly, but you (and perhaps even the folks debugging) don't know which information is critical and which is not. As someone who was trying to hunt down bugs, I'd much rather have had an excess of information than a shortage of it.

Of course, if your experience really is "No bugs encountered." I am fairly sure whoever you're dealing with would love to read that in a review or something.

[ADDENDUM]: And if a developer or maintainer asks a question, PLEASE actually answer the question rather than fill in the background with so much other stuff that foreground goes underground. That was question wasn't asked for nothing.

Comments

[vakkotaur.livejournal.com]

That sort of story is a classic "How NOT to handle bug reports." Solved stuff doesn't always stay solved, and sometimes supposedly solved stuff isn't. It would not surprise if they stopped getting useful bug reports with an attitude like that. And got more bugs as a result, since they left them in.

I'm not sure how it is now, but some years ago Opera did it right. I got the response, and later I even got the second response to say something was being done. Another time I was pointed to the right forum (I don't mind RTFM, but I need to *find* TFM to R it.)

It does vary immensely by project. But yeah, I can see getting burned by stupid responses leading to giving up. I just LOVE it (not) when I get a forum reply that tells me to try what I've clearly said did not work.

[yakko.livejournal.com]

A good bug report is not "No bugs found." as wonderful as that would be. It's "I'm using this hardware, in this configuration, connected this way, to that thing, and when I do *THIS ACTION* on this program, *THIS* happens rather than *THAT* which was supposed to happen."

This is how I filed a bug in ADTpro awhile back. Basically, there was this evil bug where some disks were incorrectly formatted. It would have been useless to say "some disks are incorrectly formatted when the block count reads 0", so I filed a bug with the type of disks I was having the issue with, and which disks seemed to be unaffected. It turned out that that was important, because I had the problem when I used a SmartPort device, and the bug turned out to be an extra SmartPort call being made when it shouldn't have been. And it was fixed about a day after I filed the bug.

For car service, I've heard about service writers translating the customer's "It has a squeak in the front passenger's side when I hit bumps" to "CUSTOMER STATES THAT THERE IS A SQUEAK". Of course it won't get fixed. That job will be closed out with "NO TROUBLE FOUND".