[This has been sitting my drafts for a while, time to hit send. Long time ago I wrote something like this to a Red Hat internal mailing list, thought I’ll put it out here too.]
A week ago, I was participating in one of the regular upstream OpenStack bug triage sessions (for Nova project) to keep up with the insane flow of bugs. All open source projects are grateful for reported bugs. But more often than not, a good chunk of bugs in such large projects end up being bereft of details.
Writing a good bug report is hard. Writing a coherent, reproducible bug report is much harder (and takes more time). Especially when involved in complex cloud projects like OpentStack where you may have to care about everything from Kernel to user space.
A few things to consider including in a bug report. The following are all dead obvious points (and are usually part of a high-level template in issue trackers like Bugzilla). That said, however many times you repeat, it’s never enough:
- Summary. A concise summary — is it a bug? an RFE? a tracker-bug?
- Description. A clear description of the issue and its symptoms in chronological order.
- Version details. e.g. Havana? IceHouse?, hypervisor details; API versions, anything else that’s relevant.
- Crystal clear details to reproduce the bug. Bonus points for a reproducer script.
- Test environment details. (This is crucial.)
- Most of “cloud” software testing is dearly dependent on test environment. Clearer the details, fewer the round-trips between developers, test engineers, packagers, triagers, etc.
- Note down if any special hardware is needed for testing, e.g. an exotic NAS, etc.
- If you altered anything — config files, especially code located in
/usr/lib/pythonx.x/site-packages/, or any other dependent lower-layer project configurations (e.g. libvirt’s config file), please say so.
- Actual results. Post the exact, unedited details of what happens when the bug in question is triggered.
- Expected results. Describe what is the desired behavior.
- Additional investigative details (where appropriate).
- If you’ve done a lot of digging into an issue, writing a detailed summary (even better: a blog post) while its fresh in memory is very useful. Along with addition info like – configuration settings, caveats, relevant log fragment, _stderr_ of a script, or a command being executed, adding trace details — all of which would be useful for archival purposes (and years later, the context would come in very handy)
Why?
- Useful for new test engineers who do not have all the context of a bug.
- Useful for documentation writers to help them write correct errata text/release notes.
- Useful for non-technical folks reading the bugs/RFEs. Clear information saves a heck of a lot of time.
- Useful for folks like product and program managers who’re always not in the trenches.
- Useful for downstream support organizations.
- Should there be a regression years later, having all the info to test/reproduce in the bug, right there makes your day!
- Reduces needless round-trips of NEEDINFO.
- Useful for new users referring to these bugs in a different context.
Overall, a very fine bug report reading experience.
You get the drift!
Kashyap Chamarthy 