$FreeBSD: doc/en_US.ISO8859-1/articles/problem-reports/article.sgml,v 1.9 2002/01/20 22:30:35 des Exp $ This article describes how to best formulate and submit a problem report to the FreeBSD Project. Dag-Erling Smørgrav Contributed by problem reports One of the most frustrating experiences one can have as a software user is to submit a problem report only to have it summarily closed with a terse and unhelpful explanation like not a bug or bogus PR. Similarly, one of the most frustrating experiences as a software developer is to be flooded with problem reports that are not really problem reports but requests for support, or that contain little or no information about what the problem is and how to reproduce it. This document attempts to describe how to write good problem reports. What, you ask, is a good problem report? Well, to go straight to the bottom line, a good problem report is one that can be analyzed and dealt with swiftly, to the mutual satisfaction of both user and developer. Although the primary focus of this article is on FreeBSD problem reports, most of it should apply quite well to other software projects. Note that this article is organized thematically, not chronologically, so you should read through the entire document before submitting a problem report, rather than treat it as a step-by-step tutorial. There are many types of problems, and not all of them should engender a problem report. Of course, nobody is perfect, and there will be times when you are convinced you have found a bug in a program when in fact you have misunderstood the syntax for a command or made a typo in a configuration file (though that in itself may sometimes be indicative of poor documentation or poor error handling in the application). There are still many cases where submitting a problem report is clearly not the right course of action, and will only serve to frustrate you and the developers. Conversely, there are cases where it might be appropriate to submit a problem report about something else than a bug—an enhancement or a feature request, for instance. So how do you determine what is a bug and what is not? As a simple rule of thumb your problem is not a bug if it can be expressed as a question (usually of the form How do I do X? or Where can I find Y?). It is not always quite so black and white, but the question rule covers a large majority of cases. Some cases where it may be appropriate to submit a problem report about something that is not a bug are: Requests for feature enhancements. It is generally a good idea to air these on the mailing lists before submitting a problem report. Notification of updates to externally maintained software (mainly ports, but also externally maintained base system components such as BIND or various GNU utilities). Another thing is that if the system on which you experienced the bug is not fairly up-to-date, you should seriously consider upgrading and trying to reproduce the problem on an up-to-date system before submitting a problem report. There are few things that will annoy a developer more than receiving a problem report about a bug she's already fixed. Finally, a bug that can not be reproduced can rarely be fixed. If the bug only occurred once and you can not reproduce it, and it does not seem to happen to anybody else, chances are none of the developers will be able to reproduce it or figure out what's wrong. That does not mean it did not happen, but it does mean that the chances of your problem report ever leading to a bug fix are very slim, and you should consider letting the matter drop. A good rule to follow is to always do a background search before submitting a problem report. Maybe your problem has already been reported; maybe it is being discussed on the mailing lists, or recently was; it may even already be fixed in a newer version than what you are running. You should therefore check all the obvious places before submitting your problem report. For FreeBSD, this means: The FAQ. The mailing lists—if you're not subscribed, use the searchable archives on the FreeBSD web site. If your problem has not been discussed on the lists, you might try posting a message about it and waiting a few days to see if someone can spot something you have overlooked. Optionally, the entire web—use your favorite search engine to locate any references to your problem. You may even get hits from archived mailing lists or newsgroups you did not know of or had not thought to search through. Finally, the FreeBSD PR database. Unless your problem is recent or obscure, there is a fair chance it has already been reported. Next, you need to make sure your problem report goes to the right people. The first catch here is that if the problem is a bug in third-party software (a port or a package you've installed), you should report the bug to the original author, not to the FreeBSD Project. There are two exceptions to this rule: the first is if the bug does not occur on other platforms, in which case the problem may lie in how the software was ported to FreeBSD; the second is if the original author has already fixed the bug and released a patch or a new version of his software, and the FreeBSD port has not been updated yet. The second catch is that FreeBSD's bug tracking system sorts problem reports according to the category the originator selected. Therefore, if you select the wrong category when you submit your problem report, there is a good chance that it will go unnoticed for a while, until someone re-categorizes it. Now that you have decided that your issue merits a problem report, and that it is a FreeBSD problem, it is time to write the actual problem report. Make sure your VISUAL (or EDITOR if VISUAL is not set) environment variable is set to something sensible, and run &man.send-pr.1;. The &man.send-pr.1; program has provisions for attaching files to a problem report. You can attach as many files as you want provided that each has a unique base name (i.e. the name of the file proper, without the path). Just use the -a command-line option to specify the names of the files you wish to attach: &prompt.user; send-pr -a /var/run/dmesg -a /tmp/errors Don't worry about binary files; they will be automatically encoded so as not to upset your mail agent. If you attach a patch, make sure you use the -c or -u option to &man.diff.1; to create a context or unified diff, and make sure to specify the exact CVS revision numbers of the files you modified so the developers who read your report will be able to apply them easily. You should also take note that unless you explicitly specify otherwise in your PR, any patches you submit will be assumed to be licensed under the same terms as the original file you modified. The template consists of a list of fields, some of which are pre-filled, and some of which have comments explaining their purpose or listing acceptable values. Do not worry about the comments; they will be removed automatically if you do not modify them or remove them yourself. At the top of the template, below the SEND-PR: lines, are the email headers. You do not normally need to modify these, unless you are sending the problem report from a machine or account that can send but not receive mail, in which case you will want to set the From: and Reply-To: to your real email address. You may also want to send yourself (or someone else) a carbon copy of the problem report by adding one or more email addresses to the Cc: header. Next comes a series of single-line fields: Submitter-Id: Do not change this. The default value of current-users is correct, even if you run FreeBSD-STABLE. Originator: This is normally prefilled with the gecos field of the currently logged-in user. Please specify your real name, optionally followed by your email address in angle brackets. Organization: Whatever you feel like. This field is not used for anything significant. Confidential: This is prefilled to no; changing it makes no sense as there is no such thing as a confidential FreeBSD problem report—the PR database is distributed worldwide by CVSup. Synopsis: Fill this out with a short and accurate description of the problem. The synopsis is used as the subject of the problem report email, and is used in problem report listings and summaries; problem reports with obscure synopses tend to get ignored. If your problem report includes a patch, please have the synopsis start with [PATCH]. Severity: One of non-critical, serious or critical. Do not overreact; refrain from labeling your problem critical unless it really is (e.g. root exploit, easily reproducible panic). Developers tend to ignore this and the next field, precisely because problem report submitters tend to overrate their problems. Priority: One of low, medium or high. See above. Category: Choose one of the following: advocacy: problems relating to FreeBSD's public image. Rarely used. alpha: problems specific to the Alpha platform. bin: problems with userland programs in the base system. conf: problems with configuration files, default values etc. docs: problems with man pages or on-line documentation. gnu: problems with GNU software such as &man.gcc.1; or &man.grep.1;. i386: problems specific to the i386 platform. kern: problems with kernel. misc: anything that does not fit in any of the other categories. ports: problems relating to the ports tree. sparc: problems specific to the Sparc platform. Class: Choose one of the following: sw-bug: software bugs. doc-bug: errors in documentation. change-request: requests for additional features or changes in existing features. update: updates to ports or other contributed software. maintainer-update: updates to ports for which you are the maintainer. Release: The version of FreeBSD that you are running. This is filled out automatically by &man.send-pr.1; and need only be changed if you are sending a problem report from a different system than the one that exhibits the problem. Finally, there is a series of multi-line fields: Environment: This should describe, as accurately as possible, the environment in which the problem has been observed. This includes the operating system version, the version of the specific program or file that contains the problem, and any other relevant items such as system configuration, other installed software that influences the problem, etc.—quite simply everything a developer needs to know to reconstruct the environment in which the problem occurs. Description: A complete and accurate description of the problem you are experiencing. Try to avoid speculating about the causes of the problem unless you are certain that you are on the right track, as it may mislead a developer into making incorrect assumptions about the problem. How-To-Repeat: A summary of the actions you need to take to reproduce the problem. Fix: Preferably a patch, or at least a workaround (which not only helps other people with the same problem work around it, but may also help a developer understand the cause for the problem), but if you do not have any firm ideas for either, it's better to leave this field blank than to speculate. Once you are done filling out the template, have saved it, and exit your editor, &man.send-pr.1; will prompt you with s)end, e)dit or a)bort?. You can then hit s to go ahead and submit the problem report, e to restart the editor and make further modifications, or a to abort. If you choose the latter, your problem report will remain on disk (&man.send-pr.1; will tell you the filename before it terminates), so you can edit it at your leisure, or maybe transfer it to a system with better net connectivity, before sending it with the -f to &man.send-pr.1;: &prompt.user; send-pr -f ~/my-problem-report This will read the specified file, validate the contents, strip comments and send it off. Once your problem report has been filed, you will receive a confirmation by email which will include the tracking number that was assigned to your problem report and a URL you can use to check its status. With a little luck, someone will take an interest in your problem and try to address it, or, as the case may be, explain why it is not a problem. You will be automatically notified of any change of status, and you will receive copies of any comments or patches someone may attach to your problem report's audit trail. If someone requests additional information from you, or you remember or discover something you did not mention in the initial report, just mail it to bug-followup@FreeBSD.org, making sure that the tracking number is included in the subject so the bug tracking system will know what problem report to attach it to. If the problem report remains open after the problem has gone away, just send a follow-up (in the manner prescribed above) saying that the problem report can be closed, and, if possible, explaining how or when the problem was fixed. This is a list of resources relevant to the proper writing and processing of problem reports. It is by no means complete. How to Report Bugs Effectively—an excellent essay by Simon G. Tatham on composing useful (non-FreeBSD-specific) problem reports.