{"id":312,"date":"2008-12-09T18:22:54","date_gmt":"2008-12-09T22:22:54","guid":{"rendered":"http:\/\/benjamin.smedbergs.us\/blog\/?p=312"},"modified":"2008-12-09T18:26:17","modified_gmt":"2008-12-09T22:26:17","slug":"laying-blame","status":"publish","type":"post","link":"https:\/\/benjamin.smedbergs.us\/blog\/2008-12-09\/laying-blame\/","title":{"rendered":"Laying Blame"},"content":{"rendered":"<p>As I mentioned <a href=\"http:\/\/benjamin.smedbergs.us\/blog\/2008-12-03\/parsing-compiler-errors\/\">last week<\/a>, I&#8217;ve been resurrecting a project to report on compiler warnings. A basic form of this buildbot is now operational on the <a href=\"http:\/\/tinderbox.mozilla.org\/Firefox\/\">Firefox tinderbox tree<\/a> (look to the far right for the static-analysis-bsmedberg column). It prints a summary of the total number of warnings on the summary page: in the full tinderbox log, it lists each warning and who can be &#8220;blamed&#8221; for that warning:<\/p>\n<pre>\/builds\/static-analysis-buildbot\/slave\/full\/build\/memory\/jemalloc\/jemalloc.c:177:1: warning: C++ style comments are not allowed in ISO C90 blamed on Taras Glek <tglek@mozilla.com> in revision hg:36156fbf817d8a0e2d54a271cf0bff94a1c41c13:memory\/jemalloc\/jemalloc.c\r\n\/builds\/static-analysis-buildbot\/slave\/full\/build\/js\/src\/jsdbgapi.cpp:712: warning: ISO C++ forbids casting between pointer-to-function and pointer-to-object blamed on brendan@mozilla.org in revision cvs:3.36:js\/src\/jsdbgapi.c<\/pre>\n<p>Assigning blame can be a tricky process. In order to figure out the blame for a warning, the code uses the following steps:<\/p>\n<ul>\n<li>Resolve relative paths against the current working directory, using GNU make &#8220;Entering\/Leaving directory&#8221; markers as a guide.\n<li>Dereference symlinks to find the source tree location of an error. For instance, Mozilla headers which produce warnings often do so via paths in dist\/include. We have to resolve these to their original source tree location in order to find blame.\n<li>Using mercurial APIs (through python), find the mercurial changeset which introduced the line in question.\n<li>If the code dates back to Mercurial revision 9b2a99adc05e, which is the original import of CVS code to Mercurial, use a database of CVS blame to find the original CVS checking which was responsible for introducing that line of code.\n<\/ul>\n<p>If you&#8217;re interested, take a look at the <a href=\"http:\/\/hg.mozilla.org\/users\/bsmedberg_mozilla.com\/static-analysis-buildbot\/file\/ef65f2093ed3\/warning-parser.py\">build log parsing code<\/a>, or see the scripts which <a href=\"http:\/\/hg.mozilla.org\/users\/bsmedberg_mozilla.com\/create-cvsblame-db\/\">save CVS blame to a database<\/a> (thanks <a href=\"http:\/\/blog.mozilla.com\/ted\/\">Ted<\/a>!).<\/p>\n<p>The current reporting system for warnings is very primitive. I&#8217;m currently working on a second version of this code which will provide additional features:<\/p>\n<ul>\n<li>Compare warnings with the previous build and highlight &#8220;new&#8221; warnings. I do this by recording the error text and the <em>blamed location<\/em> of the warning. As lines are added and removed from the code, the reported location of the warning changes, but the location of Hg\/CVS blame doesn&#8217;t. This means it is a stable location which can be used for comparisons across runs. It even works across file renames\/moves!\n<li>Web frontend to the warning database to allow users to query warnings by user or directory.\n<li>Classify warnings by &#8220;type&#8221;. This is not a simple process, because GCC mixes distinctive error text, such as &#8220;may be used uninitialized in this function&#8221; with variable names; and the granularity of -fdiagnostic-show-option is low enough that it&#8217;s not very useful by itself. Oh, I wish GCC had error codes like MSVC does: C1234 is easy to recognize!\n<\/ul>\n<p>At one point, I thought I could implement all of the warning mechanism on the buildbot server by parsing the BuildStep logs. It quickly became clear that I couldn&#8217;t do that, because I couldn&#8217;t resolve symlinks, and getting Mercurial blame was difficult or impossible. My new version actually uses a hybrid mechanism where the build log is parsed on the buildbot slave: this parses out warnings, resolves symlinks, and looks up blame. It then sends the results back via stdout to the master. A custom build step on the master parses this log, saves the information to a database, and does the checking for new warnings and prints various results to custom build logs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As I mentioned last week, I&#8217;ve been resurrecting a project to report on compiler warnings. A basic form of this buildbot is now operational on the Firefox tinderbox tree (look to the far right for the static-analysis-bsmedberg column). It prints a summary of the total number of warnings on the summary page: in the full [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,6],"tags":[30,82,17,41,183],"class_list":["post-312","post","type-post","status-publish","format-standard","hentry","category-mozilla","category-untagged","tag-build-system","tag-buildbot","tag-gcc","tag-mercurial","tag-warnings"],"_links":{"self":[{"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/posts\/312","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/comments?post=312"}],"version-history":[{"count":2,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/posts\/312\/revisions"}],"predecessor-version":[{"id":314,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/posts\/312\/revisions\/314"}],"wp:attachment":[{"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/media?parent=312"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/categories?post=312"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/benjamin.smedbergs.us\/blog\/wp-json\/wp\/v2\/tags?post=312"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}