<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>erik dörnenburg</title>
	<atom:link href="http://erik.doernenburg.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://erik.doernenburg.com</link>
	<description>Just another WordPress weblog</description>
	<lastBuildDate>Wed, 18 Nov 2009 01:22:29 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.1</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Making build pain visible</title>
		<link>http://erik.doernenburg.com/2009/11/making-build-pain-visible/</link>
		<comments>http://erik.doernenburg.com/2009/11/making-build-pain-visible/#comments</comments>
		<pubDate>Tue, 03 Nov 2009 02:28:04 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=334</guid>
		<description><![CDATA[The practice of continuous integration is gaining widespread adoption and almost every project I was involved in over the past few years used a continuous integration server to maintain an up-to-date view on the status of the build. Developers can look at the status page of the server or use tools such as CCTray and [...]]]></description>
			<content:encoded><![CDATA[<p>The practice of continuous integration is gaining widespread adoption and almost every project I was involved in over the past few years used a continuous integration server to maintain an up-to-date view on the status of the build. Developers can look at the status page of the server or use tools such as CCTray and CCMenu to find out whether a recent check-in has broken the build. Some teams also use build lights, like <a href="http://www.delcomproducts.com/products_USBLMP.asp">these</a> for example, or other information radiators to make the status of the build visible.</p>
<p>The reason why developers need an up-to-date build status is a common, and good, practice: new check-ins are only allowed when the build is known to be good. If it is broken chances are that someone is trying to fix it and dumping a whole new set of changes onto them would undoubtedly make that task harder. Similarly, when the server is building nobody knows for sure whether the build will succeed, and checking in changes would make fixing the build harder, should it fail.</p>
<p>To recap: the build must be good for a developer to be able to check in. On one of our projects this was becoming a rare occurrence, though. In fairness, the build performed fairly comprehensive checks in a complex integration environment, involving an ESB and an SSO solution. The team had already relegated some long-running tests to a different build stage, and they had split the <i>short build</i>, ie. the build that determines whether check ins are allowed, into five parallel builds, bringing build time down from over 45 to under ten minutes. Still, developers often found themselves waiting in a queue, maintained with post-its on a wall, for a chance to check in their changes. Not only that but everybody felt the situation was getting worse, that the build was broken more often. This was obviously a huge waste and I was keen to make it visible to management using a visualisation.</p>
<p><span id="more-334"></span></p>
<h4>Buildlines visualisation</h4>
<p>After some experimentation I decided on a variation of <a href="http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001OR">spark lines</a> for the build status. Each of the builds gets its own line and the colour shows the status of the build at any given point in time, green for good, red for broken, and blue for building. I blanked out the weekends and stretched the time during the days so that  only the hours between 8am and 7pm are visible. The resulting visualisation looked like this: (Click through for the full-size version.)</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2009/11/buildlines.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2009/11/buildlines-600x65.png" alt="" title="Buildlines" width="600" height="65" class="alignnone size-medium wp-image-336" /></a></p>
<p>The zoomed out version clearly shows that matters were getting worse. The first two weeks in this diagram only show a few broken builds that, with the exception of one episode on build number 4 on Aug 11/12, get fixed relatively quickly. Looking at the last week and a half the picture changes quite dramatically:</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2009/11/buildlines-end.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2009/11/buildlines-end-600x343.png" alt="Buildlines (last week)" title="Buildlines (last week)" width="600" height="343" class="aligncenter size-medium wp-image-345" /></a></p>
<p>On Monday, Sep 7, build 1 is broken for a while, which can happen. Later in the day build 2 breaks and is only fixed early on Tuesday. Again, this alone would not concern me, and while it is generally not such a great idea to leave a broken build behind, sometimes it is important to go home and get some rest.</p>
<p>At this point I should probably clarify that for a developer to be able to check in all the builds in this diagram need to be green. Each test in each build of this stage is significant. Long running, brittle tests, which include end-to-end integration tests with a mainframe system, have been moved to a different build stage and are treated separately.</p>
<p>A real reason for concern is that that around noon on Sep 8, all builds break at the same time. This is unlikely to be the result of a code change, because for a single change to break all builds it would have had to affect at least one test in each of the builds. Possible but not likely. A more reasonable explanation for this failure is a problem with the environment, maybe a database that is not responding or an SSO server that cannot authenticate the test users any longer. Similar problems can be seen Sep 14 during the morning, and Sep 15 in the afternoon. They are not completely new, either, as looking at Aug 28 reveals.</p>
<p>Environment problems like these can be extremely frustrating for a development team because a build that is broken not because of a code change but because of a problem with the  environment leaves the developers in an awkward position. They can either wait until the environment is fixed, but that often relies on a separate team that may have different priorities, or they can continue to check in based on the assumption that the build isn&#8217;t <em>really</em> broken. The latter is, of course, playing with fire as the team now effectively works without continuous integration.</p>
<p>Visualisations such as this one can help management get clarity on environment problems, and hopefully support a case for improving the build environment.</p>
<h4>The buildines script</h4>
<p>Unlike the visualisations I wrote about on this blog so far, this time I needed to write a bit more code as I could not find a tool to draw the spark lines for me. The data acquisition step was easy, though, because the team was using the <a href="http://studios.thoughtworks.com/cruise-release-management">Cruise</a> continuous integration server, which allowed me to get information about past builds through a web-based API. I used the cURL command, which, by the way, is available for Windows, too.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">curl <span style="color: #660033;">-o</span> Shortbuild1.csv <span style="color: #ff0000;">&quot;http://servername:8153/cruise/properties/search?pipelineName=WebFrontEnd-Dev&amp;stageName=Shortbuild&amp;jobName=Shortbuild1&amp;limitCount=1000&quot;</span>
curl <span style="color: #660033;">-o</span> Shortbuild2.csv <span style="color: #ff0000;">&quot;http://servername:8153/cruise/properties/search?pipelineName=WebFrontEnd-Dev&amp;stageName=Shortbuild&amp;jobName=Shortbuild2&amp;limitCount=1000&quot;</span>
...</pre></div></div>

<p>Normally I&#8217;m advocating to split the data processing stage from the actual visualisation but in this case, because I was going to write the visualisation from scratch and the data coming from Cruise was in a pretty good format already, I decided to put everything into a single script. For that I chose Ruby and <a href="http://rmagick.rubyforge.org/">RMagick</a>. The latter can be a pain to set up but there are installers for Windows, on the Mac it can be installed using MacPorts, and presumably the Linux package management systems include it, too.</p>
<p>The script is available from this <a href="http://bitbucket.org/erikdoe/buildlines/">Bitbucket repository</a>. It is relatively straight-forward and I am not going to examine it in great detail in this post. What is noteworthy is that, internally, the script shows a separation of concerns, with one class reading the Cruise files, a second class scaling and drawing the lines and labels, and a main class that holds everything together. If you were to adapt this to a different continuous integration server it should be possible to achieve this by simply writing a different DataFile class.</p>
<p>To run the script simply pass the list of build status files as command line parameters to it:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">ruby buildlines.rb Shortbuild<span style="color: #000000; font-weight: bold;">*</span>.csv</pre></div></div>

<p>The output will be written into a file named <i>buildlines.png</i> in the working directory. If you prefer a different format you can change the extension of the filename in the script, and RMagick, provided it supports the format, will magically write the corresponding format.</p>
<p><b>The visualisation script can be downloaded from <a href="http://bitbucket.org/erikdoe/buildlines/downloads/">Bitbucket</a>.  Simply select one of the zipped versions of the &#8220;tip&#8221; snapshot.</b></p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/11/making-build-pain-visible/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>The language of SOA</title>
		<link>http://erik.doernenburg.com/2009/10/the-language-of-soa/</link>
		<comments>http://erik.doernenburg.com/2009/10/the-language-of-soa/#comments</comments>
		<pubDate>Fri, 23 Oct 2009 06:17:52 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Architecture]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=323</guid>
		<description><![CDATA[To begin with an antipodean metaphor, using a service-oriented architecture, when it works out, can be as elegant and exhilarating as surfing on a 10 meter wave, exploiting huge uncontrollable forces to move forward at great speed. However, unfortunately, and maybe much like in the surfing world, the SOA wave often mercilessly rolls over a [...]]]></description>
			<content:encoded><![CDATA[<p>To begin with an antipodean metaphor, using a service-oriented architecture, when it works out, can be as elegant and exhilarating as surfing on a 10 meter wave, exploiting huge uncontrollable forces to move forward at great speed. However, unfortunately, and maybe much like in the surfing world, the SOA wave often mercilessly rolls over a hapless development team, leaving them confused and wondering whether to give up on the idea or to paddle out again.</p>
<p>Despite its complexity a service-oriented architecture is high on almost every project&#8217;s wish list. I have been involved in eight projects over the last year, sometimes directly delivering working software, sometimes reviewing or advising, and out of these eight projects six were, according to the respective architects, building a system with a service-oriented architecture. On these projects I have come across two recurring uses of language around SOA that I found noteworthy.</p>
<p>Firstly, SOA used to, and still should, stand for an architecture that is service-oriented, with &#8220;architecture&#8221; being the noun. What I have found, though, is that more and more often people use &#8220;SOA&#8221; as an adjective and then add it to a more or less random noun, such as &#8220;approach&#8221; or &#8220;model&#8221; or even &#8220;architecture&#8221; again. Yes, I have heard people talk about an SOA Architecuture more than once.</p>
<p>This use of language highlights a problem. Sticking the three letters &#8220;SOA&#8221; onto a random noun, without pronouncing what the letters stand for, makes it easy to forget what this was all about in the first place. In fact, it seems that &#8220;SOA&#8221; is now put in front of words such as &#8220;approach&#8221; simply to make something sound contemporary; as in: &#8220;How&#8217;s your project?&#8221; &#8212; &#8220;Oh, we follow an SOA approach.&#8221; Given my experience, chances are that the project&#8217;s architecture is not service-oriented. It might use WS-* standards, though.</p>
<p>Secondly, there are people who are referred to as SOA professionals or similar. It usually sounds like it&#8217;s their job to build SOAs. But what does that really mean? Shouldn&#8217;t developers built software according to a service-oriented architecture? Are the SOA professionals merely advising teams on how to build service-oriented architectures correctly?</p>
<p>In my experience the SOA professionals have a different agenda, and that is to sell SOA. Unfortunately, in the words of my colleague <a href="http://jim.webber.name/">Jim Webber</a>, there are two things money can&#8217;t buy: love and an SOA. Again, using the three letters one might think &#8220;why not?&#8221; but spelled out, would anybody argue you can buy a service-oriented architecture? Or any architecture for that matter? So, what is really being sold is usually a set of products, software and/or hardware, that may or may not help a team implement a service-oriented architecture. </p>
<p>The key problem in this case is that when you &#8220;buy SOA&#8221; the success criteria shift. It seems that success is defined based on whether something that can be labelled &#8220;SOA&#8221; is in place, and not whether actual business value is delivered. What is often completely forgotten is an evaluation of the suitability of the architecture, but that&#8217;s a topic for another post.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/10/the-language-of-soa/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>OCMock 1.55 released</title>
		<link>http://erik.doernenburg.com/2009/10/ocmock-1-55-released/</link>
		<comments>http://erik.doernenburg.com/2009/10/ocmock-1-55-released/#comments</comments>
		<pubDate>Tue, 20 Oct 2009 06:03:04 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Releases]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=319</guid>
		<description><![CDATA[The main features of this release are partial mocks and method swizzling. Sometimes it&#8217;s just easier to use a real object rather than setting up a complex mock from scratch, but often in such cases there is at least one method on the real object that has undesirable side effects, or a method returning a [...]]]></description>
			<content:encoded><![CDATA[<p>The main features of this release are partial mocks and method swizzling. Sometimes it&#8217;s just easier to use a real object rather than setting up a complex mock from scratch, but often in such cases there is at least one method on the real object that has undesirable side effects, or a method returning a value that we would like to change for a test. With the new features in OCMock it is now possible to selectively replace individual methods on existing objects. Did I mention that I love the Objective-C runtime?</p>
<p>As usual the release also includes many contributions and bug-fixes from the community. More details on the <a href="http://www.mulle-kybernetik.com/software/OCMock/">OCMock page</a> at Mulle Kybernetik.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/10/ocmock-1-55-released/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Moose MSE for Java and C#</title>
		<link>http://erik.doernenburg.com/2009/07/moose-mse-for-java-and-cs/</link>
		<comments>http://erik.doernenburg.com/2009/07/moose-mse-for-java-and-cs/#comments</comments>
		<pubDate>Fri, 10 Jul 2009 01:58:00 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Releases]]></category>
		<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=276</guid>
		<description><![CDATA[Over the past years I have shown everyone who could not run fast enough some of the tools based on Moose. And even now I cannot resist putting a screenshot of CodeCity into this post.

Most of the Moose tools now use the MSE file format as an interchange format. By the way, if you are [...]]]></description>
			<content:encoded><![CDATA[<p>Over the past years I have shown everyone who could not run fast enough some of the tools based on <a href="http://moose.unibe.ch/">Moose</a>. And even now I cannot resist putting a screenshot of <a href="http://www.inf.unisi.ch/phd/wettel/codecity.html">CodeCity</a> into this post.</p>
<p><a href="http://www.inf.unisi.ch/phd/wettel/codecity-wof.html"><img src="http://erik.doernenburg.com/wp-content/uploads/2009/07/azureus-2500-coarse-cropped.png" alt="Part of the Azureus city" title="Part of the Azureus city" width="244" height="132" class="alignright size-full wp-image-288" /></a></p>
<p>Most of the Moose tools now use the <a href="http://moose.unibe.ch/docs/mse">MSE file format</a> as an interchange format. By the way, if you are interested in writing your own visualisations or analysis tools it is probably worthwhile looking at MSE, reading this format is so much more convenient than parsing source code.</p>
<p>In <strong>Java</strong> it was always relatively easy to create MSE files. Among many other things, <a href="http://loose.upt.ro/iplasma/">iPlasma</a> can read Java source code and export to MSE. That said, iPlasma has so many interesting features itself that oftentimes no export to an external tool is necessary.</p>
<p>For <strong>C#</strong> the story was different and for one reason or another no tool existed that could create MSE files for C#. This has changed now. As a student project at the University of Lugano such a tool was written and, thanks to <a href="http://www.inf.unisi.ch/faculty/lanza/">Michele Lanza</a>, then donated for general use. I&#8217;ve made a few improvements and put the code into this <a href="http://bitbucket.org/erikdoe/pmcs/">Bitbucket repository</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/07/moose-mse-for-java-and-cs/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Making ESB pain visible</title>
		<link>http://erik.doernenburg.com/2009/07/making-esb-pain-visible/</link>
		<comments>http://erik.doernenburg.com/2009/07/making-esb-pain-visible/#comments</comments>
		<pubDate>Wed, 01 Jul 2009 05:40:39 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Architecture]]></category>
		<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=236</guid>
		<description><![CDATA[At some point last year I was asked to review the architecture of the software behind a large and popular website. The resident architect explained how he had followed a modern approach, decoupling the web front-end from back-end services that provide  content. To gain further flexibility he had put the front-end and the services [...]]]></description>
			<content:encoded><![CDATA[<p>At some point last year I was asked to review the architecture of the software behind a large and popular website. The resident architect explained how he had followed a modern approach, decoupling the web front-end from back-end services that provide  content. To gain further flexibility he had put the front-end and the services on an ESB, mostly to cater for the possibility to make the content available to other consumers. In short, the architecture diagram looked a lot like many others: nothing to see here, move on.</p>
<p><img src="http://erik.doernenburg.com/wp-content/uploads/2009/06/high-level-esb.png" alt="Website Architecture" title="Website Architecture" width="320" height="212" class="alignnone size-full wp-image-237" /></p>
<p>The diagram above only shows one of the content services, which for the sake of this article is a service that provides contact details for a person.</p>
<p>Based on conversations with the project sponsors I began to suspect that at least the introduction of the ESB was a case of RDD, ie. Resume-Driven Development, development in which key choices are made with only one question in mind: how good does it look on my CV? Talking to the developers I learned that the ESB had introduced &#8220;nothing but pain.&#8221; But how could something as simple as the architecture in the above diagram cause such pain to the developers? Was this really another case of architect&#8217;s dream, developer&#8217;s nightmare?  </p>
<p><span id="more-236"></span></p>
<p>Often a picture says more than a thousand words, but at the same time traditional architecture diagrams simply do not provide enough information. (I&#8217;ve argued this before, <a href="http://97-things.near-time.net/wiki/show/Get%20the%201000ft%20view">here</a> and <a href="http://www.infoq.com/interviews/erik-doernenburg-software-visualization">here</a> for example.) The answer to my questions, and the anxious project sponsors&#8217; question whether the project would deliver on time, lay in a different picture.</p>
<p>The picture should not only show the layering of the system in more detail than the architecture diagram but it should also visualise the complexity introduced by the ESB. It&#8217;s often possible to produce more detailed diagrams automatically from source code, configuration, or runtime inspection but in this case I couldn&#8217;t find a way to do that, which is which I drew the following:</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2009/06/low-level-esb.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2009/06/low-level-esb-600x533.png" alt="" title="Detailed ESB diagram" width="600" height="533" class="alignnone size-medium wp-image-237" /></a></p>
<p>In the centre column this diagram shows the layers involved, from the Struts action that needs to look up details by an ID down to the back-end that provides a &#8220;RESTful&#8221; service to retrieve the details. The top box contains the front-end layers while the box in the middle represents the ESB. The layers labelled in italics are frameworks or infrastructure software, the rest is project-specific code.</p>
<p>Key to this diagram are obviously the coloured representations of data travelling up and down the stack. Primitive types are grey, Java objects are purple circles, XML documents are shown as turquoise hexagons, and HTTP messages as green blobs. Mule messages, which are Java objects, are shown in dark purple. If one representation is wrapped in another it&#8217;s displayed inside the wrapper. </p>
<p>The diagram highlights that it&#8217;s a simple piece of data that travels all the way down the stack (actually further until it hits a relational database at some point) and, equally, that the data needed by the action is available in the right form from the service. All that the intermediate layers are doing is wrapping and unwrapping the data in various ways. In fact, out of the four layers of project-specific code three layers do nothing but wrapping. No wonder the developers felt some pain.</p>
<p>As a side note: the diagram also makes it obvious that the wrapping is asymmetric between the WsToRestHandler and the RestServiceWrapperImpl, which makes the point of this layer boundary even more questionable.</p>
<p>This diagram made people more interested in the cost of what had just been a simple, innocuous line in the architecture diagram; which was the point of the exercise. So, what would <em>this</em> diagram look like without the ESB? Easy:</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2009/06/Low-Level-ESB2.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2009/06/Low-Level-ESB2-600x305.png" alt="" title="Without ESB" width="600" height="305" class="alignnone size-medium wp-image-237" /></a></p>
<p>Apart for the obvious simplicity the diagram highlights that every layer has a purpose and there is only one wrapping, which is not undone either. Would it surprise you if I told you that the project went live without the ESB?</p>
<h3>Footnotes</h3>
<p>1) This article is by no means implying that any of the mentioned technologies are useless or harmful. Their use was harmful in this case.</p>
<p>2) The pattern of wrapping and unwrapping the same data through multiple layers is not new, it was also quite common in early 3-tier architectures. (In fact, some people called their architectures n-tier; I guess to show that the number of tiers was somewhat arbitrary but large.) We had a term to describe this pattern: the wormhole pattern.</p>
<p>3) This project is not an isolated incident of a Wormhole ESB. I have personally seen several in the past years.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/07/making-esb-pain-visible/feed/</wfw:commentRss>
		<slash:comments>16</slash:comments>
		</item>
		<item>
		<title>97 Things Every Software Architect Should Know</title>
		<link>http://erik.doernenburg.com/2009/03/97-things/</link>
		<comments>http://erik.doernenburg.com/2009/03/97-things/#comments</comments>
		<pubDate>Thu, 05 Mar 2009 15:48:41 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Architecture]]></category>
		<category><![CDATA[Clippings]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=211</guid>
		<description><![CDATA[ Admittedly, I&#8217;ve been struggling with the &#8220;Architect&#8221; title in the IT world. It is not that I think there&#8217;s no role for architecture, far from it, but too often I&#8217;ve encountered architects who focus too narrowly on architecture, losing track of the realities of actual software development and the context in which the software [...]]]></description>
			<content:encoded><![CDATA[<p><img src="http://erik.doernenburg.com/wp-content/uploads/2009/03/97thingscover-100x150.jpg" alt="97 Things..." title="97 Things..." width="100" height="150" class="alignleft size-thumbnail wp-image-220" /> Admittedly, I&#8217;ve been struggling with the &#8220;Architect&#8221; title in the IT world. It is not that I think there&#8217;s no role for architecture, far from it, but too often I&#8217;ve encountered architects who focus too narrowly on architecture, losing track of the realities of actual software development and the context in which the software will be used. I wonder, if there was no &#8220;Architect&#8221; title and people who are responsible for architecture would be called guide or coach or tech lead, or just <em>the senior developer</em>, whether things would be better.</p>
<p>About a year ago, in a discussion about architects, rather than trying to define what an architect is or does, we looked at what he or she should know. We expressed our ideas as mini-essays, strictly limiting ourselves to one per essay, and it turned out that, at least to my surprise, there was a lot of agreement; maybe because we hadn&#8217;t come up with hard and fast rules but with ideas and guidelines.</p>
<p>Luckily Richard Monson-Haefel was part of that discussion and he had the resolve and means to make our thoughts more widely available. Our list of the 97 things every architect should know was collected and refined on <a href="http://97-things.near-time.net/wiki">this wiki</a>, and is available under a creative commons license. For a more convenient read it has now also been published as a <a href="http://oreilly.com/catalog/9780596522698/preview.html">book by O&#8217;Reilly</a>. As expected, <a href="http://www.theserverside.com/news/thread.tss?thread_id=53819">the discussions have begun</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/03/97-things/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>JAOO Nights</title>
		<link>http://erik.doernenburg.com/2009/03/jaoo-nights/</link>
		<comments>http://erik.doernenburg.com/2009/03/jaoo-nights/#comments</comments>
		<pubDate>Thu, 05 Mar 2009 06:56:38 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Conferences]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=202</guid>
		<description><![CDATA[The JAOO conference is coming to Australia again, and I think it will be the best technical conference in Australia this year. (I admit that I&#8217;m somewhat biased as a member of the programme committee.) In the lead up to the conference we have organised what we call JAOO Nights in Brisbane, Melbourne, and Sydney. [...]]]></description>
			<content:encoded><![CDATA[<p><img src="http://erik.doernenburg.com/wp-content/uploads/2009/03/speaking_at_jaoo-150x102.png" alt="speaking_at_jaoo" title="speaking_at_jaoo" width="150" height="102" class="alignright size-thumbnail wp-image-225" />The <a href="http://jaoo.dk/">JAOO conference</a> is coming to Australia again, and I think it will be the best technical conference in Australia this year. (I admit that I&#8217;m somewhat biased as a member of the programme committee.) In the lead up to the conference we have organised what we call JAOO Nights in Brisbane, Melbourne, and Sydney. These are free events, starting at 5pm with two speakers presenting on topics that give a flavour of what the JAOO conference itself will be like.</p>
<p>Check out the <a href="http://www.jaoo.com.au/?cid=435752">JAOO Australia site</a> for information on the sessions, and sign up for your city if your interested!</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2009/03/jaoo-nights/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>How toxic is your code?</title>
		<link>http://erik.doernenburg.com/2008/11/how-toxic-is-your-code/</link>
		<comments>http://erik.doernenburg.com/2008/11/how-toxic-is-your-code/#comments</comments>
		<pubDate>Fri, 21 Nov 2008 00:54:46 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=133</guid>
		<description><![CDATA[If you are somebody who writes code you probably know that moment when you look at some code you didn&#8217;t write, or some code you wrote a long time ago, and you think &#8220;that doesn&#8217;t look good.&#8221; Ok, more realistically, you probably think &#8220;WTF? I wouldn&#8217;t want to touch that with a barge-pole!&#8221; It is [...]]]></description>
			<content:encoded><![CDATA[<p>If you are somebody who writes code you probably know that moment when you look at some code you didn&#8217;t write, or some code you wrote a long time ago, and you think &#8220;that doesn&#8217;t look good.&#8221; Ok, more realistically, you probably think &#8220;WTF? I wouldn&#8217;t want to touch that with a barge-pole!&#8221; It is not even so much about whether the code does what it should do&#8212;that takes a bit longer to figure out&#8212;or whether the code is too slow. Even if it&#8217;s perfectly bug free and performs well, there&#8217;s something to the way it&#8217;s written. This is part of the internal quality of a software system, something that the users and development managers can&#8217;t observe directly; yet, it still affects them because code with poor internal quality is hard to maintain and extend.</p>
<p>Now, as a developer, how do you help managers and business people understand the internal quality of code? They generally want a bit more than &#8220;it&#8217;s horrible&#8221; before they prioritise cleaning up the code over implementing new features that directly deliver business value. Or even: how do you figure out for yourself how bad some code actually is in relation to some other code? These were questions that <a href="http://jroller.com/zoonabar/">Chris Brown</a>, <a href="http://darrenhobbs.com/">Darren Hobbs</a>, and myself were asking ourselves a couple of years ago.</p>
<p>The answer came in the form of a simple bar chart, arguably not the most sophisticated visualisation but a very effective one. And our colleague <a href="http://www.alphaitjournal.com/">Ross Pettit</a> had the perfect name for it: The <strong>Toxicity Chart</strong>. Read on to see what it is and how it&#8217;s created.</p>
<p><span id="more-133"></span></p>
<p>As usual, let&#8217;s start with an example. What follows is the toxicity chart for a version of the Hibernate framework.</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2008/11/toxicity_hibernate.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2008/11/toxicity_hibernate-600x331.png" alt="Toxicity Chart for Hibernate" title="toxicity_hibernate" width="600" height="331" class="size-medium wp-image-143" /></a></p>
<p>In a toxicity chart each bar represents a class and the height of the bar shows the toxicity score for that class. The score is based on several metrics (more on that below) and the higher the score the more toxic the class. The individual components of the score are coloured. For example, the contribution of the method length metric to the overall score is shown in orange. This makes it possible to see at a glance not only how toxic a codebase is but also how the problems are distributed. If there is a lot of purple and orange in the chart, this indicates long and complex methods, which means that the code is probably hard to test on a unit level. Lastly, classes that score zero points are left off the chart.</p>
<p>The calculation of the toxicity score is based on metrics and thresholds. For example, the method length metric has a threshold of 30. If a class contains a method that is longer it gets points proportional to the length of the method in relation to the threshold, e.g. a method with 45 lines of code would get 1.5 points because its length is 1.5 times the threshold. The score for all elements is added up. So, if a class comprises two methods, one that is 45 lines and another that is 60 lines long, the method length component of the score for the class will be 3.5 points. This means that elements are not just classified as toxic but the score reflects how toxic the element is.</p>
<p>The following table shows the metrics that make up the toxicity score and the corresponding base threshold on which the multipliers are based.</p>
<table border="0" cellpadding="2">
<tr>
<th>Metric</th>
<th>Level</th>
<th>Threshold</th>
</tr>
<tr>
<td>File Length</td>
<td>file</td>
<td>500</td>
</tr>
<tr>
<td>Class Fan-Out Complexity</td>
<td>class</td>
<td>30</td>
</tr>
<tr>
<td>Class Data Abstraction Coupling</td>
<td>class</td>
<td>10</td>
</tr>
<tr>
<td>Anon Inner Length</td>
<td>inner class</td>
<td>35</td>
</tr>
<tr>
<td>Method Length</td>
<td>method</td>
<td>30</td>
</tr>
<tr>
<td>Parameter Number</td>
<td>method</td>
<td>6</td>
</tr>
<tr>
<td>Cyclomatic Complexity</td>
<td>method</td>
<td>10</td>
</tr>
<tr>
<td>Nested If Depth</td>
<td>statement</td>
<td>3</td>
</tr>
<tr>
<td>Nested Try Depth</td>
<td>statement</td>
<td>2</td>
</tr>
<tr>
<td>Boolean Expression Complexity</td>
<td>statement</td>
<td>3</td>
</tr>
<tr>
<td>Missing Switch Default</td>
<td>statement</td>
<td>1</td>
</tr>
</table>
<p>At this point you might wonder where the selection of metrics and thresholds comes from. Well, we made them up. When we designed the toxicity chart we made a call on what constitutes &#8220;toxic&#8221; as opposed to just bad. Of course, staying with the method length metric, we normally wouldn&#8217;t want to see a 15-line method but that might be disputed. However, we hope that nobody thinks that a 30-line method is acceptable. And in case you&#8217;re really uncomfortable with the thresholds, you can obviously change them to build your own toxicity score. We do suggest, though, that you try the values presented here first.</p>
<p>Like many visualisations the creation of the toxicity chart falls into two steps: data acquisition and rendering.</p>
<p><strong>Step 1:</strong> For Java projects <a href="http://checkstyle.sourceforge.net/">Checkstyle</a> is the easiest way to get the metrics. The  score table above easily translates into a Checkstyle configuration file, which, by the way, is included in the Zip archive attached to this page. Assuming this file is named &#8220;metrics.xml&#8221; Checkstyle can be invoked like this:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">java <span style="color: #660033;">-jar</span> checkstyle.jar <span style="color: #660033;">-c</span> metrics.xml <span style="color: #660033;">-r</span> <span style="color: #000000; font-weight: bold;">&lt;</span>source_dir<span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #660033;">-f</span> xml <span style="color: #660033;">-o</span> toxicity.xml</pre></div></div>

<p>Afterwards, the file &#8220;toxicity.xml&#8221; contains a list of all components for the toxicity score in an XML format.</p>
<p>Similar tools for other languages should be able to create similar output. It is not a huge problem if the output is somewhat different as it&#8217;s always possible to change step 2 to handle different input formats.</p>
<p><strong>Step 2:</strong> What is required next is to read the data from XML, aggregate it on a per-class level, and then render the bar chart. Microsoft Excel is good at the latter two but it needs a bit of help to read the XML file. This help comes in the form of a small piece of VBA code. The attached Excel workbook contains a sheet to load the data, an &#8220;Open XML&#8221; button backed by said VBA code, a pivot table to do the aggregation, and the chart based on the table. So, step 2, really is to open the Excel workbook and load the XML file. That&#8217;s all.</p>
<p>Now, if you know me you know that I&#8217;ve been working with Macs for a very long time. So, naturally, I&#8217;d like to use Excel on the Mac but, alas, in its current version it <a href="http://www.schwieb.com/blog/2006/08/08/saying-goodbye-to-visual-basic/">does not support VBA</a> anymore. So, unfortunately, it&#8217;s VMWare Fusion for this one.</p>
<p><strong>This <a href="http://erik.doernenburg.com/wp-content/uploads/2008/11/toxicitychart.zip">Zip archive</a> contains everything needed to create a toxicity chart for Java. The spreadsheet can be adapted to read input file formats created by tools for different languages; just edit the VBA behind the &#8220;Data&#8221; sheet.</strong></p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2008/11/how-toxic-is-your-code/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>Call Graph Visualisation with AspectJ and Dot</title>
		<link>http://erik.doernenburg.com/2008/09/call-graph-visualisation-with-aspectj-and-dot/</link>
		<comments>http://erik.doernenburg.com/2008/09/call-graph-visualisation-with-aspectj-and-dot/#comments</comments>
		<pubDate>Sat, 27 Sep 2008 17:26:28 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Coding]]></category>
		<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=101</guid>
		<description><![CDATA[One of my favourite tools to render graphs is GraphViz Dot and in an earlier entry I described how to use it to visualise Spring contexts. Today I want to showcase a different application.
Call graphs show how methods call each other, which can be useful for a variety of reasons. The example I use here [...]]]></description>
			<content:encoded><![CDATA[<p>One of my favourite tools to render graphs is <a href="http://www.graphviz.org/">GraphViz Dot</a> and in an <a href="http://erik.doernenburg.com/2008/08/springviz/">earlier entry</a> I described how to use it to visualise Spring contexts. Today I want to showcase a different application.</p>
<p>Call graphs show how methods call each other, which can be useful for a variety of reasons. The example I use here is the graph rooted in a unit test suite, and in this case the graph gives an understanding of how localised the unit tests are, how much they are real unit tests or how close they are to mini-integration tests. In an ideal case the test method should call the method under test and nothing else. However, even with mock objects that&#8217;s not always practical. And if, like myself, you fall into the classicist camp of unit testers, as described by Martin Fowler in <a href="http://martinfowler.com/articles/mocksArentStubs.html">Mocks aren&#8217;t Stubs</a>, you might actually not be too fussed about a few objects being involved in a single test. In either case, looking at the call graph shows you exactly which methods are covered by which unit tests.</p>
<p>There are several ways to generate calls graphs and I&#8217;m opting for dynamic analysis, which simply records the call graph while the code is being executed. A good theoretical reason is that dynamic analysis can handle polymorphism but a more practical reason is that it&#8217;s actually really easy to do dynamic analysis; provided you use the right tools. The approach I describe in this article uses <a href="http://www.eclipse.org/ajdt/">Eclipse AJDT</a> to run the unit tests with a simple Java aspect that records the call graph and writes it out into a format that can be rendered more or less directly with Dot. Of course, this technique is not limited to creating graphs for unit test; it only depends on weaving an AspectJ aspect into a Java application.</p>
<p><span id="more-101"></span></p>
<p>Let&#8217;s start with the result. The following diagram shows the call graph for a few simple unit tests in the test suite for the CruiseControl dashboard. The tests methods are in the leftmost column and method invocations occur from left to right. It&#8217;s clearly visible that the tests are quite localised.</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_simple.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_simple-600x250.png" alt="" title="CC Test Suite - Normal Call Graph" width="600" height="250" class="alignnone size-medium wp-image-113" /></a></p>
<p>The next example shows a section that&#8217;s a bit more intertwined. Some tests exercise the same method, some tests call more than one method, and some methods are called indirectly through different paths. All in all pretty reasonable, though.</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_normal.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_normal-600x138.png" alt="" title="CC Test Suite - Normal Call Graph" width="600" height="138" class="alignnone size-medium wp-image-110" /></a></p>
<p>Now, I don&#8217;t want to showcase only the good parts. There are also sections in the graph that show how some of the test and dependencies are, well, a bit messy. In fairness, though, this is partly caused by some domain objets that are used across multiple tests.</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_knot.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2008/09/cc_calls_knot-600x178.png" alt="" title="CC Test Suite - The Knot" width="600" height="178" class="alignnone size-medium wp-image-114" /></a></p>
<p>How hard can it be to create these graphs? Not very, as it turns out.</p>
<p><strong>Step 1</strong> is to create an aspect that intercepts all interesting method calls, which normally means calls to methods that are part of the project. It&#8217;s also possible, though, to include calls to libraries and frameworks. Depending on your experience with AspectJ this is as simple as follows:</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;"><span style="color: #000000; font-weight: bold;">package</span> <span style="color: #006699;">com.example.callgraph</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">public</span> aspect CallInterceptor
<span style="color: #009900;">&#123;</span>
    pointcut allInterestingMethods<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">:</span>
        <span style="color: #339933;">!</span>within<span style="color: #009900;">&#40;</span>CallInterceptor<span style="color: #009900;">&#41;</span> <span style="color: #339933;">&amp;&amp;</span> <span style="color: #339933;">!</span>within<span style="color: #009900;">&#40;</span>CallLogger<span style="color: #009900;">&#41;</span> <span style="color: #339933;">&amp;&amp;</span>
          execution<span style="color: #009900;">&#40;</span><span style="color: #000000; font-weight: bold;">public</span> <span style="color: #339933;">*</span> com.<span style="color: #006633;">example</span>..<span style="color: #339933;">*</span><span style="color: #009900;">&#40;</span>..<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
    before<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">:</span> allInterestingMethods<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        CallLogger.<span style="color: #006633;">INSTANCE</span>.<span style="color: #006633;">pushMethod</span><span style="color: #009900;">&#40;</span>thisJoinPointStaticPart.<span style="color: #006633;">getSignature</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        CallLogger.<span style="color: #006633;">INSTANCE</span>.<span style="color: #006633;">logCall</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    after<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">:</span> allInterestingMethods<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        CallLogger.<span style="color: #006633;">INSTANCE</span>.<span style="color: #006633;">popMethod</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>The first part of the aspect is the definition of the pointcut: it matches all public methods in &#8220;com.example&#8221; and sub-packages but excludes calls to the two classes that form part of the call graph logger itself.</p>
<p>The <em>before</em> advice, which is run before each method that matches the pointcut, places the signature of the current method on a stack maintained by the graph logger class. With the current method signature as the topmost element and the one that called it as the second from the top, the advice asks the logger to log the corresponding call. The <em>after</em> advice simply pops the topmost call from the stack.</p>
<p>This is almost all the excitement. The implementation of <code>CallLogger</code> deals with maintaining the stack and writing out the call in the format required by Dot. The complete implementation with some comments follows below.</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;"><span style="color: #000000; font-weight: bold;">package</span> <span style="color: #006699;">com.example.callgraph</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.io.BufferedWriter</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.io.FileWriter</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.io.IOException</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.io.Writer</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.util.Stack</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.util.HashSet</span><span style="color: #339933;">;</span>
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">java.util.Set</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">import</span> <span style="color: #006699;">org.aspectj.lang.Signature</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000000; font-weight: bold;">class</span> CallLogger
<span style="color: #009900;">&#123;</span>
    <span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000000; font-weight: bold;">static</span> CallLogger INSTANCE <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> CallLogger<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span></pre></div></div>

<p>An instance is made available as a public singleton. The main reason for using a singleton, rather than creating an instance in the aspect, is that if coherent logging is required across multiple projects in Eclipse it&#8217;s easiest to add a copy of the aspect to each project and have all aspects use the same logger instance.</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;">    <span style="color: #000000; font-weight: bold;">private</span> Stack<span style="color: #339933;">&lt;</span>String<span style="color: #339933;">&gt;</span> callStack <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> Stack<span style="color: #339933;">&lt;</span>String<span style="color: #339933;">&gt;</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #000000; font-weight: bold;">private</span> Set<span style="color: #339933;">&lt;</span>String<span style="color: #339933;">&gt;</span> callLog <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> HashSet<span style="color: #339933;">&lt;</span>String<span style="color: #339933;">&gt;</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #000000; font-weight: bold;">private</span> <span style="color: #003399;">Writer</span> writer<span style="color: #339933;">;</span></pre></div></div>

<p>The fields hold the actual stack of method names, a set that contains all calls that have been logged already, and a writer to which the actual output is written. The set is a simple optimisation to prevent the same call from being written over and over again.</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;">     <span style="color: #000000; font-weight: bold;">public</span> CallLogger<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #000000; font-weight: bold;">try</span> <span style="color: #009900;">&#123;</span>
            writer <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> <span style="color: #003399;">BufferedWriter</span><span style="color: #009900;">&#40;</span><span style="color: #000000; font-weight: bold;">new</span> <span style="color: #003399;">FileWriter</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;calls.txt&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span> <span style="color: #000000; font-weight: bold;">catch</span> <span style="color: #009900;">&#40;</span><span style="color: #003399;">IOException</span> e<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
            <span style="color: #000000; font-weight: bold;">throw</span> <span style="color: #000000; font-weight: bold;">new</span> <span style="color: #003399;">RuntimeException</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;Cannot open 'calls.txt' for writing.&quot;</span>, e<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000066; font-weight: bold;">void</span> pushMethod<span style="color: #009900;">&#40;</span><span style="color: #003399;">Signature</span> s<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #003399;">String</span> type <span style="color: #339933;">=</span> s.<span style="color: #006633;">getDeclaringType</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span>.<span style="color: #006633;">getName</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #003399;">String</span> method <span style="color: #339933;">=</span> type.<span style="color: #006633;">substring</span><span style="color: #009900;">&#40;</span>type.<span style="color: #006633;">lastIndexOf</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">'.'</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">+</span> <span style="color: #cc66cc;">1</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">+</span> <span style="color: #0000ff;">&quot;.&quot;</span> <span style="color: #339933;">+</span> s.<span style="color: #006633;">getName</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        callStack.<span style="color: #006633;">push</span><span style="color: #009900;">&#40;</span>method<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span></pre></div></div>

<p>If anyone knows a better way to turn the AspectJ signature into a pretty string, please leave a comment.</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;">    <span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000066; font-weight: bold;">void</span> popMethod<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        callStack.<span style="color: #006633;">pop</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000066; font-weight: bold;">void</span> logCall<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #000000; font-weight: bold;">if</span><span style="color: #009900;">&#40;</span>callStack.<span style="color: #006633;">size</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">&lt;</span> <span style="color: #cc66cc;">2</span><span style="color: #009900;">&#41;</span>
            <span style="color: #000000; font-weight: bold;">return</span><span style="color: #339933;">;</span>
        <span style="color: #003399;">String</span> call <span style="color: #339933;">=</span> <span style="color: #0000ff;">&quot;<span style="color: #000099; font-weight: bold;">\&quot;</span>&quot;</span> <span style="color: #339933;">+</span> top<span style="color: #009900;">&#40;</span><span style="color: #cc66cc;">1</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">+</span> <span style="color: #0000ff;">&quot;<span style="color: #000099; font-weight: bold;">\&quot;</span> -&gt; <span style="color: #000099; font-weight: bold;">\&quot;</span>&quot;</span> <span style="color: #339933;">+</span> top<span style="color: #009900;">&#40;</span><span style="color: #cc66cc;">0</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">+</span><span style="color: #0000ff;">&quot;<span style="color: #000099; font-weight: bold;">\&quot;</span>&quot;</span><span style="color: #339933;">;</span>
        <span style="color: #000000; font-weight: bold;">if</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">!</span>callLog.<span style="color: #006633;">contains</span><span style="color: #009900;">&#40;</span>call<span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
            write<span style="color: #009900;">&#40;</span>call<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
            callLog.<span style="color: #006633;">add</span><span style="color: #009900;">&#40;</span>call<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span>
    <span style="color: #009900;">&#125;</span></pre></div></div>

<p>Following a guard statement that protects the code from trying to log a call when only one method is on the stack, the code creates a description of the call in the format required by Dot for a graph edge, ie. the origin node in double quotes, followed by a stylised arrow, followed by the target node, again in double quotes.</p>
<p>If this call is not in the call log, the code writes it to the writer and then stores the call in the log so that it does not get written again.</p>

<div class="wp_syntax"><div class="code"><pre class="java" style="font-family:monospace;">    <span style="color: #000000; font-weight: bold;">private</span> <span style="color: #003399;">Signature</span> top<span style="color: #009900;">&#40;</span><span style="color: #000066; font-weight: bold;">int</span> i<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #000000; font-weight: bold;">return</span> callStack.<span style="color: #006633;">get</span><span style="color: #009900;">&#40;</span>callStack.<span style="color: #006633;">size</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #339933;">-</span> <span style="color: #009900;">&#40;</span>i <span style="color: #339933;">+</span> <span style="color: #cc66cc;">1</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
    <span style="color: #000000; font-weight: bold;">private</span> <span style="color: #000066; font-weight: bold;">void</span> write<span style="color: #009900;">&#40;</span><span style="color: #003399;">String</span> line<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        <span style="color: #000000; font-weight: bold;">try</span> <span style="color: #009900;">&#123;</span>
            writer.<span style="color: #006633;">write</span><span style="color: #009900;">&#40;</span>line <span style="color: #339933;">+</span> <span style="color: #0000ff;">&quot;<span style="color: #000099; font-weight: bold;">\n</span>&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
            writer.<span style="color: #006633;">flush</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span> <span style="color: #000000; font-weight: bold;">catch</span><span style="color: #009900;">&#40;</span><span style="color: #003399;">Exception</span> e<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
            <span style="color: #000000; font-weight: bold;">throw</span> <span style="color: #000000; font-weight: bold;">new</span> <span style="color: #003399;">RuntimeException</span><span style="color: #009900;">&#40;</span>e<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        <span style="color: #009900;">&#125;</span>
    <span style="color: #009900;">&#125;</span>
&nbsp;
<span style="color: #009900;">&#125;</span></pre></div></div>

<p>Running some code, the unit tests in this example, will now create a file named &#8220;calls.txt&#8221; that contains all calls to methods that match the pointcut description in the aspect. The calls are written in Dot format but to make the output a valid input file for Dot a header and a closing bracket at the end are required.</p>
<p>It would be easy to write the required header when the writer is initialised but realistically it&#8217;s not possible to know when <code>logCall</code> is called for the last time, and thus it&#8217;s not possible to write the closing bracket from the Java code. Therefore&#8230;</p>
<p><strong>Step 2</strong> is a small shell script, or a script in your favourite scripting language, that wraps the required header and footer around the raw graph edges.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;-EOS
strict digraph G {
    graph [ ratio=&quot;auto compressed&quot;,
            rankdir=&quot;LR&quot;,
            ranksep=2,
            remincross=&quot;true&quot;,
            ];
    node  [ shape=box,
            style=filled,
            fillcolor=white,
            fontname=helvetica
            fontsize=10,
            fontcolor=black
            ];
EOS</span>
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">cat</span> calls.txt
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;-EOS
    }
EOS</span></pre></div></div>

<p>I have an admission to make: This script is simplified and would not create the output I&#8217;ve shown further up. With a graph definition like this Dot would not put all unit tests nodes into the leftmost column but instead it would move them just left of the method they call. If, for example, test 1 called method A, which in turn called method B, and test 2 called method B directly, then test 1 would be in the first column, method A and test 2 in the second column and method B in the third column. The following UNIX command line wizardry, when added just below the <code>cat calls.txt</code> line in the script above, solves this problem. The explanation is left as an exercise to the reader&#8230;</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'{ rank = same; '</span>
<span style="color: #c20cb9; font-weight: bold;">fgrep</span> Test.test calls.txt <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">awk</span> <span style="color: #660033;">-F</span> <span style="color: #ff0000;">&quot; -&gt; &quot;</span> <span style="color: #ff0000;">'{ print $1 }'</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">sort</span> <span style="color: #660033;">-u</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">awk</span> <span style="color: #ff0000;">'{ printf &quot;%s; &quot;, $1 }'</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'}'</span></pre></div></div>

<p>I hope this article shows how two simple but powerful tools combined make it easy to create a really useful visualisation; and how clever but cryptic UNIX command lines can be.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2008/09/call-graph-visualisation-with-aspectj-and-dot/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>SpringViz and the 1000ft view</title>
		<link>http://erik.doernenburg.com/2008/08/springviz/</link>
		<comments>http://erik.doernenburg.com/2008/08/springviz/#comments</comments>
		<pubDate>Wed, 13 Aug 2008 02:07:30 +0000</pubDate>
		<dc:creator>Erik Doernenburg</dc:creator>
				<category><![CDATA[Coding]]></category>
		<category><![CDATA[Visualisation]]></category>

		<guid isPermaLink="false">http://erik.doernenburg.com/?p=47</guid>
		<description><![CDATA[The Spring framework has become ubiquitous in the Java world, and there are a large number of tools supporting developers of Spring-based applications. In this post I describe SpringViz; or, more accurately, my variant of it.
SpringViz helps developers with what is at the heart of a Spring-based application, the container and the contexts files that [...]]]></description>
			<content:encoded><![CDATA[<p>The <a href="http://www.springframework.org/about">Spring framework</a> has become ubiquitous in the Java world, and there are a large number of tools supporting developers of Spring-based applications. In this post I describe <a href="http://www.samoht.com/wiki/wiki.pl?SpringViz">SpringViz</a>; or, more accurately, my variant of it.</p>
<p>SpringViz helps developers with what is at the heart of a Spring-based application, the container and the contexts files that describe the beans. In larger projects these context files can grow quite a bit. Newer versions of the Spring framework introduced features that help reduce the clutter and there are vast numbers of blog posts voicing different opinions on what should and what shouldn&#8217;t be in a context file but, no matter what, the number of beans in the context files will grow with the size of a project, and at some point it becomes difficult to understand the overall structure. This is no different from trying to maintain an understanding of a large codebase. In fact, I consider the context files to be code rather than configuration.</p>
<p>I&#8217;ve argued before (<a href="http://97-things.near-time.net/wiki/show/Get%20the%201000ft%20view">here</a> and <a href="http://www.infoq.com/interviews/erik-doernenburg-software-visualization">here</a> for example) that to deal with the complexity and sheer size of software systems we need a 1000ft view. This is a view that uses visualisation techniques to aggregate large amounts of data and multiple metrics into one big picture. SpringViz provides that 1000ft view for Spring context files.</p>
<p><span id="more-47"></span></p>
<p>Let&#8217;s look at an example first. The following diagram shows the context files for the Spring JPetStore example application (click on the image for the full-size version):</p>
<p><a href="http://erik.doernenburg.com/wp-content/uploads/2008/08/jpetstore_beans.png"><img src="http://erik.doernenburg.com/wp-content/uploads/2008/08/jpetstore_beans_small.png" alt="Diagram showing the context files and beans contained in them for the Spring JPetSore example application" title="JPetStore Beans" width="600" height="424" class="size-full wp-image-81" /></a></p>
<p>I find that this diagram presents the architecture of the application quite nicely. The web controllers are defined in the petstore-servlet context in the top left and many of them have a dependency on the petStore bean, defined in the main application context. The petStore bean in turn has dependencies on all the DAOs, which are defined in the data access context. In that context we can clearly see how the top four DAOs share the same data source while the <em>Order</em> and <em>Sequence</em> DAOs use a different one. In addition to the individual dependencies we also get an impression of the overall architecture: a simple, multi-layered design with a single central choke point that acts as an adaptor between the presentation and data access layers.</p>
<p>From personal experience I can say that the bigger and more complex a system gets, the more valuable these diagrams become. In many cases they not only help share a common understanding of the architecture in the development team but they also help spot inconsistencies in the design and often trigger and support discussions around important restructurings. Ok, this is all I&#8217;m going to say to convince you to try out SpringViz. In the following paragraphs I&#8217;ll explain how to use it on your own projects.</p>
<p><strong>Step 1:</strong> You need a copy of SpringViz. The version I&#8217;m discussing in this article can be downloaded <a href="http://erik.doernenburg.com/wp-content/uploads/2008/08/springviz-20080813.zip">here</a>. When you unzip the archive you should have an Ant build file, a <em>springviz</em> folder with a few files in it and an <em>example</em> folder that contains the context files for JPetStore.</p>
<p><strong>Step 1b:</strong> SpringViz uses <a href="http://www.graphviz.org/">GraphViz Dot</a> to render the diagram. So, if you don&#8217;t have this on your system you need to download and install it. You should also open the build file and change the <em>dot.command</em> property to point to the Dot executable. On the Mac you should really get <a href="http://www.pixelglow.com/graphviz/">Pixelglow&#8217;s version</a> of GraphViz.</p>
<p><strong>Step 2:</strong> You can now run SpringViz by calling Ant with the default target. This will create a <em>target</em> folder, process the context files with the SpringViz XSLT sheet to create an input file for GraphViz Dot, and then invoke Dot to create <em>beans.png</em>, the final diagram.</p>
<p><strong>Step 3:</strong> To integrate SpringViz into your own projects you can simply add the contents of the build file to your build file, place the <em>springviz</em> folder in an appropriate place, and adjust the configuration properties at the top of the build file.</p>
<p>If you are adventurous you can have a look at the Dot documentation and change the XSLT sheet to create slightly different input for Dot. You could colour the beans according to all sorts of criteria or you could change other aspects of their appearance depending on bean attributes.</p>
<p>There are a few limitations that you should be aware of: In its current version the system can&#8217;t deal with multiple beans with the same name defined in different contexts. So, if you have a <em>defaultUrlMapping</em> bean or similar in multiple files and you have dependencies to/from these the resulting diagram will be wrong. In this case it&#8217;s easiest to modify the style sheet to filter out the problematic beans. Another important point is that in its current form the sheet simply tries to use a bean&#8217;s id and name, which usually works out but will cause problems when beans have both, name and id. And lastly, the XSLT processor will need access to the web to fetch the schema files. I guess this can be avoided but so far this hasn&#8217;t been a problem for me.</p>
<p><strong>If you missed the link above you can download the specific version of SpringViz discussed in this article from <a href="http://erik.doernenburg.com/wp-content/uploads/2008/08/springviz-20080813.zip">this site</a>. The latest version of SpringViz as well as more documentation and resources are available on <a href="http://www.samoht.com/wiki/wiki.pl?SpringViz">Mike Thomas&#8217;s site</a></strong>.</p>
]]></content:encoded>
			<wfw:commentRss>http://erik.doernenburg.com/2008/08/springviz/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>
