Sitemap on the wall

Labels: commutineer, sitemap

Posted by Simon Baker @ permalink

Links to this post 

0 Comments

Petition against recurring Government IT incompetence

Isn't it about time we started calling the civil service and the Government to account for the repeated failures and wasted money in Public IT projects?

Don't delay! Sign the petition to the PM.

Posted by Simon Baker @ permalink

Links to this post 

1 Comments

Integration Testing: The Story Continues

Over the past few months I've been reading the 'Integration Tests Are A Scam' serious of articles by J.B. Rainsberger and following some of the responses to it such as this one by Steve Freeman. I put in my 2 cents a few days ago which I've reproduced here:

Interesting series of articles & comments. I also read Steve Freeman’s article in response to the same topic. It’s got me thinking about how we work and I thought I’d take the time to describe it here.

You define an integration test as “… any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior.” We have many such components that exhibit such non-trivial behaviour in the products we create, many of which are not developed by us. And we have integration tests to verify they work. I’m not just talking about 3rd party libraries and frameworks here, I’m referring to the whole system: caching layers. load balancers, DNS servers, CDNs, virtualization etc. When we build software it only becomes a product or service for our users when it has been deployed into a suitable environment; an environment that typically contains more than just the software we have written and packaged. Since our users’ experience and perception of quality result from their interaction with a deployed instance of the whole system, not just their interaction with the software at a unit level, we have come to value end-to-end integration testing. I believe there’s merit in testing these components in symphony and will attempt to clarify what kind of integration testing I’m talking about.

For a given piece of functionality we write an executable acceptance test in human readable form (for web projects we typically use some domain-specific extensions to selenium, for services we have used FIT and it’s ilk, sometimes we roll our own if there’s nothing expressive enough available). We run it against a deployed version of the application (usually local though not always) which typically has a running web/application server and database. The test fails. We determine what endpoint needs to be created/enhanced and then we switch context down into unit-test land. A typical scenario would involve enhancing a unit test for the url mappings, adding one for the controller, then one for any additional service, domain object etc. When we’re happy and have tested and designed each of the required units we jump back up a level and get our acceptance test to progress further. The customer steers the development effort as he sees vertical ‘slices’ of functionality emerge. The acceptance test is added to a suite for that functional area. The continuous build system will then execute that test against a fully deployed (but scaled down) replica of the production environment, with hardware load balancer, vlans, multiple nodes (session affinity) and so forth. Any additional environmental monitoring (e.g. nagios alerting) is also done as part of this development effort and is deployed into the test environment along with the updated code.

Setting up the infrastructure to do this kind of testing takes investment, both initial and ongoing. The continuous build needs to be highly ‘parallelized’ so you get feedback from a checkin in 10 mins or less (we’re heavy users of virtualization, usually VMWare or OpenVZ). The individual acceptance test suites need to be kept small enough to run quickly before check-in.

Benefits of this approach

• The continuous context-switch between acceptance test and unit test is key to our staying focused on delivering what the customer actually wants.


• The customer has multiple feedback points that he can learn from and use to steer the development effort.


• It confirms that the whole system works together – networking, DNS, load balancing, automated deployment, session handling, database replication etc.


• We create additional ‘non-functional’ acceptance tests that automatically exercise other aspects of the system such as fail-over and recovery.


• Upgrades to parts of the system (switches, load balancers, web caches, library versions, database server versions etc.) can be tested in a known and controlled way.

We’ve caught a number of integration-related issues using this approach (a few examples: broken database failover due to missing primary keys, captcha validation not working due to a web cache not behaving correctly, data not persisting because one database server had the wrong locale) and stopped them before they have reached our users. We have used the feedback as a basis for improving our products and their delivery at a system level.

OK this reply has now become far too long :-/ It would of course be good to discuss this in person sometime :)

J.B.'s taken the time out to respond and it seems that there's a lot of common ground. Maybe there's a language problem here in developer land? Do we need some clear common definitions in this area?

Labels: end-to-end testing, integration tests, testing

Posted by Gus Power @ permalink

Links to this post 

0 Comments

Don't aim at the target

Without numerical measures we wouldn't know what to do. The problem is, when numerical measures are used as targets they cause people to think their sole purpose is to achieve them, usually to the detriment of everything else. When managers own the targets and use them to force performance they bring out the wrong behaviors. People cut corners to meet the targets. And targets are everywhere. We blinker ourselves to everything except our targets and forget about the real needs of users. In pursuit of our targets we make local optimizations that are suboptimal for the throughput of the whole system, the wider organization.

Measures should reflect the true purpose of the people doing the work, which is to improve service and quality and satisfy users, and should therefore measure the improvements directly experienced by users. These people are in the best position to decide how to improve quality and performance and they should own the measures and use them to understand their work as a system. As part of a plan-do-check-act cycle, they should study the actual results of changes aimed at improvement, comparing them to expectations, analyzing the differences to determine cause, and then identify further opportunities to improve the system.

Managers shouldn't use their measures as targets to control our performance. Instead, we should use our measures to continuously improve how we work so that our system performs better.

Labels: continuous improvement, measures

Posted by Simon Baker @ permalink

Links to this post 

1 Comments

Bad posture

Jeff Patton recently tweeted:

“I see agile process practiced with waterfall posture. By posture I mean the values, principles, and thinking processes with which you approach software development.”

I like this.
It's a simple way to explain many of the things I see.
'Posture'.

Posted by Simon Baker @ permalink

Links to this post 

0 Comments

What's distinctive about Lean Thinking and where is it going

At the UK Lean Conference, my brother Marc Baker talked about Lean Thinking, how it has developed into a complete business system and where it's going. He also shares some insights from lean transformations he has been part of in Healthcare.

Labels: lean thinking

Posted by Simon Baker @ permalink

Links to this post 

0 Comments

Lean Software and Systems Conference in Atlanta

David Anderson invited us to speak at the first Lean Software and Systems Conference in Atlanta from April 21st to 23rd 2010. We'll be talking about our evolving 'system' for software product development, which David saw when he visited us at BSkyB, and more. That was nearly two years ago. Since then we've continued to develop the approach and techniques, applying it more recently on a project with a couple of consultants from the Womack and Jones crew at the Lean Enterprise Academy.

Here's the abstract for our session:

Product Development in the Land of the Free
The Test-driven Organization

Creating and sustaining a 'system' for effective product development is neither easy nor commonplace. If we were to pull together the lessons we've learned from eXtreme Programming and Scrum with systems approaches such as Lean Thinking and the Theory of Constraints to build such a 'system' what would it look like? Where would we start? How would we organize ourselves? And what would be our approach?

The fact that so many information technology projects are still failing tells us that we should be doing something very different. This session will explore some of the things we've been doing beyond the agile comfort zone to improve the effectiveness and throughput of product development and realize business agility.

Labels: lean thinking, systems thinking

Posted by Simon Baker @ permalink

Links to this post 

0 Comments

Invite us around for a cup of tea

Since we won the Gordon Pask Award we've been wanting to get out and about to meet people, visit organizations or usergroups, see how people are working, understand the problems overcome and the problems still faced, and learn some new things.

We're happy to share our experiences through an informal 2-hour get-together, be it a brown-bag session, a meeting during office hours or an after-work gathering. We'd welcome the opportunity to answer questions, talk about a topic of your choosing, participate in a discussion, provide a sounding board, or offer advice. And we're more than happy to consider other suggestions.

This is without obligation on your part and free of charge if you're located in London. If you're outside London we'll probably ask you to cover our travel costs.

If you think this would be useful to you, your team or your organization please email simon at energizedwork dot com.

Labels: gordon pask

Posted by Simon Baker @ permalink

Links to this post 

3 Comments

Reports are waste and a reason for poor decision-making

Myron Tribus said:

"Managing a company by means of a monthly report is like trying to drive a car by watching the yellow line in the rearview mirror."

Periodic progress reports are a symptom of hierarchical thinking. They make everybody 'look up' at managers rather than 'out' to users.

Progress reports don't tell you what's going on. They tell you what people want to tell you. And decisions based on such incomplete information are risky and likely to be poor. Producing reports is waste; it’s not adding value.

As John Seddon says "reports are substitutions for action". To really know what's going on you must go see the real thing for yourself. And do it regularly. Talk face-to-face with the people doing the work and observe the actual process at the actual place to obtain actual data. Learn from each visit and make your decisions on the facts you have gathered yourself.

Labels: gemba gembutsu, hierarchical thinking, john seddon, reports, waste

Posted by Simon Baker @ permalink

Links to this post 

9 Comments

Getting past design to design thinking

I previously mentioned that we've been applying iterative development to grow better user experiences so I found the following words and the video particularly interesting.

"Instead of starting with technology start with people and culture. If human need is the place to start then design thinking rapidly moves on to learning by making. Instead of thinking about what to build - build in order to think. Prototypes speed up the process of innovation because it's only when we put ideas into the world that we really start to understand their strengths and weaknesses. And the faster we do that the faster our ideas evolve.

Instead of seeing the primary objective as consumption, design thinking is starting to explore the potential for participation. The shift from a passive relationship between consumer and producer to the active engagement of everyone in experiences that are meaningful, productive and profitable."

Tim Brown of IDEO urges designers to think big:

Labels: design thinking

Posted by Simon Baker @ permalink

Links to this post 

0 Comments