Sarah Taraporewalla's Technical Ramblings

Working With Integration Points: Simulators

| Comments

In my last post about working with integration points I described how I use anti-corruption layers (wrappers) to hide away any warts of the integration point. In this post, I will explain how I used these wrappers to help develop against an integration point which was also under development.
The problem with two systems being developed concurrently is that it is rare that they are developed at the same rates, or even in the same priority. That means that the consumer can often be left waiting for functionality to be delivered before they can start developing against it. And that assumes that it all works (not a wise assumption if my previous projects are anything to go by).
There is another way however - by using simulators to develop against. The simulators can be as sophisticated as you need them be - either returning pre-canned values or randomising the responses. They can live in-memory or they can be across the wire depending on what your focus is. Or have both types and simulate less and less and you move through the CI pipeline. An in-memory simulator can replace your wrapper easily using your DI container (it also helps draw the distinction between logic for the business service and logic for the wrapper).
The main importance of simulators is that they let you develop independently of other teams. An added benefit is that they help you define a consumer-driven contract (if you are the consumer). You can also use the simulators to help identify problems with the system - if it works against the simulator and not against the real integration system, then there is a problem with the integration system. If there is a problem even with the simulator then there is a problem in your system. If it works against the in-memory simulator but not against the simulator across the wire, then your problem lies somewhere on the wire.
Simulators can really help you in your development - don't be afraid by the effort you need to spend to create them. Obviously you need to weigh up the benefits of the sophistication of the simulator with the effort to maintain them but there is often a nice balance to strike.

Working With Integration Points: Anticorruption Layer

| Comments

On my last few projects we had a requirement to integrate our application with another system. Adding to this complexity was the fact that these other systems where also still being developed. I learned a few tricks on the first of these types of projects which really helped the latter projects which I shall share with a wider audience here.
The first of these tricks was to enforce a strict layering of my application and introduce an anti-corruption layer. This layer was responsible for dealing with the interface to the other system, and often hiding any crapiness of either the api or transport protocol from the rest of the application. It ensured that any business logic remained in the business services layer and any api/transport logic remained in the anti-corruption layer (which we called a wrapper).
The main reason for this was to limit the amount of damage that would occur should the api change (i.e. try to decrease the amount of coupling between the two systems). To support this, the wrapper would talk to the business services in domain language and pass around domain objects which it would transform into the required transport objects.
There are a few cases where the wrappers proved their worth:
  • when the service did not populate all the necessary fields for the domain object
  • when the domain object did not logically contain a field necessary for the transport
  • when the business layers dealt in synchronous calls, and the api dealt with asynchronous calls
  • when the a wcf object type changed
It was sometimes hard, especially when dealing with a crappy api that we could not alter, to know if a piece of logic should reside in the business service or in the wrapper. The general rule of thumb we followed was to ask ourselves "if we ever changed this integration point to a different system, would we still need this bit of logic?". [The crappier the api the harder it was to answer].
However, despite what I considered to be the obvious benefits of introducing this layer, I faced a lot of resistance introducing it, especially when it looked like the business service was just a pass through to the wrapper i.e. when the method only had one line in it _wrapper.Foo() (actually it turned out that the only reason the service was pass through was because there were fat controllers - pushing the logic from the controller into the service made it look a lot healthier but that is a whole other discussion). What irked me the most is when people invoked YAGNI as a reason not to add a new object. Although that debate is a whole entire post in itself, all I am going to mention here is that YAGNI does not trump SOLID and that anti-corruption layers is nothing more than Single Responsibility Principle (i.e. Separation of Concerns) [and seriously - are you really telling me that adding another object is time consuming, in the world of resharper/intelli-j].

I Don't Believe in NFRs

| Comments

There is something about the phrase Non Functional Requirements that I don’t really like. It’s the Non bit which gets me - I think that word makes them feel unimportant and therefore there is no pressing need to explore how they will affect the system. I have seen on many projects teams procrastinating in defining how the “NFRs” will affect what they are building - usually in the metrics around performance.

On my current project we have decided to appease my sensibilities and have called these Cross Functional Requirements to better express what they truly represent - requirements which cross all the functions we are building. So far the wording works - but we have only finished the 2nd week of inception.

The Importance of Words

| Comments

The english language is a very complex beast which can easily strike confusion in the most unsuspecting moment. Take, for example, the word "lollies". Did your mind immediately jump to a bag of gummy bears, redskins, bananas or milk bottles that you get from the garage? Or instead did you start thinking of a frozen dessert on a stick? If you thought the former, then, like me, you probably are Australian. Because the English term for it is sweets, and the American version is candy (although I doubt that they have such wonderful lollies as bananas and milk bottles). And the frozen-dessert-on-a-stick - thats an iceblock for me!
Take another example - when Sisqo's Thong Song came out I didn't really understand why someone dedicate an entire song to $1 rubber plugs - until I watched the video clips and I realised that he wasn't referring to "flip flops".
The lesson that I have learnt living in the UK is that I have to be very careful with the words that I use, lest I cause confusion by saying that someone was running around with their pants* off**.
And this is also applies to the words and language that we use around our business, between developers and in our code. Too often, we describe concepts by words like "that thing" or "this stuff" which can obfuscate what it really is, and hence hinder our ability to model it. I often see complexity, misunderstandings and hacks creep into code bases when the original designer has referred to an information concept as "this thing" without taking the time to name it. I find that spending the energy, talking to the business to understand what you are really producing helps clarify understanding which in turn distills the model into a simpler, easily understood model.
I had this demonstrated to me yesterday, when I came across a pair discussing how to "shove" this concept of "a thing that when clicked on opens up a popup with stuff in it" onto the model. As I listened to the conversation, I noticed just how muddled the model and the situation was becoming, so I decided to intervene with some of my words of wisdom about what they were trying to model. As soon as we worked out that the "a thing that when clicked on opens up a popup with stuff in it" translated into "a help message", we were easily able to see exactly where in our model it should go.
So, never underestimate the power of the words we use. When faced with a confusing concepts strive to distill it into easily understood, well defined names.
* pants in Australia are trousers - pants in the English vernacular are what I call underpants.
** yes I have once said that at work after I saw a pair of jeans on the table and was a little confused by what someone must be wearing!

The English language is a very complex beast which can easily strike confusion in the most unsuspecting moment. Take, for example, the word “lollies”. Did your mind immediately jump to a bag of gummy bears, redskins, bananas or milk bottles that you get from the garage? Or instead did you start thinking of a frozen dessert on a stick? If you thought the former, then, like me, you probably are Australian. Because the English term for it is sweets, and the American version is candy (although I doubt that they have such wonderful lollies as bananas and milk bottles). And the frozen-dessert-on-a-stick - thats an iceblock for me!

Take another example - when Sisqo’s Thong Song came out I didn’t really understand why someone dedicate an entire song to $1 rubber plugs - until I watched the video clips and I realised that he wasn’t referring to “flip flops”.

The lesson that I have learnt living in the UK is that I have to be very careful with the words that I use, lest I cause confusion by saying that someone was running around with their pants* off**.

And this is also applies to the words and language that we use around our business, between developers and in our code. Too often we describe concepts by words like “that thing” or “this stuff” which can obfuscate what it really is, and hence hinder our ability to model it. I often see complexity, misunderstandings and hacks creep into code bases when the original designer has referred to an information concept as “this thing” without taking the time to name it. I find that spending the energy talking to the business to understand what you are really producing helps clarify understanding which in turn distills the model into a simpler, easily understood model.

I had this demonstrated to me yesterday, when I came across a pair discussing how to “shove” this concept of “a thing that when clicked on opens up a popup with stuff in it” onto the model. As I listened to the conversation, I noticed just how muddled the model and the situation was becoming, so I decided to intervene with some of my words of wisdom about what they were trying to model. As soon as we worked out that the “thing that when clicked on opens up a popup with stuff in it” translated into “a help message”, we were easily able to see exactly where in our model it should go.

So, never underestimate the power of the words we use. When faced with a confusing concepts strive to distill it into easily understood, well defined names.

  • pants in Australia are trousers - pants in the English vernacular are what I call underpants. ** yes I have once said that at work after I saw a pair of jeans on the table and was a little confused by what someone must be wearing!

JAOO and Women: Attendance at Conferences

| Comments

I was going to start this series off talking about filters and the way that connections are made in your brain, but after hearing about the lap dances at the Taiwan Yahoo! Hack Day, I thought I might begin with addressing the issue of low attendance at conferences.

I think one of the biggest concerns that conference organisizers should be worrying about (and luckily JAOO did) is that the ratio of women at the conferences does not reflect the industry - I have heard that between 5-10% is considered good, where as I believe the industry is at about 20% (unconfirmed sources). I don’t want to start of speculating at why the numbers are so low, but what I can share is why I don’t attend conferences.

Conferences are quite expensive for your employer - not only do they need to purchase entry, but accommodation, meals and transfer to and from the conference, not to mention loss of “billable” work. So, usually they can only afford to send one or two people there. I have seen two ways that these tickets are distributed. The first is that the same people always seem to go to the conference. It is noticeably remarkable how many people you see at conferences that seem to be on the “conference circuit” - they have seen each other every month, following the conferences around the world. So, unless you already are in the conference circuit, it is really hard to enter it - as there are currently 5-10% women normally at conferences, and the same people keep attending, then this number won’t change. The other way that I have seen tickets distributed is by everyone-gets-a-turn selection. Lets say the industry average is 20% - then its only fair that for every five conferences that are attended (given one ticket per conference), a company would send a female to only one of them. From that point of view, it does not seem that unbelievable to me that the number of women is low in comparison.

So, I think it is up to conference organisers to think outside the box for different ways to attract females to conferences. For this, I would like to commend the people at Trifork who organised JAOO Aarhus 2009 as they did something a little different. For every full-paying ticket that you bought for JAOO, you were entitled to a free day pass for a person of the opposite sex. This wonderful scheme saw registrations jump from a lowly 3.7% to about 14%. I think that this shows that the arguments around women not able to travel to be not necessarily the driving factor behind the low attendance, and perhaps the key lies in being selected in your company to attend. For me it is certainly true that the main reason I don’t attend conferences is that I believe that the bought tickets should be distributed fairly.

I think, however, speaking at a conference is slightly a different manner. I read today (I tried to find the source, but I can’t seem to find it again) that the difference between a would-be male speaker and a would-be female speaker is that the female thinks “I don’t know everything about [topic], I don’t think I can speak” whereas a male thinks “I know a bit about [topic], I think I will speak”. I obviously don’t know if that is true for all men/some men/no men in as much as I can’t say that it speaks for all women/majority of women. What I can say is that it certainly rings true for me.

As anyone who has known me since I was three can testify, I like public speaking. I am comfortable on a stage, talking about what I feel passionate about. As a female software dev who loves public speaking, I feel that I should be one of those who help increase the numbers - but here’s the catch. I don’t want to get up and talk just because I am a woman. If I ever present at a conference, I want it to be primarily because I have something important to say about technology and -oh yeah- I happen to be a women. But I also don’t feel that I have anything that interesting worth presenting - I don’t know as much as those guys already presenting, so what would I have to give. I don’t know if this is a female trait - I would be interested to find out if others feel like this.

If this assertion is correct (that women don’t feel that they have anything to say), perhaps the conference old-timers could help these would-be presenters in recognising material that could be presented. Another way to know if you have something valuable to say is to go and attend a few conferences yourself - see what other newbies are presenting, and see if you can do as good a job (lets face it - there is no sense in comparing yourself to a Martin Fowler or a Neal Ford - compare yourself to someone who is in the same position as you are).

So, there are the reasons why I don’t attend or present at conferences. How do we adjust this? Well, JAOO certainly has set the standard - I hope other conferences follow suit and find new and innovative ways to get women to their conferences.

JAOO and Women

| Comments

A few weeks ago, I was fortunate enough to attend JAOO, an awesome conference for software developers, architects and PMs, for one day. How did I get to? Well, a few weeks before the conference, the organisers (Trifork) noticed that while conferences usually have around 5-10% women attend (and that is considered good), JAOO only had about 3.7%. So, the organisers did a really cool thing…they decided that for any full paying pass that you bought, you could bring a person of the opposite sex free for a day! They also added a Women in IT meeting to their user geek night. As a result, they managed to get a whopping 14% women attendance!

The Women in IT meeting was quite fun and well attended - it seemed like it was the first time many of the people there met with others to talk about being a women in IT. I was also glad to see that this was not attended only by women - four of our ThoughtMen came along. I thought this was fantastic - how cool is it that I work for a company where the guys not only want to find out more about the problem and help where they can but are also willing to miss out on other user groups, such as the Java and Ruby groups which also met at the same time.

During the session, we had three of the conferences speakers (who were also women) form a panel to share experiences and exchange ideas. These speakers were Rachel Reinitz, Linda Rising and Rebecca Parsons. I have been to many of these sessions before, but I was entered a little skeptical and a bit jaded but I left reenergized to helping solve some of the problems.

During this discussion, I think I realised that the topic “Women in IT” was such a broad one, and there are so many issues surrounding it, that when you get a group of people to talk about it as a whole you get too many cross cutting conversations. So, I am going to attempt to list the issues that I see comprise the broader “Women in IT” ‘problem’.

  • Not enough women entering maths/science/technology
  • Lack of self confidence to speak up
  • Lack of involvement in the community and open source projects
  • Lack of women at conferences (attendees as well as speakers)
  • Lack of role models
  • Perceived glass ceiling and salary offsets
  • Perception around women as mothers vs women as employees
  • Employees don't do right for humans
  • Women are not networking as well as men are
  • Treatment of women at social events/in the workplace/lack of respect for other humans
  • When you try to do something positive, you get hit with people crying discrimination
  • Filters women apply to situation to make them seem worse than they actually are
  • Subvert harassment through jokes and conversations
  • Lack of awareness with men to understand what is OK
  • Women not respecting the problems other women have faced.

I hope to build out a series of posts around these points, explaining what I believe is the issue, suggestions of some solutions, and what I have seen the community do to try to help address these problems.

If you think there are parts to the problem that I have missed, tell me about them in the comments section!

Slippers: How to Click Your Heels

| Comments

In my last post about Slippers, I introduced it’s philosophy and the places that you could find it. In this post, I will introduce some of its constructs.

Rendering template of a string without any holes

1
2
3
template = "This is a string without any holes in it"
engine = Slippers::Engine.new(template)
engine.render #=> "This is a string without any holes in it"

Filling in a hole within a template

1
2
3
template = "This is a string with a message of $message$"
engine = Slippers::Engine.new(template)
engine.render(:message => "hello world") #=> "This is a string with a message of hello world"

Rendering a subtemplate within a template

1
2
3
4
5
subtemplate = Slippers::Template.new("this is a subtemplate")
template_group = Slippers::TemplateGroup.new(:templates => {:message => subtemplate})
template = "This is a template and then $message()$"
engine = Slippers::Engine.new(template, :template_group => template_group)
engine.render #=> "This is a template and then this is a subtemplate"

Applying an object to a subtemplate

1
2
3
4
5
subtemplate = Slippers::Template.new("this is a subtemplate with a message of $saying$")
template_group = Slippers::TemplateGroup.new(:templates => {:message_subtemplate => subtemplate})
template = "This is a template and then $message:message_subtemplate()$!"
engine = Slippers::Engine.new(template, :template_group => template_group)
engine.render(:message => {:saying => 'hello world'}) #=> "This is a template and then this is a subtemplate with a message of hello world!"

Applying an object to an anonymous subtemplate

1
2
3
template = "This is a template and then $message:{this is a subtemplate with a message of $saying$}$!"
engine = Slippers::Engine.new(template)
engine.render(:message => {:saying => 'hello world'}) #=> "This is a template and then this is a subtemplate with a message of hello world!"

Render a subtemplate using a different rendering technology

1
2
3
4
5
6
age_renderer = AgeRenderer.new
subtemplate = Slippers::Engine.new('$first$ $last$')
person = OpenStruct.new({:name => {:first => 'Fred', :last => 'Flinstone'}, :dob => Date.new(DateTime.now.year - 34, 2, 4)})
template_group = Slippers::TemplateGroup.new(:templates => {:name => subtemplate, :age => age_renderer})
engine = Slippers::Engine.new("Introducing $name:name()$ who is $dob:age()$.", :template_group => template_group)
engine.render(person) #=> "Introducing Fred Flinstone who is 34 years old."

Select a renderer based on the type of the object to render

1
2
3
4
person = OpenStruct.new({:name => {:first => 'Fred', :last => 'Flinstone'}, :dob => Date.new(DateTime.now.year-34, 2, 4)})
template_group = Slippers::TemplateGroup.new(:templates => {:name => Slippers::Engine.new('$first$ $last$'), Date => AgeRenderer.new})
engine = Slippers::Engine.new("Introducing $name:name()$ who is $dob$.", :template_group => template_group)
engine.render(person) #=> "Introducing Fred Flinstone who is 34 years old."

Use the specified expression options to render list items

1
2
3
template = 'This is a list of values $values; null="-1", seperator=", "$'
engine = Slippers::Engine.new(template)
engine.render(:values => [1,2,3,nil]) #=> "This is a list of values 1, 2, 3, -1"

Slippers: Introduction

| Comments

There are many template engines that you can choose for the generation of views in your mvc application. The problem with most of the them, however, is that they are too permissive. These turing-complete engines allow for many complex constructs within the template, which begin at simple if statements and for loops, and expand to complex object traversal. While these permissive languages are intended to offer great flexibility, in reality they promote bad practices. Allowing logic to permeate your view is bad for many reasons: firstly, the code in views is rarely tested; secondly, the separation between the models and the view blurs and business logic creeps into the view.

All we want our template engine to do is read a string which has holes in it, and replace those holes with the desired string, much like mail merge. Luckily for me, there is already a templating engine which enforces strict separation of model and view by only supporting these strings with holes - String Template. String Template is originally created in Java, however there has been subsequent ports to C# and python. It is opinionated, and I like it’s opinions. (And if you prefer permissive views, I would recommend you reading about the origins of String Template before completely ruling it out.)

Unfortunately, when I started to try out Ramaze (a great web framework for ruby - much better than rails imho) I looked around for a port of String Template to ruby, but couldn’t find out. So, I decided that if no-one else would port it, I would…and so Slippers was born.

Slippers is a strict template engine for ruby, supporting the syntax from String Template including anonymous template, named templates and template group directories but also goes beyond this to allow you to use your own renderers. Nearly all the useful constructs from String Template have been ported, and you can see the comparison on the Slippers vs String Template page.

In creating Slippers, I have been surprised to find how many development tools for ruby can now be found “in the cloud” (ie on the web):

  • Slippers code repository is hosted on github.
  • Slippers has been test-driven, and I have a CI build at runcoderun.com. As you can see, I have also made it Ruby 1.9 compatible.
  • The gem can be found on gemcutter. To install:
    1
    2
    3
    4
    
    # Run the following if you haven't done so before:
    $ gem sources -a http://gemcutter.org
    # Install the gem:
    $ sudo gem install slippers
    

Ramaze now supports Slippers as a view engine, as of gem version 2009.10 (also on gemcutter).

More information about Slippers can be found at http://slippersrb.com

How to Refactor Like a Something Star

| Comments

Refactoring a piece of code can be such a thrilling experience when it's finished, but at the same time can also act as the blackhole of time. Recently, I have been observing how other people refactor, watching their looks of happiness when they finish it and also their despaired faces when they have realized they have spent a week trying to add one new argument to their method.
I love refactoring and I think I do a good job at it now, but there was a time in the not-so-distant past where I was one of those despaired faces. I got to where I am now thanks to pairing with refactoring gurus and practice. Here are some of the tricks that I have picked up:
  • Plan your refactoring Planning should never be considered overrated. Before I start a refactor, I like to assess everything that the existing code does, and map where I want that to sit. I also determine which angle is the best to start from after looking at all the possible combinations, so that I am always working with minimal changes. I like to use post-its to guide my work, so I usually start with a handful of post-its to show where I am going. If there are too many post-its, I also start questioning the scope of the refactor: what could I get done if I only had 2 days to do it in.
  • Parallelize your work This really works well when, instead of shuffling bits around, you are actually doing a rewrite of the current functionality. For instance, the other day I rewrote the legacy login code to the application. Obviously I couldn't be too ruthless as everyone needed to still login, and this code was not under test. So, I wrote the new login section alongside the existing code, and just swapping in my login code in the DI container on my local machine. When it was time to switch it over, we checked in the DI container change, and voila, everyone started using the new login code. This allowed us to check in frequently without breaking anyone regardless of it being incomplete. It also meant that we always had the original version to refer to as we proceeded
  • Checkin often This is helped by either doing minimal changes at a time, or by parallelizing the work. Checkin frequently, checkin small. That way if you ever need to revert your changes, you know the brilliant work that you did before the "screwup" won't be wasted. If you are working with a source control like mercurial or git, you are in an even better situation, as you still have source control without affecting the rest of your team with your breaking changes.
  • Never be afraid to revert your changes This one is such a valuable lesson to learn and it feels so wrong when your doing faced with the possibility but when you have painted yourself into a corner, the best thing to do is to revert all changes. What it does mean, however, is that all the lessons that you learned from your first attempt will help guide your decisions the next time round.
  • Prepare your system before you add new concepts I know that refactoring is all about improving your world without adding new functionality but sometimes people start adding new features before the world is setup for them. Ensuring that your world is good before you add new features will make your life simpler.
  • Introduce new concepts top down Obvious? I thought so, but I have seen a refactoring go horribly wrong because it was started from the wrong place. When you change a class, you can isolate the changes within it, but every consumer of it will be affected. This bubbling effect can be catastrophic if you start from objects in the lowest level.
  • Only have top level tests This one is slightly controversial, and I don't know if I completely agree with it anyway, but if you only test your entry-point object, and never use fake objects during your testing (except to simulate external dependancies like databases or external services) then you will not have the annoyance of changing loads of unnecessary tests. You will also find that you can be 100% certain that your tests are still relevant without touching every single test file.
  • Start with tests which pass Very important to ensure that you don't break anything, so start with passing tests on the highest layer, whether that means browser based tests or controller tests. Make sure you have a test for all scenarios as that way you can easily test that your refactor has not changed the world.
  • Making sure your world better than you found it (or at least no worse) reduces the number of big risky refactor adventures
Do you have any other tricks that you use when you refactor?

Pairing 101: Skills Matter

| Comments

A few weeks ago, my good buddy Christian Blunden and myself presented at a London Geek Night. For us, it was quite fun and I hope people not only picked something up from it but also had a good time. The presentation was recorded and should be on skills matter so check it out if your interested (she says cringing at watching herself on video).