"As Above, So Below", (Macrocosm and microcosm)

Often, doing something small gives an opportunity to understand the mechanics of larger and more complex processes. Let's take a closer look at the relationship between a Kata and how to tackle larger problems.

The Kata

Every two weeks on a Friday afternoon, we have been running a Code Dojo at Simply Business. The idea, if you are not familiar with the concept, is to hone your developer skills through practice and repetition.

Pair programming and test-driven development are two important muscles that get worked during the execution of the kata, both very valuable techniques in writing good code.

At the last Dojo, we practised using the "Counting Code Lines" kata. We had 2 hours, and we were swapping the keyboard every 30 minutes with a break of 5 mins in between - the Pomodoro technique.

The usual flow is something like:

• Write the simplest test to get you going
• Write the simplest implementation that makes your test pass
• Repeat :)

At the end of the session, the different pairs came up with different implementations, and we had a chat about our solutions and the thought process we had used to get there. It was a good learning experience and good fun!

My brain

For a few days after the kata, I had been thinking about what had happened in my brain during the coding session. My colleague Nigel-Runnels Moss (aka Fox) - Principal Engineer at Simply Business, presented the problem. My brain had started two processes:

• Identifying the different parts of the problem (e.g. recognising a comment, single line, multi-line, parsing, performance, etc.)
• Scanning similar problems I'd encountered in the past (a bit like pattern matching)

At that moment, I felt that the "hardest" part of the problem was around the multi-line comments, so the pattern matching algorithm my brain was running was on that particular aspect. What my brain uncovered, without any guarantee of correctness was: Simple API for XML (SAX), abstract syntax tree (AST), finite state machine (FSM), and a bunch of other things.

My hands

A few seconds later, with Punit, my partner in the pairing, we started writing the first simple test to get us going (writing something down as soon as possible is the best icebreaker). I switched my brain into "build mode", silencing the research, confident that the process would guide us to a good place.

The snippet of Java code we were using to test our Thing did not contain any comments or empty lines, so the only thing our algorithm had to do was string.split.count. To reduce the number of concerns, the Thing that was counting did not care for File IO but accepted a simple string.

def count(string_containing_java_code)
    string_containing_java_code.split.count
end

We continued to add additional test cases. The implementation relied heavily on select and reject until we came to write the multi-line test case :).

At that point, our simple approach started to reach its limitations. The base unit of our algorithm was a line (because of the split), but what we were trying to do there needed a different level of granularity.

We tried a couple of different things until we landed on a Regex expression that removed the blocks of text between /* */ replacing it with a \n.

  SINGLE_LINE_COMMENT_DELIMITER = "//"
  MULTILINE_COMMENT_LIMITER=/\/\*.+\*\//m

  def count(string_containing_java_code)
    string_containing_java_code
          .gsub(MULTILINE_COMMENT_LIMITER, "\n")
          .split("\n")
          .map(&:strip)
          .reject(&:empty?)
          .reject{|line| line.start_with? SINGLE_LINE_COMMENT_DELIMITER }
        .count
  end

It worked ... at least until we stress-tested the implementation with some additional tests!

Our implementation could not handle something like System.out.println("/* This is not a comment */"). The bottom line is that the algorithm is not aware of what is surrounding the characters that identify a multi-line comment (e.g. is the comment within double quotes?).

At that point, we ran out of time, but we really had two options:

1. Try hard to make the current implementation work
2. Throw everything away, keep the tests, and rewrite the implementation

Zooming out: "the Above"

Zooming out from the kata, it is not uncommon to find yourself at a fork in the road when working and evolving towards a more complex system. Either option has pros and cons, and in some of the worst cases, you might not even be able to consider the second one.

1) Try hard to make the current implementation work

"The term "Concorde fallacy" derives from the fact that the British and French governments continued to fund the joint development of the costly Concorde supersonic airplane even after it became apparent that there was no longer an economic case for the aircraft."

Wikipedia, Sunk cost

"Let's make it work" is probably the first thing that comes to mind, in the end we are only adding (another) small feature, what could possibly go wrong?

The sunk cost here is that this approach, most of the time, increases the complexity of the system up to a point where it is impossible to make changes. In the worst-case scenario, tech bankruptcy is around the corner.

A dutiful note: systems should always be evolved, refactored, maintained and adapted, and consistency and correctness must be kept in mind at all times.

2) Throw everything away, keep the test and rewrite the implementation

You won the lottery ticket! Let's rebuild it :) Some very good tests, decoupled from the implementation, are in place and you can design a solution that solves the problems you encountered.

This is the best option... if you managed to convince the business to pay for it. The usual issues here are, it takes time & money, no additional features are delivered during the rewriting, and usually, complexity goes up before going down again.

There is also the risk that if you rewrite the system to do exactly what it did before, it will suffer from the exact same problems.

The other option: Think about design before starting

That means, take some time to review the different options you have available, spike with code, use tracer bullets, share your ideas (within the pair or within the wider team).

If you know what the challenges are, face them at the beginning, don't hope for the best. Try to avoid getting to the fork where you have to make some tough decisions.

In the context of creating a new system, often (incorrectly in my opinion) the YAGNI or Agile arguments are used. It is important to notice that Agile or YAGNI don't prescribe avoiding analysing the problem, designing for solutions, or are against architecture.

My approach to architecture is summarised in "the best architect removes architecture" (Martin Fowler. In other words, reducing the need of taking the decision up-front for things that are hard to change (eg. a database, programming language, interfaces, etc.)

For YAGNI, something that often gets missed is the fact that YAGNI is meant to be used in combination with several other practices, such as continuous refactoring, continuous automated unit testing, and continuous integration. Used without continuous refactoring, it could lead to disorganised code and massive rework, known as technical debt. (YAGNI, Wikipedia)

To me this option is a must, in particular when:

• You know the system is going to be part of the production environment and it is going to stay i.e. not an experiment
• It covers a number of functionalities that needs to be evolved over time
• Multiple people are working on it
• It is not easy to replace or is hard to change

Zooming in, "the Below"

In the context of the kata, you don't have to make that tough decision. You simply throw everything away and try a new option because it is cheap.

The main idea is that you are training your brain to recognise common patterns when you are working on some problems and rely on that muscle memory. You already explored the pros and cons of different approaches and it will become more obvious as to how to choose what to do next when you are faced with a work-related challenge.

Big problems are usually the sum of a series of small problems, so it "always works!".

Takeaway

I hope you enjoyed reading this article. Some practical takeaways are:

• Practice on small problems to build your muscle memory (do you need a Dojo?)
• When tackling a new challenge, split it into sub-problems
• Face the big unknown first, don't hope for the best
• Take your time to think, discuss and share your options before starting something that is going to be hard to change later
• If you went down the "wrong path", listen to early feedback (how complex/slow is it to add/change a feature). Don't follow the white rabbit, go back to the drawing board and start changing the design.

You can contact us on the social media links on this page if you'd like to share your experiences of using the coding techniques discussed in this article or if you have any tips to share with the community.