August 12, 2013

Legacy project - improving code quality by identifying technical debt

I have always worked on new projects where we decide what technologies/framework/language to use. It is easier to follow good development practices from day one and Continuous delivery in a project is the new norm. More often large enterprises have a legacy projects that has to be maintained and enhanced with new features. 

I recently had an opportunity to work on a consulting assignment where we had to work with client team, up skill them with good development practices and move towards continuous delivery model for their business critical application. It was a large program with 7 scrum teams and each team having 6-8 members. We initially started with couple of scrum teams as a POC(Proof of Concept) and tried to identify the issues/blockers in the process. These are the steps we followed while we tried to improve the code quality of a legacy project.

1. Identifying what to fix
Every project that has good and bad parts of the code. First step in improving a legacy project is to identify what needs to be fixed in the existing system.  
  • List the high risk areas in the codebase based on experience - There will be some parts of the code that no developer wants to touch. Every time we make a change to that piece of code hell breaks loose. Identify those classes/modules and categorize it as High/Medium/Low.
  • Run static code analysis tool (like Sonar for Java project) - Static code analysis tools immediately shows you the current state of the code.
  • Generate Heatmap from Version Control - There are few parts of the code that gets modified very frequently. It is important to identify those classes so that it can be refactored to smaller classes, wrapped with unit tests to improve resilience. 
  • Heatmap - Identify frequent code checkins for fixing bugs. Based on the commit message identify the classes that get frequent bug fixs.     
2. Prioritizing the changes
  • Code which causes most of the defects
  • Code where the new enhancement work is going to happen
  • Bugs identified by the static code analysis tools
  • Refactor code to fix the cyclomatic complexity, large classes/methods, etc based on static code analysis (if it satisfies 1 & 2 conditions)
3. Items to exclude from scope
 This is one of the important lessons I learnt while dealing with legacy projects. If you come across a piece of code that is "working" and if there is no foreseeable changes happening in that code, don't fix/clean the code. Add it to the backlog of "code to be refactored" with a low priority. As a developer it is common to have an urge to clean everything, but we should keep in mind that lot of companies/projects have gone bust by spending their time and money on rewriting software. 
4. Scheduling the changes as part of on going work
Once the backlog of tech debt has been identified, we need to plan it as part of on-going project work. Avoid the idea of spending dedicate time (few sprints/months) just for refactoring. This would not fly well with business as they don't see any returns after few months of refactoring.     
I would recommend spending 10% - 40% of each sprint capacity to fixing the Tech debt based on the backlog you had come up with.
Also, In agile world, don't add story points to tech debt stories. You are not delivering new "business value" and it should reflect in the velocity. It would make it obvious to the business on how much time is invested in tech debt and over time velocity would increase as the tech debt decreases.

5. Prepare the team 
If the team is not familiar with TDD and other development practices, it would help in training the team before venturing into refactoring. 
          - TDD techniques
          - Design patters
          - Pair programming
          - Brown bag sessions
          - Code showcase
Few recommended books:
  • Refactoring
  • TDD
  • Working with Legacy code
  • Refactoring DB
  • Pragmatic Programmer 
6. Working on the improvements
  • Hire expert/coach to help team on the job (CRITICAL)
  • Break the silos (Architects, DBA, etc)
  • Retrospect frequently
  • Identify, raise and fix the blockers.
  • Improve efficiency
    • Invest in new infrastructure (Cloud - is not a differentiator, a necessity)
    • Find alternate tools (Tomcat instead of Websphere, Weblogic, etc, Local database for each developer)
    • Good IDE environment
      • Emphasis on using shortcuts
      • Never write code if IDE can generate
    • Faster build
    • Identify bottlenecks
7. Track the improvements
  • Review the static code analysis metrics
  • Review the code coverage metrics
  • Introduce code coverage ratchet (Good and bad.. )
  • Track individual commits
  • Build a dashboard
  • GREEN build is a must
  • Don't refactor if it is not in the prioritized list. This is a important item to track as it would increase the scope of work.
Testing Process & Deployment Process improvements:
Will be covered in the next blog post.