Agile Development

Process

QBurst uses agile development methodologies to improve quality while reducing cost and time-to-market.
Why Agile?
Traditionally, software applications are developed using waterfall methodology. A large percentage of big corporations still continue to follow the waterfall model. As of 2008, surveys show that agile penetration in large enterprise is less than 25%, but growing fast as more and more enterprises understand its benefits. A majority of ISVs and startups already use agile methodologies.
Agile is gaining momentum because of several factors:
  • Businesses operate in a fast changing environment, where they have to respond to market changes quickly by introducing new products or changing existing ones. Typical projects are now 3 months or less and not years as it used to be.
  • New technologies are coming to market every other day, so if a product takes years to build, it sure will get outdated before it launches.
  • The web-based model allows companies to launch a bare bones version of the product first, get feedback from end-users and then incrementally add new features based on that feedback.
Thus launching early, collecting feedback from real users and performing frequent releases is the new way of product development. And agile is closely aligned with this new way.
What is Agile Development?
In the waterfall method, a software development project goes through multiple sequential phases starting with requirements analysis, high-level and low-level design, coding, system and acceptance testing and finally deployment. Requirements are typically frozen after requirements analysis. Any change in requirements after the freeze will typically add to the cost of the project. The cost of making a change progressively increases as the development moves further downstream. In order to avoid this, end-users are asked to state all the features they need upfront resulting in bloated requirements. It does not matter even if those features are rarely, if not ever used.
Agile methodologies take an iterative approach and does not require a requirements freeze upfront. Work is carried out in iterations, which typically last one to six weeks. All of the features required for the project are put in a wish-list. Then depending on business priority, these features are assigned to releases which are tied to iterations.
There are several popular agile methodologies like Scrum and Extreme Programming (XP). We at QBurst create a tailor-made process (typically based on and derived from the Scrum process) for each client taking their specific needs into account. We understand that one size does not fit all and what may work for one may not work for all.
The Scrum Process
Scrum is the default process we use. A custom-made process for a client is usually a variant of Scrum.
Scrum, though often written using upper case letters, is not an acronym. It derives its meaning from a play in Rugby in which the two sets of forwards mass together around the ball and, with their heads down, struggle to gain possession of the ball. Scrum could be used in any project, but is mostly applied in software product development.
The requirements for the product are collected into what is called the product backlog, which is a prioritized wishlist of features. Features may be described using user stories. The Product Owner maintains the product backlog. The Product Owner is a Scrum role that represents the interests of the customer. It could be the customer or a business analyst representing the customer. The product backlog is never frozen, users can suggest new requirements and the Product Owner can add them to the backlog and prioritize it.
The product is developed iteratively in multiple sprints. A sprint is a time-boxed iteration, which usually lasts 30 days. The duration of a sprint can be fixed based on the nature of the project. Typically a release follows a sprint, though a release can be made after multiple sprints. Each sprint will implement a set of features. A sprint backlog is created prior to the beginning of a sprint during the sprint planning meeting. The sprint backlog contains all the tasks that need to be done to implement the features included in that sprint. Once a sprint starts, the scope of the sprint cannot be changed by adding new features. This allows the developers to complete a sprint without external interference. The Scrum Master is responsible for making sure that the process is followed. Typically, this is the Project Manager.
The level-of-effort required for each feature is estimated by the Team, which involves designers, developers and QA. They do this by identifying the tasks that need to be performed to implement a feature and estimating the level-of-effort involved for each task. The estimate is team member's best guess from their prior experience. The more granular the work breakdown, the better the estimate.
A burn down chart is used to track progress of a sprint. It shows the number of hours of work remaining to complete the sprint. As the sprint progresses, developers report, on a daily basis, the amount of hours required to complete each task. Ideally, the work remaining should come down, but it could also go up if the initial estimates were wrong. The slope of the graphs shows the velocity of the project and is helpful in predicting if the project will meet the deadline.
Scrum requires that the Team, Product Owner and Scrum Master have a daily stand-up meeting for 15 minutes. Each team member will say what they have accomplished after the last meeting, what they plan to do before the next meeting and what the impediments are. It is the responsibility of the Scrum Master to remove the impediments and clear the way for the team.
Remote Development and Agile
Most agile methodologies, especially XP, stress the importance of face-to-face meetings between users and developers and also co-locating developers. Having everyone in the same room is always good and paves the way for highly efficient development. But the reality is that development almost always happens in a distributed fashion. All the skills required for executing a project are often not available in the same geographical location.
The current reality is that development happens in multiple locations located in different timezones and with team members from different cultural backgrounds. Is it possible to go agile with such teams? The proof that distributed teams can create great products is discernible from the success of open source projects. Most open source products are created by community developers who work from home and barely see each other.
Modern web-based project management and collaboration tools allow developers and users to work from different locations. Most of these tools are free. Instant messaging, Skype video calls, desktop sharing tools, task tracking appplications like Trac, wiki, version control systems like Subversion and GIT and web-based build and deployment tools like CruiseControl allow seamless communication and collaboration. In recent years, these tools have helped drastically improve the efficiency of remote development. Having overlap time for teams working in different timezones is important too.
Sample Process for a Customer
A tailor-made process devised for a specific customer is described below. This process is derived from Scrum, but is more specifc and detailed to suit the needs of the project.
Development is done in 4 week iterations. Prior to the beginning of an iteration, several things happen. We call these pre-iteration planning. Unlike the Scrum sprint planning meeting, which normally takes place in one long session, this is a more drawn out process that could last weeks. The actors involved in this phase are Customer, Project Manager, Business Analyst and to a lesser extend Developers and QA. Note that during the pre-iteration phase, the developers are working on the previous iteration.
Pre-iteration:
  • Identify features to be implemented in the next iteration from the Product backlog - Customer, Project Manager.
  • Write use cases, create wireframes - Business Analyst, Customer.
  • Identify sprint tasks for implementing the features / use cases - Project Manager, Team.
  • Estimate Level-of-effort (LOE) for each task - Team.
  • Revise scope based on LOE (so that it fits in the 4 week time box) - Customer, Project Manager.
  • Create sprint backlog - Project Manager.
  • Create tasks in Trac (task tracking tool) and assign to team members.
  • Document use cases in Trac wiki.
  • Project Manager creates burn down chart for the sprint.
iter1
Week 1: Days 1 & 2:
  • Team clarifies use cases with customer and BA. They further elaborate use cases and factor-in exceptional and failure conditions.
  • Team refines the wireframes.
  • Creative designer comes up with a design using Photoshop.
  • QA comes up with acceptance criteria.
  • Customer approves use cases, wireframes, creative design and acceptance criteria.
Week 1: Days 3, 4 and 5:
  • Developers create XHTML mocks. The mocks are navigable with dummy data. There is no backend connectivity.
  • The team does UML designs by working together. They create/update class, sequence and ER diagrams.
  • Designs are uploaded to wiki.
  • Proof-of-concepts done.
  • The mocks and designs are approved by customer.
  • Creative Designer provides Photoshop design.
  • QA works on test plans.
iter2
Week 2 and Days 1, 2, 3 of Week 3
  • Team writes unit tests and code. The cycle of test, code, test, code, continues.
  • Scrum-style updates to customer every day through video conference calls.
  • Burn-down charts are updated.
  • End-of-day posts on wiki.
  • Comments are entered against Trac tasks.
  • Code is checked-in to repository and integrated daily.
  • Builds and unit tests are run daily. If those are successful, code is deployed to the integration server. If build or unit tests fail, developer has to fix it the same day, before leaving.
  • QA creates test-cases.
  • QA and Customer: test the product in integration environment (this is more for getting a feel of the new features).
End of week 2
  • Meeting to discuss iteration progress. Items may be de-scoped if Project Manager and Customer agrees that they cannot be completed as planned within the 4 week timebox.
iter3
Week 3: Days 4 and 5
  • Create release documentation and database migration scripts.
  • QA to deploy in staging.
  • Run migration trials in staging.
  • QA starts testing in staging.
iter4
Week 4: Days 1 and 2
  • QA tests in staging and reports bugs.
  • QA runs automated regression tests.
  • QA performs performance and stress tests.
  • Development fixes bugs.
Week 4: Days 3, 4 and 5
  • Acceptance testing - Customer tests in staging and reports bugs.
  • Development fixes bugs.
  • Final regression tests by QA.
Post-iteration
  • QA works on test automation (new test cases are automated to the extend possible).
  • Release code to live.
  • Smoke tests in live environment by QA.
process