What are spillovers in Sprint/Scrum?

In scrum, if all the sprint backlog items are not completed by end of the sprint, the incomplete sprint backlog items are moved back to the product Backlog and is called spillover. The spillover backlog items will be taken up in a future sprint. Having spillovers in sprint means the development team could not complete the product backlog item they have committed during sprint planning. Such a situation is not desirable to the team. The Scrum team strives to avoid spillover as far as possible and as the team metures, the development team is able to forecast the effort better avoiding spillovers.

Following are some of the main reasons for spillovers in a sprint.

• Underestimation of effort
• Over estimation of team capacity
• Unexpected extra work or blockers
• Lack of coordination in team
• Lack of focus
• Taking up an unclear product backlog item in the sprint and so on..

Let us see how the spillover can be avoided.

Do an effective capacity planning 

Capacity planning is an assessment done by the team during sprint planning to see how many story points can be taken up by the team in that particular sprint. For those who are new to the world story point, it is a unit of measure for the effort required to complete a feature using relative sizing. If a team is not able to reasonably assess their capacity it is likely that they will either over comment or under commit on the features they will be able to complete in the sprint. In order to effectively plan the team capacity following factors needs to be considered

• Number of working days in the Sprint
• Any planned leaves or vacations for the team members
• Availability of shared resources
• Hours to be dedicated in the sprint for backlog refinements and other events like Sprint planning, daily scrum, sprint review and sprint retrospective
• Average velocity of previous sprints

Capacity of the team is calculated in story points. It is a common practice to calculate the capacity of a team by projecting the man hours or days available in this sprint and deriving a proportionate story point based on average velocity of the team and average man hours or day of previous sprint. It would be like:

Capacity= Projected man hours for this sprint * ( Average velocity of the team/ average man-hours spent in previous sprint)

Take up only thoroughly refined product backlog items in a sprint

Developers should take up product backlog items in a sprint only if it is clear and all dependent information required for its implementation is available. Backlog items created by the product owner become metur only when it is explained to developers and is discussed during the refinement process. This brings required clarity, enabling the development team to come up with an effort estimate. In order to make sure only right backlog items are brought up in sprint planning often the team agrees on a definition of ready(DOR) which sets criterias that needs to be fulfilled for a product backlog item to be taken up in a sprint. Following are few examples of what all can be included in Definition of ready :

• The backlog item is explained by product owner to the development team and all questions are answered 
• The product backlog item has an acceptance criteria based on which the feature can be validated
• The product Backlog item is estimated in story points
• Estimate of the Backlog item does not exceed 8 story points. If it exceeds it is subdivided into smaller stories or backlog items
• There is no blocker dependency

Effectively plan the work during sprint planning

Sprint planning consists of two main activity

• Determining What to do-Deciding which product Backlog item is to be taken up in the sprint
• Determining How to do- analysing the backlog items and coming up with a plan of how to get it done

Initial two points that we discussed take care of the ‘what to do’ part, however the ‘how to do’ part is not given the importance it deserves by many teams, which could lead to spillover in the sprint. Planning involves two main functions 

• Analyzing the backlog item and dividing it into smaller tasks.
• Estimating the effort required for each task in hours

Following are some points to be considered while creating tasks

• Try to divide the backlog items to as small tisk never exceeding one day of effort
• If multiple developers can work on a task, try to create separate task for each person instead of multiple people working on single task
• Look into ‘Definition of Done’ for the backlog item and create a task for each point in the DOD in addition to the regular development task. For example there could be criteria like code review, automated tests, performance testing , Product owner review etc. There should be separate task for each of these points 

Proper plan forces the developer to think about practical aspects of requirements which may be overlooked during the refinement process. At this stage, if the developer feels that the initial assessment of effort was not enough, there is an option to negotiate with the product owner and make room for it. This will reduce the chances of underestimation and uncertainties and will help avoid spillover in a sprint.

Learn from past mistakes

It is normal to have spillovers in a sprint in the initial days. However after a few sprints the team learns from their mistakes and matures, enabling them to avoid spillovers in future sprints. We know Inspection and adaptation are two of the three pillars of scrum framework and sprint retrospective is a scrum event specifically used for inspecting and adapting processes. If spillovers are frequently occurring,it should be taken up in the retrospective meeting, where the root cause of the spillovers should be analyzed. This would help to identify and fix issues related to processes that lead to the spillovers. 

Reduce work in progress

Spillover of the sprint backlog happens when the team does not have enough time to complete all the agreed backlog items. The reason for this shortage of time could vary, but if such situations arise, the focus should be to reduce the number of product backlog items that spills over. A tip to handle this situation is to reduce the work in progress or number of backlog items on which the team is working parallely . 

Let’s try to understand this with an example. Assume a scenario where in a sprint the team has committed 5 backlog items totalling 25 story points . For simplicity let’s assume that all backlog items have an estimate of 5 points each. The team starts working on all backlog items simultaneously. As the sprint is about to end the team realizes that their actual capacity was only 20 story points. Since work on all backlog items was going on in parallel, by the end of the sprint none of the product backlog items gets completed. The team’s velocity for the sprint will be 0 as a result. This was a scenario where the work in progress was maximum. 

Now let’s assume the same team has decided to cap the maximum number of backlog items to be worked on parallel to 3. In this case the team takes up 3 backlog items first and once those are completed the next 2 are taken . In this case the team will be able to complete the first set of 3 backlog items which adds up to 15 story points, but will not be able to complete the next 2. So the velocity of the team is 15 in place of 0 in the previous case.

Now if the team focuses all its effort only on one backlog item at a time, the team will be able to complete 4 backlog items with only one spillover. Here we saw three scenarios where we saw how spillovers in sprint can be reduced just by reducing the work in progress even if all other factors are the same. 

In the real world it may not be efficient for the entire team to work on just one backlog item at a time. The team will have to find an optimal level so that efficiency is not impacted much, at the same time work in progress is also reduced.

Effectively use the process of Inspect and Adapt in Daily scrum

The daily scrum meeting is an important scrum event where,  on a daily basis  the team inspects and adapts their progress towards their scrum goal and takes corrective action if there is any deviation. Many times the daily scrum meeting is conducted mechanically without understanding its core principles. This could result in the team losing focus from the bigger picture and resulting in spillovers. Spillovers could be avoided, if deviations are detected early in the sprint and corrective actions are taken. Following are some good practices to be followed in daily sprint to avoid spillovers:

• Keep the burn down chart visible to all team members during the daily Scrums, review it and make it a discussion point
• Scrum masters need to make sure that all the impediments are sorted out and the team members are not stuck.
• The team members should be transparent and should ask help if they need it.

Stay focused on the sprint goal

Every sprint needs to have a sprint goal. Sprint goals are often neglected and in many cases are just given for name sake. Purpose of the sprint goal is to keep the effort of the team focused and as a result increase efficiency. Not having a proper sprint goal results in unrelated backlog items pulled into the sprint and the effort of team members are not focused. Having a good sprint goal would also ensure that the team members do not Just focus on the task they are working on and work to achieve the sprint goal. If anyone is facing an issue that compromises the sprint goal, that becomes the team’s issue and the entire team puts a conscious effort to sort that out. This ensures that everyone completes their work and spillover can be avoided in the sprint

Protect the team from external distraction 

If the team.members are constantly distracted it is unlikely that they will be able to focus and complete their work . This will lead.to spillover in sprint . It is the job of scrum master to ensure that there are no external distractions for the developer that hampers the work.  Following are the factors that could distract the developer:

• Work allocated in other projects in case of shared resources
• Too much interference in ‘how to’ part from product owner
• Too much request for help by an inefficient team member
• Other non project responsibilities assigned

With no external distraction the developers will be able to do their job and fulfill their commitment. This will help avoid spillover in the sprint.

Leave enough room for contingencies

As the team matures they are able to refine the backlog well, do better estimation, plan their capacity well and so on, resulting in reduced spillovers. However there could be unplanned activities or contingencies the team may have to handle during the course of sprint which may result in spillovers. Examples of some of these contingencies are:

• The team had to work on critical production issues reported during the course of the sprint
• Unavailability of developers due to unplanned leaves 
• Team members falling ill
• Blockers related to dependent third party softwares  or components
• Delays in inputs from dependent team
• Infrastructure issues like network disruption or server issues

These are events outside the control of the tem.To handle this It is good practice to leave a small part of the team capacity unutilized while pulling backlog items to sprint. For example if the capacity of the team is 50 story points, the team will pull backlog items of only say 40 story points to the sprint backlog. The margin here depends on project to project and teams expectation on the likelihood of these disruptions.

Here the obvious question that arises is , will that not affect the velocity of the team. In the above example where the team had the capacity to deliver 50 story points they only deliver 40. The answer here is No, there is nothing stopping the development team from pulling more stories from the product Backlog to sprint backlog once all sprint backlog items are completed early.

Effectively handling change requests

Sometimes a product backlog item could not be closed due to time taking issues which could not be fixed by the end of the sprint and that backlog item is spilled over. Here it is very important to distinguish between genuine issues and issues that are reported by product owner as bugs, but are really change requests. 

Sometimes it is very tricky to distinguish between these two. Obviously if there is a bug, it has to be fixed and the backlog items should not be closed without fixing its issues and closing it. However in case of change requests it is not necessary to keep the original product Backlog open because the change requests based on the product owner’s review is not completed. It is sometimes seen that there is a disagreement between the developers and product owner on whether an issue is a bug or change request, where what the product owner considers as a bug, the development team considers as change requests. Again, even in case of a change request there is a confusion whether to take it up on current sprint or the next one. The agile principle guides us to welcome changes even if it comes late . Taking up changes by the end of the sprint could cause the backlog item to spillover. Following are some guidelines that can be adopted by the scrum team to deal with these confusions:

• Acceptance criteria should be clearly defined so that developers understands the expected behaviour clearly and can be used as a guide for testing
• If there is a confusion on whether an issue is a bug or change requests, refer acceptance criteria on what it says about the behaviour. If the issue is breaking acceptance criteria it should be considered a bug. If it is not covered in acceptance criteria, the benefit of doubt goes to the developer and the issue should be considered a change requests
• If there is a change request which does not affect the estimate substantially, it can be taken up in the current sprint itself.
• If the change request requires a re estimation of the backlog item and the new estimate can be accommodated within the current sprint, it should be taken up in the current sprint.
• If after re estimation it is assessed that the change cannot be completed within the current sprint, the change can be added to the product backlog as a new product backlog item.
• If the change requests make the existing backlog item invalid the original backlog item can be removed from the sprint. And a new backlog item for the change request can be created. Whether this new backlog item needs to be taken up in the current sprint will depend on its estimate and whether it can be completed in the current sprint.

Always automate testing

Automated testing is a must in agile product development. Having  an eight to hundred percentage coverage of automated test cases may often seem donating and unnecessary. Infact during the initial stage of product development writing automated unit and functional test cases may seem an extra overhead, however the value of writing automated test cases will be apparent as the sprints proceeds. Remember the product increment that is expected after the end of each sprint consist of the features completed in the current sprint along with features completed in all previous sprints. All increments should ideally be in a state that can potentially be deployed to production, so there is no compromise on testing. This means there should be a thorough testing of new features along with regression testing of features developed in all previous sprints. Imagine the situation in a project with no automated test case in its 40th sprint,a substantial portion of the sprint will be required just to do regression testing. There will be situations where not even a single story could be closed following a standard definition of done. Only option to avoid spillover here will be to have maximum coverage of automated testing. 

Closing note

Avoiding spillovers in sprint is necessary and we saw tips to avoid spillovers in a sprint. However this does not mean that the team tries to game the process in order to mark sprint backlog items as complete.  The scrum team should strive to close all the committed backlog items in a sprint , but the product backlog items should be complete in its true sense and should deliver value to its users.