Introduction to Agile Development and Scrum

It has these modules…

Introduction to Agile and Scrum

> 70% of organizations have incorporated some Agile approaches. — Project Management Institute
28% more successful using agile than traditional projects. — Price Waterhouse Coopers
47% agile transformations’ failure rate. — Forbes
- #1 reason is inexperience with implementing and integrating the Agile methodology. — VersionOne
Agile is a mindset that requires culture change.
It’s hard to learn Agile from just reading a book.
Recognizing when something is wrong is just as important as knowing how to do something right.

Introduction to Agile Philosophy: Agile Principles

What is Agile?

Agile is an iterative approach to project management that helps teams be responsive and deliver value to their customers faster

Agile defining characteristics:

Agile emphasizes:

Adaptive planning
Evolutionary development
Early delivery
Continual improvement
Responsiveness to change

Agile Manifesto:

We have come to value:

Individuals and interactions over processes and tools.

Working software over comprehensive documentation.

Customer collaboration over contract negotiation.

Responding to change over following a plan.

That is, while there is value in the items on the right, we value the items on the left more.

Agile Software development:

An iterative approach to software development consistent with Agile Manifesto
Emphasizes flexibility, interactivity, and a high level of transparency
Uses small, co-located, cross-functional, self-organizing teams

Key takeaway:

Build what is needed, not what was planned.

Methodologies Overview

Traditional Waterfall Development:

Problems with waterfall approach:

No provisions for changing requirements
No idea if it works until the end
Each step ends when the next begins
Mistakes found in the later stages are more expensive to fix
There is usually a long time between software releases
Teams work separately, unaware of their impact on each other
The people who know the least about the code are deploying it into production

Extreme Programming (XP)

In 1996 Kent Beck introduced XP
Based on an interactive approach to software development
Intended to improve software quality and responsiveness to changing customer requirements
One of the first Agile method

Extreme Programming values:

Simplicity
Communication
Feedback
Respect
Courage

Kanban

What is Kanban?

Kanban | ‘kanban | noun

(also Kanban system) a Japanese manufacturing system in which the supply of components is regulated through the use of an instruction card sent along the production line.

An instruction card used in a Kanban system.

Origin

1970s: Japanese, Literally mean ‘billboard, sign’

Core principles of Kanban:

Visualize the workflow
Limit work in progress (WIP)
Manage and enhance the flow
Make process policies explicit
Continuously improve

Working Agile

Working in small batches
Minimum Viable Product (MVP)
Behavior Driven Development (BDD)
Test Driven Development (TDD) (Gherkin Syntax — Developed by Cucumber Company)
Pair programming

Introduction to Scrum Methodology

Agile and Scrum:

Agile is a philosophy for doing work (not prescriptive)
Scrum is a methodology for doing work (add process)

Scrum Overview

Scrum:

Is a management framework for incremental product development
Prescribes small, cross-functional, self-organizing teams
Provides a structure of roles, meeting, rules, and artifacts
Uses fixed-length iterations called sprints
Has a goal to build a potentially shippable product increment with every iteration
Easy to Understand – Difficult to master

Sprint:
A sprint is one iteration through the design, code, test, and deploy cycle
Every sprint should have a goal
2 weeks in duration

Steps in the Scrum process:

Agile development is iterative:

The 3 Roles of Scrum

Scrum roles:

Product owner
Scrum master
Scrum team

Product owner:

Represents the stakeholder interests
Articulates the product vision
Is the final arbiter of requirements’ questions
Constantly re-prioritizes the product backlog, adjusting any expectations
Accepts or rejects each product increment
Decides whether to ship
Decides whether to continue development

Scrum master:

If your team is experienced, you might skip this role, but if you have a team new to Scrum, you require an experienced Scrum master.
- Facilitates the Scrum process
- Coaches the team
- Creates an environment to allow the team to be self-organizing
- Shields the team from external interference to keep it “in the zone”
- Helps resolve impediments
- Enforces sprint timeboxes
- Captures empirical data to adjust forecasts
- Has no management authority over the team

Scrum Team:

A cross-functional team consisting of
- Developers
- Testers
- Business analysts
- Domain experts
- Others
Self-organizing
- There are no externally assigned roles
Self-managing
- They self-assign their own work
Membership: consists of 7 ± 2 collaborative members
Co-located: most successful when located in one team room, particularly for the first few Sprints
Dedicated: Most successful with long-term, full-time membership
Negotiates commitments with the product owner – one sprint at a time
Has autonomy regarding how to reach commitments

Artifacts, Events, and Benefits

Scrum Artifacts:

Product backlog
Sprint backlog
Done increment

Scrum events:
Sprint planning meeting
Daily Scrum meeting (a.k.a. daily stand-up)
Sprint
Sprint review
Sprint retrospective

Benefits of Scrum:
Higher productivity
Better product quality
Reduced time to market
Increased stakeholders satisfaction
Better team dynamics
Happier employees

Scrum vs. Kanban:

Organizing for Success: Organizational impact of Agile

Organize for success:

Proper organization is critical to success
Existing teams may need to be reorganized

Conway’s Law:

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.”

— Melvin Conway, Datamation, 1968

Examples of Conway’s Law:

If you ask an organization with four teams to write a compiler … you will get a 4-pass compiler!

How teams should be aligned?
Teams are loosely coupled, tightly aligned
Each team has its own mission aligned with the business (like a “mini startup”)
Teams have end-to-end responsibility for what they build
The long-term mission is usually around a single business domain

Autonomy is important:
It’s motivating – and motivated people build better stuff
It’s fast – decisions happen locally in the team
It minimizes handoffs and waiting, so teams don’t get bogged down

The Agile dilemma!

The entire organization must be Agile:

Agile + DevOps = Alignment

Mistaking Iterative Development for Agile

The biggest pitfall for companies is that, they think they’re agile, but actually they’re just doing an iterative work.

Agile is not…

Agile isn’t a new version of a waterfall, software development life cycle (SDLC), where you do legacy development in sprints
Agile isn’t just the developers working in each sprint, it involves a cross-functional team
The Agile Manifesto doesn’t include the term “Agile project management” (and so there are no “project managers” in Agile)

Agile Planning

Planning to be Agile: Destination Unknown

Deadlines:

“I love deadlines… I like the whooshing sound they make as they fly by”. – Douglas Adams

How do you avoid this?

Plan iteratively:
Don’t decide everything at the point when you know the least
Plan for what you know
Adjust as you know more
Your estimates will be more accurate

Agile Roles and the Need for Training

Formulas for failure:

Product manager becomes product owner
Project manager becomes scrum master
Developers (alone) become scrum team

Product Manager vs. Product Owner:

Project Manager vs. Scrum Master:

Development Team vs. Scrum Team:

“Until and unless business leaders accept the idea that they are no longer managing projects with fixed functions, timeframes, and costs, as they did with waterfall, they will struggle to use agile as it was designed to be used.”

— Bob Kantor, Founder Kantor Consulting Group, Inc.

The roles have changed:

You cannot put people in new roles without the proper training and mindset
This mindset must come down from upper management

Kanban and Agile Planning Tools

Agile planning tools:

Tools will not make you Agile
Tools can support your Agile process
Many Agile planning tools
ZenHub is one of them

ZenHub:
Plug-in to GitHub
Provides a kanban board and project management reporting
Customizable and integrated with GitHub

Why use ZenHub?
Helps you manage where you are in a project based on GitHub Issues
Provides an easy way to let management know how you are doing
Maintains up-to-date status due to integration with GitHub
Allows developers to only use one tool – GitHub

What is Kanban Board?

Real World Example:

Default ZenHub pipelines:

User Stories: Creating Good User Stories

What is a user story?

A user story represents a small piece of business value that a team can deliver in an iteration.

Story contents:

Stories should contain:

A brief description of the need and business value
Any assumptions or details
The definition of “done”

Story description:
User stories document a persona requesting a function to achieve a goal:

As a <some role>

I need <some function>

So that <some benefit>

Assumptions and details:

It’s important to document what you know;
List any assumptions
Document any details that may help the developer

Acceptance criteria:
It is critical to document the definition of “done”
I like to use the Gherkin syntax

Given <some precondition>

When <some event happens>

Then <some outcome>

Sample Story:

Bill Wake’s INVEST:

Independent
Negotiable
Valuable
Estimable
Small
Testable

Epic:
A big idea
A user story that is bigger than a single sprint
A user story that is too big to estimate on its own

When to use an epic?
When a story is too large in scope it is considered an epic
Backlog items tend to start as epics when they are lower priority and less defined
For sprint planning, epics should be broken down into smaller stories

Effectively using Story Points

What are story points?

Story point;

A metric used to estimate the difficulty of implementing a given user story
An abstract measure of overall effort

What does a story point measure?

Relative T-Shirt sizes

Story points acknowledge that humans are bad at estimating time-to-completion
Instead, story points use relative T-Shirt sizes (S, M, L, XL)
Most tools use Fibonacci numbers (1, 2, 3, 5, 8, 13, 21)

Agree on what “medium” means:
Since story points are relative, it’s important to agree on what “medium” is
Then, evaluate from there
Is it the same, larger, or smaller than medium

Story size:
A story should be small enough to be coded and tested within a single sprint iteration – ideally, just a few days
Large stories should be broken down into smaller ones

Story point antipattern
Equating a story point to wall-clock time
Humans are bad at estimating wall-clock time
Don’t do it!

Building the Product Backlog

Steps in the Scrum process:

Product Backlog:

A product backlog contains all the unimplemented stories not yet in a sprint
Stories are ranked in order of importance and/or business value
Stories are more detailed at the top, less detailed at the bottom

Sample requirements:

What: A service for counting things
Must allow multiple counters
Counters must persist across restarts of service
Counters can be reset

ZenHub Kanban board:

Creating new stories

Story Template:

As a <some role>

I need <some function>

So that <some benefit>

Need a service for counting things:

As a User

I need a service that has a counter

So that I can keep track of how many times something was done

Creating the next story:

Must allow multiple counters:

As a User

I need to have multiple counters

So that I can keep track of several counts at once

Creating the next story:

Persist counters across restarts:

As a Service Provider

I need the service to persist the last known count

So that users don’t lose track of their counts after the service is restarted

Creating the last story:

Counters can be reset:

As a System Administrator

I need the ability to reset the counter

So that I can redo counting from the start

Stories in the backlog:

Prioritize the product backlog:

The Planning Process

Backlog refinement:

Keep the product backlog ranked by priority so that the important stories are always on the top
Break large stories down into smaller ones
Make sure that stories near the top of the backlog are groomed and complete

Backlog refinement meeting:

Who should attend?
Product owner
Scrum master
Development team (optional)
- Lead developer/architect
  
  What is the goal?
Groom the backlog by ranking the stories in order of importance
Make sure the story contains enough information for a developer to start working on it

Backlog refinement workflow:

New issue triage:

Start with new issue triage
Goal: At the end of backlog refinement, the New Issues column is empty

Take stories from new issues and…
Move them into the product backlog if they will be worked on soon
Move them into the icebox if they are a good idea but not now
Reject them if they are not where you want to go

Backlog refinement workflow:

Product owner sorts the product backlog in order of importance
The team may provide estimates and other technical information
Large vague items are split and clarified
The goal is to make the stories “sprint read”

Complete the story template:

As a <some role>

I need <some function>

So that <some benefit>

Assumptions and Details:

<anything you already know>

Acceptance Criteria:

Given <some precondition>

When <some event>

Then <some measurable outcome>

Need a service that has a counter:

Must Persist counter across restarts:

Deploy service to the cloud:

Ability to reset the counter:

Label:

Help visualize the work

Labels in GitHub

Need a service that has a counter:

Must persist counter across restarts:

Deploy service to the cloud:

Ability to reset the counter:

Technical debt

Technical debt is anything you need to do that doesn’t involve creating a new feature
Technical debt builds up when you take shortcuts, but may also occur naturally

Examples of technical debt:
Code refactoring
Setup and maintenance of environments
Changing technology like databases
Updating vulnerable libraries

You should refine the backlog every sprint to ensure the priorities are correct
Have at least two sprints’ worth of stories groomed
The more time you spend refining the backlog, the easier sprint planning will be

Sprint Planning

The purpose of sprint planning is to define what can be delivered in the sprint and how that work will be achieved
This is accomplished by producing a sprint backlog

Sprint planning meeting

Who should attend?

Product owner
Scrum master
Development team

Sprint planning goals:
Each sprint should have a clearly defined business goal
The product owner describes the goal and product backlog items supporting it
It’s important for the whole team to understand why they are building the increment

Mechanics of sprint planning

The development team;
Takes stories from the top of the product backlog and assigns them to the sprint backlog
Assigns story points and labels
Ensures each story contains enough information for a developer to start working on it
Stops adding stories when the team’s velocity is reached

Team velocity:
The number of story points a team can complete in a single sprint
This will change over time as the team gets better at estimating and better at executing
The velocity is unique to the team because the story point assignment is unique to the team

Create a sprint milestone:
Create a sprint milestone to start the sprint
The milestone title should be short
The description should document the milestone goal
The duration should be 2 weeks

Create a milestone:

Executing the Plan

Workflow for Daily Plan Execution

Steps in the Scrum Process:

The Sprint:

A sprint is one iteration through the design, code, test, deploy cycle
It is usually 2 weeks in duration
Every sprint should have a goal

Daily Execution:
Take the next highest priority item from the sprint backlog
Assign it to yourself
Move it in process

No one should have more than one story assigned to them unless they are blocked
When you are finished, move the story to Review/QA and open a PR
When the PR is merged, move the story to the Done column

The Daily Stand-Up

Occurs every day at the same time and place
Sometimes called the “daily Scrum”
Each team member briefly reports on their work
Called a “stand-up” during the meeting to keep it short
Timeboxed to 15 mins
Not a project status meeting – all status should be tabled for later discussion

Daily stand-up meeting:

Who should attend?
Scrum master
Development team
Product owner (optional)

Daily stand-up question:

Each team member answers three questions:
1. What did I accomplish the previous day?
2. What will I work on today?
3. What blockers or impediments are in my way?
Impediments and blockers:
Impediments identified by the team should be unblocked by the scrum master
Developers that are blocked should work on the next story

Tabled topics:
Topics raised during the daily stand-up should be held until the meeting has ended
Anyone interested in those topics can stay to discuss

Completing the Sprint

Using Burndown Charts

Milestones and burndowns:

Milestones can be created for anything in your project
- sprint, beta drop, demo, release…
Burndown charts can be used to measure your progress against a milestone

Burndown chart:
The measurement of story points completed vs. story points remaining for a sprint
Over time the story points remaining should go down, hence the name: burndown

Burndown chart examples:

The Sprint Review

Live demonstration of implemented stories
Product owner determines if stories are done based on acceptance criteria
Done stories are closed

Sprint Review meeting:

Who should attend?
Product owner
Scrum master
Development team
Stakeholders
Customers (Optional)

Sprint review:
Feedback gets converted into new product backlog stories
This is where iterative development allows the creation of products that couldn’t have been specified up-front in a plan-driven approach

Rejected Stories:
What about stories that are not considered done?
Add a label to indicate this and close them
Write a new story with new acceptance criteria
This will keep the velocity more accurate

The Sprint Retrospective

A meeting to reflect on the sprint
Measures the health of the process
The development team must feel comfortable to speak freely

Who attend the meeting:
Scrum master
Development team

A time for reflection:

Three questions are answered:
1. What went well? (keep doing)
2. What didn’t go well? (stop doing)
3. What should we change for the next sprint?
The goal is improvement:
This is critical for maintaining a healthy team
The scrum master must ensure that changes are made as a result of the feedback
The goal is to improve for the next sprint

Measuring Success

Using Measurements Effectively

Measurements and metrics:

You can’t improve what you can’t measure
High performing teams use metrics to continually improve
They take baselines and set goals and measure against them
Beware of vanity metrics
Look for the actionable metrics

Baselines and Goals:

Baseline:
It currently requires 5 size team members, 10 hours to deploy a new release of your product
This costs you $X for every release

Goals:
Reduce deployment time from 10 hours to 2 hours
Increase percentage of defects detected in testing from 25% to 50%

Top 4 actionable metrics:

Mean Lead Time

How long does it take from the idea to production?

Release Frequency

How often can you deliver changes?

Change Failure Rate

How typically do changes fail?

Meantime to Recovery (MTTR)

How quickly can you recover from failure?

Example metrics:

Reduce time-to-market for new features
Increase overall availability of the product
Reduce the time it takes to deploy a software release
Increase the percentage of defects detected in testing before production release
Provide performance and user feedback to the team in a more timely manner

Getting Ready for the Next Sprint

End of sprint activities:

Move stories from done to closed
Close the current milestone
Create a new sprint milestone
Adjust unfinished work

Handling untouched stories:
Stories not worked on can be moved to the top of the product backlog
Resist the urge to move them to the next sprint
Remember to unassign them from the sprint milestone

Handling unfinished stories:
Don’t move unfinished stories into the next sprint!
Give the developers credit for the work they did
This will keep your velocity more accurate
Adjust the description and story points of the unfinished story, label it unfinished, and move it to done
Write a new story for the remaining work
Assign remaining story points and move it to the next sprint

Ready for the next sprint:
All stories assigned to the current sprint are closed
All unfinished stories are reassigned
The sprint milestone is closed
A new sprint milestone is created

Agile Anti-Patterns and Health Check

Agile Anti-Patterns:

No real product owner/Multiple product owners
Teams are too large
Teams are not dedicated
Teams are too geographically distributed
Teams are siloed
Teams are not self-managing

YOU WILL FAIL!

…and you should not wonder why.

Scrum health check:

The accountabilities of product owner, development team(s) and Scrum master are identified and enacted
Work is organized in consecutive sprints of 2–4 weeks or fewer
There is a sprint backlog with a visualization of remaining work for the sprint
At sprint planning a forecast, a sprint backlog, and a sprint goal are created
The result of the daily Scrum is work being re-planned for the next day
No later than by the end of the sprint, a Done increment is created
Stakeholders offer feedback as a result of inspecting the increment at the sprint review
Product backlog is updated as a result of the sprint review
Product owner, development team(s) and Scrum master align on the work process for their next sprint at the sprint retrospective