Keeping track of frequent releases by independent teams is just too much cognitive load. We need to Work in Small Batches. The DevOps Release Convoy™ (DORC) simplifies things. Why burden ourselves tracking 5 or 10 releases per quarter if we can use 1 easily managed DORC™?
Set sail with the DORC!
Why Convoys?
Other frameworks have tried to coordinate things with more linear transportation metaphors. However, they are constrained to a single direction. With DORCs™ we have more options. We can turn left or right or even make a U-turn!
1 - Convoy Ceremonies
The meetings required to manage changes in the SADMF way!
The meetings required to manage changes in the SADMF way!
Captain’s Mast
In this ceremony, anyone wishing to change the priorities set in Convoy Alignment must file a PCR and represent it for approval. This allows the Chief Signals Officer to adjust the Completed vs Committed goal to ensure it does not reflect poorly on the Commodore.
Captains’ Meeting
Meeting of the Feature Captains to plan the date when the DORC™ will be assembled.
Coding
The group of Code Engineers assigned to each feature will implement the requirements they are given as rapidly as possible using the industry best practice, Continuous Isolation. This ensures it can be tested and delivered on the Convoy it is planned for. Once the feature is coded, the Code Engineers can return to the coding pool to await their next adventure!
Code Inspection
The CSET will review the completed code before it is tested to ensure it is compliant with CSET and EARB standards.
Convoy Alignment
This is a 5-day meeting held every 6 weeks for planning the next 8 quarters of features to ensure the critical paths are aligned. See the agenda for details.
Dry Dock
It’s normal for a little damage to occur when we are moving so quickly. Defects will accumulate. The Dry Dock is the process of halting feature development for a few weeks so that repairs can be made.
Manifest Approval Ceremony
Before any convoy may “leave port”, it’s critical we ensure that all Scaled Agile DevOps processes have been followed. The Convoy Steering Committee is convened on the day the fleet sets sail. See Deploy the Fleet for the detailed process.
Post-Standup Standup
Only the most important status updates are given in Standup and some defects are lower priority than some features. To address this, we use the Post Standup Standup ceremony. Here everyone on the Convoy participates to provide status updates to the Commodore on the lower priority work that is not being worked on.
Post Standup Standup Review
To ensure no information is lost through insufficient documentation, each Feature Captain will submit a daily report using the SAD Update form and email it to the Commodore who will consolidate it and file it.
Press Gang
In the Press Gang ceremony, each Feature Captain chooses 2 to 20 Code Engineers from the coding pool to deliver the next feature. This ensures each Code Engineer is allowed to work on new and interesting things and that all are fully utilized.
Scrum of Scrum of Scrums
To effectively scale communication, we use the Scrum of Scrum of Scrums. First thing in the morning after the daily scrum, each team selects a Tribute to attend the daily Scrum of Scrums (SOS). At noon your Tribute attends a Scrum with the Tributes from the other teams. They select a Tribute of Tributes from the SOS meeting who, at 3 pm, attends a meeting with the Tribute of Tributes of the Scrums of Scrums from the broader organization. They then reverse the process to pass down direction. May the odds be ever in your favor!
Testing
To keep the Code Engineers productive, we need them to focus only on coding. For unit testing, the Feature Captain will assign the complete feature to the Unit Testing Team to ensure 100% test coverage.
System Integration Testing
If two teams validating a change are good, three teams are better! This SIT team tests all of the new features of the Convoy together.
Rota Fortunae
We know that a static organizational structure limits improvement and also becomes boring for some executives. While many outside the SADMF community may propose Value Stream Mapping to identify constraints, we know that the process is time-consuming and uses too many Post-Its. To resolve these we introduced the Rota Fortunae ceremony where we “spin the wheel”, restructure, and then see if we are delivering better. If we fail, then we convene a Tribunal to address it.
Tribunal
We must Build Quality In by removing things that cause poor quality. In this monthly ceremony, we identify and remove the person who created each defect.
2 - Convoy Alignment Agenda
Every journey must begin with meticulous, even pedantic, planning
Every journey must begin with meticulous, even pedantic, planning
Attendance
All teams should assemble in the Convoy Alignment room for this multi-day celebration of detailed planning. It is important that the alignment happens in person and that everyone travels to a remote location to eliminate distractions from less important things such as family or pets.
Establish a Focused Environment
We need to maximize our planning time. We only have 5 days to plan the quarter, after all. Schedule what is ideally 9 hours of work each day. Because we have eliminated any distractions not related to work, we can easily extend the daily meetings if we don’t meet each day’s planning goals. Plan the planning! If we cannot meet our commitments, we cannot go fast.
Feel comfortable that since there are no extra-work distractions, the teams will all be able to provide their utmost focus and attention to the work at hand, even if it gets extended by a few hours due to the executive management team’s desire to exceed expectations.
Breaks and lunch can eat into our schedule. Additionally, people enjoy a change in diet. Take a collection at the beginning of each day and order sandwiches. Not only will this eliminate the need to stop for lunch, but it will also result in the team-building results of a shared meal!
Wrapping Up Alignment
On the last hour of the last day, while all of the alignment is fresh in our minds, hold a “Fist of Five” confidence vote on our ability to deliver the next 8 quarters’ worth of work. We must all be very confident, so make sure everyone is aware that anything less than a score of 4 will result in additional planning until we get to the minimum confidence level. If there is not a unanimous vote of 4 or better, take another collection for sandwiches and continue until everyone is confident.
All stakeholders must be included in the vote. Alignment requires unanimity. Letting the developers see the votes of their managers and vice-versa will ensure that accountability is maintained. In the event of disparity between the developers and managers, ensure that the managers’ votes receive preference since they can influence the developers to work harder.
3 - Deploy the Fleet
Delivering software is scary. We need layers of process to feel better.
Delivering software is scary. We need layers of process to feel better.
We need to nail the fundamentals of speed, innovation, and impact. We need to ensure we go fast and innovate but very cautiously to prevent solutions that do not work as designed. To do this, we need a “Zero Defects” approach to delivery.
To maintain “Zero Defects”, centralized control of all DORC™s is critical. We need to “slow down to go fast” by ensuring we have an effective inspection process at the end!
Let’s Get READY to sail!
All changes should use the READY release process. READY stands for:
Review
Evaluate
Approve
Deploy
Yield results.
This will ensure we are 100% READY to launch each convoy! All production changes or config changes should follow READY.
Convoy Steering Committee (CSC)
The Committee is made up of senior leaders and architects across all engineering functions to centralize review and preparation for changes. The CSC will ensure that changes, big or small, are approved at the highest levels and are READY to go! This centralized review by the CSC will shine a light on dependencies, secondary effects, and other aspects that are not always clear when reviewed closer to where the work happens. The CSC is also responsible for reviewing all of the documentation contained in the Convoy Manifest to ensure all Scaled Agile DevOps processes were followed to the letter before allowing the convoy to sail. This is a crucial step to assure our customers we are as SAD as possible. Since the CSC consists of representatives who are as far removed from the work as possible, they can have the utmost objectivity.
Approval Process
All requests must be peer-reviewed and approved in duplicate and also approved by the Convoy Commodore at least 3 days before being presented to the CSC. Each Feature Captain will also present a PowerPoint presentation detailing the effort required to build the feature to ensure that the importance of the feature is understood.
The CSC will meet each workday to review high-risk, or Commodore escalated changes to the production environment. Requests will be approved, sent back for more information, or rejected.
After the CSC approves the Convoy’s release there will be much rejoicing!
4 - Convoy Manifest
The critical documentation required for every DORC
The critical documentation required for every DORC
A Convoy doesn’t navigate by dead reckoning. We require proper documentation to keep from leaving any Release Captain behind!
Priority Change Request
A Priority Change Request must be created and approved during the Captain’s Mast before changing the priority of any feature. This request must contain the following critical information:
