Narrathon: Creating & Sharing Probes for Nourishing Learning


A Narrathon, is a peer online-learning experience, where we guide and mentor each other in capturing our learning as probes.

As a coach, you’re often the one who needs to drive and sustain the agile transformation. Yet, how do you do this? What has helped and hindered you doing so and how can you best pass your experience on to other coaches or how can you best learn from other coaches?

This is the focal point of the Narrathon and to tackle that challenge we want to invite you to try something new:

  • We face currently the following situation: To be able to change and to drive change gets more and more important for companies to survive in this VUCA world. At the same time, it is (or should be) the core skill of every coach – no matter if you coach individuals, teams, or organizations. However, sometimes it seems every coach has to make his / her own experience on how to drive change and especially the change for an agile transformation successfully.
  • Our hypothesis is that for driving & sustaining an agile transformation, every coach uses so-called probes, that are defined by small, safe-to-fail experiments based on hypotheses derived from reflection on the current situation as well as on theory. So, probing allows discovering (based on the hypothesis) what’s working and what is not through one or several experiments. This allows us to make sense of both the current situation but even more important of the situation we are aiming for. And if we create a knowledge base of our collective wisdom on probes we used (or intend to use) we can learn from each other.
  • Therefore, as an experiment we want to invite you to jointly discover, share, create, improve, and finally publish probes that help(ed) driving and sustaining an agile transformation. We will use the new format of a Narrathon to guide the creation of the probes. We anticipate as a result that the discovered and created probes will provide a foundation for such a knowledge base.

Find here a report of the first Narrathon participants.


This is an online program running over the course of 10-12 weeks, where between session 2 and 3 we allow several weeks to improve the probes (and asynchronous online assistance). Every session lasts 90 minutes.

The agenda is as follows:

  • Session 1:
    – Defining the context
    – Example probe
    – Elements of a probe
    – Hints on creating probe
    – Improving your probe
  • Session 2:
    – Reports (by team) on improving your probes
    – Sharpening up your hypotheses & experimental designs
    – Complexity and probes
    – BOSSA nova and hypotheses
    – Improving your probe
  • Session 3:
    – Reports (by team) on improving your probes
    – Importance of publications
    – Measurements & Controls
  • Session 4:
    – Recap on your probes & learning
    – Planning your next steps
    – Community of Practice


Definition and structure of a probe:
Probes are defined by small, safe-to-fail experiments based on hypotheses derived from reflection on the current situation as well as on theory.

Use the below structure to create your Probe:

  • Name:
  • Background:
  • Hypothesis:
  • Safe-to-fail experiment:

In summary, a probe is defined by a name, background, hypothesis, and one or several experiments.

Probes nourishing continuous learning:

Example Probe:
Here’s a real-world example probe. And note, it’s not spectacular (yet probes don’t need to be spectacular.):

  • Name: Will consent decision-making improve involvement and agreement?
  • Background: Not everyone can be equally involved in decision-making. Individuals dominate decision-making.
  • Hypothesis: With consent decision-making in everyone’s’ toolbox especially difficult decisions can be made by mutual agreement
  • Experiment: EGs [these are specific kinds of teams] where decision-making is repeatedly described as difficult (based on a survey) are trained in consent decision-making. After e.g. 6 weeks, a new survey should be carried out. If consent decision-making is considered as a useful tool, all EGs will be trained in their use.

Results of the first Narrathon: The Probes Created

  • Name of the Probe
    How can we enable teams to discuss openly their impediments and improvements in retrospective?
  • Abstract
    Agile is being followed in most teams and organizations now. They are also doing retrospectives. But most of the time, the right set of impediments and improvements are not the output of these retrospective meetings as they have a fear that if they speak up it would impact them in appraisals.
    If we do not have managers in the retrospectives, then the team will open up with their impediments and improvements as they will feel safe. So, this probe talks about how removing managers from retrospectives impacts the team and how they bring the impediments/improvements and also question each other openly making each other accountable.
  • Think about your situation. Describe background
    Currently in retrospectives, team members are bringing up only positive or happy points and are not talking about the impediments/improvements of themselves and other team members even after they are asked powerful questions by the scrum master. Team feels that if they open up in retrospective then they would be impacted in appraisals as their manager will just talk about their impediment and improvement as an individual rather than team. Appraisal is currently totally focussed on individual performance.
  • State your hypothesis
    Under the same team setup, if we do not have managers attending the retrospective, then we will observe that team members are opening up in retrospectives which can be measured by more impediments / improvements discussed by team.
  • Design experiment
    • Control Group: One Team, i.e., Team A
    • Duration: 2 Months
    • Steps to keep the experiment safe: Results of the experiment duration will not impact appraisals of the team. Appraisal of the team will be done based on work and performance of 10 months only.
    • Measurement:
      • Pre-Experiment state: Very few improvement/action items being brought up by team in retrospective around 1-2 which are of less impact.
        These points being brought up by the team are very superficial and general and do not talk about concrete steps. Team performance metrics:

        • Velocity – 12 points per Week
        • Time taken to revert to a query – Around 4 days
        • Issues reported in delivered items – 3-4 issues per sprint
      • Expected Result: More improvement/action items being brought up by the team in retrospective may be at least 5-6 and these are having a good impact on team health and performance. This will result in increase in team performance over a period of 2 months and improvement in team performance metrics.
    • Procedure:
      • Discuss with the manager to not to come to retrospectives explaining the experiment to him.
      • Declare to the team about the manager not coming in retrospective.
      • Conduct retrospectives as usual.
      • Observe the team about their changes, if any, in interaction.
    • Observation/Actual Result:
      • Pre-Experiment state: Very few improvement/action items being brought up by team in retrospective around 1-2 which are of less impact.
        These points being brought up by the team are very superficial and general and do not talk about concrete steps. Team performance metrics:

        • Velocity – 12 points per Week
        • Time taken to revert to a query – Around 4 days
        • Issues reported in delivered items – 3-4 issues per sprint
    • Conclusion: 
      • Results after 2 months reflect that if the manager is not present in retrospective, then the team feels that they have a safe environment. So, they bring all the impediments/improvements for the team. They question each other openly and make each other accountable.
  • Next Steps
    As the results are positive and show improvement for team A, so we can apply this hypothesis across all the teams in the account.
    We can also look if we can establish a safe environment with the manager being present in retrospective.
  • Publication
    I am planning to get this published in Narrathon / BOSSAnova
  •  

  • Name of the Probe
    Does dedicated time for self-learning improves team happiness and product initiatives?
  • Abstract
    Mostly team members are just allowed to focus on deliveries and not on learning and improvement. They are mostly given so much delivery related work that they are not able to take up some new things and learn.
    If we allow the team to have dedicated time (10% of available capacity) for self-learning in each sprint, then the team’s learning and new ideas proposal increase resulting in improvement of product and their satisfaction level. Self-learning opportunity helps in improving ideation and improvement of product. Giving dedicated time for self-learning and improvement helps in new initiatives from them and improves their happiness index. This probe talks about experimentation of giving 10% of capacity time for dedicated self-learning.
  • Think about your situation. Describe background
    Currently teams are doing their usual stuffs. There are almost no new learning and new initiatives being worked upon by teams. In terms of new ideas for product features, there are none. The team’s happiness index is also low due to lack of personal learning and growth. Teams are generally focused on assigned delivery only and are loaded with features/items to work upon which are part of releases.
  • State your hypothesis
    Under the same team setup, if we have dedicated time for learning for team members, then we will observe that team members are learning more and growing resulting in an increase in their happiness and motivation. They are also taking initiatives which will help in improved product.
  • Design experiment
    • Control Group: One Team, i.e., Team A
    • Duration: 6 Months
    • Steps to keep the experiment safe: Results of the experiment duration will not impact appraisals of the team. Appraisal of the team will be done based on delivery they have done and considering that team may deliver 10% less as compared to their usual.
    • Measurement:
      • Pre-Experiment state: Team members are rarely working on new improvement ideas. They are not bringing any new technical challenges and upgrades. Work for them is monotonous and thus the happiness index for the team is low. Team performance metrics:
        • Happiness Index – Low (around 4 on scale of 10)
        • No of new ideas/features proposed for product improvement – 0
        • No of new initiatives – 0
        • No of learning sessions – 0-1 per sprint on an average
      • Expected Result: Team happiness index improves. They are ready and enthusiastic to take up new things like automation, upgrade in technical stack etc. They are also working on new ideas which have the potential of being implemented as a new feature in the product. They are having more learning sessions amongst them for team learning and improvement. This will result in increase in team performance, new patent filings over a period of 6 months and improvement in team performance metrics.
    • Procedure:
      • Discuss with Delivery Head/leadership and explain them the experiment and its objective. Discuss with them to give 10% time of sprint for self-learning aligned to organizational needs.
      • Allocate 1 day in sprint (2 weeks sprint) for each team member for self-learning.
      • Announce to the team about this new change.
      • Let usual work go on and promote team members to complete self-learning story in each sprint.
      • Observe the team about their changes, if any, and motivate them regularly.
    • Observation/Actual Result:
      • Post-Experiment state: Initially the team did not have much focus on self-learning rather they were aligned to pull more items of delivery. So, helping them understand that if they are not able to complete their self-development story, then they will have to split it for next sprint and will result in impact on sprint completeness. So, they started focusing on learning also and started coming up with new things like automation, upgrade in technical stack etc. There was an increase in proposal for new ideas/features for product and new initiatives on which team was working on. They started having more community learning sessions. Overall, their happiness index improved. Team performance metrics:
        • Happiness Index – Improved to better (around 7 on scale of 10)
        • No of new ideas/features proposed for product improvement – 2
        • No of new initiatives – 4
        • No of learning sessions – 2-3 per sprint on an average
    • Conclusion: 
      • Results after 6 months reflect that if the team is given dedicated time for self-learning, then they are happier and more motivated and coming up with new patents, ideas and improving the overall product.
  • Next Steps
    As the results were positive and showed improvement for team A and corresponding product, so we applied this hypothesis across all the teams in the account.
    Next step can be applying this hypothesis across organization.
  • Publication
    I am planning to get this published in Narrathon / BOSSAnova and was presented in townhall.
  •  

  • Name of the Probe
    How can we foster an environment which motivates everyone in the team to focus more on delivering value?
  • Abstract
    Teams are not aligned on how they are contributing in enhancing business value. In the long run this might lead to lack of motivation and disengagement in teams.
    If sprint goals will be aligned with business value, it will not only bring motivation in the team but customer delight as well.
    Reserve some time in sprints, where the team can spend time in brainstorming and coming up with ideas to add business value which further can be converted into features.
  • Think about your situation. Describe background
    Teams are focused on the sprint goals but they are not aligned with the underlined business value and hence are unable to comprehend how their contribution is enhancing the business value. In the long run this might lead to lack of motivation and disengagement in teams. Rather than merely completing stories and delivering features, focus should be on delivering values.
  • State your hypothesis
    If a team gets some reserved time within the sprint to spend on whatever they feel important (i.e., business plan, strategy, roadmap, clients, etc.), the team will be able to help in adding business value and its productivity will increase based on innovative solutions and ideas.
  • Design experiment
    • Experiment: Reserve one day in 2-weeks sprint as ‘me-time’
    • Duration: One Quarter of the year
    • Steps:
      • Conduct ‘Innovation Market’ once in every two sprints, which is the part of ‘me-time’ itself. (This could be one of the team building activities)
      • Team members will bring their ideas (which they think can bring more business value) and try to sell in the market.
      • With the consensus of team members choose one idea that gets sold and now this idea will be worked upon. (Other stakeholders can also be part of this ‘Innovation Market’ this will bring more transparency and buy-in from them)
      • Along with idea-originator, if some other team member also wants to contribute then that team member can use his/her ‘me time’ in implementing the innovation.
      • The ideas that get converted into features and released can be tagged with ‘Innovation’.
      • Observers from outside could also be the part of the activity e.g. stakeholders, leadership and customers to make their observations.
    • Measurement:
      • Following are the parameters should be considered Pre and Post experiment:
        • The number of features tagged with ‘innovation’ in every quarter.
        • Measuring CSAT (Customer Satisfaction Score)
        • Motivation level (using team survey)
        • Happiness Index (using Team Radar)
    • Control:
      • Consider a different team where ‘me-time’ is not introduced.
    • Anticipations:
      • Trust from leadership towards teams – Is ‘me-time’ actually being utilized to bring value.
      • Innovations do not result in success every time.
  • Next Step
    If experiment yields positive results in terms of increased productivity, engagement level and motivation. The scope of experiment can be widened with multiple teams contributing for a system, so as to introduce it at system level.
  • Publication
    The experiment can be shared across and outside the organization e.g. Narrathon/ BOSSANova etc. well as where others can adapt and learn from the take-aways.
      •  

  • Name of the Probe
    Can we trust the team more and give more authority to make decisions?
  • Abstract
    Client was not trusting the new vendor team, not giving them authority for change movement to the next environment; this was causing delay in work as it was being done by a member from client’s side. Our hypothesis was that if this activity is done by team members, change will move to the next environment quickly, there will be no dependency and delay in delivery. We propose to experiment it for 2 sprints (with pre and post measurements) and see if this is making delivery faster and the team is winning client’s trust.
  • Background
    • In SAP every change made is moved as a ‘Transport’ to another environment
    • Current scenario: team (vendor) make change in development environment, test it and then send a request to IT lead (from client side) to move the change to the next environment where business users start testing from their end.
    • IT Lead moves the change as per his/her availability/bandwidth. So the team has external dependency.
    • This delays work which is quite frustrating for team.
    • Earlier there was a person from the team’s side, who was trusted because he was working in the account for a long time. He built trust with client. Deployment process was quite fast at that time. Now that he is gone, the client has not been able to develop enough trust on the new team to give them authority to complete deployment.
    • The new person (from client’s side) who is given authority for Deployment, is also overburdened and is not very happy about this new responsibility. This combination slows down the overall deployment timeline.
    • The impact of the above is delay in value generation for business and demotivation in a team.
  • Hypothesis (why do we think what we think?)
    1. Under the same setup if the team is given ownership to the change movement activity, we can move change quickly to the next environment for production release, there will be no waiting for 3rd party, no delay.
    2. This will help us to complete our work faster.
    3. We as a team will also feel more empowered and responsible.
  • Experiment
    • Duration: 2 sprints (1 month)
    • Pre-experiment Measurement: Conduct a survey within team members (in one of the retrospective meetings) to know how long it takes to move one change to the next environment for business simulation testing, what is the current delay time / wait time for each change to move?
    • Expected result: delay time/wait time should not be more than 1 hour (the actual work is just 15 minutes of work)
    • Procedure: 
      1. We propose client to give authority to one team member who can move the change for next 2 iterations (1 month)
      2. Team contact that authorized person as soon as the change is ready to move
      3. Transport gets moved quickly (no waiting on third party, no delay)
    • Post-experiment Measurement: After 4 weeks, again take a survey of how fast change is moving, what is the delay time/wait time now? Is it happening within an hour?
  • Next Step
    If the result is positive and improvement is clearly measured, we will adopt the new way of working.
  • Publication
    I would like to publish this probe in my peer group and this Narrathon/Bossanova.
    I would also like to publish this in my organization’s Agile community Wiki.
    •  

  • Name of the Probe
    Does too much work make someone inefficient?
  • Background
    • In the previous probe we saw that delivery of a Scrum Team is delayed because of dependency on IT Lead (who was from client side) who was doing the work of change movement. Client didn’t trust the team as that was a new team. They trusted the person from their side and she was given that responsibility.
    • However the IT Lead who was doing this activity is also a full time Scrum team member of another team and is working in her full capacity in a complex product feature. She was doing this change movement work as additional stretch activity.
    • Since this was a stretch work for her she was not giving it a priority or high importance always. This was causing delay in hours and sometimes days. The actual work is just 15 minutes work.
    • The IT Lead was getting 2-3 change movement requests every day from the Scrum team but could not focus on them at that time. She was doing it at the end of day or next day.
  • Hypothesis (why do we think what we think?)
    Under the same setup if the IT Lead is given a certain amount of time every day for this activity and her regular work is reduced (which is her primary work) it will help her to do both activities efficiently and on time.
    This will also reduce delay for the Scrum team which is dependent on this activity.
  • Experiment
    • Duration: 2 sprints (1 month)
    • Pre-experiment Measurement: Conduct a survey within team members (in one of the retrospective meetings) to know how long it takes to move one change to the next environment for business simulation testing, what is the current delay time / wait time for each change to move?
    • Expected result: delay time/wait time should not be more than 1 hour (the actual work is just 15 minutes of work per change)
    • Procedure: 
      1. We propose client to give 2 hours of time to IT Lead for change movement work for next 2 iterations (1 month). Her regular workload is reduced by 2 hours per day.
      2. As soon as the change movement request comes from the team, she can perform it within an hour.
      3. Transport gets moved within an hour
    • Post-experiment Measurement: After 4 weeks, again take a survey of how fast change is moving, what is the delay time/wait time now? Is it happening within an hour?
  • Next Step
    If the result is positive and improvement is clearly measured, we will adopt the
    new way of working.
    •  

  • Name of the Probe
    Is in-sprint defect logging affecting morale and productivity of a team?
  • Abstract
    In organizations, teams are demotivated/demoralized and low work satisfaction due to multi-channel communication within the scrum team created due to logging/managing, fixing in-sprint defects.
    As per our hypothesis, If we stop logging in-sprint defects and focus on collective delivery the team will be more effective and will express less frustration which can be effectively measured by timeliness of delivery and improved quality.In one team of the organization, avoid logging in-sprint defects and create a more collaborative environment. Encourage the team to use techniques like Pair Programming and Test first approach before delivering any product backlog item. Keeping all other parameters constant measure parameters for the team before and after the experiment and compare the results.
  • Background
    1. Scrum Teams are already demotivated/demoralized due to low velocity.
    2. Teams unwittingly spend much more time in communication and logging/fixing in-sprint defects.
    3. Team members had a culture of pinpointing each other based on roles and responsibilities of resources on a particular Product Backlog Item (PBI), which caused delay in delivery and low velocity of the team.
    4. This also creates some casual behavior in teams, as they are not collectively responsible for delivery.
    5. Devs are more focused on moving items to QA ASAP even sometimes without completing the work. QAs focus is on logging defects as it will help them in their appraisal. This is resulting in not finishing work when the issue is reported and leading to not building the right product first time. Team find themselves applying all sort of workarounds/excuses that lowers the morale of the team in long run
  • Hypothesis (why do we think what we think?)
    If we stop logging in-sprint defects and focus on collaborative and collective delivery, the team will be more effective and will express less frustration. This will result in building the feature right the first time and its effectiveness can be measured in terms of

    • timeliness of delivery,
    • reduction of technical debt,
    • reduction in defect leakage
    • improved team morale / work satisfaction
  • Experiment
    • Control Group: Team Phoenix
    • Duration: 4 sprints (2 Weeks / sprint)
    • Procedure: 
      • Discuss with managers/leadership team and explain the experiment and its importance.
      • Coach the team and help them identify the importance of teamwork.
      • Conduct a series of Team building activities to have a more collaborative environment within the team.
      • With the above steps, help the team understand, if anything is delayed or developed incorrectly then the team is held responsible collectively rather than some roles.
      • Declare that we will no longer log Sprint Defects and focus on collective development and testing before moving the story to the next stage using techniques like Pair Programming, Test First Approach.
    • Measurement: For our experiment we measured Velocity, Defect Leakage, Team Morale (using internal Survey), and feedback from stakeholders – to keep track and check on the experiment. We measured these parameters after every sprint and compared results after the end of the experiment to derive the conclusion. 
    • Conclusion:
      • After running the experiment and seeing the results it was observed that team motivation has increased resulting in reduction of frustration, tech debt & defect leakage. Which resulted in improvement of work satisfaction, delivery quality and timeliness of delivery.
      • For reference below are readings from experiment conducted:
        • Pre Experiment state:
          • Velocity: 17
          • Defect Leakage: .38 (Defect / Story Point)
          • Team Morale (using team survey): 2.3 (Out of 10)
          • Feedback from Stakeholders (Using Survey): 2 (Out of 5)
        • Post Experiment state::
          • Velocity: 23
          • Defect Leakage: .12 (Defect / Story Point)
          • Team Morale (using team survey): 7.8 (Out of 10)
          • Feedback from Stakeholders (Using Survey): 3.9 (Out of 5)
  • Next Step
    As data shows improvement in the team performance and motivation, a complete report will be shared at organization level and with help of leadership will replicate in other teams as well.
    Once other groups started and agreed on adopting the approach for them. Few questions were raised

    • Would encouraging team self-learning and enabling cross skills improve team morale and feedback from stakeholders?
    • Would focusing on individual and interaction over process and tools increase team morale?
  • Publication
    I would like to get this probe published in Organisation NewsLetter and Narrathon/BOSSAnova.
    •  

  • Name of the Probe
    What we have to do to prepare well for the PI planning?
  • Abstract
    Agile teams working in SAFe framework struggle to get the features ready for PI planning. This is the core/root cause of many issues which agile release train faces during program increment execution.
    If the agile release train follows proper process/method to groom the features, it avoids substantial risks/impediments/issues during program increment execution. In one team of the organization, avoid logging in-sprint defects and create a more collaborative environment. We are going to share the vision early so that the features are derived and prioritized early enough. Each team will have enough time to discuss the features and bring in clarity in all aspects in line with the definition of ready  ( business & technical requirements, architecture enabler, dependencies, risks). Post experiment, we are going to measure the % of PI objectives delivered, number production defects,   impediments, System  issues, risks encountered during program increment before and after the experiment.
  • BackgroundAgile teams in SAFe framework struggle to get ready for PI planning.

    There are a lot of things to take care of before teams claim that they are ready for PI planning. Things like feature/requirements readiness, taking care of dependencies, impediments, architecture enablers, issues, risks.

  • Hypothesis (why do we think what we think?)
    If our agile teams follow proper method/process to get the features ready for PI planning, then:

    1. Agile Release Train (ART) will have enough  confidence in whatever they are committing as a program increment (PI) objective.
    2. ART will discuss and be aware of risks/impediments/issues before committing to the PI goals/objectives. PBIs are created for impediments and the timelines are agreed to deliver the dependent work.
    3. ART will know their features (Objectives) very well and their acceptance criteria based on the definition of ready.
    4. ART will have clarity on the vision set for the upcoming program increment.
    5. The frustration and demotivation would be avoided.
    6. The program predictability increases.
    7. PI execution will be smooth without much impediments.
    8. Product increment will be high quality as per the acceptance criteria.
    9. ART will encounter less surprises/shockers in the PI execution phase.
    10.  Risks and dependencies will reduce over the period of time.
  • Experiment
    1. Product Roadmap/Vision is being shared 1 month before PI planning now. It will be shared 2 months before PI planning by the management team.
    2. Having the features finalized using prioritization techniques like WSJF, MoSoCoW and distributed very early.
    3. A dedicated meeting/s held to discuss the next PI feature.
    4. Grooming the features with whole team and other required stakeholders
    5. Promoting SoS meetings to discuss dependencies/impediments.
    6. Defining the workflow of the feature/functionality to identify all the dependencies.
    7. Architect enablers discussed in parallel to features grooming.
    8. A clear and complete information (Objective, Benefit of hypothesis, Acceptance criteria, Prerequisites for the feature) captured  as part of the feature in the project management tool ( Jira, Rally, AzureDevOps etc)  Holding regular features evaluation meetings.
    9. Define DoR and DoD for features.
    • Measurement:
      1. How much % of risks/dependencies are missed before and after the experiment?
      2. Production defects should decrease.
      3. System issues should reduce during the PI execution.
      4. How many features are groomed with finalized DoR and DoD?
      5. Spikes and NFRs are identified for features.
      6. Number of Scrum of Scrum (SoS) calls should ideally increase.
      7. Confidence level of the team to achieve PI objectives before and after the experiment.
      8. The number of dependencies / risks / issues discovered during the PI planning and discovered after the PI planning
    • Conclusion:
      • Production defects should decrease. From current 20% to 10%.
      • How many features are groomed with finalized DoR and DoD? It should increase from current 50% to 100%
  • Next Step
    I would try to implement this experiment for our next PI planning preparation phase which will start in May-2020 month.
  • Publication
    Post implementation of this experiment in my project and based on the results which I will observe, I am planning to get this published in Narrathon / BOSSAnova.

    •