Flow metrics – build or buy?

When working with our clients, we frequently advise on one of the first critical steps in applying flow metrics – how to visualise the data. The key question is, ‘Should we invest in building a custom flow metrics dashboard tailored precisely to our needs or should we opt for an off-the-shelf product that promises immediate insights and ease of implementation?’.

This isn’t a one-size-fits-all situation; decisions will be based on a range of considerations, each carrying significant weight and differing for each use case.

A recent discussion within HYPR looked into these different considerations and the pros and cons of custom-built dashboards versus off-the-shelf products. During the discussion, we tackled the question based on:

• Ease of use
• Maturity of the team/organisation
• Ability to readily identify actionable insights
• Overall cost implications

What is the purpose of a flow dashboard?

A flow dashboard is intended to bring together your current work practices and workflow management systems to enable visibility of how your work (value) flows through your system. Visualising flow should be relatively low effort and designed not to force change into your system and instead, through visualisation, identify opportunities for improvement. Flow metrics can be as simple or as complicated as you want to make it. There are many ways to visualise the data but the important thing is ensuring the ease of gaining actionable insights – then actually doing something with those insights.

The lay of the land: defining ‘custom’ and ‘purpose-built’

Before diving into the nuances of decision-making, we must establish a shared understanding of what we mean by ‘custom’ and ‘purpose-built’ in the context of flow metrics dashboards.

Purpose-built products: Dedicated tools intentionally designed to visualise and analyse flow metrics. Think of solutions like Planview’s Tasktop Viz and Nave – each is designed to provide a comprehensive view of your value stream analytics. Plugins are third-party extensions to existing Application Lifecycle Management (ALM) platforms. By seamlessly integrating with platforms such as Jira, ADO or Trello, plugins like FlowViz or PowerUp extend the functionality of your current systems. Think of these as ‘purpose built lite’ – as in they are intentionally designed to visualise various elements of flow metrics without the full feature functionality behind them.

Custom-built dashboards: This involves leveraging data visualisation tools such as Power BI, EazyBI, Tableau or even Excel to construct bespoke dashboards. These dashboards are directly connected to diverse data sources, enabling you to create a tailored representation of your flow metrics that aligns with your unique requirements.

At HYPR, we have experience directly building custom dashboards on behalf of clients, supporting client in-house development and implementing off-the-shelf options. The solutions listed above are all ones that HYPR has direct experience with but this is not an exhaustive list of options available in the market.

Now we’ve established the purpose of this capability and the different options available, let’s examine each’s suitability based on the above criteria.

Ease of use: navigating the user experience landscape

The user experience is paramount. If a dashboard is clunky, difficult to navigate or requires a PhD to decipher, its adoption will likely be limited. Think about who will be accessing the data. Are they part of a delivery team, value stream or leadership looking across the organisation? Do they have access to the data, knowledge and understanding of what the metrics mean or do they have the ability to look at the visualisations and gain insights on their own?

The beauty of a custom dashboard lies in its ability to mould the visualisations precisely to your organisation's specific needs. You can choose what visualisations to show and how to draw in the data. HYPR’s experience has shown that building a dashboard deepens one’s understanding of the importance of the underlying data and the mechanisms for extracting meaningful insights. However, this advantage comes with a caveat: it necessitates in-house expertise in data visualisation, user experience and the capacity to build knowledge throughout the organisation.

In contrast, off-the-shelf products often pride themselves on intuitive interfaces and seamless integration with widely used ALM tools. Coupled with advanced widgets or AI, the promise is a hassle-free experience, enabling users to access flow metrics with minimal effort.

In either option, users can quickly disengage if they cannot find insights or don’t know what to do with them. Therefore, the capability to build insights must be available either within the organisation or provided by the tool.

Maturity of team and organisation: gauging readiness for flow metrics

An organisation’s maturity level significantly influences the optimal choice between a custom and an off-the-shelf solution.

A custom-built dashboard thrives in environments where teams possess a high degree of maturity, fully understand their processes and have a clear vision of the metrics they need to track. The inherent flexibility of a custom solution allows for precise tailoring to accommodate specific workflows, ensuring that the dashboard aligns with the team’s unique way of working. This works well when the scale of the dashboard is relatively small. When beginning to scale, the specific workflows across multiple teams or changes to workflows as teams mature can quickly become a challenge to maintain.

Off-the-shelf solutions are suitable for all organisations regardless of their maturity level. The advantage here is that solutions provide sensible defaults and industry-standard metrics. This provides a solid foundation for understanding flow and identifying initial areas for improvement for those earlier in their agile journey or with lower overall maturity. For those with more advanced knowledge, if the visualisations aren’t providing the whole gamut of information you’d like to see, it may be worth checking if data extraction is an option to enhance what you are already measuring. For organisations at scale, flexibility in ways of working can easily be supported with simple configurations that consolidate that complexity into standardised language and metrics.

Scale and variety: other aspects of context that influence a decision to build or buy

The number of teams or varieties of types of work may also be factors in a decision. More homogeneity may make getting meaningful data more practical and quicker with a bespoke solution. For instance, teams using similar software development methodologies for similar purposes – such as squads all focused on various aspects of the user experience of a SaaS product – might move work through a very similar workflow.

Of course, the greater the number of teams, the more likely there will be specialisation and variation of approaches. In many cases, the variation of approach has been shaped by the nature of challenges those specific teams have faced and may reflect important differences that should be maintained. For instance, the metrics that may be very meaningful for a team focused on building web frontends may be very different for teams running a messaging service that powers high-volume email and web notification systems.

We discussed this with HYPR team members experienced in measuring flow in organisations to improve productivity and impact. Some said that if the scale of a company they guided was larger (say, from hundreds of individuals to thousands) or the variety of teams’ work practices was greater, beginning with a third-party tool may be the better option because those tools can quickly bring more visibility across many teams.

Identifying insights: uncovering actionable intelligence

The primary goal of any flow metrics dashboard is to illuminate actionable insights that drive continuous work. Metrics are just the opening questions, not the whole story; they are a starting point for further investigation, not an end in themselves.

Constructing a custom dashboard empowers you to design visualisations that directly address the questions you need to answer. By tailoring the dashboard to your precise requirements, you can unlock insights that would remain hidden within a generic, off-the-shelf solution. However, this advantage is contingent on the team’s ability to interpret the data accurately and translate it into concrete action plans.

Many off-the-shelf products have built-in insight generation capabilities, often leveraging artificial intelligence (AI) to automatically identify trends, anomalies and potential bottlenecks. However, there’s a risk that teams may become overly reliant on these pre-packaged insights and passively accept the tool’s analysis without genuinely understanding the underlying dynamics.

Cost: weighing financial and opportunity costs

The cost of a flow metrics solution extends beyond the initial purchase price or development budget. It encompasses ongoing maintenance, training and the potential opportunity cost of resources diverted from other strategic initiatives.

While a custom flow metrics dashboard may appear to be the more budget-friendly option upfront, it’s crucial to factor in the long-term costs associated with development, maintenance and ongoing enhancements. Consider customising the complexity of multiple value streams, with teams across the organisation with different ways of working. Rolling that up the organisation becomes a lot more challenging. Furthermore, consider the opportunity cost. While you may have the internal capability to build flow visualisations, do they have the capacity? What other high-value activities could the team be engaged in if they weren’t dedicating their time to building and maintaining the dashboard?

Off-the-shelf products offer a predictable and transparent price point, simplifying budget planning. However, costs can vary significantly depending on the chosen product, the number of users and the level of support required. Some solutions like Tasktop, have been deemed prohibitively expensive for smaller organisations. Additionally, assessing the cost of change is vital – how easily can the tool be adapted to accommodate evolving needs and priorities?

HYPR’s recommendations and guiding principles: a compass for navigating the flow metrics landscape

To help our clients make informed decisions, we’ve developed a set of recommendations and guiding principles that serve as a compass in the often complex world of flow metrics:

• Start small: Resist the temptation to embark on a large-scale, all-encompassing implementation from the outset. Instead, begin with a lightweight, targeted initiative to facilitate learning and uncover initial insights. This iterative approach allows you to refine your understanding of your needs before investing significantly in a particular tool or solution
• Context matters: Tooling should complement the people and practices of the team and how they work. Tooling should not be used to change your practices; however, visualising how work flows through your system may identify improvements you can make. You don’t aim for consistency for consistency’s sake
• Address inconsistencies: Acknowledge that inconsistencies in processes and workflows are inevitable, particularly in larger organisations. Rather than attempting to impose a rigid, top-down standard, focus on exposing these differences and fostering open communication about their implications
• Visualisation is key: One key consideration is to be clear about what flow metrics visualisation you need and choose whether to use an out-of-the-box or custom build. You can put any metrics on a dashboard but it depends on what you need from a visualisation perspective
• Flexibility: It’s essential to ensure that any tooling solution should not define or dictate process or practice and that it’s there to supplement, support and visualise what currently exists, not to dictate process or anything

Ultimately, the decision to build or buy is strategic and requires careful consideration of your organisation’s unique context, capabilities and objectives. A custom solution offers unparalleled flexibility and the potential for deeper insights, while an off-the-shelf product provides ease of use and access to industry-standard metrics.

By carefully weighing these factors and adhering to HYPR’s guiding principles, you can confidently navigate the flow metrics landscape and choose the solution that best empowers your teams to deliver value more efficiently.