In this post, we want to highlight key considerations across two foundational domains that we identified as part of the taxonomy of Observability:
Observability Operating Model (OOM)
Observability Pipeline
As stated in our earlier post, [part 2] Approaching your Observability Strategy, strategy is highly dependent on characteristics that are unique to each individual organization, though we do advocate that there are common components and starting points when building an Observability strategy.
Domain: Observability Operating Model (OOM)
Establishing an Observability Operating Model (OOM) is easily the most fundamental and critical element to success and every organization should consider this carefully.
Establish clear boundaries of responsibility between Product Teams, Operations/Global Command Center (OCC), Strategy & Architecture, Site Reliability Engineering (SRE), and Platform Engineering/Shared Services. Here we consider the role of centralized and decentralized behaviors as it relates to Observability
Establish prioritization regarding Product Team operationalization, including alignment with Developer and/or Platform Engineering advocacy and adoption programs, architectural goals, integration to operational capabilities around notification, ticketing systems, dashboard creation, KPI onboarding/gates, etc.
Determine guidance around operational support patterns that provide select Product Teams greater autonomy to introduce/manage observability tooling and processes, and define what contracts must be established with the organization to support such patterns
Develop maturity and discipline regarding Observability practice capability (introduction and deprecation) through introduction and refinement of architecture patterns, framework(s), tools, and ready-to-use code libraries
Develop a formal and consistent commitment to measuring lost engineering cycles across Product Teams (e.g., partnering with Dev/Platform advocacy) for continuous process improvement/refinement
As Observability is a journey, establishing clear operational motives and common goals will allow many teams to self-identify and co-create value - which are two necessary ingredients in developing this discipline.
Domain: Observability Pipeline
This is often a starting point in terms of strategy implementation and like all other domains, this should align with the Observability Operating Model (OOM) where careful consideration was weighed when contemplating a buy vs build approach and centralized vs decentralized deployment patterns.
An Observability Pipeline establishes guardrails (i.e., it represents a compliant facility that aligns with IT control narratives) and advanced data controls (e.g., obfuscation, removal of superfluous data, enrichment) while promoting vendor neutrality to any number of observability platforms (WORM - write-once-route-many).
The benefits of creating this beachhead are critical if building toward vendor independence as this domain not only prevents vendor lock-in but allows for safe & rapid tools experimentation while potentially reducing the costs of existing tooling.
Note: Establish and manage this Observability Pipeline as a Product and build a robust set of advocacy programs to ensure community engagement.
Building this agnostic facility to route Metric, Event, Log, and Trace (MELT) data with guardrails opens the proverbial spigot to expanded tools usage. As per the Observability Operating Model (OOM), clear definitions of 'acceptable’ must be established.
