Introduction to Data Infrastructure Strategy Investment Rationale
Inaccurate dashboards to work from. Pipeline failures. Stakeholder queries about the reliability of their data.
It all leads up to needing a budget.
Then your CFO says:
"Why should I spend more money investing in our data?"
It's a question almost all data leaders ask themselves at least once every year.
As a VP or Head of Data in your organization, you're not just building data infrastructure systems, you're creating the business case around those systems as well, and without a sound ROI narrative, even if your technology decisions are impeccably built, they will encounter considerable resistance from the folks who control the funding,” determines Keith O'Dell, co-founder of the Data Warehouse Institute (DWI). It is from this perspective, this guide was developed.
By the end of this guide, you will understand why many of the data infrastructure systems being created today are never approved for funding; will have a step-by-step quantitative-based framework to present your spend request; and have a clearly defined ROI narrative that the financial part of the organization can relate to.
AI Velocity Blueprint
Measure and multiply engineering velocity using AI-powered diagnostics and sprint-aligned teams.
The core of the problem for most organizations
The lack of justification by teams for their investment into data infrastructure is not tied to the identifiable need for the investment. It really boils down to the value of the investment being unquantified.
1. Ad-Hoc Decision Making
Teams within the organization:
- Add new tools (reactively)
- Solve immediate problems
- Do not develop long-term strategies
This creates many independent/fragmented systems with no measurable ROI.
2. Lack of Defined Ownership
Without defined ownership, most teams see increased cost related to the decisions made regarding their systems and decreased accountability (i.e., who is responsible for what happens).
Without a defined owner, no accurateFinancial teams lack an understanding of the technical debt incurred by engineering teams.
CFO's focus on the rising costs and lack of clarity regarding returns.
The value that will stem from the technical debt will be felt by:
- the decision making process
- the performance of products
- the customer experience
The challenge in providing clarity surrounding the value of an investment in technical debt is that the value will be intangible.
What you should provide during this process:
You must have a clear data infrastructure strategy. Connecting technical improvements to the business outcome, quantifying the cost savings and revenue; providing your team with a roadmap.
Realizing the challenge to provide evidence of the investment in technical debt action will not be met until you provide the infrastructure necessary to value.
This should include:
- Establishing a clear ownership model (who owns the data quality, the pipelines, who is responsible for the outcome)
- Establishing baseline metrics (pipeline failure rate, data latency, engineering time spent supporting the maintenance of technical debt)
- Understand your current spend on Infrastructure, Engineering, Tooling
- Define what your Business Goals will be (revenue growth, cost savings, risk reduction)
- Ensure there is agreement between Engineering and Business as to priority and against the leadership's understanding of the problem
The preparation you provide will drive your ability to build a credible business case for the investment of technical debt.
Phase one of the process is assessing your current state. You cannot defend the investment until
Phase One: Review Current Data Systems
Step One: Identify Your Data Landscape
Reconcile:
- Total number of/Identify All Data Processing Pipelines
- How many Tools You Have and What Kind (Tools Used) and What Tools (Where Do They Come From?)
- How Systems Are Related to Other Systems
Step Two: Quantify Your Issues
Examples:
- No. Of Hours To Debug Your Pipelines (How Many?)
- Delays In Your Reporting. (What Is The Average Delay?)
- NUMBER OF Data Quality Issues
Step Three: Map Out Your DATA FLOW
Document Where Your Data Comes From (Sources), How It's Transformed(Transformations), And Where It Goes (Outputs)
Step Four: Identify Key GAPS
Common Gaps:
- Lack of Visibility (i.e. is there a system in a separate place that does not allow you to see how it works). Also Good Data Quality Is Not Measured
- Pipelines That Are Not Efficient (Too Many Steps To Get From A To B)
Output-What It Costs Now To Be In The Current State
Take The Above Issues and Turn Them Into An Amount Of Money:
- Engineering Cost/COST FOR OPPORTUNITY /COST OF BUSINESSES $$$$$$
OR
NOTE:
Current State Will Be Your Base For Measuring ROI
Phase 2: Designing The Future State Architecture
Now Defining What The Future Looks Like Divided Into 3 Strategies
1. Define Your Values
- Independency
- Scalability
- Visibility
- Reliability
2. Select Your Components On Purpose
What To Avoid
- Tool Overpopulate/Sprawl
- Duplicate Components/System
3. Design For Visibility
What To Include:
- Monitoring
- Alerts
- Metrics
4. Estimate Future Costs
What You Should Include:
- Infrastructure
- Engineering
- Maintenance
5. Document Assumptions
Explicitly State Expectations Of Improvements And Risks/Project Dependencies
NOTE:
Document Your Future Plans Will Allow You To Effectively Measure ROI
Phase Three: Build, Test And Implement Incrementally
Because Your CFO Does Not Want You To Attempt This Process
1. Start With High Impact
- Critical Data Pipeline/S
- High Cost/Premium Data Pipeline/S
2. Implement Parallel Processing
- No Data Loss
- No Downtime
3. Automate Your Test
- Validate Your Data
- Check Your Data Pipeline
4. Instrumentation/Track Everything Possible
Track:
- Latency
- Errors
- Freshness
5. Show Early Results
- Fewer Failures
- Decreased Pipeline Speed
- Decreased Cost
NOTE:
Smaller, incremental successes Increase Confidence.
Measuring success and communicating return on investment (ROI) is where teams tend to struggle
1. Identify Service Level Objectives (SLOs)
Examples include:
- 99.9% uptime for pipelines
- < 5 minutes for data latency
- < 1% error rate
2. Convert metrics into dollar impact
Example:
- 20% fewer failures = 200 hours of engineering time
- Faster to have insights translates to faster decision making
3. Create dashboard visible to CFOs
Dashboard should include:
- Cost savings
- Increased efficiency
- Reduction in risk
4. Calculate ROI
Basic formula:
ROI = (Benefit - Cost) / Cost
Example of ROI Breakdown by Category
CategoryImpactEngineering hours saved$200K/yearReduced Downtime$150K/yearImproved Decision Making$300K/year
5. Leverage system-level intelligence
Teams should leverage system-level intelligence rather than manual methods for reporting.
Systems will:
- Automatically record metric information
- Surface insight based on recorded metric
- Optimize performance based on metrics recorded
This is where Logiciel provides value.
Teams receive up to date visibility of both ROI and system performance in near real-time without having to manually create reports.
The key take away from these processes is ROI needs to be measurable, visible and linked back to the businesses goals.
Conclusion
Justifying an organization's data infrastructure strategy is not simply about technology, it is about the impact on business.
Key takeaways for justifying data infrastructure investments:
- Translating technical issues into dollars
- Providing structure to develop your case
- Demonstrating ROI through metrics and early success
This is not an easy process. It will require effort in aligning, gathering, clarifying and communicating data/information.
However this is true, if done correctly, it will provide organizations with faster business approval, better operating systems and stronger alignment with their senior leadership.
Evaluation Differnitator Framework
Why great CTOs don’t just build they evaluate. Use this framework to spot bottlenecks and benchmark performance.
Call to Action:
If you are working towards justifying an organization’s investment into a Data Infrastructure, your next step is to develop a strong measurable business case.
Learn More:
- Root Causes of Why Data Infrastructure Failure Occur and How to Fix
- How to Create a Proof of Concept for a Data Infrastructure Solution
- Guide for Evaluating Data Infrastructure Vendors
At Logiciel Solutions, we assist organizations in building AI-first data infrastructures that produce a measurable ROI.
Frequently Asked Questions
What is a data infrastructure strategy?
Developing a plan for the design, management and optimization of an organization’s Data Infrastructure in order to achieve the goals of the business.
Why do CFO’s challenge an organization’s investment in a data infrastructure?
CFO’s challenge organization’s investment in data infrastructures because their value is typically indirect and are not easily quantified.
How can you measure the ROI of an organization’s Data Infrastructure?
You measure the ROI of an organization’s Data Infrastructure by documenting the monetary value of Cost savings, Efficiency improvements and Impact on Business.
The biggest mistake an organization makes when trying to justify an investment?
Not connecting the financial benefits of technical improvements to an organization’s overall objectives.
When will I see a return from my investment?
You can expect to see tangible results in approximately 3-6 months and full ROI in approximately 12-18 months.
