Five Years of Bringing DataOps to Power BI and Fabric

  8 mins read  

It has been five years since I started this blog, but somehow it feels like almost a decade has passed since 2021.

When I launched the blog, my goal was simple: bring DataOps principles to Power BI. Today, with Microsoft Fabric and AI reshaping the landscape, the application of those principles has expanded significantly.

Over the past month, I’ve been reflecting on how DataOps principles are showing up throughout Fabric and Power BI.

Improving Cycle Times

Whether it was intentional design or a happy accident, the introduction of Power BI Project files at the dawn of mainstream AI adoption dramatically accelerated how we build Power BI solutions.

Today, we’re not only developing semantic models and reports more efficiently, but we’re also building user interfaces with Fabric Apps and custom JavaScript applications layered on top of Power BI and Fabric.

AI has further accelerated solution development across the Fabric platform. It has enabled my teams and me to build projects such as Power Query Lint and Power Query Escape Room, both powered by Fabric Real-Time Intelligence and Fabric SQL.

Reuse

Outside of AI, features such as User-Defined Functions (UDFs), TMDL View, and DAX Query View are finally making it easier to reuse logic and share code across teams.

For years, reusability has been a challenge in Power BI development. These capabilities represent meaningful progress toward standardizing and sharing patterns, which is a core DataOps principle.

Quality Is Paramount

Testing is now becoming a first-class citizen within semantic models.

User-Defined Functions have opened the door for automated testing scenarios, and they enabled my team to develop PQL.Assert, which we shared with the community to help automate semantic model testing.

As DataOps practitioners know, quality cannot be an afterthought. The faster we can build, the more important automated validation becomes.

Room for Improvement

With all these advances, it’s easy to become complacent. Success can sometimes blind us to the opportunities that still exist.

However, complacency eventually leads to missed opportunities. While Microsoft has made tremendous progress, there are still several improvements that could further strengthen the application of DataOps principles within Fabric and Power BI.

Here is my wish list for 2026.

1. Azure Key Vault as a Fabric Artifact

This has been a frustration of mine for years, and it is clearly something many in the community have been asking for.

When Variables were introduced, I hoped they would become the foundation for securely storing secrets within Fabric tenants. Unfortunately, that hasn’t happened.

Today, obtaining and managing an Azure Key Vault can still be a bureaucratic process or a bridge too far for many organizations. As a result, some practitioners take the path of least resistance and store secrets directly in Power Query, Dataflows, or other places where they do not belong.

Yes, secrets can be passed from a pipeline, but that process is still cumbersome and ultimately depends on having a Key Vault.

In the age of AI, this becomes even more important.

Imagine if Fabric included a native Key Vault artifact. We could reference secrets securely within the Fabric ecosystem, and AI-powered code reviews could identify exposed credentials and automatically recommend moving them into a governed secret store. It would improve security while simplifying development.

2. Break Up expressions.tmdl

Let’s face it: tokens are the currency of AI.

When an agent has to read a 2,000-line expressions.tmdl file just to update a single function or table, we’re wasting tokens and increasing processing overhead.

The rapid growth of User Defined Functions has made this file even larger, which can create challenges in Visual Studio Code and increase merge conflicts during parallel development.

A more modular approach that stores tables, parameters, and functions in separate files would reduce merge conflicts, improve Git workflows, and make AI-assisted development significantly more efficient.

3. Native Regular Expression Support in Power Query

The community first requested regex support years ago, and it remains one of the most obvious gaps in the Power Query language.

There are workarounds involving HTML and JavaScript, but they do not work reliably in the service. Many of my teams have resorted to using Azure AI Foundry or external services simply to perform regex operations.

This feels unnecessary when .NET already provides robust regex functionality that could be surfaced directly through Power Query.

Regular expressions are a foundational capability for data transformation and should not require external dependencies.

4. OAuth Support for POST Requests in Web.Contents

If we cannot have a native Key Vault artifact and we still need external services for capabilities like regex, then we need better authentication support in Web.Contents.

Today, many APIs require POST requests that include payloads and OAuth authentication. In practice, developers often end up using anonymous connections alongside API keys and then face the challenge of securing those keys.

It is a cascading problem that creates unnecessary complexity and security concerns.

Supporting OAuth-authenticated POST requests natively would remove a significant friction point for Power BI and Fabric developers.

5. Skills as a Fabric Artifact or Environment Resource

Skills have quickly become an essential part of AI-enabled development.

However, they have also created a version-control challenge.

Today, skills files are often copied to individual developer machines. While convenient, this also means that different developers can experience different outcomes based on local configuration differences.

This is not an unsolved problem. The software industry has managed package dependencies for decades through ecosystems like npm and PyPI. In fact, Microsoft has produced the agent package manager to solve this problem for agents and skills.

I’d love to see skills, agents, and MCP configuration become first-class artifacts, or perhaps part of a Fabric Environment, so they can be versioned, managed, shared, and governed consistently across teams in the service and locally.

Even more importantly, I would love Fabric Copilot and related AI experiences to automatically leverage those skills so that AI interactions align with established standards and best practices.

6. GitHub App Connections for Fabric Git Integration

Git integration is critical for DataOps.

While Azure DevOps integration is excellent, GitHub integration still relies heavily on Personal Access Tokens (PATs).

I would love to see support for GitHub App connections that eliminate the need to manage, rotate, and secure PATs manually. It would simplify administration and align better with modern GitHub security practices.

7. Extend User Data Functions to Be More Like Azure Functions

I am a big fan of User Data Functions (not to be confused with User Defined Functions in DAX) and their writeback capabilities.

However, as Fabric increasingly becomes the backend platform for storing, processing, and orchestrating data for front-end applications, something Fabric Apps is making increasingly apparent, we need more of the capabilities offered by Azure Functions.

I would love to see support for additional runtimes such as Node.js, TypeScript, and PowerShell, along with features like deployment slots.

Azure Functions are incredibly powerful, but many organizations struggle with the overhead of provisioning Azure resources. Bringing more of those capabilities directly into Fabric would lower barriers to adoption and accelerate innovation.

8. Expand Fabric Apps Beyond Entra-only Internal Applications

Fabric Apps have introduced an exciting possibility: JavaScript applications backed entirely by Fabric.

With existing data layers, GraphQL support, and User Data Functions acting as an API layer, Microsoft has an opportunity to think beyond internal enterprise applications.

Fabric Apps already provide application hosting and static content hosting. The next step is enabling organizations to build truly customer-facing applications within Fabric.

Imagine being able to designate an application or workspace for public consumption while supporting authentication providers beyond Microsoft Entra ID. Organizations could build and publish customer-facing data applications directly from Fabric without introducing additional hosting platforms.

Today, solutions like Azure Static Web Apps and Vercel fill this gap. However, once a project extends beyond the Fabric boundary, adoption often slows as teams encounter additional security reviews, coordination efforts, and infrastructure costs.

Bringing these capabilities closer to Fabric could dramatically simplify the path from analytics solution to external-facing application.

Looking Back and Forward

Despite the opportunities I’ve outlined, I am genuinely pleased with the progress Microsoft has made over the past five years.

Many of the capabilities we have today, from Power BI Project Files to TMDL, DAX Query View, User-Defined Functions, and AI-assisted development, have fundamentally improved how we apply DataOps principles in Power BI and Fabric.

I also deeply appreciate the generosity of the data community. Whether through open-source projects, blog posts, conference sessions, or GitHub repositories, practitioners continue to share best practices and push the ecosystem forward.

The next five years will likely move even faster than the last five.

What would you like to see improved in Fabric and Power BI? Let me know on LinkedIn or Twitter/X.