ExpiredMsBIP møde nr. 67

Hvis det er første gang du tilmelder dig et møde via MsBIP.dk vil du automatisk få oprettet en MsBIP profil. Alternativt skal du logge ind med den profil du fik oprettet ved tidligere tilmelding og derefter tilmelde dig mødet.

Endnu et møde, hvor vi sætter fokus på Azure Synapse Analytics.

Agenda:

• Velkomst, Just Blindbæk
• Presentation of Cloud Solution Architects, Microsoft
• Synapse Analytics: Now and the future roadmap, Morne Blake
• Pause – netværk og let forplejning, Microsoft
• Creating a Metadata Driven Framework Using Azure Data Integration Pipelines, Paul Andrew

Sessions

Abstracts of the two sessions

Synapse Analytics: Now and the future roadmap, Morne Blake

What does it mean for the future of data delivery, data mesh, data lake house, and other services? Why is Synapse a good fit and how are we planning to continue to be the best data platform out there.

Come and hear about how the Power BI team now owns the Synapse product and why it is good for all of us.

Creating a Metadata Driven Framework Using Azure Data Integration Pipelines, Paul Andrew

Azure Data Factory and Synapse Integration Pipeline are the undisputed PaaS resources within the Microsoft Cloud for orchestrating data workloads. With a 100+ Linked Service connections, a flexible array of both control flow and data flow Activities there isn’t much these pipelines can’t do as a wrapper over our data platform solutions. That said, the service may still require the support of other Azure resources for the purposes of logging, monitoring, compute and storage. In this session we’ll will focus on exactly that point and explore the problem faced when structuring many integration pipelines in a highly scaled architecture.

Once coupled with other resources, we’ll look at one possible solution to this problem of pipeline organisation to create a dynamic, flexible, metadata driven processing framework that complements our existing solution pipelines. Furthermore, we will explore how to bootstrap multiple orchestrators (across tenants if needed), design for cost with nearly free Consumption Plans and deliver an operational abstraction over all our processing pipelines.

Finally, we’ll explore delivering this framework within an enterprise and consider an architect’s perspective on a wider platform of ingestion/transformation workloads with multiple batches and execution stages.