ExpiredMsBIP møde nr. 78

Hvis det er første gang du tilmelder dig et møde via MsBIP.dk vil du automatisk få oprettet en MsBIP profil. Alternativt skal du logge ind med den profil du fik oprettet ved tidligere tilmelding og derefter tilmelde dig mødet.

Agenda:

• Velkomst og nyt fra MsBIP
• Præsentation af InCommodities
• (Hyper)Scaling with Hyperscale
• Pause – let forplejning og netværk
• 1001 sources – ingestion & scraping of data
• Push based Power BI Data Refresh

Indlæg:

Tre case baserede indlæg fra InCommodities.

(Hyper)Scaling with Hyperscale

How do you ingest a quarter billion rows a day while servicing more than 2 million daily user queries?

How do you avoid that your resource heavy import queries from Power BI blocks ingestion and other users?

In this session we cover InCommodities’ tried and tested Azure SQL Database Hyperscale setup, including considerations behind avoiding Noisy Neighbors.

1001 sources – ingestion & scraping of data

How do you ingest data from thousands of sources, 24/7/365?

What role does caching and deduplication play in a real-time-always-on setup ?

In this session we cover the data ingestion setup of InCommodities.

Which methods and technologies are employed and how do they each play a role in ensuring that data moves as fast as possible from market signal and in to the data platform.

Push based Power BI Data Refresh

In this session we cover InCommodities’ Near-Real-Time PowerBI refresh setup – which allows them to constantly trigger new Data Imports whenever new data is available at the source.

Utilizing the XMLA Endpoint and a self-developed framework, the session covers the thoughts behind, methodology and what this setup brings to the table.