Approx. Running order for the night

Featured Presentation:

Using Azure Data Factory for data integration with a Data Vault data warehouse

Dan Galavan, Data Architect GSD

Summary

In this session you will receive an introduction to both Azure Data Factory v2 and the Data Vault data modelling methodology. We will start with an overview of both followed by creating a Data Vault data warehouse along with an associated virtualized access layer in Azure. Finally, the pipelines needed to integrate data from an upstream application database will be implemented using Azure Data Factory v2.

Azure Data Factory

Azure Data Factory is a cloud-based data integration service that allows you to create workflows for orchestrating and automating data movement and data transformation. The data-driven pipelines (workflows) can ingest data from disparate data stores. Azure Data Factory can process and transform data by using compute services such as Azure HDInsight Hadoop, Spark, Azure Data Lake Analytics, and Azure Machine Learning. Additionally, you can publish data to data stores for business intelligence (BI) applications to consume. Ultimately, through Azure Data Factory, raw data can be organized into meaningful data stores for better business decisions.

Data Vault data modelling:

Over 80% of data warehouses in the Netherlands have been implemented using the Data Vault methodology, and has been touted as the next generation data modelling approach. The Data Vault makes a clear distinction between data acquisition & auditing vs. data interpretation & interpolation. This talk will show you how a single version of the facts may just edge out a single version of the truth. The knock-on effect is high adaptably to changing business requirements, something that can be a challenge with traditional data warehousing approaches. This also leads to faster sprints. Additionally, the Hub (core business concept), Link (relationship) and Satellite (context and history) design lends itself well to massively parallel ETL processing.

About Dan:
A Data Architect who has been working with databases since 1998. Has delivered solutions in a variety of industries including Banking, Investment Banking, Aerospace, CRM, Social Protection, Telecoms, and Health Product Regulation - primarily using the Microsoft Data Platform. While living and working abroad, on St. Patrick’s Day 2003 Dan came face to face with MS SQL Server for the first time - it’s not a national holiday in Germany for some reason! Starting with SQL Server Database design and development, Dan branched into ETL, Reporting, Data Modelling, DB optimization, Data Quality Management and Master Data Management. Regular contributor to the Dublin SQL Server User Group.