Performing Big Data Engineering on Microsoft Cloud Services

Microsoft

Microsoft

Streamtech Knowledge provides an end to end Microsoft training solution across all technologies. Along with delivering high-quality training across the core range of Microsoft technologies.

As the training provider to many of Australia’s large corporate organizations, Streamtech Knowledge is the one most highly skilled training institution for Microsoft technologies in Australia. Offering a wider variety of Certified Microsoft courses than any other IT training organization, scheduled more often and with more certified and highly experienced MCT trainers on our staff, Streamtech Knowledge likes to make the business of acquiring skills as easy, flexible and convenient as possible.

Code

20776

Revision

A

Duration

5 Days

Audience

Data Professionals

Level

Expert

Price

$3,500.00

This five-day instructor-led course describes how to process Big Data using Azure tools and services including Azure Stream Analytics, Azure Data Lake, Azure SQL Data Warehouse and Azure Data Factory. The course also explains how to include custom functions, and integrate Python and R.

The primary audience for this course is data engineers (IT professionals, developers, and information workers) who plan to implement big data engineering workflows on Azure.

After completing this course, students will be able to:

  • Describe common architectures for processing big data using Azure tools and services.
  • Describe how to use Azure Stream Analytics to design and implement stream processing over large-scale data.
  • Describe how to include custom functions and incorporate machine learning activities into an Azure Stream Analytics job.
  • Describe how to use Azure Data Lake Store as a large-scale repository of data files.
  • Describe how to use Azure Data Lake Analytics to examine and process data held in Azure Data Lake Store.
  • Describe how to create and deploy custom functions and operations, integrate with Python and R, and protect and optimize jobs.
  • Describe how to use Azure SQL Data Warehouse to create a repository that can support large-scale analytical processing over data at rest.
  • Describe how to use Azure SQL Data Warehouse to perform analytical processing, how to maintain performance, and how to protect the data.
  • Describe how to use Azure Data Factory to import, transform, and transfer data between repositories and services.

In addition to their professional experience, students who attend this training should already have the following technical knowledge:

  • A good understanding of Azure data services.
  • A basic knowledge of the Microsoft Windows operating system and its core functionality.
  • A good knowledge of relational databases.

1 Architectures for Big Data Engineering with Azure

This module describes common architectures for processing big data using Azure tools and services.

Lessons
  • Understanding Big Data
  • Architectures for Processing Big Data
  • Considerations for designing Big Data solutions
Labs
  • Design a big data architecture

2 Processing Event Streams using Azure Stream Analytics

This module describes how to use Azure Stream Analytics to design and implement stream processing over large-scale data.

Lessons
  • Introduction to Azure Stream Analytics
  • Configuring Azure Stream Analytics jobs
Labs
  • Create an Azure Stream Analytics job
  • Create another Azure Stream job
  • Add an Input
  • Edit the ASA job
  • Determine the nearest Patrol Car

3 Performing custom processing in Azure Stream Analytics

This module describes how to include custom functions and incorporate machine learning activities into an Azure Stream Analytics job.

Lessons
  • Implementing Custom Functions
  • Incorporating Machine Learning into an Azure Stream Analytics Job
Labs
  • Add logic to the analytics
  • Detect consistent anomalies
  • Determine consistencies using machine learning and ASA

4 Managing Big Data in Azure Data Lake Store

This module describes how to use Azure Data Lake Store as a large-scale repository of data files.

Lessons
  • Using Azure Data Lake Store
  • Monitoring and protecting data in Azure Data Lake Store
Labs
  • Update the ASA Job
  • Upload details to ADLS

5 Processing Big Data using Azure Data Lake Analytics

This module describes how to use Azure Data Lake Analytics to examine and process data held in Azure Data Lake Store.

Lessons
  • Introduction to Azure Data Lake Analytics
  • Analyzing Data with U-SQL
  • Sorting, grouping, and joining data
Labs
  • Add functionality
  • Query against Database
  • Calculate average speed

6 Implementing custom operations and monitoring performance in Azure Data Lake Analytics

This module describes how to create and deploy custom functions and operations, integrate with Python and R, and protect and optimize jobs.This module describes how to create and deploy custom functions and operations, integrate with Python and R, and protect and optimize jobs.

Lessons
  • Incorporating custom functionality into Analytics jobs
  • Managing and Optimizing jobs
Labs
  • Custom extractor
  • Custom processor
  • Integration with R/Python
  • Monitor and optimize a job

7 Implementing Azure SQL Data Warehouse

This module describes how to use Azure SQL Data Warehouse to create a repository that can support large-scale analytical processing over data at rest.

Lessons
  • Introduction to Azure SQL Data Warehouse
  • Designing tables for efficient queries
  • Importing Data into Azure SQL Data Warehouse
Labs
  • Create a new data warehouse
  • Design and create tables and indexes
  • Import data into the warehouse.

8 Performing Analytics with Azure SQL Data Warehouse

This module describes how to import data in Azure SQL Data Warehouse, and how to protect this data.

Lessons
  • Querying Data in Azure SQL Data Warehouse
  • Maintaining Performance
  • Protecting Data in Azure SQL Data Warehouse
Labs
  • Performing queries and tuning performance
  • Integrating with Power BI and Azure Machine Learning
  • Configuring security and analysing threats

9 Automating the Data Flow with Azure Data Factory

This module describes how to use Azure Data Factory to import, transform, and transfer data between repositories and services.

Lessons
  • Introduction to Azure Data Factory
  • Transferring Data
  • Transforming Data
  • Monitoring Performance and Protecting Data
Labs
  • Automate the Data Flow with Azure Data Factory