Azure Data Engineer

Learn how to analyze and interpret data to drive better business decisions. This course covers data analysis fundamentals, descriptive statistics, and visualization techniques to help you transform raw data into meaningful insights

Course Content

Basics of Cloud Computing

What is Cloud Computing?
Types of Cloud Deployment Models (Private, Public, Hybrid)
Types of Cloud Services (IaaS, PaaS, SaaS)

Introduction to Big Data

Introduction to Azure

Create an Azure Account
Overview of Azure Portal
Subscription & Resource Group
Blob Storage, Data Lake Storage
Azure SQL Server & Database
Azure Data Factory
Azure Databricks
Azure Key Vaults
Azure Logic Apps
GitHub Repository

Data Lake Storage (ADLS)

Azure SQL

Azure Data Factory (ADF)

ADF UI Walkthrough
Components of ADF
Integration Runtime (Azure Auto, Self-Hosted, Azure SSIS)
Linked Services (Blob, ADLS, Azure SQL, On-Prem SQL)
Datasets (CSV, Parquet, Excel, Avro, JSON, Azure SQL, On-Prem SQL)
Pipelines (Create, Execute via REST API, Debug, Publish)
Activities (Copy Data, Delete, Stored Procedures, Get-Metadata, Lookup, ForEach, If, Switch, Until, Wait, Fail, Variables, Databricks Notebook, Execute Pipelines)
Data Flows (Filter, Select, Sort, Aggregate, GroupBy, Joins, Lookup, Exists, Union, AlterRow, Rank, Pivot/Unpivot, Flowlets)
Parameters (Dynamic ADLS/SQL Mgmt, Pipeline Execution, Global Parameters)
Triggers (Schedule, Tumbling Window, Storage Events)
Monitoring & Notifications (Monitor Jobs, Expression Language, Failure Alerts via Logic Apps, Key Vault Integration)
Repository & Change Mgmt (GitHub Repo, ARM Templates Export/Import, Migration)

Databricks + PySpark

Introduction to Spark & Architecture
Transformations & Actions
Introduction to Databricks
Workspace & Cluster Setup
Databricks Notebooks
Databricks File System (DBFS) Management
File Handling (Multiple Files/Folders)
Python Basics (Variables, Datatypes, Operators)
PySpark Basics
Read/Write Data (CSV, Azure SQL, DBFS, ADLS)
Run SQL Queries in Databricks using Spark SQL

A course by

educlassixglobal

Student Ratings & Reviews

No Review Yet

What Will You Learn?

Course Content

Basics of Cloud Computing

What is Cloud Computing?

Types of Cloud Deployment Models (Private, Public, Hybrid)

Types of Cloud Services (IaaS, PaaS, SaaS)

Introduction to Big Data

What is Data?

What is Big Data?

Types of Data: Structured, Semi-Structured, Unstructured

Introduction to Azure

Create an Azure Account

Overview of Azure Portal

Subscription & Resource Group

Blob Storage, Data Lake Storage

Azure SQL Server & Database

Azure Data Factory

Azure Databricks

Azure Key Vaults

Azure Logic Apps

GitHub Repository

Data Lake Storage (ADLS)

Create a Storage Account

Types of Storage Accounts

Create an ADLS Account

Configure Access to ADLS

Load Data to ADLS

Read & Write Data to ADLS

Configure Backup & Disaster Recovery

Azure SQL

Create Azure SQL Server & Database

Configure Elastic Pools

Configure Compute Resources

Configure Access & Security

Configure Azure SQL Connection to Data Factory & Databricks

Azure Data Factory (ADF)

ADF UI Walkthrough

Components of ADF

Integration Runtime (Azure Auto, Self-Hosted, Azure SSIS)

Linked Services (Blob, ADLS, Azure SQL, On-Prem SQL)

Datasets (CSV, Parquet, Excel, Avro, JSON, Azure SQL, On-Prem SQL)

Pipelines (Create, Execute via REST API, Debug, Publish)

Activities (Copy Data, Delete, Stored Procedures, Get-Metadata, Lookup, ForEach, If, Switch, Until, Wait, Fail, Variables, Databricks Notebook, Execute Pipelines)

Data Flows (Filter, Select, Sort, Aggregate, GroupBy, Joins, Lookup, Exists, Union, AlterRow, Rank, Pivot/Unpivot, Flowlets)

Parameters (Dynamic ADLS/SQL Mgmt, Pipeline Execution, Global Parameters)

Triggers (Schedule, Tumbling Window, Storage Events)

Monitoring & Notifications (Monitor Jobs, Expression Language, Failure Alerts via Logic Apps, Key Vault Integration)

Repository & Change Mgmt (GitHub Repo, ARM Templates Export/Import, Migration)

Databricks + PySpark

Introduction to Spark & Architecture

Transformations & Actions

Introduction to Databricks

Workspace & Cluster Setup

Databricks Notebooks

Databricks File System (DBFS) Management

File Handling (Multiple Files/Folders)

Python Basics (Variables, Datatypes, Operators)

PySpark Basics

Read/Write Data (CSV, Azure SQL, DBFS, ADLS)

Run SQL Queries in Databricks using Spark SQL

Tags

A course by

Student Ratings & Reviews

Course Includes:

Share On:

Sign in

Sign up