Senior Data Engineer - Azure

Our client within the Education sector are looking for a Senior Data Engineer to join them on a contract basis.

Inside IR35

6 month contract

Hybrid 1 - 2 days per week onsite

Technical Skills:

• In-depth knowledge of relevant Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, Azure Stream Analytics, etc.
• Strong Python or Scala with Spark, PySpark experience
• Experience with relational databases and NoSQL databases
• Significant experience and in-depth knowledge of creating data pipelines and associated design principles, standards, Datamodelling concepts, ETL procedures, and all steps of data production process
• Experience with unit and integration testing, and data quality frameworks such as Deequ, Great Expectations or Delta Live
• Experience working with continuous integration
• Good understanding of Datamodelling (medallion architectures, Kimball architectures), data schemas (Avro Schema, JSON Schema) and data serialisation (Parquet, ORC, Avro)

Azure Services Knowledge:

• Azure Data Factory: Advanced knowledge of Azure Data Factory for orchestrating and automating data workflows.
• Azure SQL Database: Expertise in SQL and T-SQL for data querying and manipulation.
• Azure Synapse Analytics: Ability to design and implement data warehousing solutions using Azure Synapse Analytics.
• Azure Databricks: Proficiency in using Azure Databricks for data processing and analytics.
• Apache Spark: Deep understanding of Apache Spark for large-scale data processing.
• Azure Blob Storage and Azure Data Lake Storage: Expertise in setting up and optimizing Azure Data Lake Storage for storing and managing large volumes of data.

Cloud Experience:

• Azure Resource Manager (ARM): Understanding how to deploy, manage, and monitor Azure resources using ARM templates.
• Azure Portal and CLI: Proficiency in using the Azure Portal and Command-Line Interface for managing Azure resources.
• Azure Security and Compliance: Knowledge of Azure security features and compliance standards.
• Monitoring and Logging: Familiarity with Azure Monitor and Azure Log Analytics for monitoring and logging.
• Ways of working:
• Experience in Data Migration, Data Profiling and Data Quality enhancement.
• Experience in Agile (Scrum) way of working
• Understanding of CI/CD process & ways of working including Git
• Excellent Problem-solving skills
• Excellent Communication skills
• Strong understanding of software development life cycle
• Knowledge of software engineering best practices for the full software development life cycle, including Agile methodologies, coding standards, code reviews, source control management, build processes, testing, etc

If you believe you have the right skills for this role please submit your CV.

Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation to this vacancy.