Data Engineer

Preferred Experience: 4-6 years

Role Summary

We are seeking a Data Engineer to build and support cloud-based data pipelines, database solutions, integrations, and data-processing workflows.

Key Responsibilities

• Develop and maintain data pipelines, integration workflows, and scheduled data processes.

• Write SQL queries, stored procedures, and database objects.

• Build solutions using Azure Data Factory, Logic Apps, and Synapse.

• Support cloud deployments, automated testing, and CI/CD pipelines.

• Monitor data processes and troubleshoot data quality, performance, and integration issues.

Required Skills

• Strong SQL, stored procedure, database architecture, and data integration experience.

• Azure Data Factory, Logic Apps, Synapse, or related Azure data services.

• APIs, Entity Framework, cloud technologies, and source-control practices.

• CI/CD, Docker, infrastructure-as-code exposure, automated testing, and NoSQL are preferred.

Technical Screening Questions

1. How would you build an Azure Data Factory pipeline for incremental loading from a source database into a target data platform?

2. Explain how you would handle schema changes, late-arriving records, duplicates, and failed records in a data pipeline.

3. Write or describe the SQL approach you would use to identify duplicate business records and retain the most recent valid record.

4. When would you use Azure Logic Apps instead of Azure Data Factory for an integration requirement?

5. How do you monitor scheduled pipelines, investigate failures, and ensure that a rerun does not create duplicate or inconsistent data?