Company Details

Job Title

Job Description

Role description: We are seeking a talented and motivated Data Engineer with expertise in Python, Azure (Data Factory), and data engineering tools to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support our data analytics initiatives. You will collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to ensure the availability, reliability, and scalability of our data solutions.

Key responsibilities:
•    Responsible to build and monitor the ADF pipelines on various source systems e.g. on-prim, cloud, sftp etc.
•    Responsible to build solution on synapse.
•    Design, develop, and maintain data pipelines and ETL processes using Python and related frameworks.
•    Implement data processing and transformation workflows to cleanse, enrich, and aggregate data from various sources.
•    Collaborate with data scientists and analysts to understand their data requirements and implement efficient data models and structures.
•    Work with cloud-based technologies, specifically Azure, to build scalable and reliable data solutions.
•    Develop and optimize data storage and retrieval mechanisms, such as databases and data lakes, to support large-scale data analytics.
•    Implement data quality and validation processes to ensure the accuracy and consistency of data.
•    Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
•    Perform performance tuning and optimization of data processing and storage systems.
•    Stay up to date with the latest advancements in data engineering tools, techniques, and frameworks.
•    Collaborate with cross-functional teams to ensure the successful implementation and integration of data solutions into business applications.
•    Will to learn new tools and technologies as an when required.  

Key stakeholders:
•    Data Analytics team
•    Data Scientist/ ML Engineer
•    Product and Implementation partners
•    Product Managers and delivery leads from IT horizontal teams within SAP etc.

Skills required:
•    Technical Skills:
o    Deep understanding of Microsoft Azure stack – ADF, Synapse, Databricks, Azure monitoring, ADLS etc
o    Good exposure in Python, SQL, Pyspark 
o    Good understanding of cloud architecture
o    Extensive technical knowledge to help decide on escalated troubleshooting issues across domains
o    Experience with data modeling and ETL processes.
o    Strong analytical and problem-solving skills.
o    Excellent communication and teamwork abilities.

Competencies