Sr.Data DevOps Engineer

Mid / Senior

|

In Office

Meytier Premier Employer

Working there

About This Workplace

Meytier Partner

This role requires expertise in managing data infrastructure, ensuring best practices for CI/CD, and troubleshooting technical issues related to databases, web services, Kafka, Spark, Fabric, and Databricks. The candidate will play a critical role in streamlining release management, ensuring seamless migration of product releases from development to upper environments, and maintaining robust exception management processes.


Key Responsibilities:

CI/CD Pipeline Automation:

  • Design, develop, and maintain automated CI/CD pipelines for migrating product releases across multiple environments (Dev, Test, UAT, Prod).
  • Implement and manage pipelines for Azure Data Factory (ADF), WhereScape, Fabric, Semarchy MDM, and other data platforms.
  • Automate deployment processes for products which use databases, Kafka, Spark, and other data infrastructure components.

Infrastructure Management:

  • Manage and optimize data infrastructure to ensure scalability, reliability, and performance.
  • Configure and manage connections to databases, web services, Kafka, Spark, Fabric, and Databricks.
  • Abstract and automate infrastructure configurations to enable repeatable and scalable deployments.

Scripting and Development:

  • Develop and maintain Python scripts for automation, deployment, and configuration management.
  • Write and maintain Windows and Unix shell scripts for infrastructure and deployment tasks.
  • Collaborate with development teams to integrate scripts and automation into the CI/CD process.

Release and Configuration Management:

  • Implement best practices for release management, configuration management, and exception management.
  • Ensure version control, change management, and auditability across all deployments.
  • Troubleshoot and resolve deployment issues across environments. 5. Technical Support and Troubleshooting:
  • Provide technical expertise in configuring and managing connections to various data platforms and services.
  • Diagnose and resolve technical issues related to pipelines, infrastructure, and deployments.
  • Collaborate with cross-functional teams to ensure smooth delivery of product releases.

Tooling and Best Practices:

  • Utilize Azure DevOps to build, test, and deploy pipelines.
  • Stay updated with industry best practices and emerging technologies in DevOps, data engineering, and cloud platforms.
  • Advocate for and implement DevOps best practices across the organization.

Desired Profile:

  • Proven experience in DevOps roles, with a focus on Azure Data Factory (ADF), WhereScape, Fabric, Semarchy MDM, and Microsoft Purview.
  • Strong expertise in Azure DevOps for building and managing CI/CD pipelines.
  • Proficiency in Python scripting for automation and deployment tasks.
  • Experience with Windows and Unix shell scripting.
  • Experience with databases, Kafka, Spark, Fabric, and Databricks.
  • Knowledge of configuration management, release management, and exception management practices.
  • Ability to troubleshoot and resolve technical issues related to data infrastructure and deployments.
  • Strong understanding of cloud platforms (Azure preferred) and data engineering concepts.
  • Excellent communication and collaboration skills, with the ability to work in a cross-functional team environment.

Preferred Skills:

  • Experience with Semarchy MDM and Microsoft Purview for data governance and master data management.
  • Familiarity with Kubernetes and Docker for containerization and orchestration.
  • Knowledge of IaC (Infrastructure as Code) tools such as Terraform or ARM templates. Understanding of data security and compliance best practices.

© 2025 Meytier - All Rights Reserved.
   Privacy Policy    Terms Of Use