This position is no longer open for applications

Senior Data Engineer

Senior Data Engineer (32689) Singapore

Senior Data Engineer



What’s on offer:

  • Job Type: 12-months Fixed Term Contract (with possibility of extension)
  • Industry: Shipping / Oil & Gas
  • Location: Singapore
  • Competitive Compensation and Comprehensive Insurance Coverage

 

Job Summary:

The Senior Data Engineer will be part of data engineering team which is creating, maintaining, scaling, and improving the enterprise data platform providing the data for AI/Data Science solutions, applications/tools, and other digital use cases.


Job Responsibilities:

  • Design, develop, and maintain robust, scalable, and sustainable data products and build and optimize data pipeline and infrastructure.
  • Collaborate with stakeholders to understand their data requirements and translate them into technical solutions.
  • Identify and implement data quality monitoring and validation processes to ensure data integrity.
  • Implement data quality frameworks and ensure data governance best practices, including data lineage, data documentation, and data security.
  • Build out the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Synapse, ADF, Spark, Kafka, or similar technologies.
  • Work closely with Data Analysts and Data Architects to support their data needs and enable advanced analytics and machine learning initiatives.
  • Contribute to the development of the organization's data strategy, including evaluating new technologies, tools, and frameworks to enhance the data engineering ecosystem.


Educational Qualification and Experience Required:

  • Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related technical field.
  • 4 to 8 years to total IT experience preferably in field of data engineering.
  • 4 years or more of experience with Azure services including IAM, Synapse, Data Lake, SQL Server, ADF etc.
  • 2 years or more of experience in creating and deploying docker containers on Kubernetes.
  • 2 years or more of experience in supporting development teams on Kubernetes best practices, troubleshooting, and performance optimization.
  • 2 years or more of experience with CI/CD pipelines toll such as Jenkins and GitHub Actions.
  • 2 years or more of experience with Synapse data warehousing and Data Lake solutions
  • Strong programming skill in Python, PySpark and SQL.
  • 4 years or more of experience in scripting and automation using languages such as Bash, Python, or Go.
  • 2 years or more of experience with infrastructure-as-code tools such as Terraform, Ansible, or CloudFormation and containerization technologies (e.g., Docker, Kubernetes).
  • Knowledge of Agile methodologies and software development lifecycle processes.
  • Proven experience in designing and implementing large-scale data solutions, including data pipelines and ETL processes on Azure.


Required Knowledge:

  • Troubleshoot and resolve data-related issues, performance bottlenecks, and scalability challenges on Azure.
  • Solid understanding of DevOps principles and experience with infrastructure automation using tools like Terraform or CloudFormation.
  • Hands-on experience with cloud platforms - Azure, and related services (e.g., Synapse, Data Lake, etc.).
  • Understandings of data warehousing concepts and best practices.
  • Work closely with stakeholders to understand business requirements and translate them into data solution designs.
  • Strong understanding of data architecture principles, data modelling techniques, and data integration patterns.

 

 

© 2023 Basecamp. All rights reserved.
Base Camp Recruitment PTE Ltd
EA License: 19S9816 Reg No: R1986523

;