Senior Data Engineer – Dublin · (LRSDE) Dublin, Republic of Ireland
| Salary: | EUR442 - EUR557 per day + PAYE/Umb/Irish Ltd |
Senior Data Engineer – Dublin
Gibbs Hybrid is looking for an accomplished Data Engineer to support a Dublin based finance client on a large risk consolidation project. 2-3 days onsite work Dublin. Work options: PAYE, umbrella or Irish Limited company.
· Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system
· Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers.
· Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data.
· Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines. Implement data governance in line with company standards.
· Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability.
Essential:
• Extensive experience designing data solutions including data modeling.
• Extensive hands-on experience developing data processing jobs (PySpark / SQL) that demonstrate a strong understanding of software engineering principles.
• Experience orchestrating data pipelines using technology like ADF, Airflow etc
• Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another.
• Experience building data pipelines on either AWS, Azure or GCP, following best practices in Cloud deployments
• Experience working with Hive /HBase / Presto
• Fluent in SQL (any flavor), with experience using Window functions and more advanced features.
• Understanding of DevOps tools, Git workflow and building CI/CD pipelines
• Experience supporting big data pipelines.
• Experience applying data governance controls within a highly regulated environment.
• Data Engineering experience
• Experience working in projects with agile/scrum methodologies.
• Familiar with Azure Data Factory or Apache Airflow
• Familiar with Azure Databricks or Snowflake
• Experience with shell scripting languages
• Well versed in Python, in fulfilling multiple general-purpose use-cases, and not limited to developing data APIs and pipelines.
• Experience with Apache Spark and related Big Data stack and technologies
• Experience working with Apache Kafka, building appropriate producer/consumer apps.
• Familiarity with production quality ML and/or AI model development and deployment.
• Experience working with Kubernetes and Docker, and knowledgeable about cloud infrastructure automation and management (e.g., Terraform)
Click Apply now/contact Lianne to be considered for the Senior Data Engineer – Dublin role
Gibbs Hybrid is looking for an accomplished Data Engineer to support a Dublin based finance client on a large risk consolidation project. 2-3 days onsite work Dublin. Work options: PAYE, umbrella or Irish Limited company.
· Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system
· Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers.
· Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data.
· Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines. Implement data governance in line with company standards.
· Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability.
Essential:
• Extensive experience designing data solutions including data modeling.
• Extensive hands-on experience developing data processing jobs (PySpark / SQL) that demonstrate a strong understanding of software engineering principles.
• Experience orchestrating data pipelines using technology like ADF, Airflow etc
• Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another.
• Experience building data pipelines on either AWS, Azure or GCP, following best practices in Cloud deployments
• Experience working with Hive /HBase / Presto
• Fluent in SQL (any flavor), with experience using Window functions and more advanced features.
• Understanding of DevOps tools, Git workflow and building CI/CD pipelines
• Experience supporting big data pipelines.
• Experience applying data governance controls within a highly regulated environment.
• Data Engineering experience
• Experience working in projects with agile/scrum methodologies.
• Familiar with Azure Data Factory or Apache Airflow
• Familiar with Azure Databricks or Snowflake
• Experience with shell scripting languages
• Well versed in Python, in fulfilling multiple general-purpose use-cases, and not limited to developing data APIs and pipelines.
• Experience with Apache Spark and related Big Data stack and technologies
• Experience working with Apache Kafka, building appropriate producer/consumer apps.
• Familiarity with production quality ML and/or AI model development and deployment.
• Experience working with Kubernetes and Docker, and knowledgeable about cloud infrastructure automation and management (e.g., Terraform)
Click Apply now/contact Lianne to be considered for the Senior Data Engineer – Dublin role