Note! Apply link will take you to an external website.
Data Engineer (Spark Scala DevOps)
Unison Consulting Pte Ltd, Taiwan
Experience
1 Year
Salary
Job Type
Job Shift
Job Category
Traveling
No
Career Level
Telecommute
Qualification
As mentioned in job details
Total Vacancies
1 Job
Posted on
Feb 19, 2024
Last Date
Mar 16, 2024
Location(s)
Job Description
- Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
- Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
- Ensure data quality and integrity throughout the data processing lifecycle
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
- Optimize data engineering workflows for containerized deployment and efficient resource utilization
- Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability
- Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
- Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
- Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
- Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
- Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps
- Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank
Requirements
- 5+ Years of experience as Data Engineer(Spark, Scala and DevOps)
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
- Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
- Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank
Job Specification
Unison Consulting Pte Ltd
Information Technology and Services - Singapore, Singapore