Data Engineer

xpatjobs       |    Location:Taguig , Metro Manila       |    Country:Philippines

Data Engineer [for Project Domino preferably senior 5 years and up experience] Responsibilities Design, architect and maintain secure data platforms such as setting up data storage solutions like SQL databases Implement and maintain optimized workflows, ETL / ELT processes, and pipelines deliver flexibility for data consumption by designing batch and streaming data use cases Ensure support for traceability and reproducibility through enabling of data lineage features for ingested data Work with colleagues such as Data Scientists, Data Analysts, software and data engineers, and architects on implementing solutions involving data on distributed systems and platforms Create highly available and scalable capabilities that deliver data related tooling and manage data platforms and pipelines for stakeholders Qualifications Proficiency and experience in software engineering programming languages such as Python. Experience in Golang is preferred Expertise in SQL / RDBMS databases and proficiency in setting up these resources. Additional experience in NoSQL databases and setting up these is preferred Extensive experience in design and implementation of data handling processes and pipelines (ingestion, transformation, modelling, and storage) using Python, Apache Airflow, Spark Expertise in cloud platforms and technologies (Azure, AWS, GCP) is a plus but not required Experience with streaming platform setup and consumption such as Kafka, Flink Experience with consumption from other storages for streaming such as S3, Spark Experience in containers and container orchestration platforms such as Docker and Kubernetes Experience in CI / CD and Devops, scripting languages such as shell and automation Preferably a degree in the related fields of Computer Science Strong problem solving and solutions engineering mindset Experience in working and coordinating remotely