Help the refugees

If you move around the world by choice, consider helping those forced from their homes by conflict. Donate to the UN Refugee Agency today.

Back

Senior Data Engineer

Visit website

SENIOR DATA ENGINEER – Remote Start Date: ASAPDuration: 6 months rolling project/ high chances going permanent.Scope: Transformation project; Oil & Energy client – they are looking for a Specialists who will assist with the design, development, and management of the company Digital Center. Role:- Strong programming experience with Redshift/SQL, Plsql Procedures and Functions.- Development experience in AWS Serverless services implementation (Lambda, Step function)- Development experience in AWS services like – Redshift, Cloud Formation, EMR.- Strong experience in code optimisation using spark SQL and pyspark.- AWS Architecture experience using S3, EC2, Lambda, Redshift , EMR, Cloud Formation etc- Experience in Big Data services (Glue, Athena)- Strong programming experience with PySpark / Python.- Understanding of AWS infrastructure and security services (AWS IAM, Cloud Watch, Config)- AWS DevOps CICD (Code Commit, Code Build, Code pipeline) and Git repo/JFrog Antifactory.- Experience of writing complex pipelines with huge volumes of data.- Ability to debug pyspark code Must haves:· Strong coding in PySpark/Python/SQL/Spark SQL· Scripting experience and be able to lead a team of 5 data engineers/set market standards with their scripting· 3+ years’ experience· Consultant must have worked in complex environment/multiple instances of data (eg across several continents). Experienced of high volume of data processing (50-100 million records)

Apply now