Help the refugees

If you move around the world by choice, consider helping those forced from their homes by conflict. Donate to the UN Refugee Agency today.

Back

Senior Data Engineer

Visit website

hackajob is partnered with a business leading the way in changing how organisations think about digital, working collaboratively with customers around the world to deliver truly intelligent and innovative transformative solutions. With a user-first approach, their solutions are secure, accessible, and cost-effective. As for their credentials, they've delivered more than 70 digital transformation projects in the public sector.Essential Experience:Software development experience with distributed data storage and processing technologies including Hadoop and Spark and using JVM languages.Experience in leading the development of substantial components for large-scale data processing solutions, taking responsibility for non-functional needs of ETL/ELT data processing pipelines such as robustness, performance and security. ‘Development’ incorporates design, code, test defect resolution and operational readiness, and includes setting the standards for these activities.Coaching and mentoring experience in data development disciplines with the ability to gain the respect of a junior team.Ability to advise architects and other stakeholders on detailed technology and development practice and on development estimates.Ability to make effective decisions within fast-moving Agile delivery and to lead on troubleshooting.Strong understanding of tools, design methodologies and best practices.Ability to clearly communicate technical design both written and verbally.Desirable Experience:Software development experience with Cloudera’s distribution of Apache Hadoop and with Python.Experience in data visualisation and data complex data transformations, including ETL tools such as Talend.Able to productionise machine learning algorithms.Understanding of text processing including Natural Language Processing.Experience with steaming and event-processing architectures including technologies such as Kafka and change-data-capture products.Data modelling experience with data storage technology, such as document, graph, log stores and other non-relational platforms.Open source contributorIf you are interested in finding out more about this fantastic opportunity, please get your application in and we can arrange a call.

Apply now