Senior Data Engineer

Our client is an innovative consultancy specialising in the delivery of custom software solutions for blue-chip enterprise and public sector clients across a range of industry sectors, seeking a Senior Data Engineer. They work on important projects for their clients, deliver market-leading propositions across the digital, online and mobile channels, and are recognised for their leadership in the application of agile and lean delivery methods to assure delivery. They have offices in Australia, India, South Africa, the UK, the EU and the USA.

Senior Data Engineer : Job Description

Their Data Engineers excel at designing and implementing data pipelines for efficient data ingestion, transformation, and storage, often at scale. They are also proficient in setting up analytics clusters.

Senior Data Engineers leverage modern agile practices such as CI/CD, TDD, and XP to deliver timely and practical solutions. They also contribute to improving team efficiency and effectiveness.

Responsibilities:

• Collaborating with Clients and Teams: Collaborating with clients and team members to define data processing and access requirements, and identifying optimal delivery strategies.
• Building Data Infrastructure: Designing and implementing robust, highly available data pipelines and storage solutions, such as data warehouses.
• Agile Software Delivery: Delivering software solutions using agile practices like pair programming, TDD, and CI/CD.
• Automating Data Infrastructure: Automating data infrastructure and deployment processes to improve efficiency and reliability.
• Promoting Agile Practices: Advocating for the adoption of agile practices within client organizations.
• Mentoring Team Members: Mentoring and coaching team members to enhance their skills and knowledge.
• Enhancing Data Capabilities: Contributing to the improvement of team and client data capabilities.

Required Skills and Experience: (8 – 10 years)

• Proven Experience in Data Engineering: Significant experience working on data pipelines, platforms, and projects.
• Cloud Proficiency: Experience in developing solutions on major cloud platforms like AWS, GCP, or Azure.
• Programming Expertise: Strong background in at least one programming language, preferably Python or Scala.
• Data Storage and Processing: Experience working with various databases, data warehouses, and distributed file systems in production environments.
• Data Modeling and SQL: Strong SQL skills and experience in data modeling.
• Scalable Data Processing: Experience with large-scale data processing.
• Data Security and Compliance: Knowledge of data privacy regulations like GDPR and data security best practices.
• Infrastructure as Code and CI/CD: Proficiency in using source control (Git/GitHub), infrastructure as code tools (Terraform, Ansible), and CI/CD pipelines.
• Data Science and BI Support: Ability to support data scientists and business intelligence teams through data preparation, mentoring, and technical assistance.
• Effective Communication and Collaboration: Excellent communication and collaboration skills, including the ability to work effectively with remote and globally distributed teams.

 Other Perks

• Remote First Culture
• Great leave benefits
• Competitive Salaries

*This role is only open to candidates that are based in South Africa