Senior Data Engineer
South Africa (Hybrid)Full-timeSeniorEngineering
About the Role
We need a Senior Data Engineer to build and scale the data infrastructure that underpins the entire APEX platform. Zenith — our data collection and aggregation layer — is the foundation everything else runs on. You will design ETL/ELT pipelines that ingest structured and unstructured data from diverse sources across agriculture, mining, retail, and banking, transforming it into the intelligence layer that powers Quantum’s AI analytics, Prism’s visualisations, and Vector’s verification processes.
Qualifications
Minimum Requirements
- 5+ years experience in data engineering or data platform development
- Strong proficiency in Python and SQL
- Experience designing and operating ETL/ELT pipelines at scale (Airflow, dbt, Spark, or similar)
- Deep knowledge of data warehousing concepts and dimensional modelling
- Experience with cloud data platforms (AWS Redshift/Glue, Azure Data Factory/Synapse, GCP BigQuery)
- English language fluency, written and spoken
Preferred Skills
- Experience with real-time streaming architectures (Kafka, Kinesis, Event Hubs)
- Familiarity with geospatial data processing and GIS systems
- Background in data governance, data quality frameworks, and metadata management
- Experience with NLP or unstructured data processing pipelines
- Knowledge of graph databases or knowledge graph patterns
Responsibilities
- Design, build, and maintain the Zenith data aggregation layer — the core data infrastructure for the APEX platform
- Build scalable ETL/ELT pipelines that ingest data from APIs, databases, file systems, web scrapers, and IoT sources
- Implement data quality checks, monitoring, and alerting to ensure accuracy and reliability
- Optimise data storage, partitioning, and query performance for both analytical and operational workloads
- Collaborate with ML engineers to prepare training datasets and serve model outputs through the data layer
- Design data models that support multi-tenant, multi-industry use cases (agriculture cooperatives, mining operations, banking compliance)
- Contribute to data governance policies, lineage tracking, and access control frameworks
- Mentor junior engineers on data engineering best practices and pipeline design patterns
Education & Experience
- Bachelor’s or Master’s in Computer Science, Data Science, Engineering, or equivalent practical experience
- Cloud data certifications (AWS Data Analytics, Azure Data Engineer, GCP Data Engineer) are advantageous
- Track record of building and operating data platforms serving enterprise-grade workloads
Why Join Us?
- Own the data foundation of an enterprise intelligence platform — everything starts with the data you engineer
- Work with diverse, high-impact datasets across agriculture, mining, banking, and government
- Directly influence the architecture of a product suite used by institutional decision-makers
- Hybrid work from South Africa with flexible scheduling