Ingesting data from multiple external vastly different sources at hundreds of rich data points per second, moving terabytes of data while processing it in real-time, running complex and complicated prediction and forecasting AI models while coupling their output into a hybrid human-machine data refinement process and presenting the result through a nimble low-latency SaaS solution used by customers around the globe is no small feat of science and engineering. This processing requires a highly reliable, stable, fault-tolerant infrastructure that can withstand multiple and varied uses and abuses by data analysts, data scientists, industry experts, and the end-users. Fast product iteration cycles necessitate a platform that allows engineers to deploy frequently, independently, and without in-depth knowledge of infrastructure complexities.
The Data Services Team is responsible for the developer platform, and the impressive Amazon AWS estate Vortexa uses to achieve its purpose. This includes two major areas of concern: infrastructure availability, security and costs, as well as developer experience. We use a wide variety of technologies like Kubernetes, Kubevela, Golang, Kafka, Opensearch, Airflow, Athena, and others.
You will be a key member of the small, globally distributed team, instrumental in assessing current technologies, delivering the best possible engineering experience with the shortest development iterations, helping crystallise, converge and standardise technology stacks and implementing the infrastructure roadmap, ensuring 100% uptime, availability, and fault-tolerance of every component of the platform, while optimising AWS costs. You will provide your customers such as data analysts, data scientists, and other engineers with a delightful developer experience tooling boosting their productivity, and allowing them to concentrate on business logic. You will codify and automate any operational knowledge gathered so that R&D can focus purely on creating value for our customers.
