· Administer and manage Redis clusters for low-latency caching and real-time transaction processing.
· Manage MongoDB clusters (replication, sharding) for scalable transaction and semi-structured data storage.
· Working knowledge of Hadoop ecosystem (Hadoop, Hive, Pig, Oozie, Hbase, Flume, sqoop) using both automated tool sets as well as manual processes.
· Support and maintain Hadoop (HDP)clusters for batch processing, analytics, and regulatory reporting.
· Perform cluster lifecycle management provisioning, scaling, patching, and decommissioning nodes.
· Ensure 24x7 availability and resilience of production systems supporting payment flows.
· Manage and optimize Apache Kafka for high-throughput, real-time payment event streaming.
· Support Apache NiFi for ingestionpipelines from upstream payment systems and external partners.
· Work with AWS EMR for scalableprocessing of transaction data and reconciliation workloads.
· Administer HDFS, ensuring optimal replication,storage utilization, and fault tolerance.
· Manage OpenSearch / Elasticsearchclusters for transaction search, audit trails, and operational dashboards.
Required Skills & Experience
· 6–8 years of experience in Big Data / DataPlatform Engineering in enterprise environments.
· Strong hands-on experience with:
· Hadoop ecosystem (HDFS, YARN, MapReduce, HDP)
· Apache Kafka (high-throughput environments)
· Redis and MongoDB clusters
· OpenSearch / Elasticsearch
· Apache NiFi
· AWS EMR + good knowledge of AWS Cloud.
· Strong expertise in Linux system administration and scripting (Shell/Python)
· Experience with Kerberos, data security, and access governance
· Proven experience in handling high-volume, low-latency systems (preferably payments/trading)
EA License No: 11C4879 / Registration ID : R1218583
Apar Technologies Pte Ltd, Singapore