Responsible for managing and improving data access and processing systems, collaborating with engineers and traders, and implementing automation in a data platform.
Join our dynamic and high-impact Data team as a Data Engineer, where you'll be responsible for safely receiving and storing trading-related data for the India teams, as well as operating and improving our shared data access and data processing systems. This is a critical role in the organisation as the data platform drives a huge range of trader analysis, simulation, reporting and insights. The ideal candidate should have work experience in systems engineering, preferably with prior exposure to financial markets and with proven working knowledge in the fields of Linux administration, orchestration and automation tools, systems hardware architecture as well as storage and data protection technologies.
Your Core Responsibilities:
Your Skills and Experience:
Your Core Responsibilities:
- Manage and monitor all distributed systems, storage infrastructure, and data processing platforms, including HDFS, Kubernetes, Dremio, and in-house data pipelines
- Drive heavy focus on systems automation and CI/CD to enable rapid deployment of hardware and software solutions
- Collaborate closely with systems and network engineers, traders, and developers to support and troubleshoot their queries
- Stay up to date with the latest technology trends in the industry; propose, evaluate, and implement innovative solutions
Your Skills and Experience:
- 5-7 years of experience in managing large-scale multi-petabyte data infrastructure in a similar role
- Advanced knowledge of Linux system administration and internals, with proven ability to troubleshoot issues in Linux environments
- Deep expertise in at least one of the following technologies: Kafka, Spark, Cassandra/Scylla, or HDFS
- Strong working knowledge of Docker, Kubernetes, and Helm
- Experience with data access technologies such as Dremio and Presto
- Familiarity with workflow orchestration tools like Airflow and Prefect
- Exposure to cloud platforms such as AWS, GCP, or Azure
- Proficiency with CI/CD pipelines and version control systems like Git
- Understanding of best practices in data security and compliance
- Demonstrated ability to solve problems proactively and creatively with a results-oriented mindset
- Quick learner with excellent troubleshooting skills
- High degree of flexibility and adaptability
Top Skills
Airflow
AWS
Azure
Cassandra
Docker
Dremio
GCP
Git
Hdfs
Helm
Kafka
Kubernetes
Linux
Prefect
Scylla
Spark
Similar Jobs at IMC Trading
.png){kind=avatar}
Fintech • Machine Learning • Software • Financial Services
The Commercial Legal Counsel will draft, review, and negotiate contracts, provide legal advice across departments, and support general in-house legal matters.
Top Skills:
Commercial ContractingLegal Analysis
Fintech • Machine Learning • Software • Financial Services
The Linux Engineer will manage and improve large-scale Linux systems, focusing on automation, troubleshooting, and collaborating with teams to enhance server performance and reliability.
Top Skills:
AnsibleBashCi/CdDockerGitGrafanaHieraKubernetesLinuxPrometheusPuppetPython
Fintech • Machine Learning • Software • Financial Services
Design, develop and maintain Big Data solutions, build data pipelines, collaborate on data modeling, and ensure data integrity.
Top Skills:
Alert ManagerAlertaDatabricksDockerDremioFlinkGrafanaHadoopHdfsIcebergJavaKafkaKubernetesOpsgeniePrometheusPythonS3SparkSQL
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
