Design, implement, and optimize Apache DolphinScheduler workflows and ETL pipelines. Integrate with Hadoop, Spark, Flink, Hive, and Kafka. Develop custom plugins, troubleshoot performance and scalability issues, monitor job execution and resource allocation, and maintain CI/CD, version control, and infrastructure automation while collaborating with data engineers and DevOps teams.
Key Responsibilities:
- Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
- Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
- Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
- Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
- Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
- Develop custom plugins and extensions for DolphinScheduler as needed.
- Monitor and optimize job execution, resource allocation, and workflow efficiency.
- Maintain best practices for CI/CD, version control, and infrastructure automation.
Required Skills & Qualifications:
- Strong experience with Apache DolphinScheduler in a production environment.
- Proficiency in Java, Python, or Scala for workflow scripting and automation.
- Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
- Understanding of workflow orchestration, DAG execution, and scheduling strategies.
- Strong problem-solving skills and ability to work in a fast-paced environment.
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Manage EHS programs at a semiconductor manufacturing site: perform hazard identification and risk assessments, oversee PTW/LOTO and contractor safety, conduct audits and incident investigations, run trainings and drills, support ISO 45001/14001 compliance, track safety metrics, and help develop AI-enabled EHS reports and process improvements.
Top Skills:
AIConfined Space EntryHot WorkIso 14001Iso 45001Lockout-Tagout (Loto)Permit-To-Work (Ptw)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Performs complex semiconductor manufacturing operations independently, serves as a subject matter expert, troubleshoots equipment and process issues, and supports yield, scrap, cycle-time, and productivity improvements. Analyzes manufacturing data, trains production associates, supports engineering evaluations and product introductions, investigates root causes, implements corrective actions, and maintains safety, cleanroom, quality, and operational standards while working rotating 12-hour shifts.
Top Skills:
Cleanroom OperationsHigh-Volume Production OperationsManufacturing EquipmentManufacturing SystemsProcess ControlsSemiconductor Manufacturing
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Lead mechanical design reviews and coordination for large-scale semiconductor/pharma cleanroom facilities. Perform HVAC, exhaust, utility and piping analyses, CFD review, pipe stress and vibration checks, equipment specification and procurement support, clash detection, BIM coordination, and technical oversight during construction to ensure compliant, energy-efficient mechanical systems and on-time project delivery.
Top Skills:
AutocadBimBim 360ClaudeComputational Fluid Dynamics (Cfd)CopilotGen-Ai (ChatgptLlms)NavisworksP&Ids
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
