Design, implement, and optimize Apache DolphinScheduler workflows and ETL pipelines. Integrate with Hadoop, Spark, Flink, Hive, and Kafka. Develop custom plugins, troubleshoot performance and scalability issues, monitor job execution and resource allocation, and maintain CI/CD, version control, and infrastructure automation while collaborating with data engineers and DevOps teams.
Key Responsibilities:
- Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
- Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
- Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
- Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
- Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
- Develop custom plugins and extensions for DolphinScheduler as needed.
- Monitor and optimize job execution, resource allocation, and workflow efficiency.
- Maintain best practices for CI/CD, version control, and infrastructure automation.
Required Skills & Qualifications:
- Strong experience with Apache DolphinScheduler in a production environment.
- Proficiency in Java, Python, or Scala for workflow scripting and automation.
- Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
- Understanding of workflow orchestration, DAG execution, and scheduling strategies.
- Strong problem-solving skills and ability to work in a fast-paced environment.
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Sells enterprise cybersecurity software to new and existing customers across assigned accounts. Responsibilities include prospecting, qualification, negotiation, closing, cross-selling, upselling, renewals, forecasting, account planning, CRM documentation, customer onboarding coordination, and collaboration with internal teams and channel partners. The role also requires promoting products, communicating customer feedback, and using AI technologies to improve decision-making and business outcomes.
Top Skills:
Ai TechnologiesCRMCybersecurityEnterprise Security SoftwareFalcon FlexMarketing SoftwareSaaS
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Develops end-to-end product design solutions and high-fidelity designs for ServiceNow products. Collaborates with engineering and product management, presents ideas, gathers feedback, reviews proposals, identifies user needs, validates solutions with end users, and contributes throughout the product development lifecycle.
Top Skills:
AIWeb Applications
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Oversee private equity fund accounting, NAV calculations, capital accounts, waterfalls, carried interest, fees, expenses, transactions, valuations, and financial reporting. Review statements and investor packages, manage audits, ensure LPA and accounting-standard compliance, coordinate with auditors and tax advisors, and lead a fund accounting team. Drive controls, process standardization, automation, and system enhancements while supporting clients, GPs, and internal stakeholders.
Top Skills:
GenevaIfrsInvestranLux GaapExcelUs Gaap
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
