Similar Jobs
Software
Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, PySpark, SQL, GCP, BigQuery, and Delta Lake. Build agentic workflows that automate data validation, troubleshooting, and operational processes. Develop reusable frameworks, implement data quality and monitoring controls, optimize workloads, and support testing, deployment, productionization, and ongoing enhancements across data and AI-enabled solutions.
Top Skills:
Agentic WorkflowsAi AgentsBigQueryData PipelinesDatabricksDelta LakeDeploymentEltETLGoogle Cloud Platform (Gcp)Lakehouse ArchitectureMonitoringOrchestration FrameworksPysparkSQLVersion ControlWorkflow Automation
Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Administer SolarWinds and monitor Windows, Linux, network, and application infrastructure. Configure monitoring modules, devices, alerts, dashboards, polling engines, agents, credentials, reports, and platform maintenance. Respond to 24x7 alerts, investigate issues through RDP and SSH, perform first-level troubleshooting, and escalate incidents. Manage ITSM tickets, documentation, runbooks, handovers, audits, patching windows, and maintenance schedules while supporting client-facing infrastructure operations.
Top Skills:
Active DirectoryApacheAppdynamicsDatadogDnsDynatraceExchangeGrafanaIcmpIisItilJIRALinuxLog AnalyzerNagiosNcmNew RelicNpmNtaPrometheusPrtgRdpSamScmScomServicenowSnmpSolarwinds Hybrid Cloud ObservabilitySolarwinds OrionSplunkSQL ServerSshTcp/IpTomcatWindows ServerWmiWpmZabbix
Artificial Intelligence • Automotive • Computer Vision • Information Technology • Internet of Things • Logistics • Software
Lead software engineer designing and developing scalable systems that enrich and publish global points-of-interest data. Responsibilities include full lifecycle development, automation, testing, deployment, configuration management, production support, performance optimization, AI/ML algorithm implementation, deep learning optimization, and collaboration with users and engineering teams. The role also involves translating requirements into reliable software designs, improving development practices, and mentoring team members.
Top Skills:
SparkAWSAws KiroAzureDbmsDeep LearningGithub CopilotJavaJSONMicroservicesNeural NetworksPostgisPostgresPysparkPythonRestScalaSoaSQLXML
Design and optimize scalable ETL/ELT pipelines using Databricks, PySpark, SQL, GCP, BigQuery, and Delta Lake. Build agentic workflows that automate data validation, troubleshooting, issue identification, and operational processes. Develop reusable frameworks, implement data quality and production controls, optimize workloads, and support testing, deployment, monitoring, and production support. Collaborate with business, data engineering, and platform teams to identify automation opportunities.
We are looking for a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.
The mandatory requirements are 5+ years of experience with Databricks and PySpark, advanced SQL, hands-on GCP experience, and experience with Delta Lake and modern lakehouse architectures.
RECRUITMENT PROCESS
1. Application — Share a few details about your experience and background.
2. Coding Challenge — If applicable, complete it in the coding language you are most comfortable with.
3. Video Interview — Record a short video introduction in English.
4. Technical Interview or Hiring Manager Interview — Discuss your experience and fit for the role.
5. Offer — If it’s a match, you’ll receive an offer.
WHAT YOU’LL GAIN
• Remote work — Work from where you feel most productive.
• Local presence in India — Work in a structured and compliant environment aligned with Indian regulations.
• Competitive compensation in INR — Receive compensation in INR, plus support for learning, education, and wellness.
• Exciting projects — Work with modern technologies for global clients and fast-growing companies.
MUST HAVES
• 5+ years of strong hands-on experience with Databricks and PySpark.
• Advanced SQL and data-processing skills.
• Hands-on experience with GCP, particularly BigQuery.
• Experience with Delta Lake and modern data lake/lakehouse architectures.
• Strong understanding of ETL/ELT, data pipeline design, performance optimization, and data quality.
• Experience building reliable, scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of software engineering practices, including testing, version control, deployment, monitoring, and production support.
• Upper-intermediate English level.
NICE TO HAVES
• Experience developing or integrating AI/agentic workflows, AI agents, or workflow automation solutions.
• Experience applying AI to automate data engineering, validation, troubleshooting, or operational processes.
• Familiarity with orchestration and automation frameworks.
• Experience developing reusable data engineering frameworks and platform components.
• Exposure to productionizing AI-enabled solutions with appropriate validation, monitoring, and human oversight.
WHAT YOU WILL DO
• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes such as data validation, issue identification, troubleshooting, and workflow execution.
• Integrate agentic capabilities with existing Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and reusable frameworks and components to support new business requirements and datasets across multiple use cases.
• Implement data quality checks, monitoring, validation, exception handling, and production controls.
• Optimize PySpark and SQL workloads for performance, reliability, and scalability.
• Support testing, deployment, productionization, troubleshooting, and ongoing enhancement of data and agentic solutions.
• Collaborate with business, data engineering, and platform teams to identify further automation opportunities.
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
