Lead and manage complex data engineering projects, develop innovative solutions, and mentor team members while ensuring optimal data processing and security.
Distinguished Tech Innovator:3Pillar warmly extends an invitation for you to join an elite team of visionaries. Beyond software development, we are dedicated to engineering solutions that challenge conventional norms. Envision you: steering projects that redefine urban living, establish new media channels for enterprise companies, or drive innovation in healthcare. Your invaluable expertise will serve as the cornerstone in shaping the future direction of our endeavors.This role transcends the ordinary realms of coding; it's about orchestrating technological marvels that disrupt industries. Seize this extraordinary opportunity to lead a team that is actively shaping the tech landscape for our clients, and sets global standards along the way.
Minimum Qualification
- 8+ Years of Exp into IT
- Experience: 5 of experience in data engineering,
- Strong exp in Snowflake.
- Programming & Scripting: Strong programming skills in Python and Linux Bash for automation and data workflows.
- Framework Proficiency: Hands-on experience with Luigi for orchestrating complex data workflows.
- Data Processing & Storage: Expertise in Hadoop ecosystem tools and managing SQL databases for data storage and query optimization.
- AWS Cloud Services: In-depth knowledge of AWS EC2, S3, RDS, and EMR to deploy and manage data solutions.
- Monitoring & Alerting Tools: Familiarity with monitoring solutions for real-time tracking and troubleshooting of data pipelines.
- Communication & Leadership: Proven ability to lead projects, communicate with stakeholders, and guide junior team members.
Additional experience
- Data Architecture: Experience designing or optimizing data lake solutions.
- Security Practices: Understanding of data security practices, data governance, and compliance for secure data processing.
- Automation & CI/CD: Familiarity with CI/CD tools to support automation of deployment and testing.
- Big Data Technologies: Knowledge of big data processing tools like Spark, Hive, or related AWS services.
- Advanced Analytics: Background in analytics or data science to contribute to more data-driven decision-making.
- Cross-Functional Collaboration: Experience collaborating with non-technical teams on business goals and technical solutions.
Regards
Rajan
3Pillar Global
Top Skills
Aws Ec2
Emr
Hadoop
Hive
Linux Bash
Luigi
Python
Rds
S3
Snowflake
Spark
SQL
Similar Jobs
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The Unit Manager will lead business process excellence activities, implement lean methodologies, manage stakeholder relationships, and drive continuous improvement initiatives within the organization.
Top Skills:
Generative AiLean Six SigmaMicrosoft Power Platform ToolsMS OfficePower AppsPower AutomatePower BIPower PagesRpa ToolsTableauVba Macros
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The Manager of Business Excellence will oversee operations, manage teams, and ensure optimal performance to enhance service delivery and efficiency within MetLife.
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
In this role, you will integrate AI into security processes, advocate for security, manage projects, and work with AWS, Docker, and Kubernetes.
Top Skills:
Ai-Powered ToolsAWSDockerJenkinsKubernetesServicenow
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
