Lead development and deployment of end-to-end ML and NLP solutions using Azure and Databricks. Build and optimise embeddings, semantic matching, RAG and hybrid retrieval pipelines, perform vector index tuning, migrate cloud-native pipelines to Azure Databricks, and develop forecasting and topic models for client use cases.
Job Description:
Senior Data Scientist — ML & Semantic AI
Technologies: Azure · NLP · RAG · Semantic Matching · Python
Role SummaryWe are looking for a Data Scientist with expertise in Python, Azure Cloud, and NLP to build and enhance machine learning models at scale. The role includes embedding optimisation, semantic matching, LDA and RAG architectures, dense and sparse retrieval pipelines, and migration of cloud-native data pipelines to Azure Databricks.
Core Requirements- Design and execute end-to-end machine learning pipelines including data extraction, preprocessing, feature engineering, model development, tuning, and deployment.
- Develop machine learning pipelines using Azure Synapse, Databricks, and Snowflake.
- Build and deploy classification, regression, and clustering models.
- Develop and deploy proof-of-concept solutions for client use cases.
- Implement semantic matching and similarity search using cosine similarity, dot-product scoring, and bi-encoder/cross-encoder architectures (e.g., SBERT, sentence-transformers).
- Build embedding models by fine-tuning pre-trained models and optimising embedding storage in vector databases such as Chroma DB, FAISS, and Azure AI Search.
- Train and optimise models for new data providers with dynamic input handling.
- Improve LDA model performance for large-scale topic modelling.
- Implement hybrid semantic search by combining dense and sparse retrieval methods.
- Optimise RAG architectures and retrieval QA systems for chatbot and recommendation performance.
- Enable semantic query understanding using intent classification and query expansion techniques.
- Develop forecasting models for marketing, demand prediction, and trend analysis.
- Apply NLP-based forecasting techniques using sentiment and external data.
- Use semantic similarity for audience intelligence, including zero-shot and few-shot classification techniques.
- Migrate data pipelines from Azure Synapse to Azure Databricks and retrain models accordingly.
- Optimise embedding storage and retrieval within Azure AI Search.
- Perform vector index tuning including HNSW optimisation and ANN benchmarking for production systems.
Python, Azure Databricks, Azure ML, Azure Synapse, Azure Blob Storage, Scikit-learn, NumPy, Pandas, Hugging Face, sentence-transformers, FAISS, Chroma DB, Azure AI Search, LangChain, TensorFlow, PyTorch, Statsmodels, Azure OpenAI.
Location:
DGS India - Mumbai - Thane Ashar IT ParkBrand:
MerkleTime Type:
Full timeContract Type:
PermanentSimilar Jobs
AdTech • Marketing Tech
Design, develop, and deploy end-to-end ML pipelines and semantic AI solutions (RAG, embeddings, dense/sparse retrieval). Migrate cloud pipelines to Azure Databricks, optimise embedding storage and vector indexes, build forecasting and NLP models, and deliver production-grade solutions for classification, clustering, and recommendation use cases.
Top Skills:
AnnAzureAzure Ai SearchAzure Blob StorageAzure DatabricksAzure MlAzure OpenaiAzure SynapseChroma DbCosine SimilarityDot-ProductFaissHnswHugging FaceLangchainLdaNumpyPandasPythonPyTorchRagSbertScikit-LearnSemantic MatchingSemantic SearchSentence-TransformersSnowflakeStatsmodelsTensorFlow
Healthtech • Biotech • Pharmaceutical • Manufacturing
Build and operationalize predictive and prescriptive models for HR workflows at enterprise scale. Deploy, monitor, and improve production models; embed intelligence into products and APIs; ensure model explainability, fairness, privacy, and responsible AI; partner with product, engineering, and experience teams to deliver actionable, governed data products.
Top Skills:
DatabricksPython
Artificial Intelligence • HR Tech • Professional Services • Software
Senior Data Scientist responsible for analyzing large datasets, building predictive and statistical models (fraud, risk, customer analytics), conducting EDA and A/B testing, creating dashboards, and communicating insights. Collaborates with product, engineering, and operations to improve decision-making, automation, and regulatory/compliance analytics; documents methodologies for scalability.
Top Skills:
AWSAzureGCPLookerNlpNumpyPandasPower BIPythonScikit-LearnScipySparkSQLStatsmodelsTableau
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
