Evaluates AI-generated data science and quantitative reasoning outputs; develops prompts, datasets, reference solutions, and objective evaluation benchmarks. Identifies statistical errors, flawed methodologies, and unsupported conclusions while providing structured feedback to improve model accuracy. Responsibilities include dataset validation, statistical and quantitative assessment, quality assurance, technical documentation, and clear communication of analytical findings.
This role is for one of our clients
Pay: 150–200/hour
Role Type: Contractor
Location: Remote
We are seeking a detail-oriented Junior Data Scientist to contribute to the evaluation and improvement of advanced artificial intelligence systems. This opportunity is ideal for early-career data science professionals with strong analytical skills and a solid understanding of statistics, machine learning, experimentation, and quantitative reasoning.
In this role, you will evaluate AI-generated responses to data science and quantitative problems, develop expert-level prompts and datasets, and create reference solutions to assess the accuracy and reliability of AI models. You will identify statistical errors, flawed methodologies, and weaknesses in quantitative reasoning while providing structured feedback to improve model performance.
The ideal candidate combines practical data science knowledge with clear technical communication and a rigorous, evidence-based approach to problem-solving. Previous experience in AI development is not necessarily required.
Requirements
Key Responsibilities
- AI Model Evaluation: Evaluate AI-generated outputs on data science, statistics, machine learning, experimentation, and quantitative analysis tasks.
- Problem and Prompt Development: Create challenging, well-defined prompts and technical problems that assess analytical reasoning and practical data science skills.
- Dataset Preparation: Prepare and validate relevant datasets for testing AI models across statistical and machine learning use cases.
- Reference Solution Development: Develop accurate reference answers, calculations, and analytical solutions to establish reliable evaluation benchmarks.
- Statistical Validation: Identify statistical errors, incorrect assumptions, flawed methodologies, and inappropriate interpretations of data.
- Quantitative Reasoning Assessment: Assess whether AI-generated solutions demonstrate sound mathematical reasoning, appropriate analytical methods, and evidence-based conclusions.
- Structured Feedback: Provide clear, actionable feedback highlighting errors, missing steps, and opportunities to improve the accuracy and reliability of AI-generated outputs.
- Quality Assurance: Validate analytical results, review solution consistency, and ensure that evaluation criteria are objective and reproducible.
- Technical Documentation: Explain statistical concepts, analytical approaches, and evaluation findings in a clear and structured manner.
Required Qualifications
- At least one year of professional experience at a well-established company in technology, finance, research, or a related field.
- At least part of the relevant professional experience should have been gained within the past three years.
- Foundational knowledge of data science, statistics, machine learning, and quantitative analysis.
- Strong analytical and problem-solving skills, with the ability to evaluate data-driven conclusions critically.
- Understanding of statistical methods, experimental design, data interpretation, and model evaluation.
- Ability to identify incorrect calculations, methodological weaknesses, and unsupported conclusions.
- Strong written communication skills, particularly when explaining technical and statistical concepts.
- Ability to work independently, follow evaluation guidelines, and deliver accurate, high-quality assessments.
Must-Have Skills
- Data Science Fundamentals
- Statistics and Probability
- Machine Learning Fundamentals
- Quantitative Reasoning
- Data Analysis and Interpretation
- Statistical Methodology
- Analytical Problem-Solving
- Technical Writing and Documentation
Good-to-Have Skills
- Python Programming
- SQL and Data Manipulation
- Pandas, NumPy, and scikit-learn
- Experimental Design and Hypothesis Testing
- Predictive Modeling and Model Evaluation
- Dataset Creation and Validation
- AI Model Evaluation and Benchmarking
- Experience in Technology, Finance, or Research Organizations
Preferred Qualifications
- Experience working at a recognized technology company, financial institution, or research organization.
- Professional exposure to statistical analysis, machine learning projects, quantitative research, or data-driven decision-making.
- Familiarity with evaluating AI-generated answers or validating analytical solutions.
- Ability to communicate complex quantitative concepts in clear, concise English.
- Candidates based in English-speaking countries are preferred.
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Leads high-volume semiconductor module and SSD manufacturing operations across assembly, testing, burn-in, inspection, labeling, and packing. Owns safety, quality, delivery, cost, productivity, capacity utilization, and workforce optimization for 24/7 operations. Directs supervisors and production teams, drives Lean, Six Sigma, Kaizen, AI, and smart manufacturing initiatives, and uses factory data to improve yield, OEE, cycle time, and labor productivity.
Top Skills:
Artificial IntelligenceKaizenLean ManufacturingMesOeePower BISAPSix SigmaSmart Manufacturing
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Leads semiconductor test process engineering for Component Test manufacturing. Responsibilities include managing engineers, developing and qualifying processes, improving yield and quality, driving NPI readiness, establishing process controls, integrating equipment and technologies, applying statistical and digital analytics, and advancing Lean and Six Sigma initiatives. The role partners across engineering, manufacturing, quality, suppliers, and global sites while ensuring safety, compliance, operational efficiency, and talent development.
Top Skills:
Ai-Assisted AnalyticsAutomated Test Equipment (Ate)Design Of Experiments (Doe)JmpManufacturing AutomationMinitabPower BIPythonSemiconductor Test ManufacturingSQLStatistical Process Control (Spc)Tableau
Cloud • Information Technology • Security • Software
Lead cloud platform reliability across observability, FinOps, incident management, AI-driven automation, business continuity, and disaster recovery. Build production tooling in Go or Python across AWS and GCP, improve availability and recovery metrics, optimize cloud costs, design and test DR environments, and establish incident processes. Partner with engineering, finance, support, and leadership while mentoring teams and driving technical direction across the organization.
Top Skills:
AthenaAWSAws Compute OptimizerAws Cost And Usage Report (Cur)Aws Cost ExplorerBigQueryCloudabilityCloudhealthCommitted-Use DiscountsDatadogEbpfGCPGoGrafanaKubecostKubernetesLarge Language Models (Llms)LokiMimirOpentelemetryPrometheusPythonReserved InstancesSavings PlansTempoTerraformThanosVantage
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
