Prolific is seeking AI and Machine Learning Engineers to join their Expert Network to train and evaluate LLMs. Participants will review model architectures, audit code, provide RLHF feedback, and benchmark performance. Qualified candidates should have deep expertise in ML, experience with PyTorch/TensorFlow, and knowledge of LLM workflows like RAG and RLHF. This is a remote, contract role with flexible hours and competitive pay up to $80/hour.
Responsibilities
Evaluate LLM Architecture Logic: review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy.
Audit Code & Notebooks: validate ML-specific code (e.g., training loops, data preprocessing scripts) for efficiency and correctness.
Refine RLHF Frameworks: provide high-quality human feedback to align models with human intent, safety, and helpfulness.
Analyze Model Reasoning: critically assess AI model's chain-of-thought (CoT) prompts and identify reasoning breakdowns.
Benchmark Performance: conduct comparative testing between model outputs based on specific technical taxonomies and performance metrics.
Requirements
BS, MS, or PhD in Computer Science, Artificial Intelligence, Robotics, or related quantitative field with focus on Machine Learning.
Experience building, deploying, or fine-tuning ML models in a production environment.
Professional-level understanding of neural network architectures (Transformers, CNNs, RNNs) and optimization techniques.
Prolific is a research technology firm that provides a SaaS platform connecting academic and industry researchers with a global network of vetted research participants, facilitating human feedback for AI systems.