Talent.com
MCP & Tools Python Developer - Agent Evaluation Infrastructure

MCP & Tools Python Developer - Agent Evaluation Infrastructure

MindriftNew York, NY, US
6 days ago
Job type
  • Part-time
  • Remote
  • Quick Apply
Job description

At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We’re on the hunt for hands-on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team.

Who we're looking for :

Calling all security researchers, engineers, and penetration testers with a strong foundation in problem-solving, offensive security, and AI-related risk assessment.

We’re looking for someone who can bring a hands-on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability.

What you’ll be doing :

  • Developing and maintaining MCP-compatible evaluation servers
  • Implementing logic to check agent actions against scenario definitions
  • Creating or extending tools that writers and QAs use to test agents
  • Working closely with infrastructure engineers to ensure compatibility
  • Occasionally helping with test writing or debug sessions when needed

Although we’re only looking for experts for this current project, contributors with consistent high-quality submissions may receive an invitation for ongoing collaboration across future projects.

Requirements

  • 4+ years of Python development experience , ideally in backend or tools
  • Solid experience building APIs , testing frameworks, or protocol-based interfaces
  • Understanding of Docker, Linux CLI, and HTTP-based communication
  • Ability to integrate new tools into existing infrastructures
  • Familiarity with how LLM agents are prompted, executed, and evaluated
  • Clear documentation and communication skills - you’ll work with QA and writers
  • We also value applicants who have :

  • Experience with Model Context Protocol (MCP) or similar structured agent-server interfaces
  • Knowledge of FastAPI or similar async web frameworks
  • Experience working with LLM logs, scoring functions, or sandbox environments
  • Ability to support dev environments (devcontainers, CI configs, linters)
  • JS experience
  • Benefits

  • Get paid for your expertise, with  rates that can go up to $55 / hour  depending on your skills, experience, and project needs.
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
  • Influence how future AI models understand and communicate in your field of expertise.
  • Create a job alert for this search

    Python Developer • New York, NY, US

    Related jobs
    • Promoted
    • New!
    Compiler Engineer

    Compiler Engineer

    VirtualVocationsYonkers, New York, United States
    Full-time
    A company is looking for a Compiler Engineer - MLIR / PyTorch Infrastructure.Key Responsibilities Architect the migration of the existing compiler flow into MLIR, defining dialects, passes, and l...Show moreLast updated: 18 hours ago
    • Promoted
    Platform Integration Engineer

    Platform Integration Engineer

    VirtualVocationsJamaica, New York, United States
    Full-time
    A company is looking for a Platform Integration Engineer (Agentic Systems / Integrations).Key Responsibilities Design and validate APIs, SDKs, and event-driven pipelines for agentic workflows Revi...Show moreLast updated: 19 days ago
    • Promoted
    • New!
    Principal Analog Engineer

    Principal Analog Engineer

    VirtualVocationsJamaica, New York, United States
    Full-time
    A company is looking for a Principal Analog Mixed-Signal Engineer.Key Responsibilities Conduct low-level SPICE simulations and top-level algorithm development in Python and SystemVerilog / RNM Ens...Show moreLast updated: 18 hours ago
    • Promoted
    • New!
    Principal LLM Engineer

    Principal LLM Engineer

    VirtualVocationsPaterson, New Jersey, United States
    Full-time
    A company is looking for a Principal Large Language Model Engineer.Key Responsibilities Design and implement high-performance, scalable LLM services using Python and Cloud technologies Collabora...Show moreLast updated: 9 hours ago
    • Promoted
    Cloud Infrastructure Engineer - AWS & ML Tooling

    Cloud Infrastructure Engineer - AWS & ML Tooling

    Zelis Healthcare, LLCMorristown, NJ, United States
    Full-time
    Zelis is modernizing the healthcare financial experience for all by providing a connected platform that bridges the gaps and aligns interests across payers, providers, and healthcare consumers.This...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Infrastructure Engineer

    Cloud Infrastructure Engineer

    VirtualVocationsPaterson, New Jersey, United States
    Full-time
    A company is looking for a Staff Cloud Infrastructure Engineer to join their team.Key Responsibilities Design and build infrastructure and tooling to support the platform Automate infrastructure...Show moreLast updated: 30+ days ago
    • Promoted
    Lead DevOps Engineer

    Lead DevOps Engineer

    VirtualVocationsElizabeth, New Jersey, United States
    Full-time
    A company is looking for a Lead DevOps Engineer to lead, innovate, and drive excellence across teams.Key Responsibilities Define and implement DevOps best practices, including CI / CD pipelines and...Show moreLast updated: 30+ days ago
    • Promoted
    CRM Developer

    CRM Developer

    VirtualVocationsPaterson, New Jersey, United States
    Full-time
    A company is looking for a CRM Developer to oversee the administration, customization, and integration of their enterprise CRM platform. Key Responsibilities Manage and maintain CRM environments, ...Show moreLast updated: 30+ days ago
    • Promoted
    DevOps Engineer

    DevOps Engineer

    VirtualVocationsYonkers, New York, United States
    Full-time
    A company is looking for a DevOps Engineer.Key Responsibilities Safeguard AI / ML platforms, data pipelines, and cloud infrastructure Implement and develop monitoring strategies and controls for a...Show moreLast updated: 30+ days ago
    • Promoted
    Power Platform Developer

    Power Platform Developer

    VirtualVocationsYonkers, New York, United States
    Full-time
    A company is looking for a Power Platform Developer for a Contract-to-Hire (C2H) position, a 100% Remote role.Key Responsibilities Automate business processes using model-driven Power Apps and Po...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Python Engineer

    Senior Python Engineer

    VirtualVocationsBronx, New York, United States
    Full-time
    A company is looking for a Senior Python Engineer with a strong background in backend development, data engineering, and AI infrastructure. Key Responsibilities : Design, develop, and maintain robu...Show moreLast updated: 30+ days ago
    • Promoted
    Python Developer

    Python Developer

    Genesis10Jersey City, NJ, US
    Permanent +1
    Genesis10 is currently seeking a IT Developer with our client in their Jersey City, NJ location.This is an 11-month contract position. Summary : Finance & Risk Solutions is responsible for implementi...Show moreLast updated: 4 days ago
    • Promoted
    ML Ops Engineer

    ML Ops Engineer

    VirtualVocationsNewark, New Jersey, United States
    Full-time
    A company is looking for an ML Ops Engineer.Key Responsibilities Develop machine learning algorithms, models, and data pipelines for digital and linear advertising Deploy ML models using AWS ser...Show moreLast updated: 11 days ago
    • Promoted
    • New!
    Lead Engineer, Applications - Edifecs / QConnect - Remote

    Lead Engineer, Applications - Edifecs / QConnect - Remote

    Molina HealthcareYONKERS, NY, United States
    Remote
    Full-time
    Designs and builds company specific enterprise application systems and technology expertise across multiple disciplines.Applies and promotes key principles (e. Considers business problems “end-to-en...Show moreLast updated: 11 hours ago
    • Promoted
    Platform Engineer

    Platform Engineer

    VirtualVocationsFlushing, New York, United States
    Full-time
    A company is looking for a Platform Engineer to join their remote team in Canada.Key Responsibilities Deliver infrastructure with Terraform CDKTF in partnership with development teams Maintain a...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Python Developer

    Senior Python Developer

    VirtualVocationsYonkers, New York, United States
    Full-time
    A company is looking for a Senior Python Developer specializing in Voice AI.Key Responsibilities Architect and build the conversation orchestration service, including ASR, LLM inference, and TTS ...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Data Engineer

    GCP Data Engineer

    Tata Consultancy ServicesParsippany, NJ, US
    Full-time
    Role : GCP Data Engineer Location : Parsippany, NJ Experience : - 8- 10 Years Type : Full Time Primary Skill – Python / SQL 5 to 7 years total experience, including 5+ years on GCP / Python Strong skills i...Show moreLast updated: 4 days ago
    • Promoted
    AWS DevOps Engineer

    AWS DevOps Engineer

    VirtualVocationsYonkers, New York, United States
    Full-time
    A company is looking for a Software Engineer : DevOps.Key Responsibilities Build and maintain cloud infrastructure within AWS environments Automate infrastructure management using Terraform and A...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Forward Deployed Engineer

    Senior Forward Deployed Engineer

    VirtualVocationsBronx, New York, United States
    Full-time
    A company is looking for a Senior Forward Deployed Engineer, AI (Remote).Key Responsibilities Lead the design, development, and deployment of AI / ML-powered solutions tailored to customer needs A...Show moreLast updated: 30+ days ago
    • Promoted
    Terraform and IaC Engineer

    Terraform and IaC Engineer

    VirtualVocationsJamaica, New York, United States
    Full-time
    A company is looking for a Terraform and IaC Engineer to support a migration project.Key Responsibilities Design, author, and maintain Terraform modules / stacks for various AWS constructs and serv...Show moreLast updated: 1 day ago