Talent.com
Software Engineer - AI Infrastructure
Software Engineer - AI InfrastructureAssembledHQ, Inc • New York, NY, United States
Software Engineer - AI Infrastructure

Software Engineer - AI Infrastructure

AssembledHQ, Inc • New York, NY, United States
30+ days ago
Job type
  • Full-time
Job description

About Assembled

Great customer support requires human agents and AI in perfect balance, and Assembled is the only unified platform that orchestrates both at scale. Companies like Canva, Etsy, and Robinhood use Assembled to coordinate their entire support operation - in-house agents, BPOs, and AI - in a single operating system. With AI Agents that resolve cases end-to-end, AI Copilot for agent assistance, and AI-powered workforce management that optimizes both human and AI capacity, Assembled helps teams deliver faster, better service while making smarter decisions about how to staff and automate. Backed by $70M from NEA, Emergence Capital, and Stripe, we're building the platform that makes AI and human collaboration actually work.

The Role

We're looking for a software engineer to join our Infrastructure team-building and operating the core systems that power our rapidly growing AI agent platform for customer support. Our AI Agents automates support workflows across email, chat, and voice, and has grown from $0 to $1M in ARR in just 3 months. As adoption accelerates, we're investing deeply in scaling its infrastructure to meet increasing demand and security expectations from enterprise customers.

As part of the AI Infrastructure team, you'll be responsible for the systems that enable Assist to be fast, reliable, and secure. You'll work on foundational platform components that power real-time LLM usage at scale, while also exploring how AI can be leveraged internally to make our engineering team more productive. This team is highly cross-functional, working closely with the AI, security, and product engineering teams.

This is a high-ownership role for someone who's excited by 0-to-1 building and shaping the infrastructure backbone of our AI products.

Some projects owned by the Infrastructure team

  • Agent service reliability and scaling : We manage and scale the infrastructure that serves LLM-powered agents across chat, email, and voice. This includes selecting inference strategies, integrating with model providers (e.g. OpenAI, Anthropic), and dynamically routing traffic for performance and cost efficiency.
  • Prompt and embedding storage systems : Assist relies heavily on dynamically generated prompts and semantic search across support content. The team owns highly-available, fast-access storage and indexing layers optimized for real-time AI interactions.
  • Privacy and security : Enterprises expect strict guardrails around AI use. We're building systems like network-level intrusion detection (IDS / IPS), audit logging, and LLM usage policy enforcement to meet these expectations and unlock new sales channels.
  • Observability and usage analytics : We operate systems that surface key metrics-token usage, latency, cost per response, and quality signals-so the Assist team can continuously improve Assist's performance and accuracy.
  • AI-powered developer tools : We are beginning to explore and evangelize the use of AI to accelerate internal engineering workflows-through internal chat agents, pair programming tools, and intelligent automation for deployment, debugging, and on-call. Our goal is to empower engineers across the company to build faster and more confidently with AI.

You may be a good fit if you :

  • Have 6+ years of engineering experience, with past ownership of high-scale, production-critical infrastructure
  • Have experience with distributed systems and container orchestration (especially Kubernetes)
  • Have worked with AI / ML platforms or are excited to build foundational infrastructure for LLM-based applications
  • Thrive in fast-paced environments with shifting requirements and ambiguous problem spaces
  • Are motivated by impact, enjoy deep technical challenges, and want to work cross-functionally across security, AI, and product
  • Have strong familiarity with one or more parts of our tech stack :
  • Cloud provider : AWS

  • Orchestration : Kubernetes + Karpenter
  • LLM integration : Experience with OpenAI, Anthropic, or open-source model serving (e.g., vLLM, HuggingFace TGI, Ray Serve)
  • Prompt & embedding infrastructure : Vector databases (e.g., Pinecone, Weaviate, PGVector), semantic search, prompt templating systems
  • Datastores : Postgres + PgBouncer, Snowflake, Redis
  • Languages : Go and Python
  • Monitoring & CI / CD : Datadog, Mezmo, CloudWatch, Buildkite, CircleCI
  • Our U.S. benefits

  • Generous medical, dental, and vision benefits
  • Paid company holidays, sick time, and unlimited time off
  • Monthly credits to spend on each : professional development, general wellness, Assembled customers, and commuting
  • Paid parental leave
  • Hybrid work model with catered lunches everyday (M-F), snacks, and beverages in our SF & NY offices
  • 401(k) plan enrollment
  • Create a job alert for this search

    Software Engineer Infrastructure • New York, NY, United States

    Related jobs
    AI Engineer - NYC

    AI Engineer - NYC

    Aegis Ai • New York, New York, United States
    Full-time
    We’re a team of ex-Google engineers who built some of the largest defensive platforms on the planet —.Now, we’re striking out on our own to tackle an even bigger challenge : stopping the new wave of...Show more
    Last updated: 18 days ago • Promoted
    AI Infrastructure Engineer, Model Serving Platform

    AI Infrastructure Engineer, Model Serving Platform

    Scale AI, Inc. • New York, NY, United States
    Full-time
    As a Software Engineer on the ML Infrastructure team, you will design and build platforms for scalable, reliable, and efficient serving of LLMs. Our platform powers cutting-edge research and product...Show more
    Last updated: 30+ days ago • Promoted
    AI Platform Engineer Lead

    AI Platform Engineer Lead

    Capital Group • New York, NY, United States
    Full-time
    I can succeed as a Lead AI Platform Engineer at Capital Group.As a Lead AI Platform Engineer, you will serve as a technical SME and lead workstreams to design, build, and operate the foundational c...Show more
    Last updated: 24 days ago • Promoted
    AI Engineer - AI Platform

    AI Engineer - AI Platform

    Traversal • New York, NY, United States
    Full-time
    AI Site Reliability Engineer (SRE) for the enterprise—already trusted by some of the largest companies in the world to troubleshoot, remediate, and even prevent the most complex production incident...Show more
    Last updated: 30+ days ago • Promoted
    AI / ML Engineer

    AI / ML Engineer

    Profound • New York, New York, United States
    Full-time
    Profound is an NYC-based AI startup helping brands measure and improve their visibility in AI platforms such as ChatGPT.We partner with some of the biggest brands and marketing agencies, including ...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer

    AI Engineer

    Quantifi • New York, New York, United States
    Full-time
    Quantifi is a leading provider of analytics, trading, and risk management solutions for the global financial industry.For over two decades, we have partnered with banks, asset managers, hedge funds...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer

    AI Engineer

    Rilla • New York, New York, United States
    Full-time
    Rilla builds the world’s leading conversation intelligence software for offline commerce.By recording, analyzing, and indexing conversations, our platform supplies unparalleled alpha to the critica...Show more
    Last updated: 30+ days ago • Promoted
    Senior AI / ML Software Engineer (Remote in North Carolina)

    Senior AI / ML Software Engineer (Remote in North Carolina)

    Rocket Lawyer • New York, New York, United States
    Remote
    Full-time
    We believe everyone deserves access to affordable and simple legal services.Founded in 2008, Rocket Lawyer is the largest and most widely used online legal service platform in the world.With office...Show more
    Last updated: 28 days ago • Promoted
    Senior Software Engineer, AI Agent

    Senior Software Engineer, AI Agent

    Astronomer • New York, New York, United States
    Full-time
    Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow...Show more
    Last updated: 7 days ago • Promoted
    AI / ML Engineer (Software Engineering) - Vice President

    AI / ML Engineer (Software Engineering) - Vice President

    Icapital • New York, New York, United States
    Remote
    Full-time
    Capital is powering the world’s alternative investment marketplace.Our financial technology platform has transformed how advisors, wealth management firms, asset managers, and banks evaluate and re...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer (New York)

    AI Engineer (New York)

    Finster Ai • New York, NY, United States
    Full-time
    Compensation : $120,000-$200,000 + Equity .We’re a Series A stage firm, redefining the future of finance with our AI-native research and task automation platform, backed by leading, global venture i...Show more
    Last updated: 18 days ago • Promoted
    Sr. AI / ML Engineer

    Sr. AI / ML Engineer

    Constrafor • New York, New York, United States
    Full-time
    Constrafor is a SaaS and fintech platform purpose-built for construction.We are setting new standards of productivity and cost-efficiency for the way General Contractors and Subcontractors manage p...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer

    AI Engineer

    Zone It Solutions • New York, New York, United States
    Full-time
    We are on the lookout for an innovative and driven.In this role, you will be responsible for designing, developing, and deploying AI models that will enhance our products and improve our services.B...Show more
    Last updated: 30+ days ago • Promoted
    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    OneStream Software • New York, NY, United States
    Full-time
    Senior AI Engineer - Cloud Engineering Focus.Hybrid, USA (Birmingham, MI, Chicago, IL, New York, NY, Fort Lauderdale, FL). Range applies to US candidates only) + Benefits / Variable Comp / Equity - Rang...Show more
    Last updated: 2 days ago • Promoted
    AI Engineer

    AI Engineer

    Metropolitan Commercial Bank • New York, New York, United States
    Full-time
    Metropolitan Commercial Bank (the “Bank”) is a full-service commercial bank based in New York City.The Bank provides a broad range of business, commercial, and personal banking products and service...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer

    AI Engineer

    Oscar • New York, NY, US
    Full-time
    AI Engineer - Healthcare Automation Platform.Healthcare AI | Hybrid (NYC preferred) or Remote.AI-powered automation platform. Our system processes complex, unstructured data to ensure time-sensitive...Show more
    Last updated: 23 days ago • Promoted
    AI Engineer

    AI Engineer

    Pathwork • New York, New York, United States
    Full-time
    Location : San Francisco or New York City.Pathwork is redesigning life & health insurance jobs for the AI age.M professionals in the $1T life and health insurance industry don’t need better tools; t...Show more
    Last updated: 17 days ago • Promoted
    Software Engineer, AI Infrastructure

    Software Engineer, AI Infrastructure

    Fireworks Ai • New York, New York, United States
    Full-time
    At Fireworks, we’re building the future of generative AI infrastructure.Our platform delivers the highest-quality models with the fastest and most scalable inference in the industry.We’ve been inde...Show more
    Last updated: 28 days ago • Promoted