Senior Staff Engineer - Kafka USA, Miami

Nu is the world’s largest digital banking platform outside of Asia, serving over 105 million customers across Brazil, Mexico, and Colombia. The company has been leading an industry transformation by leveraging data and proprietary technology to develop innovative products and services. Guided by its mission to fight complexity and empower people, Nu caters to customers’ complete financial journey, promoting financial access and advancement with responsible lending and transparency. The company is powered by an efficient and scalable business model that combines low cost to serve with growing returns. Nu’s impact has been recognized in multiple awards, including Time 100 Companies, Fast Company’s Most Innovative Companies, and Forbes World’s Best Banks. Learn more :

About the Role

We are opening this position in the US to find an engineer with a highly specialized skill set that is critical to our current needs. The messaging team is responsible for maintaining and operating a high-throughput messaging platform built on Apache Kafka. This system processes trillions of messages every month and underpins mission-critical, low-latency systems used across clusters. This role is critical for the future success of the product.

As Nubank rapidly expands to support billions of customers across multiple countries, over 100 financial products, and 100+ business platforms, running on infrastructure spanning over multiple AWS accounts, we face significant scalability and operational challenges. Our current infrastructure includes approximately 200 Kafka clusters, 1500 brokers, more than 200k topics, and processes 300 TB of data daily. This role will directly address challenges such as :

• Operating and managing over 200 Kafka clusters efficiently.
• Supporting and monitoring over 1 million Kafka topics.
• Handling over 1PB of data flowing through Kafka daily, ensuring minimal latency and high reliability.
• Optimizing resource utilization and controlling costs in a smart, sustainable way.
• Extending asynchronous communication patterns beyond Kafka to meet future needs.
• Maintaining performance, reliability, and containing blast radius as message volume grows exponentially.
• In this role, you will be responsible for tackling the key challenges associated with our rapidly expanding, high-throughput Apache Kafka messaging platform. Your main activities and goals will include :
• Operating and managing a large-scale Kafka infrastructure, including hundreds of clusters and thousands of brokers.
• Ensuring high reliability and minimal latency for systems processing petabytes of data daily.
• Developing strategies to efficiently support and monitor over a million Kafka topics.
• Optimizing resource utilization and implementing cost-effective solutions for our Kafka ecosystem across multiple AWS accounts.
• Driving the evolution of asynchronous communication patterns, potentially extending beyond Kafka to meet future needs.
• Implementing solutions to maintain performance and reliability and contain potential issues as message volume grows exponentially.

We are looking for a person who has

Role Location

Apply for this job

indicates a required field

First Name

Last Name

Preferred First Name

Email

Phone

Resume / CV

Enter manually

Accepted file types : pdf, doc, docx, txt, rtf

Enter manually

Accepted file types : pdf, doc, docx, txt, rtf

Profile

Website

Location

Do you have strong experience in maintaining and operating high-throughput messaging platforms utilizing Apache Kafka, particularly in environments that are both large-scale and mission-critical?

Have you managed Kafka infrastructures that span multiple AWS accounts and involved hundreds of clusters and thousands of brokers?

Are you experienced in monitoring and supporting over a million Kafka topics while ensuring high reliability and minimal latency in data processing?

Have you developed and implemented strategies to optimize resource utilization and control costs within a Kafka ecosystem, particularly across a complex, multi-account AWS infrastructure?

Are you capable of handling data flows exceeding 1 petabyte daily within a Kafka platform, ensuring efficient performance and low latency?

Do you have experience in evolving asynchronous communication patterns and extending beyond Kafka to meet future technological and operational needs?

Do you have a proven track record of implementing solutions that maintain system performance, reliability, and resiliency, particularly when dealing with exponential growth in message volumes?

Are you currently responsible for leading strategic initiatives or making high-level architectural decisions in your role, specifically involving cross-functional collaboration or influencing the technical direction of the organization?

#J-18808-Ljbffr