102 evaluation engineer jobs found in New York

Machine Learning Engineer - Model Evaluations, Public Sector The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, and multimodal pipelines—into mission-critical government environments. We build evaluation frameworks that ensure these models operate reliably, safely, and effectively under real-world constraints. As an ML Engineer, you will design, implement, and scale automated evaluation pipelines that help customers trust and operationalize advanced AI systems across defense, intelligence, and federal missions. You will: Develop and maintain automated evaluation pipelines for ML models across functional, performance, robustness, and safety metrics, including LLM-judge–based evaluations. Design test datasets and benchmarks to measure generalization, bias, explainability, and failure modes. Build evaluation frameworks for LLM agents, including infrastructure for scenario-based and environment-based testing. Conduct...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for Accenture. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. We Are We are entering into a new decade of Data & AI that will reshape work and society. Accenture is stepping boldly into this future with a clear strategy and purpose: to help clients optimize and reinvent their business with data & AI — backed by a $3B investment and commitment to our people to do industry-defining work. With over 77,000 professionals dedicated to Data & AI, Accenture’s Data & AI organization is powered by experienced innovation, strategic...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for Stantec. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. We create great places and facilities that support our nation's economic security - including support to our US Federal and Department of Defense assets nationwide. Working within the context of the communities we serve, we provide planning, design, and construction engineering and management services that fit the needs of these unique clients. Join our team and help us keep our nation and economy strong. Your Opportunity Stantec has an opportunity for a highly motivated and...

The Frontier Data team builds the data and systems that power Scale’s most advanced Frontier AI use cases (agentic capabilities like coding agents, tool use, and GUI / computer-use automation). Our work sits at the intersection of applied AI and robust backend engineering: we turn messy, ambiguous problems into scalable platforms and pipelines that reliably produce high-quality outcomes. We’re looking for a Senior Backend Engineer who thrives in ambiguity, moves fast, and enjoys tackling daunting challenges - someone who can design and build scalable systems while partnering closely with research, product, operations, and other engineering teams. What You’ll Do Own major backend systems for frontier agentic data products, driving projects from early exploration through production deployment. Build scalable services and pipelines that support agent workflows (e.g., coding, tool-use orchestration, GUI/computer-use tasks), with strong reliability and observability....

At Scale, our Security Architecture team builds the foundations that allow engineers to ship fast without compromising security. From securing modern TypeScript services and cloud infrastructure to enabling safe adoption of AI-driven systems, our work shapes how products are designed, deployed, and operated across the company. We are looking for a Staff Security Engineer to help define and build the “paved road” for secure development at Scale. As a Staff Security Engineer, you will operate as a builder first — roughly 60% software engineering and 40% security. You’ll partner deeply with product, platform, and infrastructure teams to design secure architectures, build shared primitives, and influence how engineering teams work end-to-end. This role requires strong production software engineering DNA, architectural judgment, and the ability to lead through influence in a fast-moving, high-impact environment. You will: Design and build secure application and...

We are seeking a highly technical Security Engineer to join our Product Security team. This role is integral to ensuring the security and integrity of our products and services. You will conduct in-depth code reviews, implement security best practices, and influence the overall security strategy. Your expertise in TypeScript, Python, Kubernetes, CI/CD, SAST, DAST, and terraform orchestration will be crucial in identifying and mitigating potential security vulnerabilities. You will also structure complex problems, diagnose root causes independently, and clearly explain the mechanics and significance of security vulnerabilities, including their exploitability and potential impact. You will: Conduct in-depth code reviews to identify and remediate security vulnerabilities. Evaluate and enhance the security of our product offerings, through RFC and service review. Implement and maintain CI/CD pipelines with a strong focus on security. Perform Static Application Security...

Staff Software Engineer, Full-Stack - Enterprise Gen AI Scale GP (Scale Generative AI Platform) is an enterprise-grade AI platform providing APIs for knowledge retrieval, inference, evaluation, and more. We are looking for a frontend-focused full-stack engineer to help build AI-powered applications that redefine enterprise workflows and push the boundaries of interactive AI. This role is ideal for someone who thrives in a fast-paced environment, enjoys working on a diverse set of projects, and has a passion for crafting high-quality, intuitive user experiences. At Scale, you'll work on a mix of cutting-edge customer-facing AI applications and internal SaaS products. Our engineering team powers projects like TIME’s Person of the Year AI experience (see it in action), where our AI technology helped shape one of the most iconic features in media. You'll also contribute to Scale’s GenAI Platform (SGP), a powerful system that enables businesses to build and deploy AI agents...

Company Description Spark Foundry is a global media agency that exists to bring HEAT - Higher Engagement, Affinity, and Transactions - to brands. By combining flawless media fundamentals with aggressive innovation, Spark inspires consumers to pay more attention, to care more about our clients’ brands, and to buy more products and services from them. Balancing the nimble spirit of a startup with the powerhouse soul of Publicis Media, Spark Foundry delivers the best of both worlds to a client roster that spans some of the world’s best and most beloved brands and companies. We combine boutique-caliber insights and service with the buying clout and first-look access of a global leader, bringing the heat to challenger brands that want to act like giants, and to giant brands that want to act like challengers. With a bottom-up culture that celebrates diversity and aims for all voices to be heard, Spark has become a magnet for the industry’s best talent, with one of the best retention...

Machine Learning Engineer - Model Evaluations, Public Sector The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, and multimodal pipelines—into mission-critical government environments. We build evaluation frameworks that ensure these models operate reliably, safely, and effectively under real-world constraints. As an ML Engineer, you will design, implement, and scale automated evaluation pipelines that help customers trust and operationalize advanced AI systems across defense, intelligence, and federal missions. You will: Develop and maintain automated evaluation pipelines for ML models across functional, performance, robustness, and safety metrics, including LLM-judge–based evaluations. Design test datasets and benchmarks to measure generalization, bias, explainability, and failure modes. Build evaluation frameworks for LLM agents, including infrastructure for scenario-based and environment-based testing. Conduct...

The Frontier Data team builds the data and systems that power Scale’s most advanced Frontier AI use cases (agentic capabilities like coding agents, tool use, and GUI / computer-use automation). Our work sits at the intersection of applied AI and robust backend engineering: we turn messy, ambiguous problems into scalable platforms and pipelines that reliably produce high-quality outcomes. We’re looking for a Senior Backend Engineer who thrives in ambiguity, moves fast, and enjoys tackling daunting challenges - someone who can design and build scalable systems while partnering closely with research, product, operations, and other engineering teams. What You’ll Do Own major backend systems for frontier agentic data products, driving projects from early exploration through production deployment. Build scalable services and pipelines that support agent workflows (e.g., coding, tool-use orchestration, GUI/computer-use tasks), with strong reliability and observability....

Company Description Spark Foundry is a global media agency that exists to bring HEAT - Higher Engagement, Affinity, and Transactions - to brands. By combining flawless media fundamentals with aggressive innovation, Spark inspires consumers to pay more attention, to care more about our clients’ brands, and to buy more products and services from them. Balancing the nimble spirit of a startup with the powerhouse soul of Publicis Media, Spark Foundry delivers the best of both worlds to a client roster that spans some of the world’s best and most beloved brands and companies. We combine boutique-caliber insights and service with the buying clout and first-look access of a global leader, bringing the heat to challenger brands that want to act like giants, and to giant brands that want to act like challengers. With a bottom-up culture that celebrates diversity and aims for all voices to be heard, Spark has become a magnet for the industry’s best talent, with one of the best retention...

Scale GP (Scale Generative AI Platform) is an enterprise-grade Generative AI platform providing APIs for knowledge retrieval, inference, evaluation, and more. We are seeking a strong Senior Full-Stack Engineer to help us build, scale, and refine our rapidly growing product. The ideal candidate is deeply grounded in software engineering best practices and experienced in developing and scaling modern web applications end-to-end. You will work across the stack—from React/TypeScript frontends to Python-based backends—while integrating with LLMs and machine learning systems. You will solve complex challenges in scalability, reliability, and product experience while owning significant product areas in a fast-paced environment. What You’ll Do Own major full-stack product areas , driving features from design through production deployment. Build modern frontend experiences using React and TypeScript, ensuring performance, usability, and responsiveness. Develop...

Scale AI is seeking a highly skilled and motivated Software Engineer, AI Infrastructure & Security to join our dynamic Public Sector Engineering team. As a part of this team, you will play a critical role in delivering high-impact AI-powered mission solutions for government customers. Our scalable and high-performance platform forms the foundation for these solutions, and your expertise will be instrumental in designing and implementing systems that can handle billions of data points with exceptional performance. You will: Design and implement secure scalable backend systems for Public Sector customers, leveraging Scale's modern and cloud-native AI infrastructure. Own services or systems and define their long-term health goals, while also improving the health of surrounding components Improve our high engineering standards, tooling, and process Collaborate with cross-functional teams to define and execute the vision for backend solutions, ensuring they meet the...

Do you want to help engineer what’s next? Simpson Gumpertz & Heger (SGH) is a national engineering firm committed to delivering holistic advice for our clients’ most complex challenges. We leverage our collective and diverse experience, technical expertise, and industry knowledge of structures and building enclosures, advanced analysis, performance & code consulting, and applied science & research to deliver unrivaled, comprehensive solutions that drive superior performance. With 750 employees in ten office locations throughout the United States, SGH’s industry-leading teams constantly seek to advance the meaning of what’s possible. What makes careers at SGH so special? The only way to advance is to question and explore. Every member of the SGH team is both a learner and an educator, committed to advancing ourselves, our teams, and our industry. Together we are creating a community that never settles for what is but always seeks what could be. There Are Many...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for WSP. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. This Opportunity WSP is searching for an Associate Water/Wastewater Engineer for our Shelton, CT office. This is an exciting opportunity for a dedicated engineer to be involved in projects with our Northeast Water Engineering Team and be a part of a growing organization that meets our clients' objectives and solves their challenges. Provides support to technical staff and project managers for analysis, design, bidding, and construction phase services for water and wastewater...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for MTA, Inc.. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. Assistant Director Project Management (Stations) Job ID: 14000 Business Unit: MTA Construction & Development Location: New York, NY, United States Regular/Temporary: Regular Department: Stations Date Posted: Dec 19, 2025 Description This position is eligible for telework , which is currently one day per week. New hires are eligible to apply 30 days after their effective date of hire. Job Title Assistant Director, Project Management Agency Construction &...

Description Who We Are Enovate is an Engineering firm that specializes in Construction Management, Transportation Engineering and Monitoring services. Since its inception a few years ago, Enovate was able to position its brand as a leading WBE/DBE engineering/CM firm. We have teamed with and supported well-known Engineering and CM firms and worked on flagship projects in both the public and private sectors. Our diverse team consists of Professional Engineers, Construction Managers, Engineers and Technicians from varied backgrounds – design, project management and construction. This is an opportunity to work with Enovate as we expand to meet the demand of this wide range of services and excel in a traditional industry that is ever changing with the adoption of technology. We have been honored with a certification from Great Places to Work, listed on the FORTUNE Best Places to Work in NY and Best Small Workplaces lists, and received an ACEC NY Pioneer of Belonging Award for...

The Company PayPal has been revolutionizing commerce globally for more than 25 years. Creating innovative experiences that make moving money, selling, and shopping simple, personalized, and secure, PayPal empowers consumers and businesses in approximately 200 markets to join and thrive in the global economy. We operate a global, two-sided network at scale that connects hundreds of millions of merchants and consumers. We help merchants and consumers connect, transact, and complete payments, whether they are online or in person. PayPal is more than a connection to third-party payment networks. We provide proprietary payment solutions accepted by merchants that enable the completion of payments on our platform on behalf of our customers. We offer our customers the flexibility to use their accounts to purchase and receive payments for goods and services, as well as the ability to transfer and withdraw funds. We enable consumers to exchange funds more safely with merchants using a...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for Insight Global. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. Job Description Insight Global is looking or an AI Solutions and Analytics Director to join one of their AI Consulting clients. This person will provide technology direction, ensure project implementation, and utilize technology research to innovate, integrate, and manage technology solutions for a brand new org that is being built out. You will also significantly contribute to identifying best-fit architectural solutions for one or more projects. As an AI Solutions and...

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for Insight Global. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice. Job Description Insight Global is looking or an AI Solutions and Analytics Director to join one of their AI Consulting clients. This person will provide technology direction, ensure project implementation, and utilize technology research to innovate, integrate, and manage technology solutions for a brand new org that is being built out. You will also significantly contribute to identifying best-fit architectural solutions for one or more projects. As an AI Solutions and...

Data is at the center of everything we do. As a startup, we disrupted the credit card industry by individually personalizing every credit card offer using statistical modeling and the relational database, cutting edge technology in 1988! Fast-forward a few years, and this little innovation and our passion for data has skyrocketed us to a Fortune 200 company and a leader in the world of data-driven decision-making. As a Data Scientist at Capital One, you’ll be part of a team that’s leading the next wave of disruption at a whole new scale, using the latest in computing and machine learning technologies and operating across billions of customer records to unlock the big opportunities that help everyday people save money, time and agony in their financial lives. Team Description Join an elite Applied AI team within AI Foundations, operating at the intersection of deep research and massive real-world impact. We are pioneering the next generation of personalized customer...

Production Engineering is a hybrid software/systems group that ensures Meta's services and products run smoothly and have the capacity for future growth. Production Engineers work with Meta's product and infrastructure teams, sometimes embedded in those teams, collaborating in building and scaling technology solutions. Managing a Production Engineering team requires a comprehensive understanding of a wide range of technologies, a focus on growing and developing the skills and talents of your team, and a relentless drive toward high-value projects and ruthless prioritization. Manager, Production Engineering Responsibilities: Support and lead engineers working on Meta's products and services, at different layers of the stack, on challenges related to scalability, reliability, performance and efficiency of systems Understand and contribute to technical architectures, capacity plans, tooling needs, automation plans, product launch plans and create comprehensive plans for...

Manager, Product Management, Card Data Manager, Product Management (PXDP50) Product Management at Capital One is a booming, vibrant craft that requires reimagining the status quo, finding value creation opportunities, and driving innovative and sustainable customer experiences through technology. We believe our portfolio of businesses and investments in growth and transformation will result in a company with the scale, brand, capabilities, talent, and values to succeed as the digital revolution transforms our society and our industry. About the Team The US Card Data team is responsible for sourcing, rationalizing, standardizing, and publishing well governed Credit Card data for analytical use cases within Capital One. Every credit decision, product strategy, policy and valuations use this data to acquire and manage customer accounts while supporting monitoring, governance, and forecasting. The team manages 500+ data flows, 30+ data tables to serve thousands of...

The Guardian is a global, reader-funded news organization that delivers fearless, independent journalism. From breaking news and award-winning investigations, to in-depth coverage of technology, sports, film, culture and lifestyle, the Guardian offers a global view that deepens our audiences' understanding of America and the world. The Guardian's US edition – headquartered in New York City, with growing bureaus in Washington DC and Los Angeles – is an entirely digital operation that combines the best of the Guardian's international reporting with US voices and expertise. Core coverage areas include the climate emergency, economic and racial inequality, wellness, culture, digital privacy and sports – all highlighting the Guardian's distinctive role within the US media landscape: journalism that's global, independent, and free. It's the talent, energy and commitment our people bring to the...