Evaluating Generative AI Systems: the Good, the Bad, and the Hype

Monday, 15 April 2024

Georgetown Law, 9th floor
500 First St NW, Washington, D.C. 20001

& LIVESTREAMED

This initiative is generously supported by the K&L Gates Endowment for Ethics and Computational Technologies at Carnegie Mellon University

Due to limited, in-person capacity, join the:

We’re bringing together a fantastic program of machine-learning (ML) researchers and engineers, policymakers, lawyers, and civil society for a day of invited talks and panels on the nuts and bolts of generative AI evaluations — where the technology is, what policymakers and civil society say they want from evaluations, and concrete gaps between the two (focused on both misconceptions, and opportunities).

Topics:

1) training-data attribution

2) privacy

3) data provenance & watermarks

4) trust & safety

The goal? To go deep on our curated set of topics.

We’re discussing 1) training-data attribution, 2) privacy, 3) data provenance & watermarks, and 4) trust & safety — all in response to topics raised in President Biden’s Executive Order.

Each topic will be addressed in pairs of talks: putting an ML expert in conversation with an expert from law or civil society.

The day will also include updates from the U.S. Copyright Office, FTC, U.S. AI Safety Institute (& NIST), and U.K. AI Safety Institute.

Please note that this event will be recorded (both video & photography).

  • Zack Lipton

    Associate Professor, Machine Learning Carnegie Mellon University CTO and Chief Scientist Abridge

    [website]

  • Sejal Amin

    Sejal Amin

    Chief Technology Officer at Shutterstock

  • David Bau

    David Bau

    Assistant Professor of Computer Science at Northeastern Khoury College

    [website]

  • Emily Lanza

    Emily Lanza

    Senior Counsel at U.S. Copyright Office

  • Alicia Solow-Niederman

    Alicia Solow-Niederman

    Associate Professor at the George Washington University Law School

    [website]

  • Niloofar Mireshghallah

    Niloofar Mireshghallah

    Postdoctroal Scholar at University of Washington

    [website]

  • Kat Walsh

    Kat Walsh

    General Counsel at Creative Commons

    [website]

  • Andreas Terzis

    Andreas Terzis

    Team Lead, Google DeepMind Privacy and Security

    [website]

  • David Willner

    David Willner

    Non-Resident Fellow, Stanford Program on Governance of Emerging Technologies and Former Head of Trust & Safety at OpenAI

  • Nicholas Carlini

    Nicholas Carlini

    Team Lead, Google DeepMind Attacking Generative AI Models

    [website]

  • Erie Meyer

    Erie Meyer

    Chief Technologist of the Consumer Financial Protection Bureau

  • Elham Tabassi

    Elham Tabassi

    Chief Technology Officer at US AI Safety Institute and Chief AI Advisor, NIST Associate Director for Emerging Technologies

    [website]

  • Nitarshan Rajkumar

    Nitarshan Rajkumar

    Adviser to the UK Secretary of State. Co-creator of UK AI Safety Institute

  • Amba Kak

    Amba Kak

    Co-executive Director at AI Now

    [website]

This event is sponsored by Carnegie Mellon’s K&L Gates Initiative and run by The GenLaw Center, Carnegie Mellon’s K&L Gates Initiative, the Georgetown Institute for Technology Law & Policy, and the Center for Democracy and Technology.

As an example of prior GenLaw events, please see our most recent report and workshop. Also see our generative AI and law glossary and explainer series.