Nils Holzenberger
nils dot holzenberger at telecom-paris dot fr
Assistant Professor at Télécom Paris in the DIG team
nils dot holzenberger at telecom-paris dot fr
Assistant Professor at Télécom Paris in the DIG team
If you have access to the IP Paris Moodle, do the lab session there. There is a form to fill in with your answers.
Otherwise, click here for the slides and materials for the lab session. Put your answers to the questions (to be found in QUESTIONS.txt) in a pdf file, with your name on it, and email it to me.
The deadline for this assignment is Friday, Sep 19, 2pm.
I am generally interested in highly motivated and self-driven M1 or M2 students for internships. If your research interests align with mine, feel free to get in touch.
I was fortunate enough to attract the attention of the radio channel France Culture. You can listen to the story here (in French), which aired during the show La Science, CQFD on September 11, 2025.
OpenAI used a sample from the dataset SARA in its demo of GPT-4 on March 15, 2023. You can see the demo here.
The NLP Highlights podcast dedicated an entire episode to the SARA dataset.
My main research aim is to develop algorithms that can spot inconsistencies and loopholes in tax law. Below is a list of research topics, with selected publications. For an exhaustive list of publications, please refer to my DBLP and Google Scholar profiles. Some resources can be found here.
Andrew Blair-Stanek, Benjamin Van Durme and I are working towards finding loopholes in tax law.
Can LLMs Identify Tax Abuse? [paper]
Andrew Blair-Stanek, Nils Holzenberger and Benjamin Van Durme
arXiv preprint, Aug 10, 2025
Shelter Check: Proactively Finding Tax Minimization Strategies via AI [paper]
Andrew Blair-Stanek, Nils Holzenberger and Benjamin Van Durme
Tax Notes Federal, Dec. 12, 2022
NLLP has made huge strides, especially since the advent of Large Language Models. I've built benchmarks for NLLP and tested LLMs on legal reasoning.
CLERC: A Dataset for U. S. Legal Case Retrieval and Retrieval-Augmented Analysis Generation [paper]
Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek and Benjamin Van Durme
NAACL (Findings) 2025
The Factuality of Large Language Models in the Legal Domain [paper]
Rajaa El Hamdani, Thomas Bonald, Fragkiskos D. Malliaros, Nils Holzenberger and Fabian M. Suchanek
CIKM 2024
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [paper]
Neel Guha et al
NeurIPS 2023 Datasets and Benchmarks
Connecting Symbolic Statutory Reasoning with Legal Information Extraction [paper] [resources]
Nils Holzenberger and Benjamin Van Durme
Proceedings of the Natural Legal Language Processing Workshop, December 7, 2023, Singapore
OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax? [paper]
Andrew Blair-Stanek, Nils Holzenberger and Benjamin Van Durme
Tax Notes Federal, August 14, 2023
As an alternative to direct analysis of legal authorities via NLP, loopholes can be found by building a simulation of the legal system and letting RL-driven agents find creative combinations of laws.
Can AI expose tax loopholes? Towards a new generation of legal policy assistants [paper]
Peter Fratrič, Nils Holzenberger, David Restrepo Amariles
arXiv March 21, 2025
Rules2Lab: from Prolog Knowledge-Base, to Learning Agents, to Norm Engineering [paper]
Peter Fratric, Nils Holzenberger and David Restrepo Amariles
EUMAS 2024
Assistant Professor at Télécom Paris, Palaiseau, France, February 2023 - Present
PhD Candidate at Johns Hopkins University, Baltimore, MD, USA, September 2017 - November 2022
Montreal Institute for Learning Algorithms with Yoshua Bengio, Montréal, Canada, April - July 2017
Master's at Mines ParisTech (École Nationale Supérieure des Mines de Paris), Paris, France, September 2013 - July 2017
CoML with Emmanuel Dupoux, École Normale Supérieure (ENS Ulm), Paris, France, May - September 2016
Carnegie Mellon University with Florian Metze, Pittsburgh, PA, USA, December 2015 - May 2016
IBM Watson, Littleton, MA, USA, June - November 2015
ENDLab with Tom Peacock, MIT, Cambridge, MA, USA, September 2014 - February 2015
TE-VSC group, CERN Technology Department, Meyrin, Switzerland, February 2014
Classe préparatoire MPSI/MP, Lycée Louis-le-Grand, Paris, France, September 2011 - July 2013