LawFlow : Collecting and Simulating Lawyers' Thought Processes
Legal practitioners, particularly those early in their careers, face complex,
high-stakes tasks that require adaptive, context-sensitive reasoning. While AI
holds promise in supporting legal work, current datasets and models are
narrowly focused on isolated subtasks and fail to capture the end-to-end
decision-making required in real-world practice. To address this gap, we
introduce LawFlow, a dataset of complete end-to-end legal workflows collected
from trained law students, grounded in real-world business entity formation
scenarios. Unlike prior datasets focused on input-output pairs or linear chains
of thought, LawFlow captures dynamic, modular, and iterative reasoning
processes that reflect the ambiguity, revision, and client-adaptive strategies
of legal practice. Using LawFlow, we compare human and LLM-generated workflows,
revealing systematic differences in structure, reasoning flexibility, and plan
execution. Human workflows tend to be modular and adaptive, while LLM workflows
are more sequential, exhaustive, and less sensitive to downstream implications.
Our findings also suggest that legal professionals prefer AI to carry out
supportive roles, such as brainstorming, identifying blind spots, and surfacing
alternatives, rather than executing complex workflows end-to-end. Building on
these findings, we propose a set of design suggestions, rooted in empirical
observations, that align AI assistance with human goals of clarity,
completeness, creativity, and efficiency, through hybrid planning, adaptive
execution, and decision-point support. Our results highlight both the current
limitations of LLMs in supporting complex legal workflows and opportunities for
developing more collaborative, reasoning-aware legal AI systems. All data and
code are available on our project page
(https://minnesotanlp.github.io/LawFlow-website/).
681140d7f8ca0d9acb1690a9
Debarati Das
claimed_verified
2025-05-02T06:37:41.458Z
60f8a014595289005b768233
/avatars/73b1ed0aa6728fa04607a99fb2eb7317.svg
Debarati Das
user
das00015
681140d7f8ca0d9acb1690aa
Khanh Chi Le
681140d7f8ca0d9acb1690ab
Ritik Sachin Parkar
681140d7f8ca0d9acb1690ac
Karin De Langis
681140d7f8ca0d9acb1690ad
Brendan Madson
681140d7f8ca0d9acb1690ae
Chad M. Berryman
681140d7f8ca0d9acb1690af
Robin M. Willis
681140d7f8ca0d9acb1690b0
Daniel H. Moses
681140d7f8ca0d9acb1690b1
Brett McDonnell
681140d7f8ca0d9acb1690b2
Daniel Schwarcz
681140d7f8ca0d9acb1690b3
Dongyeop Kang
LawFlow : Collecting and Simulating Lawyers' Thought Processes
Legal practitioners, particularly those early in their careers, face complex,
high-stakes tasks that require adaptive, context-sensitive reasoning. While AI
holds promise in supporting legal work, current datasets and models are
narrowly focused on isolated subtasks and fail to capture the end-to-end
decision-making required in real-world practice. To address this gap, we
introduce LawFlow, a dataset of complete end-to-end legal workflows collected
from trained law students, grounded in real-world business entity formation
scenarios. Unlike prior datasets focused on input-output pairs or linear chains
of thought, LawFlow captures dynamic, modular, and iterative reasoning
processes that reflect the ambiguity, revision, and client-adaptive strategies
of legal practice. Using LawFlow, we compare human and LLM-generated workflows,
revealing systematic differences in structure, reasoning flexibility, and plan
execution. Human workflows tend to be modular and adaptive, while LLM workflows
are more sequential, exhaustive, and less sensitive to downstream implications.
Our findings also suggest that legal professionals prefer AI to carry out
supportive roles, such as brainstorming, identifying blind spots, and surfacing
alternatives, rather than executing complex workflows end-to-end. Building on
these findings, we propose a set of design suggestions, rooted in empirical
observations, that align AI assistance with human goals of clarity,
completeness, creativity, and efficiency, through hybrid planning, adaptive
execution, and decision-point support. Our results highlight both the current
limitations of LLMs in supporting complex legal workflows and opportunities for
developing more collaborative, reasoning-aware legal AI systems. All data and
code are available on our project page
(https://minnesotanlp.github.io/LawFlow-website/).
LawFlow, a dataset of complete end-to-end legal workflows, reveals differences between human and LLM reasoning in legal tasks, suggesting AI should support rather than execute complex workflows.
narrowly focused datasets LawFlow end-to-end legal workflows dynamic reasoning processes modular reasoning iterative reasoning ambiguity revision client-adaptive strategies LLM-generated workflows sequential reasoning exhaustive reasoning decision-point support hybrid planning adaptive execution reasoning-aware legal AI systems