TENET: Leveraging Tests Beyond Validation for Code Generation
Test-Driven Development (TDD) is a widely adopted software engineering
practice that requires developers to create and execute tests alongside code
implementation, ensuring that software behavior is continuously validated and
refined. In the era of vibe coding, where developers increasingly delegate code
writing to large language models (LLMs) by specifying high-level intentions,
TDD becomes even more crucial, as test cases serve as executable specifications
that explicitly define and verify intended functionality beyond what
natural-language descriptions and code context can convey. While vibe coding
under TDD is promising, there are three main challenges: (1) selecting a small
yet effective test suite to improve the generation accuracy and control the
execution workload, (2) retrieving context such as relevant code effectively,
and (3) systematically using test feedback for effective code refinement. To
address these challenges, we introduce TENET, an LLM agent for generating
functions in complex real-world repositories under the TDD setting. TENET
features three components: (1) a novel test harness mechanism that selects a
concise test suite to maximize diversity of target usage scenarios; (2) a
tailored agent toolset that performs efficient retrieval of relevant code with
interactive debugging; and (3) a reflection-based refinement workflow that
iteratively analyzes failures, replenishes context, and applies code
refinement. TENET achieves 69.08% and 81.77% Pass@1 on RepoCod and RepoEval
benchmarks, outperforming the best agentic baselines by 9.49 and 2.17
percentage points, respectively. In addition, this is the first study of
test-driven code generation with repository-level context, examining how
different aspects of test suites affect the performance of LLM agents under the
TDD setting.
68dc04b44159d1f2418f983b
Yiran Hu
claimed_verified
2025-10-01T15:29:31.112Z
658bb78f2e1f5bf05c2a3da9
https://cdn-avatars.huggingface.co/v1/production/uploads/658bb78f2e1f5bf05c2a3da9/c4keEyiRRGXww0kZQdQph.jpeg
Yiran Hu
user
Yiran-Hu1007
68dc04b44159d1f2418f983c
Nan Jiang
68dc04b44159d1f2418f983d
Shanchao Liang
68dc04b44159d1f2418f983e
Yi Wu
claimed_verified
2025-10-02T03:09:06.729Z
62c88b04ab9c23f5c459ed90
https://cdn-avatars.huggingface.co/v1/production/uploads/62c88b04ab9c23f5c459ed90/tEaeuKpXdXwqK-zq1H-8a.png
Yi Wu
user
yiwu
68dc04b44159d1f2418f983f
Lin Tan
claimed_verified
2025-10-01T10:23:13.627Z
65831342f9c5cda913df366a
https://cdn-avatars.huggingface.co/v1/production/uploads/65831342f9c5cda913df366a/h7MLYt--shRYQj4-q5XmR.jpeg
Lin Tan
user
lin-tan
TENET: Leveraging Tests Beyond Validation for Code Generation
Test-Driven Development (TDD) is a widely adopted software engineering
practice that requires developers to create and execute tests alongside code
implementation, ensuring that software behavior is continuously validated and
refined. In the era of vibe coding, where developers increasingly delegate code
writing to large language models (LLMs) by specifying high-level intentions,
TDD becomes even more crucial, as test cases serve as executable specifications
that explicitly define and verify intended functionality beyond what
natural-language descriptions and code context can convey. While vibe coding
under TDD is promising, there are three main challenges: (1) selecting a small
yet effective test suite to improve the generation accuracy and control the
execution workload, (2) retrieving context such as relevant code effectively,
and (3) systematically using test feedback for effective code refinement. To
address these challenges, we introduce TENET, an LLM agent for generating
functions in complex real-world repositories under the TDD setting. TENET
features three components: (1) a novel test harness mechanism that selects a
concise test suite to maximize diversity of target usage scenarios; (2) a
tailored agent toolset that performs efficient retrieval of relevant code with
interactive debugging; and (3) a reflection-based refinement workflow that
iteratively analyzes failures, replenishes context, and applies code
refinement. TENET achieves 69.08% and 81.77% Pass@1 on RepoCod and RepoEval
benchmarks, outperforming the best agentic baselines by 9.49 and 2.17
percentage points, respectively. In addition, this is the first study of
test-driven code generation with repository-level context, examining how
different aspects of test suites affect the performance of LLM agents under the
TDD setting.
TENET, an LLM agent for TDD in complex repositories, uses a test harness, efficient code retrieval, and iterative refinement to improve code generation accuracy and performance.
Test-Driven Development (TDD) large language models (LLMs) test cases executable specifications test suite context retrieval code refinement test harness reflection-based refinement RepoCod RepoEval