CodeV: Empowering LLMs for Verilog Generation through Multi-Level
  Summarization
The increasing complexity and high costs associated with modern processor
design have led to a surge in demand for processor design automation.
Instruction-tuned large language models (LLMs) have demonstrated remarkable
performance in automatically generating code for general-purpose programming
languages like Python. However, these methods fail on hardware description
languages (HDLs) like Verilog due to the scarcity of high-quality instruction
tuning data, as even advanced LLMs like GPT-3.5 exhibit limited performance on
Verilog generation. Regarding this issue, we observe that (1) Verilog code
collected from the real world has higher quality than those generated by LLMs.
(2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating
it. Based on these observations, this paper introduces CodeV, a series of
open-source instruction-tuned Verilog generation LLMs. Instead of generating
descriptions first and then getting the corresponding code from advanced LLMs,
we prompt the LLM with Verilog code and let the LLM generate the corresponding
natural language description by multi-level summarization. Experimental results
show that CodeV relatively surpasses the previous open-source SOTA by 14.4%
(BetterV in VerilogEval) and 11.3% (RTLCoder in RTLLM) respectively, and also
relatively outperforms previous commercial SOTA GPT-4 by 22.1% in VerilogEval.
6698a542d723c35c0ec09743
Yang Zhao
claimed_verified
2024-07-18T09:05:13.327Z
646f6231a6a58aa29507091a
/avatars/14454308b13968cca11d8629350ca14d.svg
zhaoyang
user
yang-z
6698a542d723c35c0ec09744
Di Huang
6698a542d723c35c0ec09745
Chongxiao Li
6698a542d723c35c0ec09746
Pengwei Jin
6698a542d723c35c0ec09747
Ziyuan Nan
6698a542d723c35c0ec09748
Tianyun Ma
6698a542d723c35c0ec09749
Lei Qi
6698a542d723c35c0ec0974a
Yansong Pan
6698a542d723c35c0ec0974b
Zhenxing Zhang
6698a542d723c35c0ec0974c
Rui Zhang
6698a542d723c35c0ec0974d
Xishan Zhang
6698a542d723c35c0ec0974e
Zidong Du
6698a542d723c35c0ec0974f
Qi Guo
6698a542d723c35c0ec09750
Xing Hu
6698a542d723c35c0ec09751
Yunji Chen
CodeV: Empowering LLMs for Verilog Generation through Multi-Level
  Summarization
The increasing complexity and high costs associated with modern processor
design have led to a surge in demand for processor design automation.
Instruction-tuned large language models (LLMs) have demonstrated remarkable
performance in automatically generating code for general-purpose programming
languages like Python. However, these methods fail on hardware description
languages (HDLs) like Verilog due to the scarcity of high-quality instruction
tuning data, as even advanced LLMs like GPT-3.5 exhibit limited performance on
Verilog generation. Regarding this issue, we observe that (1) Verilog code
collected from the real world has higher quality than those generated by LLMs.
(2) LLMs like GPT-3.5 excel in summarizing Verilog code rather than generating
it. Based on these observations, this paper introduces CodeV, a series of
open-source instruction-tuned Verilog generation LLMs. Instead of generating
descriptions first and then getting the corresponding code from advanced LLMs,
we prompt the LLM with Verilog code and let the LLM generate the corresponding
natural language description by multi-level summarization. Experimental results
show that CodeV relatively surpasses the previous open-source SOTA by 14.4%
(BetterV in VerilogEval) and 11.3% (RTLCoder in RTLLM) respectively, and also
relatively outperforms previous commercial SOTA GPT-4 by 22.1% in VerilogEval.
CodeV, an open-source instruction-tuned Verilog generation model, improves Verilog code generation by focusing on summarization, outperforming previous models and GPT-4.
instruction-tuned language models Verilog GPT-3.5 GPT-4 CodeV summarization VerilogEval RTLCoder RTLLM