A Real-World WebAgent with Planning, Long Context Understanding, and
  Program Synthesis
Pre-trained large language models (LLMs) have recently achieved better
generalization and sample efficiency in autonomous web navigation. However, the
performance on real-world websites has still suffered from (1) open domainness,
(2) limited context length, and (3) lack of inductive bias on HTML. We
introduce WebAgent, an LLM-driven agent that can complete the tasks on real
websites following natural language instructions. WebAgent plans ahead by
decomposing instructions into canonical sub-instructions, summarizes long HTML
documents into task-relevant snippets, and acts on websites via generated
Python programs from those. We design WebAgent with Flan-U-PaLM, for grounded
code generation, and HTML-T5, new pre-trained LLMs for long HTML documents
using local and global attention mechanisms and a mixture of long-span
denoising objectives, for planning and summarization. We empirically
demonstrate that our recipe improves the success on a real website by over 50%,
and that HTML-T5 is the best model to solve HTML-based tasks; achieving 14.9%
higher success rate than prior SoTA on the MiniWoB web navigation benchmark and
better accuracy on offline task planning evaluation.
64bf412046cc3cdfbb29e894
Izzeddin Gur
claimed_verified
2023-07-26T08:44:06.313Z
64c0ccc19832a455bc2a2069
/avatars/9d4fdf13cd62c6c258b5f50d67c8a1b1.svg
IZZEDDIN GUR
user
izzeddingur
64bf412046cc3cdfbb29e895
Hiroki Furuta
admin_assigned
2023-07-25T08:14:29.475Z
646c39fc3e2a7b06594bac6c
/avatars/8925fef2b6e7105a0f0a4bbf160cfda3.svg
Hiroki Furuta
user
frt03
64bf412046cc3cdfbb29e896
Austin Huang
admin_assigned
2023-07-25T08:15:21.796Z
6020b14c8345068224043bc0
/avatars/1c5c02f9165e188ab81a53efabc062e2.svg
Austin Huang
user
austinvhuang
64bf412046cc3cdfbb29e897
Mustafa Safdari
admin_assigned
2023-07-25T08:15:49.843Z
6430b08b74e7df5793edf353
/avatars/67875a201ffc22416acbf74b759b544b.svg
Mustafa Safdari
user
TheMegusAlexandrus
64bf412046cc3cdfbb29e898
Yutaka Matsuo
64bf412046cc3cdfbb29e899
Douglas Eck
64bf412046cc3cdfbb29e89a
Aleksandra Faust
admin_assigned
2023-07-25T08:17:22.594Z
64a3b828225b2121f6b3beb3
https://cdn-avatars.huggingface.co/v1/production/uploads/64a3b828225b2121f6b3beb3/44AgAQENc7cJMOWcoK4g_.jpeg
Aleksandra Faust
user
sandraorion
A Real-World WebAgent with Planning, Long Context Understanding, and
  Program Synthesis
Pre-trained large language models (LLMs) have recently achieved better
generalization and sample efficiency in autonomous web navigation. However, the
performance on real-world websites has still suffered from (1) open domainness,
(2) limited context length, and (3) lack of inductive bias on HTML. We
introduce WebAgent, an LLM-driven agent that can complete the tasks on real
websites following natural language instructions. WebAgent plans ahead by
decomposing instructions into canonical sub-instructions, summarizes long HTML
documents into task-relevant snippets, and acts on websites via generated
Python programs from those. We design WebAgent with Flan-U-PaLM, for grounded
code generation, and HTML-T5, new pre-trained LLMs for long HTML documents
using local and global attention mechanisms and a mixture of long-span
denoising objectives, for planning and summarization. We empirically
demonstrate that our recipe improves the success on a real website by over 50%,
and that HTML-T5 is the best model to solve HTML-based tasks; achieving 14.9%
higher success rate than prior SoTA on the MiniWoB web navigation benchmark and
better accuracy on offline task planning evaluation.
LLMs large language models autonomous web navigation open domainness context length inductive bias HTML canonical sub-instructions task-relevant snippets Python programs Flan-U-PaLM HTML-T5 LLM-driven agent local attention mechanisms global attention mechanisms long-span denoising objectives grounded achieving MiniWoB web navigation benchmark offline task planning evaluation