Multi-line AI-assisted Code Authoring
CodeCompose is an AI-assisted code authoring tool powered by large language
models (LLMs) that provides inline suggestions to 10's of thousands of
developers at Meta. In this paper, we present how we scaled the product from
displaying single-line suggestions to multi-line suggestions. This evolution
required us to overcome several unique challenges in improving the usability of
these suggestions for developers.
  First, we discuss how multi-line suggestions can have a 'jarring' effect, as
the LLM's suggestions constantly move around the developer's existing code,
which would otherwise result in decreased productivity and satisfaction.
  Second, multi-line suggestions take significantly longer to generate; hence
we present several innovative investments we made to reduce the perceived
latency for users. These model-hosting optimizations sped up multi-line
suggestion latency by 2.5x.
  Finally, we conduct experiments on 10's of thousands of engineers to
understand how multi-line suggestions impact the user experience and contrast
this with single-line suggestions. Our experiments reveal that (i) multi-line
suggestions account for 42% of total characters accepted (despite only
accounting for 16% for displayed suggestions) (ii) multi-line suggestions
almost doubled the percentage of keystrokes saved for users from 9% to 17%.
Multi-line CodeCompose has been rolled out to all engineers at Meta, and less
than 1% of engineers have opted out of multi-line suggestions.
65c2f4940aa2d53135b25abc
Omer Dunay
65c2f4940aa2d53135b25abd
Daniel Cheng
65c2f4940aa2d53135b25abe
Adam Tait
claimed_verified
2024-02-24T14:36:56.503Z
629e43cba5bdf6dd379af7e7
https://cdn-avatars.huggingface.co/v1/production/uploads/1654539209397-noauth.png
Adam Tait
user
adamtait
65c2f4940aa2d53135b25abf
Parth Thakkar
65c2f4940aa2d53135b25ac0
Peter C Rigby
65c2f4940aa2d53135b25ac1
Andy Chiu
65c2f4940aa2d53135b25ac2
Imad Ahmad
65c2f4940aa2d53135b25ac3
Arun Ganesan
65c2f4940aa2d53135b25ac4
Chandra Maddila
65c2f4940aa2d53135b25ac5
Vijayaraghavan Murali
65c2f4940aa2d53135b25ac6
Ali Tayyebi
65c2f4940aa2d53135b25ac7
Nachiappan Nagappan
Multi-line AI-assisted Code Authoring
CodeCompose is an AI-assisted code authoring tool powered by large language
models (LLMs) that provides inline suggestions to 10's of thousands of
developers at Meta. In this paper, we present how we scaled the product from
displaying single-line suggestions to multi-line suggestions. This evolution
required us to overcome several unique challenges in improving the usability of
these suggestions for developers.
  First, we discuss how multi-line suggestions can have a 'jarring' effect, as
the LLM's suggestions constantly move around the developer's existing code,
which would otherwise result in decreased productivity and satisfaction.
  Second, multi-line suggestions take significantly longer to generate; hence
we present several innovative investments we made to reduce the perceived
latency for users. These model-hosting optimizations sped up multi-line
suggestion latency by 2.5x.
  Finally, we conduct experiments on 10's of thousands of engineers to
understand how multi-line suggestions impact the user experience and contrast
this with single-line suggestions. Our experiments reveal that (i) multi-line
suggestions account for 42% of total characters accepted (despite only
accounting for 16% for displayed suggestions) (ii) multi-line suggestions
almost doubled the percentage of keystrokes saved for users from 9% to 17%.
Multi-line CodeCompose has been rolled out to all engineers at Meta, and less
than 1% of engineers have opted out of multi-line suggestions.
CodeCompose evolved from single-line to multi-line suggestions, reducing latency and significantly increasing user efficiency among thousands of Meta engineers.
large language models LLMs multi-line suggestions single-line suggestions model-hosting optimizations user experience keystrokes saved