Wide-scale Monitoring of Satellite Lifetimes: Pitfalls and a Benchmark Dataset
An important task within the broader goal of Space Situational Awareness
(SSA) is to observe changes in the orbits of satellites, where the data spans
thousands of objects over long time scales (decades). The Two-Line Element
(TLE) data provided by the North American Aerospace Defense Command is the most
comprehensive and widely-available dataset cataloguing the orbits of
satellites. This makes it a highly-attractive data source on which to perform
this observation. However, when attempting to infer changes in satellite
behaviour from TLE data, there are a number of potential pitfalls. These mostly
relate to specific features of the TLE data which are not always clearly
documented in the data sources or popular software packages for manipulating
them. These quirks produce a particularly hazardous data type for researchers
from adjacent disciplines (such as anomaly detection or machine learning). We
highlight these features of TLE data and the resulting pitfalls in order to
save future researchers from being trapped. A seperate, significant, issue is
that existing contributions to manoeuvre detection from TLE data evaluate their
algorithms on different satellites, making comparison between these methods
difficult. Moreover, the ground-truth in these datasets is often poor quality,
sometimes being based on subjective human assessment. We therefore release and
describe in-depth an open, curated, benchmark dataset containing TLE data for
15 satellites alongside high-quality ground-truth manoeuvre timestamps.