Objective - Advance the capabilities of remotely operated emergency response robots by developing the measurement and standards infrastructure necessary to quantitatively evaluate system capabilities.
What is the new technical idea? Emergency responders need quantitative ways to measure whether a particular robot is capable and reliable enough to perform specific missions. These missions decompose into sets of elemental robot tasks that can be represented individually as standard test methods. Categories of elemental robot tasks include maneuvering, mobility, dexterity, sensing, endurance, radio communication, durability, reliability, autonomy, logistics, and safety. Each standard test method enables repeatable testing to establish statistically significant levels of reliability and confidence that the robot can perform the task. Standard test methods essentially define the test apparatuses, procedures, and performance metrics so they can be fabricated and practiced by robot manufacturers and user groups alike. They provide a tangible language to communicate responder requirements and demonstrate robot capabilities. The apparatuses typically include increasingly challenging settings to determine the maximum capability of the robot in a given task. Examples include stair inclines of 30°, 35°, 40°, 45°. Variables are controlled initially to measure baseline performance. They are then introduced incrementally to determine the effect. Examples of variables include lighted environments (<300 lux) versus dark environments (< 0.1 lux) or dry apparatuses versus wet apparatuses. A combination of 20-30 different standard test methods can be selected with appropriate apparatus settings and environmental variables to define a particular mission profile. Repeated testing within this set of standards establishes confidence that the robot can perform the overall mission.
The project results will help inform purchasing and deployment decisions and enable specification of available combinations of robotic capabilities demonstrated to statistical significance within a set of standard test methods that represent a potential user's mission tasks. This will align emergency responder expectations prior to purchasing by fully informing them about what the robot can and cannot be expected to do in the field.
The project deliverables will help foster innovation as robot manufacturers use the standard test methods to guide research, measure progress, and make engineering decisions regarding new sensors, coordinated manipulators, autonomous behaviors, etc. This will quicken the evolution of advanced robotic capabilities toward field deployment and improve reliability. Project deliverables will also help emergency responders improve and sustain very perishable operator skills. These standards provide inherent measures of operator proficiency that can focus training and track progress. When these standards are adopted widely for training, they can begin to isolate and measure deficiencies in equipment, including operator interfaces, and help identify technology gaps.
What is the research plan? The research plan includes three phases of development for particular mission tasks:
- Generate and validate a comprehensive suite of standard test methods to quantify component level and system level capabilities.
- Foster innovation in robotics research and manufacturing by using the standards to guide and measure advancements in robotic system capabilities using new sensors, coordinated manipulators, autonomous behaviors, etc.
- Apply the standards to support responder training and tactics with measures of operator proficiency.
During the first phase, we generate and validate a series of elemental test methods to characterize robot capabilities for specific missions defined by responder organizations. Elemental test methods typically include some combination of maneuvering, mobility, dexterity, sensing, endurance, radio communication, durability, reliability, autonomy, logistics, and safety for remotely operated ground systems, small aquatic systems, and small aerial systems under 2 kg (4.4 lbs). For example, a set of standard test methods will be generated to perform missions around a burning structure to establish situational awareness, identify hazardous materials, and locate potential victims. Another set of standard test methods will include manipulator reach, strength, and dexterity to measure potential fire service robots' ability to open doors, turn valves, manipulate hose line connections, and retrieve objects. Many of the same standard test methods also apply to improvised explosive devices and other hazardous materials, and visa versa. Any given mission can provide the initial impetus to generate and validate a set of tests. The resulting standard test methods often apply to other responder missions as well.
These test methods are being standardized through the ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01) which includes equal representation of robot manufacturers, emergency responders, and civilian/military test administrators. They are agreed upon ways to measure individual capabilities without identifying what level of performance is required by a particular user community for a given mission task. That is, they are not equipment standards that typically define the necessary outcomes of testing to meet a defined "standard." Any emergency responder organization can reference these standard test methods to specify their mission-oriented equipment or training objectives. They simply need to set thresholds within each applicable standard test method to clearly articulate their mission expectations. Operational mission descriptions specified using combinations of 20-30 standard test methods provide clear guidance to robot manufacturers. They also provide clear guidance to trainees by providing targeted levels of proficiency considered necessary by their responder organizations to be effective during deployments.