OLCF staff work to create high-speed information highway in collaboration with other DOE user facilities
Advances in network technology have allowed scientists around the world to send data between countries and across oceans in mere seconds, thus increasing collaboration and discovery.
However, with the advent of petascale computing—computers capable of at least 1 quadrillion calculations per second—researchers are generating extremely large amounts of data that, in some cases, take hours to transfer, even with state-of-the-art technology. Researchers at the Oak Ridge Leadership Computing Facility (OLCF), a US Department of Energy (DOE) Office of Science User Facility, plan to speed up this process.
Firewalls serve as one of the biggest speed bumps in sending data between networks at high speed. Though they are invaluable for keeping networks secure, firewalls can slow down data transfer speed by scanning all files for potential viruses and by looking at the state of the connection.
“Basically, whenever you have firewalls or certain types of security measures, they have to look at the source destination and look at the state, and whenever you do that, there is a pause,” said Daniel Pelfrey, OLCF high-performance computing systems network administrator. “When the firewall checks traffic, it looks to see if it is a new connection coming in or the return traffic of an established connection. Whenever you’re dealing with large amounts of data, the lookup times for connection rules or scanning can result in potential performance issues.”
DOE’s Oak Ridge National Laboratory (ORNL), where OLCF is located, is part the Energy Sciences Network (ESnet), a high-performance network built specifically to help send information between all national laboratories and over 140 research and commercial institutions.
ESnet, which is based at Lawrence Berkeley National Laboratory, carries roughly 20 petabytes worth of data every month. Because computers are getting stronger and faster, that number is expected to leap to around 100 petabytes per month by next year.
To help speed up data transfers coming in and out of a research facility, ESnet encourages member organizations to create a “Science DMZ,” a term that plays off the acronym for demilitarized zone, for scientific data. In computing, a DMZ is a perimeter network exposed to the outside world. Access between the DMZ and other internal networks is limited to add an extra layer of security.
“The whole purpose of the Science DMZ is to allow data transfer nodes on computing resources unfettered access where they can access the Internet—and the Internet can access them—while still providing layers of security,” Pelfrey said.
Pelfrey has been planning, designing, and implementing the Science DMZ for ORNL with a sequence of hardware upgrades, and he plans to have it operational by the end of 2015. With the new Science DMZ, researchers will go from 10 to 40 gigabits of managing data flowing in and out of ORNL. By designing the network with the Science DMZ in mind, Pelfrey has redesigned the ORNL network while looking ahead.
The router that connects ORNL’s main network to the OLCF will be changed to a tiered architecture that will allow staff to exchange to new technologies as they come available. “Right now, one router is serving as our border firewall,” Pelfrey said. “It’s not a bottleneck today, but we can’t go forward with this hardware, because it’s only capable of 10 gigabits. To move to 40 gigabits, and eventually 100 gigabits, we need to migrate everything off of our current router, then when everything is installed, we can go to 40 and 100 gigabits in and out of ORNL.”
Oak Ridge National Laboratory is supported by the US Department of Energy’s Office of Science. The single largest supporter of basic research in the physical sciences in the United States, the Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.