Recent advances in augmented reality (AR), virtual reality (VR), computing, graphics, cameras, and sensors are revolutionizing data center (DC) spaces. Data center visualization with AR and VR provide seamless solutions for maintenance and management to operators and managers. Typically, users that utilize AR in the DC use wearable technology such as headsets to visualize workflows, errors, and system health. VR can be used in the data center for fully immersive trainings and procedure walkthroughs. This paper describes the AR/VR tools and techniques being currently used in DC’s as well as future technologies that are being developed.
Keywords Augmented Reality (AR), Virtual Reality (VR), Mobile Augmented Reality Systems (MARS), Data Center (DC)
Table of Contents
A Survey on Augmented Reality in the Data Center
Introduction to Augmented Reality in the Data Center
Early Development of AR/VR
AR/VR in Data Centers
Augmented Reality in the Data Center vs Mobile Augmented Reality
What is an Augmented Reality Data Center
Mobile Augmented Reality
Legislation, Standards, and Best Practices
System Framework for Augmented Reality
Uses of AR/VR in Data Centers
Displays
Maintenance
Smart City
User Training
Current Technology
Holoinventory
Vertiv XR App
Ekkosense
Future Work
References
1. Introduction to Augmented Reality in the Data Center
Augmented reality (AR) and virtual reality (VR) technology is rapidly evolving and has been for decades. AR/VR are a bridge technology connecting humans and computers. Humans interact with technology in many different ways. Typically, you watch a movie, or start your car, but with AR/VR you can experience a movie and engage with your car. AR/VR do not alter what you could do in the real world, but it provides an enhances experience by adding digital information you can interact with in real time. What is an augmented reality experience? Let’s take an example, for a minute, where you intend to go bowling. You go to the bowling alley but it’s empty, no score cards, no televisions, only an alley and bowling balls. Consider an augmented reality experience at this empty bowling alley but where you can see exactly how many pins you’ve knocked down, the best angles to take to get a strike, the exact path your ball took, and the speed of your throw. You can interact with the bowling alley, pick up a ball, and throw it down the track just as you could in the real world. With AR/VR you have the power to do these things that you wouldn’t be able to do in the real world, perhaps you want to instantly change the weight of the ball without picking up a new one. This is all possible with augmented and virtual reality.
Humans have always been interested in improving our environment. We built roads to increase efficiency in getting from place to place, and bridges to help us move over water. As technology progressed, the need and ability to store, transfer, manipulate, and retrieve information developed quickly. With the development of 3D computer graphics, virtual scenes were able to be created that can mimic the physical world and display in front of our eyes. Different technologies can be used to display digital information, but what is important is that they don’t occupy space in the physical world. It is important to briefly discuss the difference between augmented reality (AR) and virtual reality (VR). Virtual reality creates an artificial virtual environment that a person can full submerse themselves in and experience. VR manipulates the senses using vision, sound, and other forms of feedback. Augment reality, on the other hand, is a supplement to the real world, instead of creating a completely artificial environment, the items in AR are to be used for annotation on an environment. AR and VR technology can be used in various fields from medicine (figure 1) to gaming to transportation, and as we will discuss in the paper, data centers (DC’s).
Figure 1 Doctors using VR glasses to assess the state of a human brain.
1.1 Early Development of AR/VR
Augmented reality (AR) and virtual reality (VR) technology is rapidly evolving and has been for decades. AR/VR are a bridge technology connecting humans and computers. Humans interact with technology in many different ways. Typically, you watch a movie, or start your car, but with AR/VR you can experience a movie and engage with your car. AR/VR do not alter what you could do in the real world, but it provides an enhances experience by adding digital information you can interact with in real time. What is an augmented reality experience? Let’s take an example, for a minute, where you intend to go bowling. You go to the bowling alley but it’s empty, no score cards, no televisions, only an alley and bowling balls. Consider an augmented reality experience at this empty bowling alley but where you can see exactly how many pins you’ve knocked down, the best angles to take to get a strike, the exact path your ball took, and the speed of your throw. You can interact with the bowling alley, pick up a ball, and throw it down the track just as you could in the real world. With AR/VR you have the power to do these things that you wouldn’t be able to do in the real world, perhaps you want to instantly change the weight of the ball without picking up a new one. This is all possible with augmented and virtual reality.
Humans have always been interested in improving our environment. We built roads to increase efficiency in getting from place to place, and bridges to help us move over water. As technology progressed, the need and ability to store, transfer, manipulate, and retrieve information developed quickly. With the development of 3D computer graphics, virtual scenes were able to be created that can mimic the physical world and display in front of our eyes. Different technologies can be used to display digital information, but what is important is that they don’t occupy space in the physical world. It is important to briefly discuss the difference between augmented reality (AR) and virtual reality (VR). Virtual reality creates an artificial virtual environment that a person can full submerse themselves in and experience. VR manipulates the senses using vision, sound, and other forms of feedback. Augment reality, on the other hand, is a supplement to the real world, instead of creating a completely artificial environment, the items in AR are to be used for annotation on an environment. AR and VR technology can be used in various fields from medicine (figure 1) to gaming to transportation, and as we will discuss in the paper, data centers (DC’s).
Figure 2 Milgram and Kishino's Virtual Reality Continuum
1.2 AR/VR in Data Centers
As automation and efficiency become increasingly necessary in networking, there is the need for new technologies that can transform data center analysis. In many environments, AR/VR has not reached the mainstream. One area of interest for AR/VR researchers is in networking Data Centers (DC). Paul Milgram [2] coined the term “augmented reality” and create the reality virtuality continuum (figure 2). Of which is defined as any case where a real physical environment is altered by means of computer graphics. One early experiment of AR/VR in DC’s was in 2004, The US Army Research Laboratory develop an augmented reality virtual system to aid in physics simulations. Researchers developed the system to represent complex phenomena by constructing a variety of complements for virtual/physical environments. By using low-cost bright projectors and systems similar to FlatWorld, researchers were able to combine rear-projected screen with modular theatrical components and 3-D spatial audio to simulate different training environments. Army researchers were able to collect data on techniques for adding sound to molecular visualization. This is an early example of using AR/VR to collect data, later in the paper we will explore more modern uses in networking DC’s.
2 Augmented Reality in the Data Center vs Mobile Augmented Reality
2.1 What is an Augmented Reality Data Center
An augmented reality data center is a data center that uses augmented reality for various tasks ranging from monitoring to maintenance to employee training. The technology makes data centers more efficient by allowing engineers to look up real-time records, help operators optimize performance, and make better decisions around energy and time use. With AR in the DC, engineers can walk through the DC and inspect elements, components, racks, and servers with their own eyes and not have to be physically next to the object. Data center workers can troubleshoot issues as they come up with alert systems and make quick decisions because they have a visual representation of the issue in front of their eyes. Imagine you are a data center engineer, and you’re center receives a new shipment of pods that contains 274 servers, 179 switches, 60 routers, and 32 firewalls, all needing to be connected by 2,000 miles of cable. It is your job to figure out where to place these components. With the power of augmented reality, you can visualize and map out where to place all these components without lifting a finger, send your schematic to the movers who can then move everything to it’s mapped out place. With augmented reality there is an extra level of assurance that things are in the place they should be in. There are many other examples of how to use augmented reality in the DC which we will explore more later. The most important thing to note is that augmented reality in the data center is a tool that can be leveraged to integrate human’s computer interaction in the DC.
2.2 Mobile Augmented Reality
While augmented reality and mobile augmented reality hand in hand, mobile augmented reality is a subset of augmented reality. As computers decrease in size and increase in power, wearable devices become more and more popular. The flexibility of microcomputers makes mobile augmented reality systems (MARS) intriguing. Instead of using a screen to map out a DC, an engineer could put on wearable glasses and full integrate themselves in the DC environment. With mobile augmented reality, digital information is presented via small wearable devices in real-time. A MARS system consists of four main components: a computational platform, displays, registration/tracking, interaction, wireless networking, data storage and access technology (figure 3). The most important aspect of MARS systems is that they are small often wearable, but most importantly portable, systems.
Figure 3 A data center operator wearing a small MARS headset in order to view the different components in a server
3 Legislation, Standards, and Best Practices
With the development of new DC technology comes the need for safe and useful legislation so ensure that all those that contribute and use technology can do so equitably. Standards provide guidance on how to use these AR systems while building, managing, and designing DC’s. One aspect of AR in the data center that needs standards is the real-time monitoring of systems. Real-time monitoring is integral for engineers to be able to address issues in the DC but can also be a security risk. There are many professional guidelines for DC use of technology infrastructure such as, ANSI/BICSI-002: Data Center Design and Implementation Best Practices, BICSI-009: Data Center Operations and Maintenance Best Practices, and the Data Center Site Infrastructure Tier Standard: Operational Sustainability. While these standards exist, the US government has expressed concern into privacy issues in DC’s. The government has passed legislation such as the Data Center Optimization Initiative [6] which requires agencies to report on data center strategies to consolidate inefficiency in infrastructure to reduce costs, integrate modern technology such as cloud servers, and improve efficiency. Data centers now must comply with certain standards to become more efficient, secure, and sustainable. These standards will eventually affect the implementation of VR/AR in data centers because more standards will need to be designed to monitor efficiency and sustainability in these technologies.
4 System Framework for Augmented Reality
AR/VR systems have developed various frameworks that can be separated into difference classes. Two options of classes are: systems with high precision tracking and graphics, and mobile systems with limited processing power. In a controlled environment such as a data center, these AR/VR systems require a lot of computing power which can be hard to achieve on a small microcomputer such as a Raspberry Pi. These MAPS often require tracking and rendering. These systems need to be able to access the right information efficiently at the right time. In wearable AR systems with multiple functional parts there are many units to be considered such as users’ position, direction, virtual objects, and the device itself. These systems can be complicated and often require a system administrator. Some frameworks [8] have suggested to limit inflexibility in the MARS systems by combining the camera and tracking software into one tracking module (figure 4). This framework gives the user a set of tools that can be combined seamlessly by plugging hardware units together. This allowed for distributed computing and tracking which bridges the gap between high precision tracking and limited computing power.
Figure 4 AR System for Data Center Tracking and Interaction
5 Uses of AR/VR in Data Centers
5.1 Displays
One of the most widely used technologies in AR/VR for the DC is a head worn projective display. This AR display contains a pair of micro displays, beam splitters, and miniature projection lenses. Using the wearable displays projects the physical materials of a data center outwards in front of one’s field of vision. A DC operator can walk down the isles of servers and be able to see exactly what they contain in front of them without having to touch physical devices. As mentioned earlier in the paper, with the use of head worn displays a DC operator can map the entire data center out various tasks or monitor system errors while knowing exactly where they are happening. This increases time efficiency because the engineer is not stuck wandering the isles to fix issues or set up new servers.
Another type of display is a VR display that exists as a planar eyepiece with retinal tracking. This system emits light from the display module, projects the light, and translates the size of the images to the viewer. A human’s field of view can be large so pupil, overlapping binocular vision, and intersection area of ray bundles tracking needs to be considered. These VR displays provide a fully immersive experience. While the AR displays allow the user to see the physical world and virtual at the same time, VR displays are more focused on viewing only the virtual world.
5.2 Maintenance
Once a DC is built, a large amount of time is devoted to maintenance and error correction. Maintenance is an area that benefits grossly from MARS technology. These wearable MARS devices use a tracking system and calibration to assess GPS location of the wearer and the other devices in the DC. AR is very suitable for problems that require you to see “through” a physical entity. By using direct overlays of hidden infrastructure, AR can assistant DC operators who are trying to fix a broken wire connection, or a server timeout issue. MARS systems can even help with tasks like monitoring fluid leaks on the DC floor. Sensors need to be installed to determine the exact location so that MARS knows what to display, but once installed, this provides real time updates to engineers in the DC. This is the fastest way to direct a worker’s attention to a potentially serious issue. AR can also be used in maintenance to determine the health of current operating systems, time till the parts need to be replaced, or the survival analysis of a current server.
5.3 Smart City
Data centers can be modeled as a smart city. A smart city uses 3D dynamic visualizations for buildings, trees, roads, etc. This idea can be applied to the DC but instead of buildings and roads, it is servers and wires. A smart DC, in this case, not only provides a means for DC decision making, but also managements, corporate virtual management, and general interactive platforms. The main parts of a smart DC are modeling, data transmission, and rendering/simulation. This architecture is shown in (figure 5). In the 3D modeling layer, the system performs terrain modeling and dynamic and static scene modeling. In the data layer the system performs data processing and collection/transmission. In the rendering and simulation layer the system performs interactive rendering and simulation.
Figure 5 System Framework for augmented reality, [https://ieeexplore.ieee.org/abstract/document/6376794]
5.4 User Training
Virtual reality can revolutionize data center training by providing an immersive learning experience for operators. Instead of traditional training methods where new employees shadow existing ones, virtual reality allows for more efficient, rapid, and cost-effective training. The training scenario unfolds as follows:
User assumes some technological literacy but may lack knowledge of modern VR/AR devices. The user puts on their VR headset.
In the VR scenario, the user enters the data center.
The user is tasked with picking up a wire from the ground and placing it into a storage bin.
Iterations continue with the user performing various tasks around the data center:
a. Manually turning on a server by navigating to it.
b. Addressing an overheating error by identifying the device causing the issue and adding a note to the file (see Figure 6).
c. Inspecting the rack and deciding whether to replace hardware.
Upon completion, the user gains an understanding of data center topology and potential day-to-day operational issues.
After this interactive training, users will comprehend how to navigate the data center floor. While not a complete substitute for training alongside a seasoned employee, combining virtual reality training with shorter person-to-person sessions enhances efficiency. Users can explore the data center at their own pace while receiving instructions. The simulation covers a range of operations, both critical and non-critical, contributing to increased employee confidence in using AR/VR systems daily in the data center.
Figure 6 Example VR training scenario in the data center [https://par.nsf.gov/servlets/purl/10106927]
6 Current Technology
There are many new AR technologies that companies have created to assist with data center operations which have been developed in the last 5 years. Below is a survey of the current technologies:
6.1 Holoinventory
The Holoinventory system is designed for use with the Microsoft HoloLens 2 AR headset (figure 7). It offers a novel solution to data center management problems. It provides a system to visualize and manage rack contents without the need to physically open rack doors. It also enables efficient tracking of wire connections and port statuses. This technology is particularly valuable for assisting all skill levels of technicians in locating equipment components in the data center environment. It is compatible with current Data Center Infrastructure Management software which increases its applicability in the data center.
Figure 7 technician wearing the Microsoft HoloLens using Holoinventory.
[https://tinyurl.com/beyondfig]
6.2 Vertiv XR App
The Vertiv XR App introduces an innovative approach to data center management by catering to IT manager, DC operators, and channel partners. The app allows DC employees to visualize Vertiv data center solutions within the context of their facility. It provides an easy-to-use user interface for product selection and placement as well as various visualization options that are customizable by DC management.
6.3 Ekkosense
Ekkosense is a solution focused on data center cooling and power optimization. It leverages AR, sensors, and experimental thermal services to monitor, manage, and maximize cooling and power infrastructure. They have defined a “Critical Things” family of sensors which offer a disruptive approach to real-time thermal management for data centers. The key features are: a suite of IoT wireless thermal sensors, a novel cooling duty meter that provides insight into cooling utilization, integration capabilities with 3rd party software systems, and robust security measures using encrypted data transmission.
7 Future Work
One of the latest advances in AR technology for the data center comes from MIT where researchers have built and augmented reality headset that gives the wearer X-ray vision. The X-AR headset combines computer vision and wireless perception to enable users to locate and retrieve specific items that are hidden from view, whether they are inside boxes, obscured under piles, or situated in other non-line-of-sight locations. This system leverages radio frequency (RF) signals to detect hidden items labeled with RFID tags, allowing the headset to guide users to the item's precise location in real time. The X-AR technologies uses lightweight, highly efficient antenna that communicates with RFID-tagged items. The application of this research would allow data center operators to visualize components behind the shells servers and racks. The potential applications in data centers is to aid workers in locating items on cluttered shelves, within boxes, or assisting technicians in assembling products accurately.
Milgram, Paul & Takemura, Haruo & Utsumi, Akira & Kishino, Fumio. (1994). Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies. 2351. 10.1117/12.197321.
J. Clarke, J. Vines and E. Mark, "An augmented virtuality scientific data center," 2003 User Group Conference. Proceedings, Bellevue, WA, USA, 2003, pp. 354-357, doi: 10.1109/DODUGC.2003.1253416.
Jochen Schiller, Agnès Voisard, Introduction, In The Morgan Kaufmann Series in Data Management Systems, Location-Based Services, Morgan Kaufmann, 2004, Pages 1-5, ISBN 9781558609297, https://doi.org/10.1016/B978-155860929-7/50001-7.
S. Kent, Update to data center optimization initiative (DCOI - June 25 2019). OMB Memorandum M-19–19, 2019.
Moises Levy and Anitha Subburaj. Emerging trends in data center management automation. In 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC), pages 0480–0485, 2021.
M. Bauer et al., "Design of a component-based augmented reality framework," Proceedings IEEE and ACM International Symposium on Augmented Reality, New York, NY, USA, 2001, pp. 45-54, doi: 10.1109/ISAR.2001.970514.
Seo, Jinsil & Bruner, Michael & Payne, Austin & Gober, Nathan & Mcmullen, Rick & Chakravorty, Dhruva. (2019). Using Virtual Reality to Enforce Principles of Cybersecurity. The Journal of Computational Science Education. 10. 81-87. 10.22369/issn.2153-4136/10/1/13.
Tara Boroushaki, Maisy Lam, Laura Dodds, Aline Eid, & Fadel Adib (2023). Augmenting Augmented Reality with Non-Line-of-Sight Perception. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23) (pp. 1341–1358). USENIX Association.
David Breitkreuz, Maike M¨uller, Dirk Stegelmeyer, and Rakesh Mishra. Augmented reality remote maintenance in industry: A systematic literature review. In Lucio Tommaso De Paolis, Pasquale Arpaia, and Marco Sacco, editors, Extended Reality, pages 287–305, Cham, 2022. Springer Nature Switzerland.
S. Deffeyes. Mobile augmented reality in the data center. IBM Journal of Research and Development, 55(5):5:1–5:5, 201.
Liping Wang, Dunbing Tang, Changchun Liu, Qingwei Nie, Zhen Wang, and Linqi Zhang. An augmented reality-assisted prognostics and health management system based on deep learning for iot-enabled manufacturing. Sensors, 22(17), 2022.
Cedric Westphal, Dongbiao He, Kiran Makhijani, and Richard Li. Qualitative communications for augmented reality and virtual reality. In 2021 IEEE 22nd International Conference on High Performance Switching and Routing (HPSR), pages 1–6, 2021.
Pengyuan Zhou, Wenxiao Zhang, Tristan Braud, Pan Hui, and Jussi Kangasharju. Enhanced augmented reality applications in vehicle-to-edge networks. In 2019 22nd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN), pages 167–174, 2019.
