Abstract
One of the main objectives of the METIS-II project was to enable 5G concepts to reach and convince a wide audience from technology experts to decision makers from non-ICT industries. To achieve this objective, it was necessary to provide easy-to-understand and insightful visualization of 5G. This paper presents the visualization platform developed in the METIS-II project as a joint work of researchers and artists, which is a 3D visualization tool that allows viewers to interact with 5G-enabled scenarios, while permitting simulation driven data to be intuitively evaluated. The platform is a game-based customizable tool that allows a rapid integration of new concepts, allows real-time interaction with remote 5G simulators, and provides a virtual reality-based immersive user experience. As a result, the METIS-II visualization platform has successfully contributed to the dissemination of 5G in different fora and its use will be continued after METIS-II.
1. Introduction
The fifth generation of mobile and wireless communication systems (5G) aims to satisfy the communication needs of the connected society in the years beyond 2020. 5G is planned to have commercial deployments in 2020 when, for example, Japan has pledged to provide full coverage for the Summer Olympics in Tokyo.
Today, the 5G technology is being standardized by standard developing organizations such as the Third Generation Partnership Project (3GPP). This standardization activity comes together with a huge research work conducted by the Information and Communication Technologies (ICT) industry and the academic world. Mobile and wireless communications Enablers for Twenty-twenty (2020) Information Society-II (METIS-II) was one of the 5th Generation Public-Private Partnership (5GPPP) European research projects contributing to this activity [1].
Since 5G will not only be a mere evolution of existing cellular standards but also will extend cellular communications to a wide range of novel applications and use cases involving vertical industries and other novel stakeholders, it will be especially important that the benefit of the solutions and frameworks developed in 5G is well-demonstrated to a wide audience. Note that, in parallel to the research and standardization work, the policy-makers and regulators are making decisions that will have a profound impact on the future deployment of this technology. For instance, they have to decide on the availability of more frequency spectrum in the near future for mobile communications, or on the kind of wireless technology to be promoted for the communications between autonomous cars in Europe.
To assist the work and decisions of the abovementioned actors, it is necessary to provide them meaningful insights about the technology, and one extremely useful way to do that is through the visualization of the technology operation and performance.
In the past, the visualization of the wireless communications concepts has been circumscribed to (1) the provision of static contents on research papers (such as figures, pictures, or diagrams), (2) slide presentations used in conferences or fairs showing a sequence of static materials, and (3) videos with prerecorded information. These traditional visualization options have some limitations, such as the lack of interactivity or the absence of user immersion. These drawbacks prevent them from being effective to demonstrate the 5G capabilities to today’s audiences accustomed to more appealing visual stimuli. However, one of the objectives of METIS-II was to provide a tool for training, education, and demonstration, targeting even nonexpert audience. Accordingly, the METIS-II project identified the need to work on a new paradigm for the 5G visualization without the mentioned drawbacks: the use of a 3D serious-game in which the visualization is a gaming experience where story, art, and software contribute to a pedagogic aim [2]. To succeed in this objective, METIS-II involved researchers and artists to jointly develop a visualization platform. Besides, to increase user immersion, virtual reality was considered a must for the platform.
5G is a complex technology, with many aspects to visualize whose number increases as the technology evolves and with many contributing entities. Therefore, METIS-II considered that the 5G community would obtain a great benefit from the availability of an open-source fully customizable 3D visualization platform in which new concepts could be implemented easily and inexpensively by anyone.
METIS-II realized that a significant part of the 5G information whose visualization is useful comes from computer simulations. In fact, researchers demand the possibility to interact in real time with running simulation to better understand the impact of certain concepts. This demand was not covered by a general-purpose visualization tool before METIS-II.
In summary, the target of METIS-II was to design a full-fledged 3D game-based platform with high user interactivity and high user immersion, easily customizable to show different aspects and concepts of the technology, open to third parties, and with an easy integration with computer simulators. The result of this activity is the software available in [1].
This paper, presents the visualization platform developed by METIS-II to satisfy all the abovementioned 5G visualization requirements. The aim of this paper is not to provide a manual that can be found in [3]. Instead, the main characteristics of the visualization platform, its design process, impact on the field of 5G visualization, and lessons learned in this process will be detailed. Section 2 explains what is the 5G and some of its peculiarities to understand how METIS-II conceived its visualization platform. Section 3 presents the general lines in the development process of the platform. Section 4 presents the visualization platform capabilities that make it a game-based open-source customizable platform capable of representing a plurality of 5G concepts. Sections 5 and 6 are focused on two characteristic aspects of the platform: the interaction and virtual reality capabilities. After that, we conclude with some lessons learned in the design process, the impact of the visualization platform for 5G, and the future lines of work.
2. 5G Visualization
To create an effective visualization platform, it is necessary to understand the science that is to be visualized [4]. If we asked a nonexpert about what the 5G is or will be, the answer will be likely something related to the increase in browsing speed. But the 5G is not only that. The 5G paradigm is to change many things in the system, affecting the way we will communicate and who will be the agent of communication [5]. The 5G topics are quite diverse and cover their new use cases, spectrum usage requirements, new radio interface, new radio access network architecture, and specific technological solutions that can boost system performance and radio architecture, among others.
5G visualization emerges as a unique opportunity to make pedagogy with this shift on our understanding of mobile communications and must result in a tool that allows its users to focus on different levels of conceptual complexity. To increase the educational capability of the visualization platform, METIS-II decided to present the 5G concepts using the story-telling approach [6], organizing the visualization into parts with a rational continuous flow that would start with a general view of 5G and would finish with more detailed concepts. The next paragraphs deal with the description of these parts and, at the same time, will present to the reader the main 5G concepts and how METIS-II decided to visualize them.
The story starts with the description of the service types to be enabled by the 5G deployment. The first one is the extreme Mobile BroadBand (xMBB), characterized by the provision of high data rates that will allow the users, for example, to download high-quality streaming everywhere. The second one is the Ultra Reliable and/or Low Latency Communication (URLLC). It is characterized by a low probability of failure in the data transmission and/or a strict latency requirement, needed for communication between autonomous vehicles or in mission critical situations, for instance. The third one is massive Machine Type Communication (mMTC), which is required to support the deployment of the Internet of Things (IoT) in which there will be a huge number of small devices interconnected, most of them with direct connections without going through base stations. The idea is to show clearly the difference between traditional mobile communication and the new 5G paradigm. The second message is that a new Radio Access Network (RAN) (i.e., the network that provides data transmission to the users through the air interface) able to accommodate the wide range of services and requirements of 5G is being designed, and one key feature in this design is the network slicing capability. This capability enables one mobile network operator to deploy, on demand, different instantiations of its physical network, each one isolated and dedicated to a specific vertical.
In order to give room to the traffic demands forecasted for the 5G era, 5G will need more radio spectrum and efficient means of using it. This is the second part of the 5G explanation designed in METIS-II. The key messages exposed in this second part are that the 5G success depends on the access to contiguous, wide, and globally harmonized new frequency bands. Moreover, this part highlights the spectrum rationale, that is, the usage scenarios and the coverage aspects in dependence of carrier frequency.
The part 3 presents the harmonization of air interface technologies for 5G and some slicing details. 5G will consist of different air interfaces, each one tailored to specific services and/or frequencies. The harmonization would reduce complexity, provide a lean standard, and more efficient integration of interfaces. After presenting services and spectrum, at this part, some exemplary slices are presented and the RAN architecture to support them. Some key technology enablers included in the exemplary architectures are commented paving the way for the next part.
Part 4 focuses on the explanation of some technology enablers that the research community is considering as pillars for the 5G successful design. Device-to-device communication, vehicle-to-vehicle communication, multislice management, MTC aggregation points, or handover improvements are some examples of these new technologies that should be further elaborated in this part of the 5G visualization.
The last part in the 5G explanation was allocated to how to organize the previous components in a network. The different architecture deployment scenarios under discussion in the standardization bodies and research projects should be presented. It is important to consider how the different network functions can be split over different physical architecture deployments and which interfaces between the physical entities would correspondingly be needed, the pros and cons of each option, and the scenarios in which each option is best suited.
3. Design Process Highlights
Once METIS-II project agreed upon an initial storyline for the 5G explanation, it was the moment to decide what and how to visualize each part.
We concluded that for parts 1 and 3, a detailed 3D graphical representation would be better to explain the concepts shown. A high interactivity would be required in part 3, to deepen into the technology enablers. For the rest of parts, with a more abstract conceptual level, simpler 2D schematic representations would be optimal to keep the user attention on the key concepts shown.
We started a closed interaction with artists in which the first step was to explain them all the scientific concepts required to understand the 5G. They then provided ideas on how those concepts could be visualized.
For the detailed visualization of technology enablers, it was decided to kick-off the discussion and integration with the graphic designers starting with a small group of people and institutions. Specifically, only two groups started working on graphical integration, which allowed the team to learn the optimal way of working and, above all, to design the graphic components necessary to accommodate the general needs of technologists. This first iteration gave rise to a library of graphic objects of logical representation of results that allowed the subsequent incorporation of many more components.
The design process had two significant milestones: the demonstration at the mobile world congress 2016 (MWC16) in Barcelona, about 8 months after the start of the project, and another demonstration in EuCNC 2017 (EuCNC17), at the end of the project, around 16 months after MWC16. In addition to those events, the platform had real confrontation with the public in other conferences and fairs.
At MWC16, the presenters could see the reactions of the audience, the limitations of the tool, and the parts that successfully delivered the intended message. Moreover, some ideas that could not be well visualized and were not essential were dropped. Some ideas were also empowered after its significant impact.
After MWC16, the storyline and the ideas on detailed visualization were refined, and two new lines of activity started: the interaction with simulators and the virtual reality implementation. Also, a manual was provided to the project facilitating the use of the tool. Figure 1 shows a simplified diagram of the development process.
4. Open-Source Graphical Environment
METIS-II visualization platform should be open-source with the objective of convincing 5G researchers and companies to use the tool for presenting their results with visual demonstrations and to foster collaboration between different projects.
The visualization platform was developed with the open-access Unity game engine. However, to achieve the wide use objective, allowing developers to create their visualization examples, it is also necessary to have a highly customizable platform and to have a simple and well-defined methodology to transform scientific data into visual representations.
At the beginning of the project, it was suggested that the scientific data, including the raw data coming from computer simulations, could be directly passed to the visualization platform, which would do the data processing to obtain the visualization content. In fact, it was planned to build a common database where the simulator would store its results and the visualization platform would read them. This option was withdrawn before MWC16 after realizing the diversity of the concepts to be visualized, as previously explained, and the enormous processing power needed in the machines to carry pure visualization tasks and data processing at the same time. Instead, it was decided that the visualization platform would be just a tool for visualizing concepts, responsible for data management and for turning data into visual entities. Therefore, simulation results should be processed outside the visualization tool to indicate to the visualization tool what to visualize and how. This approach requires, then, the provision of a plurality of visualization options for the user in the platform and a clear interface with the platform.
To provide the plurality of visualization options, the platform provides a library with numerous graphical assets and options that easies the setup of any potential concept to be visualized. Visualization objects are 3D or 2D. Perhaps, the most important 3D object is the scenario. In the context of the METIS-II project, a realistic urban scenario named Madrid grid, described in [7], was the elected scenario to visualize the 5G key concepts on. This scenario resembles a typical European city center consisting of multiple blocks of multistoried buildings, a park, medium-width streets, a wide avenue, and a pedestrian commercial street. One interesting characteristic of the implementation of the Madrid grid scenario is that it allows applying different skins to buildings. A bird’s eye view of the Madrid grid 3D scenario is shown in Figure 2.
The visualization platform was designed to allow the loading of any 3D scenario compatible with Autodesk® FBX® format. The key aspect of the FBX® format is its interoperability with any digital content creation software, so developers are not constrained to a single creation tool, instead, they may use their favorite one.
The visualization platform also provides a set of predefined 3D models, also known as components, for urban areas such as pedestrians, cars, buses, traffic lights, lampposts, and cellular antennas. The collection of components is completed with physical and logical networking elements models of routers, antenna beams, coverage areas, or spheres. The editor of the visualization platform may of course configure the size, color, location, and orientation of the objects through the whole visualization. An example of the scenario populated with 3D objects is shown in Figure 3.
The platform also provides 2D objects to be visualized. Consistently with how Unity manipulates those objects, 2D objects do not occupy an area in the virtual space. Instead, 2D objects are displayed over the user interface (UI) panel. The variety of 2D objects included in the visualization platform goes from simple forms like lines and circles (Figure 3) to sophisticated objects such as text boxes, buttons, bar diagrams, graphs, and widgets. The visualization platform also supports the use of external images to form small icons or larger pictures like, for example, a slide of a presentation.
The visualization platform can be understood as a window to a 3D world where different views show different scenes to illustrate the main 5G concepts. With that aim, each view contains a specific set of visualization objects and focuses on a specific area of the Madrid grid.
For each view, there is an associated button in the menu bar in the lower part of the UI. As it can be seen in Figure 3, each button has a label that unequivocally identifies the view, and the user of the visualization platform can easily jump from one view to the next one by interacting with the displayed buttons. Views can be related one to the next one as chapters in a book. However, it is possible to define views inside a view with the intention, for instance, of modifying some of the visualization content of the base view. To assist the user navigation, there is a home button that takes the user to the main view, and at its right side, the route to each view is shown.
To customize the views shown, the platform provides an XML file whose structure corresponds to the view structure shown in the computer program.
The main view is the very first view the user sees when the visualization platform starts. Its corresponding XML section includes the definition of the visualization objects that are common to all the views, including the scenario and, optionally, other visualization materials.
In general, a view section of the XML lists the visualization objects to be included, but it also contains the definition of the properties in time of the associated visualization objects. For instance, traffic lights switch their light phases following a periodic pattern that is fully customizable at the main configuration file.
Although the XML sets the initial configuration of some elements, the user may interact with the tool and change them in real time. For example, the camera setup is fully customizable at each view. By adjusting the position and orientation of the camera, the editor can control the initial perspective of the scenario that is going to be displayed. The camera can focus, for example, in one corner of the scenario as in Figure 3. In real time, the user can also zoom in and out and move around the scenario with a mouse and a keyboard. First person views such as the one shown in Figure 4 can be provided, which contributes to an enhanced immersion of the user. Another interesting aspect of the camera is the possibility to set a visualization point of interest. If the platform user selects a visualization object, the camera will focus on that object. This means that in case of selecting a moving component like a vehicle, the camera will smoothly change to a third person view and will follow the object from its rear side.
To facilitate the introduction of mobile elements such as cars, the visualization platform may read binary files, with a well-defined structure that indicates the position and orientation of each mobile element in periodic instances of time. For each view, a different mobility trace can be loaded.
Charts, graphs, arrows, and other graphical elements may appear to reflect the information from simulations. This rich information is also read from binary files with a structure well-defined to allow interoperability of the visualization platform with multiple simulators.
One good example of the use of simulation data is provided in Figure 5, which illustrates the communication between cars in 5G. In this illustration, the transmission of information between cars is shown as arrows. Green and blue arrows mean a correct transmission of information, while red arrows denote errors in the transmission and coverage problems. This visual information is enriched with a small picture on the top right corner describing the color code used and with two bar charts displaying two different network metrics (probability of reception of information vs distance). The location and size of the chart and its number of bars, colors, legends, and texts are fully customizable and defined at the main visualization configuration file.
The same applies to the diagram charts illustrated in the example in Figure 6. In this case, what is illustrated is the concept of slicing where network metrics from two slices are shown. In both examples, the values represented are periodically updated according to the update information in the binary file with the simulation data.
In Figure 7, an additional capability of the platform is shown. In this case, the buildings are made transparent to show the position of devices located indoors and the connections between those devices.
5. Interaction Capabilities
METIS-II identified two main benefits that could be provided by an interactive visualization [8]. First, the interaction keeps the user attention on the visualization thanks to an active involvement. Second, interactivity allows the user to have a customized user experience by making changes on the visualized information or the way in which information is visualized. For these reasons, the provision of user interactivity was one of the main requirements in the design of the METIS-II visualization platform.
The first degree of interactivity is achieved thanks to the use of navigation bars and menus located in the lower part of the visualization screen.
In some concepts, the visualization platform shows 3D scenes where the visualized elements change or evolve in time, for example, cars moving along the streets, or graphical representations of performance indicators, such as the download speed achieved by a smartphone. The user can control the temporal evolution of the visualization using controls located in the upper part of the UI. Specifically, the user can accelerate or decelerate the temporal evolution and jump to specific time instances thanks to the use of two sliding bars. The user can also stop and resume the visualization.
A very useful interaction capability consists of the ability of the user to change parameters of the concepts visualized. In the METIS-II platform, and this can be achieved by changing from one menu to another one when the results shown in those menus are obtained with different parameters. But, a more powerful way of interaction is the use of widgets, filled with fields where the user can select the values of some parameters. For example, Figure 6 includes a widget integrated in the right side of the UI. The visualization platform comes with a few predefined widgets for a specific use, but it also offers a modular widget that allows the editor to create a target-purpose widget at the visualization configuration file.
The inherent problem of this option is that the visualization tool should store a huge amount of information if the set of values that the user could select is high, making impossible to let the user choose the parameters freely, not from a prefixed set.
METIS-II overcame this problem by allowing the visualization platform to interact in real time with external simulators. These simulators could run in the same computer where the visualization tool is running or in remote servers. To enable this simulation-visualization interworking in real-time, Internet sockets are used, and the structure of the messages to be exchanged between both pairs have been specified in the platform manual. These messages allow the indication of the simulation parameters from the visualization platform to the simulator, with certain update period allowed, and the periodic sending of graphical representation information from the simulator to the visualization platform. This paradigm has proven to be successful and both academic partners, such as the Universitat Politècnica de València (UPV) and the University of Kaiserslautern, and industrial partners such as Deutsche Telekom, have successfully demonstrated the real-time interworking of the visualization platform and their simulators in real time. For interested readers, more technical details about the interworking are provided in [3] where instructions to work with a publicly available simulation server from UPV are provided.
6. Virtual Reality to Immerse the User in the Visualized Concepts
METIS-II considered that a virtual reality (VR) implementation of the visualization platform was necessary to completely immerse the user into the 5G concepts providing scientific insight [9]. The immersive experience creates a feeling of presence with several benefits. The first one is the power that VR has to keep the attention of the user on the visualization platform. The second one is the possibility to experience some capabilities of 5G in first person, closer to a real experience than looking at images in a screen and hearing sounds from speakers. A user could, for example, be placed into a bus and move the head down to evaluate the quality of a video running in a virtual smartphone.
One factor that contributes to the presence feeling is to have a realistic 3D world around the user. To achieve this realistic design, it is of paramount importance the work of nonscientists, artists, and designers [9], who can create scenarios where the user feels like in the real world. Artists and designers from Janmedia were in charge of this task in METIS-II.
A side benefit of VR is its commercial impact. In a fair, a demonstration based on a serious game will attract more attention than a demonstration based on prerecorded videos or posters. But, it is undeniable that if the demonstration is equipped with VR, the attraction of public increases exponentially, having more chances to disseminate the visualized concepts, as we experienced in our demonstrations.
The project identified three key requirements of the VR implementation to achieve a real immersive experience: (i) to perceive the stimuli generated in a virtual environment that surrounds the user, (ii) to interact with the virtual environment in real time, and (iii) to receive a quick feedback from the virtual environment. Therefore, the capacity of the final VR solution to fulfil all these objectives was evaluated.
Unity game engine has built-in support for the Oculus family of VR devices [10], notably the Oculus Rift Development Kit 2 (a VR headset with stereoscopic OLED display, 1080 × 1200 resolution per eye, 90 Hz refresh rate, and 110° field of view) and the consumer edition of the Gear VR mobile headset. With this equipment, it is straightforward to fulfil the first requirement, i.e., to sense the virtual environment. Indeed, using this equipment with the 3D scenario of METIS-II, the presence feeling was achieved, according to multiple user tests.
The key requirement of real-time interaction with the virtual environment needs an extension of the visualization platform. Since the non-VR UI could not be easily translated into a VR-friendly paradigm, it was necessary to create also a proper VR UI. A spatial version of the UI was designed, which positions the UI within the environment itself using Unity world space canvas render mode. The status bar, relocated as a heads-up display inside world space is in Figure 8. The lack of a mouse in VR was solved with a fixed reticle that activates the object that a user is looking at when the appropriate button of the physical controller is pressed.
A particular feature of the mobile VR version is that it relays on the sensors and touchpad integrated in the Gear VR for looking and interacting. The spatial UI is invoked and dismissed via double tapping on the touchpad, and buttons are selected via touchpad taps if they have been highlighted through the reticle previously. A device with VR rendering capability, such as the Samsung Galaxy S7, can be used to operate the platform without cables.
Concerning the Oculus Rift, a remote controller is used to interact with the spatial UI. The mapping of the controller buttons has been programmed through scripts for showing and hiding the menu and selecting menu items.
Regarding the third requirement, the users did not report any problem with the response speed of the virtual environment. In particular, there was no VR-induced motion sickness.
7. Lessons Learned
The development of the METIS-II visualization platform was a challenging process that has provided meaningful insights on several visualization aspects that will be useful in the future for the platform developers and for those making a similar exercise.
The visualization platform design was a joint effort of artists without any previous knowledge about the ICT industry and 5G experts. This joint work was facilitated by the original intuition of METIS-II that in its first phase elaborated a simple story to explain the 5G. This pedagogical work was really useful to make the artists understand the problem that they were facing. Another factor that made easy the interworking was the specific profile of the 5G researchers, many of them with grades in telecommunications and familiar, although nonexperts, with multimedia, games and other aspects involved in the development of the visualization platform. In fact, some 5G researchers who were more visualization-oriented were real catalysts of the work. Therefore, the lesson is that the more familiar are artists and researchers with the work of the other, the easier and more productive will be the joint work.
In the beginning of METIS-II, we spent too much time discussing about the interface between simulation results and the visualization platform, trying to define an interface able to accommodate all the possible results. However, the solution was really simple and defended by the visual-oriented 5G researchers: the simulation results should be processed by the researchers to tell the visualization platform what to visualize and how. This approach, although implies some work of the researchers to process their results, simplifies the development of the platform and the introduction of new concepts, since this introduction may be done in most cases without any change in the platform software. In our experience, after having a first stable version of the platform and using its manual, the time spent by researchers from the idea conception to the visualization was extremely short, even when they required from the artists new visualization options with the corresponding specification of the information format to feed the visualization platform. As a conclusion, if a visualization platform is to be used by multiple researchers with different interests, it is very useful to build the platform as a graphical general-purpose environment in which the researchers will be the ones elaborating their own instances of the visualization tool.
Some METIS-II partners were initially reluctant to make the effort to produce advanced visualizations of their research. Nevertheless, most of them changed their mind when they saw the success in the first great demonstration milestone of METIS-II, the MWC16. In that event, the visualization focused on vehicular communications got the attention of public and politics, for example. So, we learned that a successful visualization story motivates the researchers more than a large explanation.
8. Conclusion and Future Work
The visualization platform was one of the key contributions of METIS-II project to the 5G community. It was presented in different international events, with a large appreciation from the audience.
The partners that developed the platform continue the development after the project finalization ensuring a long-term impact of the platform.
Concepts related to 5G will keep on emerging in the near future, but METIS-II visualization platform is ready to face the foreseeable challenges. As an example, in the current version, it is possible to imagine any future 5G use case being added to the tool by adding new elements to the library of graphical objects or by extending the existing ones with more details from both visual and data-related perspectives. The same mechanisms could be used to produce more accurate representations of the real world and to bring the ideas closer to the user perspective.
Finally, as Unity engine can be complemented with VR and extended reality libraries, we aim at providing a yet more credible immersive experience thanks to a more detailed design and more advanced hardware interfaces, enabling future users to really enter the world of 5G even before it actually exists.
Disclosure
The views expressed are those of the authors and do not necessarily represent the project. The consortium is not liable for any use that may be made of any of the information contained therein.
Conflicts of Interest
The authors declare no conflicts of interest in the publication of this paper.
Acknowledgments
This work has been performed in the framework of the H2020/5G-PPP project METIS-II cofunded by the EU. The authors wish to thank the rest of METIS-II colleagues who contributed to the development of the METIS-II visualization platform.