By 2050, the human population will have reached 9 billion people, with 75 percent of the world’s inhabitants living in cities. With already around 80 percent of the United Kingdom’s population living in urban areas, the U.K. needs to ensure that cities are fit for purpose in the digital age. Smart cities can help deliver efficiency, sustainability, a cleaner environment, a higher quality of life and a vibrant economy.
To this end, Bristol Is Open (BIO) is a joint venture between the University of Bristol and Bristol City, with collaborators from industry, universities, local communities, and local and national governments. Bristol Is Open (www.bristolisopen.com) is propelling this municipality of a half million people in southwest England to a unique status as the world’s first programmable city.
Bristol will become an open testing ground for the burgeoning new market of the Industrial Internet of Things—that is, the components of the smart-city infrastructure. The Bristol Is Open project leverages Xilinx® All Programmable FPGA devices in many areas of development and deployment.
The vision of the smart city
A smart city utilizes information and communications networks along with Internet technologies to address urban challenges, with the objective of dramatically improving liveability and resource sustainability. It is predicted [1] that the smart-cities industry will value more than $400 billion globally by 2020, with the U.K. expected to gain at least a 10 percent share, or $40 billion. The U.K. government investment in the smart-city sector includes around $150 million for research into smart cities funded by Research Councils U.K.; $79 million over five years earmarked for the new Future Cities Catapult center being established by the Technology Strategy Board in London; $52 million invested in future city demonstrators earlier this year; and $63 million recently allocated to Internet of Things (IoT) research and demonstrator projects.
Bristol Is Open is leading the way to building a city-scale research and innovation testbed. The aim is to drive digital innovation for the smart cities of the future: the open and programmable communities that will be the norm in the latter part of the 21st century.
The BIO testbed is equipped with leading-edge programmable networking technologies, enabled by a citywide operating system called NetOS, that allow smart-city applications to interact with city infrastructure—to program, virtualize and tailor network functions for optimum performance. Xilinx devices as high-performance generic platforms are utilized at many points in the city from the wired, wireless and IoT networking infrastructure to emulation facilities.
Let’s take a tour of this new type of urban community, starting with the overall vision for programmable cities. Then we will take a deeper look at how the Bristol project is utilizing Xilinx devices to build urban “white boxes” and to deliver various networking functions.
Future smart cities
More than 100 cities of 1 million people will be built in the next 10 years worldwide [2], while the continuous influx of people to cities will grow the number of urban residents by 60 million every year during that decade. [2] The result is that more than 70 percent of the world’s population will be living in cities by 2050. Considering also that cities occupy just 2 percent of the world’s landmass while consuming about three-quarters of its resources, the ongoing urbanization presents economic and societal challenges and a strain on the urban infrastructure. Growing cities will have to deal with a variety of challenges to maintain economic advancement, environmental sustainability and social resiliency.
The solution is to make cities smarter. Although there is no absolute definition for smart cities, there are a number of key aspects widely recognized [3] for a smart city’s operations. They include:
- Citizen-centric service delivery, which involves placing the citizen’s needs at the forefront.
- Transparency of outcomes/performance to enable citizens to compare and critique performance, establishment by establishment and borough by borough.
- An intelligent physical infrastructure, enabling service providers to manage service delivery, data gathering and data analysing effectively.
- A modern digital, secure and open software infrastructure, to allow citizens to access the information they need, when they need it.
Technological enablers for smart cities are inspired by the Internet of Things, a market that, according to Gartner, [4] will grow to 26 billion units installed as of 2020. That total represents an almost thirtyfold increase from 0.9 billion in 2009, with the revenue from technologies and services exceeding $300 billion. Smart cities deploy IoT technologies on a wide scale, enabling data gathering from sensors and things present in the ecosystem, pushing them for analysis and feeding back commands to actuators, which will control city functions.
From sensing and analysis, information passes back to actuators in the city infrastructure to control operations dynamically. This arrangement is an enabler for driverless cars using smart transport facilities; greater power efficiency thanks to smart lighting; the management of network resources for different times (daily and seasonal changes); the movement of resources depending on occasions such as sports events, which require high-quality broadcast and coverage; and efficient handling of emergency situations (city evacuation).
Programmable City vs. Smart City
Smart cities aim to improve and enhance public and private service offerings to citizens in a more efficient and cost-effective way by exploiting network, IT and, increasingly, cloud technologies. To achieve this goal, smart cities rely extensively on data collected from citizens, the environment, vehicles and basically all the “things” present in the city. The more data that becomes available, the more accurately city operations can be analysed, which in turn will lead to the design and availability of smart-city services.
For the network infrastructure, citywide data retrieval and processing mean massive amounts of sensor data that needs to be collected, aggregated and transferred to computational facilities (data centers) for storage and possibly processing. The wide diversity of scenarios and applications presents major challenges regarding networking and computing infrastructure requirements in smart cities. Legacy information and communications technology (ICT) urban infrastructure can be a major bottleneck for smart-city operations, as it does not offer the capacity, flexibility and scalability desirable for the emerging, future-proof, resource-demanding and scalable smart-city applications.
Programmable networking technologies offer unique capabilities for raising the performance of smart-city operations. These technologies exploit open software and hardware platforms, which users can program to tailor network functions for different use case requirements. Improved control, monitoring and resource allocation in the network are the evident benefits of deploying programmable networks. More important, programmable technologies facilitate the integration of networks with IT facilities, which will result in greater application awareness.
Software-defined networking (SDN) is one of the main enablers for programmable networks. The SDN foundation is based on decoupling infrastructure control from the data plane, which greatly simplifies network management and application development while also allowing deployment of generic hardware in the network for delivering networking functions.
SDN-based scalable and facilitated network management also greatly empowers network virtualization. Network virtualization essentially enables multiple users to operate over shared physical resources, isolated from one another, reducing the need for installing supplementary physical hardware. Network function virtualization (NFV), a more recent innovation in virtualization technologies, offers software implementation of network functions in commodity hardware. Network functions such as firewall, deep packet inspection, load balancing and so on are deployed as pluggable software containers in generic machines, expediting network service deployments with great cost-efficiency.
In addition to software-driven networking, hardware and infrastructure programmability will progress beyond fixed-function hardware data planes. Adding high-level programmability and more sophisticated functionality to the data plane, accessed via standard software APIs, will make it possible to manage networking resources more intelligently and efficiently, increasing the rate of innovation.
Bristol is Open: vision and architecture
Launched in 2013, Bristol Is Open is a program funded by the local, national and European governments and also by the private sector. BIO is already delivering R&D initiatives that contribute to the advancement of smart cities and the Internet of Things.
BIO aims to serve as a living lab—an R&D testbed targeting city-driven digital innovation. It provides a managed multitenancy platform for the development and testing of new solutions for information and communication infrastructure, and thus forms the core ICT enabling platform for the Future Cities agenda. At the infrastructure level, BIO comprises five distinctive SDN-enabled infrastructures, as shown in Figure 1 (below):
Figure 1 – The Bristol Is Open fiber network places active core nodes at four locations in the city. HPC facilities and emulation are accessible through the network core. Wireless technologies (802.11ac, 802.11ad, LTE, LTE-A) are spread out through the center.
• Active nodes as optoelectronic-network white boxes using FPGA programmable platforms and heterogeneous optical and Layer 2/3 networking infrastructure
• Heterogeneous wireless infrastructure comprising Wi-Fi, LTE, LTE-A and 60-GHz millimeter-wave technologies
• IoT sensor mesh infrastructure
• Network emulator comprising a server farm and an FPGA-SoC-network processor farm
• Blue Crystal high-performance computing (HPC) facility
On the metro network, the infrastructure offers access to dynamic optical switching supporting multi-terabit/second data streams, multi-rate Layer 2 switching (1 to 100 GbE) and Layer 3 routing. The metro is also equipped with programmable hardware platforms and high-performance servers to allow open access to the infrastructure and a capability to create and experiment with new hardware and software solutions. This wired part of the infrastructure also connects to the Blue Crystal HPC facilities at Bristol in order to support experimentation with advanced cloud infrastructures.
The access network infrastructure includes overlapping and seamless wireless connectivity solutions (macro and small-cell radio technologies) using a combination of cellular and Wi-Fi technologies enhanced with millimeter-wave backhaul and direct connections to the optical network. The facility also supports experimentation platforms for new 5G-and-beyond access technologies such as millimeter-wave-based access solutions with beam tracking, as well as new technology enablers such as massive MIMO for ultrahigh-density networks in the 2-GHz band.
In addition, BIO provides priority access to the infrastructure (for example, lampposts) for the additional installation of sensor nodes in the area, supported by suitable data aggregators, computing and storage resources. Optionally, these resources can directly interface into the wired and wireless network. BIO has also installed a low-energy wireless-sensor mesh network. This network will support IoT-based research, with initial sensors supporting environmental monitoring (temperature, air quality, pollution levels, lighting, noise and humidity) and smart streetlights.
BIO will also provide access, through suitable secure interfaces, to IoT assets already installed elsewhere in the city, including parking sensors, traffic lights, traffic flow sensors, surveillance (safety) cameras and public-vehicle sensors. Small sensors, including the smartphones and GPS devices of willing participants, will supply information about many aspects of city life, including energy, air quality and traffic flows. All the data generated will be rendered anonymous and made public through an “open data” portal.
The entire platform uses SDN control principles and, as such, is fully programmable by experimenters and end users. Internationally, the BIO experimental network will be the first of its kind and will generate new and exciting opportunities to pioneer the development of hardware and software for future communication technologies and cloud networking.
Software-Defined Networking for City Infrastructures
The communications sector has seen a flowering of innovative solutions in recent years based on the concept of SDN, bringing advances in IT to the traditional hardware-driven telecommunications world. This decoupling of control and data through SDN enables innovative ways of controlling a network, while relying on a basic data-forwarding operation, common across all networking elements. The approach allows the integration of novel architecture concepts, such as information-centric networking (ICN), into such a software-driven network. SDN also enables continuous investment into smart infrastructure at the lowest layers of the ICT installations by driving the reduction of costs for physical components and pushing more of the operational aspects into the software.
As SDN is now reaching beyond ICT infrastructures into the IoT platforms, it creates the opportunity to realize a full circle of adaptability of computing and communication infrastructures, where sensory and real-world information drives the operation of the network. Network infrastructures in turn are utilized to provide the sensor information to applications and services in a meaningful and timely manner. At BIO, it is our vision for that programmability and adaptability across the various layers of the overall system to ultimately implement the notion of what we call a Living Network, where the Internet and things truly merge into a coherently managed and operated computing and communication environment.
Demonstrating SDN-based platforms on a citywide scale is crucial. Future Internet and 5G technologies are present in the BIO testbed, specifically an SDN-enabled optical-backbone infrastructure using current and contemporary (i.e., Wi-Fi, LTE, millimeter-wave) radio access technologies. The stimulating media and entrepreneurial community is present throughout the BIO testbed (the engine shed in Figure 1 is home to a startup incubator and the watershed is home to the media community in Bristol). Members of these communities also serve as an excellent set of early-user groups for the use case work. Their involvement in BIO allows us to capture the insights and requirements posed by the municipal communities.
The wired, wireless and RF mesh networks are technology-agnostic, built on open-network principles using SDN technologies that enable network function virtualization. A city operating system called NetOS (Figure 2), also based on SDN principles, will provide the needed programmability and adaptability for smart cities. NetOS will be an overarching and distributed operating system spanning from terminals (even the more advanced ones, e.g., mobile robots, drones) through the network elements to the cloud/IT resources. This citywide OS will cope with the heterogeneity of underlying resources based on a distributed software architecture. NetOS will act as a logical entity that is implemented in a hierarchical manner with distributed software, making it possible to map varied services on the infrastructure.
Virtualization for City Infrastructure
A large number of highly diverse city applications need to be supported on top of the city infrastructures. For example, some applications will demand high capacity and very low latency. Others will consume very little bandwidth but will need to support a very large number of endpoints. Still others will have strict requirements on resiliency or security, privacy and so on.
It is neither feasible nor cost-effective to establish dedicated infrastructures to support specific applications. Therefore, one of the key challenges for the city infrastructure operators is to offer customized, application-specific network solutions over a common ICT infrastructure. Virtualization, when integrated with an SDN-enabled control platform, is a key technical enabler for addressing this challenge. Virtualization is able to create multiple coexisting but isolated virtual infrastructures running in parallel, serving its tenant’s application requirements.
By thorough analysis of each tenant’s requirements in terms of social policy, security and resources, it’s possible to construct a virtual infrastructure with a certain network topology, indicating the way that virtual nodes are interconnected with virtual links. Performance parameters (for example, latency) and resource requirements (such as network bandwidth, compute CPU/memory) are specified in the virtual nodes and links. Generally, virtual resources (nodes and links) are obtained by partitioning or aggregating physical resources. Therefore, a programmable hardware infrastructure is essential to support the composition of virtual infrastructures with high granularity and scalability.
In the city environment, the devices deployed in the urban infrastructure are heterogeneous, including wireless/mobile, wired, optical networks, data centers/cloud and functional appliances. In order to enable seamless service provisioning, it’s mandatory to support converged virtual infrastructures enhanced with virtual network functions across the multitechnology, multidomain city infrastructure, so that each tenant can get its own slice of the city infrastructure. However, currently these technology domains are controlled and managed in silos. The NetOS with SDN capabilities at BIO provides a logically centralized control platform that can break through the management silos and bridge the different technology segments. The operating system is able to abstract the heterogeneous city devices, hide their complex technical details and expose the infrastructure in a uniform way.
The vision for the white box
Open network devices, or network white boxes, use unbranded, generic, modular and programmable hardware platforms. This type of equipment can load customized operating systems and enable on-demand redefining of network functions without the restrictions of vendor-locked devices. Network processors were the initial route to hardware programmability of the underlying network, leveraging the ease of defining functions through software APIs. Network processors are well-known hardware platforms that provide generic programmable features similar to general-purpose CPUs (with extended hardware resources), and can be programmed to perform various networking functions. The main advantage of processor-based architectures is rapid implementation of networking functions using high-level languages such as C, which is highly desirable for rapid prototyping. Network processors, however, are not optimized for parallel operations, which are essential for building high-performance data plane technologies supporting high-data-rate transport.
Field-programmable gate arrays (FPGAs) are high-performance and generic processing platforms utilizing programmability from transistor-level to IP-based function level. This makes them highly desirable platforms for designing and prototyping network technologies that must demonstrate high degrees of flexibility and programmability.
Figure 2 – NetoOS is an SDN-based platform, built in a multilayer structure, which can communicate with networking, IT and IoT technologies. This platform natively supports data collection, virtualization, information modeling and interfacing with third-party applications.
We are using Xilinx FPGAs that have evolved into system-on-chip (SoC) devices in multiple points within the BIO infrastructure: in active nodes (see Figure 2, above) as optoelectronic white boxes, emulation facilities, wireless LTE-A experimental equipment and IoT platforms. BIO uses programmable and customizable network white boxes that consist of programmable electrical (FPGA) and optical (switching, processing, etc.) parts. These boxes—which enable high-capacity data processing and transport, function programmability and virtualization—are deeply controllable through SDN interfaces. Figure 3 (below) demonstrates the FPGA-based platform, which can host multiple functions in a programmable way, and is interfaced to a programmable photonic part. [5]
Figure 3 – Bristol Is Open’s network white box is built around Xilinx FPGAs.
FPGAs offer several advantages, including hardware repurposing through function reprogrammability, easier upgradability and shorter design-to-deploy cycles than those of application-specific standard products (ASSPs).
The photonic part of the network white boxes uses an optical backplane on which a number of photonic function blocks are plugged into optical functions such as amplification, multicasting, wavelength/spectrum selection, signal add/drop, etc. Critically, the input and output links are decoupled from any of the functions that the node can offer, unlocking flexibility, efficiency and scalability, and minimizing disruptive deployment cycles with on-service hitless repurposing.
Zynq SoC-based Emulation Platform
To broaden the capabilities of BIO facilities in experimenting with larger and more-realistic scenarios, we have deployed a network emulator facility within BIO. This platform enables network emulation as well as resource virtualization and virtual-infrastructure composition techniques for advanced network, cloud and computational research. The emulation platform also utilizes local and remote laboratory-based facilities and distributed research infrastructures (networks and computing). Figure 4 (below) demonstrates the multilayer, multiplatform emulation facilities at the core of the Bristol Is Open infrastructure.
Figure 4 – The emulation facility in Bristol Is Open includes programmable hardware in the form of FPGAs and network processors.
The emulation facility offers a number of functions instrumental for enhanced network studies in conjunction with the BIO city network and other remote interconnected laboratories:
1. Node and link emulation: This plat-form can emulate network elements such as routers and switches from the wired and wireless domains, along with the interconnecting links with various physical attributes.
2. Protocol emulation: Whether centralized or distributed, network nodes rely on the protocols to communicate. The emulation facility with precise modelling of the network technologies allows the user/researcher to try out communication protocols and study their behaviour on scale.
3. Traffic emulation: Depending on the emulation scenario (wireless networks, data center networks, etc.), traffic patterns with arbitrary intervals and operating from a few megabits to multiple terabits per second can be generated and applied to the target emulated or physical network.
4. Topology emulations: Any topological formation of the desired nodes and links is possible using the BIO emulation facility. This gives the user a chance to fully examine various aspects of the desired technology on the realistic network topologies before deployment and installation.
Unlike any other existing facilities that offer computer host-based emulation environments, BIO uniquely includes programmable hardware (FPGAs, network processors) as well as dynamic and flexible connectivity to multitechnology testbeds and a rich, dedicated connectivity infrastructure. The use of programmable hardware and external interconnectivity will allow users to accurately emulate the functionality and performance of network and computing technologies in scale and use them to synthesize representative complex systems. Exploiting the FPGA’s parallel-processing capabilities and high-speed I/Os, BIO is equipped to emulate current or experimental network technologies and topologies, be they wired or wireless, precisely and at scale.
The network emulator uses a vast amount of advanced networking and IT technologies. An FPGA farm, server farm and L2/L3 programmable networking equipment are the main building blocks of the facility, enabling the users to build, experiment with and use various networking technologies in the data plane and control plane, such as virtualization, SDN and NFV, resource/workload allocation tools and algorithms, etc.
The emulator is connected to the BIO city network through 10-, 40- and 100-Gbps ports. The emulated networks can use standard data plane protocols such as Ethernet, OTN and Infiniband, or custom and proprietary protocols, to interconnect with other network domains.
The emulator uses Xilinx’s ARM®-based Zynq®-7000 All Programmable SoC platform, a single-chip implementation of processing and FPGA technologies. Algorithm acceleration is one of the target use cases for the Zynq SoC, where computationally intensive tasks for resource allocation, path calculation, load balancing and the like are offloaded to FPGA-based parallel processing. Hardware-assisted network function virtualization is another example of how we use Zynq SoC-based platforms in BIO for running performance-critical virtual network functions (VNFs) such as deep packet inspection, service control and security. Xen-based virtualization of ARM cores additionally facilitates running multiple operating systems on the same SoC chip. In this way, BIO can let multiple operators host their VNFs on the same device, and have shared and/or dedicated access to the parallel hardware computing resources to boost performance.
Experimentation as a Service
The way cities work is changing. Using digital technologies, BIO is creating an open, programmable city that gives citizens more ways to participate in and contribute to the way their city works. We call it “City Experimentation as a Service.” Being open guides our procurement, our data management and the hardware and the software we use. Being open means the stakeholders in BIO proactively share what we learn with other cities, technology companies, universities and citizens.
References
1. https://www.gov.uk/government/news/uk-set-to-lead-the-way-for-smart-cities
2. UN State of World Cities report, 2012/13, http://www.unhabitat.org/pmss/ listItemDetails.aspx?publicationID=3387
4. http://www.gartner.com/newsroom/id/2636073
5. Bijan Rahimzadeh Rofoee, George Zervas, Yan Yan, Norberto Amaya and Dimitra Simeonidou, “All Programmable and Synthetic Optical Network: Architecture and Implementation,” Journal of Optical Communications and Networking 5, 1096-1110 (2013)
Authors:
Bijan R. Rofoee, Senior Network Engineer, Bristol Is Open
Mayur Channegowda, Chief Scientist, SDN, Zeetta Networks
Shuping Peng, Research Fellow, University of Bristol
George Zervas, Professor of High-Performance Networks, University of Bristol
Dimitra Simeonidou, CTO, Bristol Is Open, Professor of High-Performance Networks, University of Bristol