We’re witnessing an exciting time of innovation in the smart building domain. Commercial buildings and residential homes are undergoing a significant transformation and becoming smarter than ever before.
But what does “smart” mean? In a nutshell, it refers to automated processes that reduce operational expenses, save energy, optimize space allocation, increase safety and security, improve occupants’ comfort, and in some cases generate business intelligence.

In order to achieve these benefits, some necessary information must be collected. Physical data, such as temperature, lighting level and humidity, are provided by simple sensors that convert analog measurements into digital values and transmit them to the control unit. It is much more challenging – and important – to gather actionable information about occupants in the building. Data about occupants’ presence, location, number and movements is essential for optimizing the performance of building subsystems and processes such as lighting, HVAC, safety, security and facility management.


Current sensors such as motion detectors fall short of providing this occupancy data reliably, and they can’t provide the required granularity of information. In commercial buildings, IP cameras combined with video analytics currently provide the most powerful sensing capability for tracking occupants. Unfortunately they also have significant drawbacks - they are expensive, require a large communications bandwidth for streaming video, and most importantly, they are perceived as privacy invasive.


A new approach to computer vision, championed by PointGrab, enables us to overcome the challenges of providing occupancy information accurately, reliably, affordably and without violating privacy. This has been achieved by utilizing an Internet of Things (IoT) image-based smart sensor that performs all analytics on board and outputs only the processed data – a kind of a “picture-less camera” if you will. Drawing on the most advanced computer vision expertise, the sensor employs deep learning and object tracking algorithms that are streamlined to run effectively on low power processors.


Mounted on a ceiling or incorporated into lighting fixtures, this smart sensor can support a wide range of applications and use cases. To name a few:

  • Space Utilization. Space utilization data is a valuable component of the IoT in commercial buildings because it can help facility managers to better optimize the facility’s layout based on current and future needs. The smart sensors track occupants’ locations and movements, providing valuable insight into how and to what extent spaces are being used.
  • Energy saving. By counting the number of occupants in a given space, demand controlled ventilation can be optimized to save energy and comply with industry standards and regulations. Incorporating the sensor into lighting fixtures yields a variety of unique insights on energy management and lighting usage in a space. Data provided by the sensor can help determine how to optimally light a space while taking advantage of natural light, thereby saving energy.
  • Safety and Security. The ability to detect falling accidents and assist in emergency building evacuations are examples of the safety capabilities enabled by the smart sensor. Access control can be enhanced by detecting events such as tailgating, and security can be further improved by detecting loitering in pre-defined zones.
  • Retail Analytics. When deployed in retail environments, smart sensors can provide a precise understanding of shoppers’ activity, including traffic, paths, display attention and dwell time to improve retail optimization. This capability can also extend to store employees, automatically prompting additional cashier support when customer queues elongate, for example.

The 'deep learning' advantage

The next generation of smart sensors, fueled by the rise of the IoT, will give us the opportunity to source and analyze ever richer levels of data. Moreover, unlike traditional analysis where data is stored and then analyzed, the IoT is moving toward decentralization and the pushing of intelligence to the edges, enabling unprecedented occupancy awareness and the execution of more sophisticated building automation tasks in real-time. These capabilities will be achieved leveraging the continued drop in processor pricing on the one hand, and the advancements in artificial intelligence technology on the other, whereby we give the sensor system the ability to acquire knowledge that ultimately helps it improve its effectiveness.


Deep Learning – the underlying technology that powers sophisticated applications like speech and image recognition – is a particularly compelling approach to artificial intelligence that essentially transfers the burden of ‘teaching’ these sensors from human engineers to the sensors themselves. With Deep Learning, all computations from raw data collection to the final output are guided by an algorithm that adapts itself to achieve desired outcomes with greater success rates. This results in a much deeper level of computation -- much more complex, and ultimately much more effective than any rule or formula used by conventional and more cumbersome Machine Learning techniques.


Simply put, machines are better than humans at identifying the most informative features of the data, and when you take humans out of the equation, system performance improves dramatically. We have seen this to be the case in Building Automation as well. In PointGrab’s case, Deep Learning algorithms have significantly improved our ability to detect people and their exact location and movement in a room. Moreover, by uniquely implementing this technology, embedded analytics is performed using low-power and low-cost processors without requiring any digital signal processor (DSP) to do the heavy lifting.


Whereas in the Machine Learning world the system engineer needs exhaustive information about the domain in order to build a good system, in the Deep Learning world this is no longer necessary. In this era of the IoT where new kinds of data are becoming available at a rapid clip, Deep Learning allows us to faster iterate on new data sources and use them to our best advantage without requiring intimate knowledge of them.


All of these properties of Deep Learning afford us great agility to develop next-generation smart sensors for advanced building automation and occupancy intelligence. We can respond quickly to new types of data, easily adapt to new scenarios, and fully utilize computational resources as they become available. The superior performance of Deep Learning enables us to achieve new levels of sensing and analytical intelligence with the most cost effective, energy efficient embedded processors. Driven by the convergence of these innovative technologies – harnessed together in a brand new way – smart sensors are taking a leading role in making intelligent buildings a reality.