Shhh! Your Things Might Be Listening!

Recent discussions highlight the fact that the domestic devices in the blossoming IoT will be listening and interacting with you as part of their normal function. This could have a number of privacy consequences, and could even spark unanticipated actions depending upon how these devices are installed and integrated.

As proposed in BJ Dooley’s IT Toons, there are many ways this could go wrong, the least of which might be a coffee overdose.

Recently, a prosecutor attempted to gain access to Amazon Alexa devices and cloud storage to obtain any recordings that might have been inadvertently made during an  incident inside the suspect’s home. Regardless of whether this is permitted, it does point to the problem of recorded interactions and privacy. What audio and video is being recorded, and for how long? Who can access it, and when?

In a multi-device household, there will also be several systems contending for attention that might be accidentally switched on with a mistaken keyword. Device coordination could become a problem. Devices will be making increasingly sophisticated decisions, and they could easily come into conflict with each other or with a user’s wishes.

While it is very early, indeed, this all points once again to the social nature of the IoT, and the need to integrate devices with the human context–as well as with the burgeoning thingaverse. If your digital assistant talks to your television, what will it say? And will the television talk back?

All of this will open up a market for personal device management, which will draw from and feed enterprise mobility management solutions that have been available for some time. The consumer versions will need to be less expensive and more user friendly, and they will feed right back to business systems–as is generally the case these days.

Devices will also need to be designed with the social context in mind. This will raise the bar a bit, and inevitably demand at least a primitive AI.

Meanwhile, the devices will keep talking. We only hope that they do not conspire against us.


IBM and ABB Collaborate to Boost Industry 4.0

Swiss-based engineering giant Asea Brown Boveri (ABB) and IBM have just announced a strategic collaboration that brings together ABB’s industry leading digital sensor and control offering, ABB Ability, with IBM Watson AI-based IoT system to provide a more comprehensive intelligent solution fir control and defect monitoring issues in utilities, industry and transport & infrastructure.

IBM Watson has been moving gradually into a range of new territories as applications for its cognitive capabilities are being explored. This move brings both companies closer to “Industry 4.0” where competition includes not only software companies, but also manufacturing firms such as GE.

The IBM/ABB solution is aimed at varying goals such as improving quality control, reducing downtime and increasing speed and yield of industrial processes by enabling current sensor and data gathering systems to become “cognitive” by using collected data to understand and take actions. ABB brings a deep domain knowledge and extensive portfolio of digital solutions to the mix, which is combined with IBM’s AI capabilities and vertical industry applications. The first two joint industry solutions will bring real-time cognitive insights to finding defects on the factory floor, and optimize maintenance of smart grids.

According to ABB CEO Ulrich Spiesshofe:

“This powerful combination marks truly the next level of industrial technology, moving beyond current connected systems that simply gather data, to industrial operations and machines that use data to sense, analyze, optimize and take actions that drive greater uptime, speed and yield for industrial customers. With an installed base of 70 million connected devices, 70,000 digital control systems and 6,000 enterprise software solutions, ABB is a trusted leader in the industrial space, and has a four decade long history of creating digital solutions for customers. IBM is a leader in artificial intelligence and cognitive computing. Together, IBM and ABB will create powerful solutions for customers to benefit from the Fourth Industrial Revolution.”

Part of ABBs business strategy is collaboration with other vendors, which includes partnering with IBM, Microsoft, and Wipro, among others in delivery of its digital solutions. Prior to the IBM deal, ABB has been using predictive and prescriptive analytics, plus customized models based extensive industry expertise, to identify and prioritize emerging maintenance needs based on probability of failure and asset criticality.

For IBM, this the collaboration aids in bringing Watson deeper into the crucial Industry 4.0 space, a key area for technological progress in IoT and AI. According to IBM CEO, Ginni Rometty:

“This important collaboration with ABB will take Watson even deeper into industrial applications — from manufacturing, to utilities, to transportation and more. The data generated from industrial companies’ products, facilities and systems holds the promise of exponential advances in innovation, efficiency and safety. Only with Watson’s broad cognitive capabilities and our platform’s unique support for industries can this vast new resource be turned into value, with trust. We are eager to work in partnership with ABB on this new industrial era.”

We can expect an onslaught of collaborations, mergers and acquisitions, and talent wars across this valuable sector as major industrial and IT forces join in the fray.


Blockchain Meets the IoT: the Video

Blockchain, the secure ledger system behind the Bitcoin cryptocurrency, is becoming increasingly important in key usage cases throughout business and industry. It provides a persistent, unalterable record system that is becoming increasingly important to security. And secure transactions are of critical importance in sensitive areas such as finance and health. But one of the most important areas that is now emerging is the connection with the Internet of Things (IoT). The surprising benefits of incorporating blockchain concepts in IoT development are creating intense interest as companies grapple with the needs of new infrastructure.

In these videos, we provide several discussions on the intersection of blockchain and the IoT. The videos are under standard YouTube license, and the description provided is from each video’s landing page (with minor edits).

Blockchain and the Internet of Things explained (IBM Internet of Things)

Published on Nov 2, 2016

A blockchain ledger can create a tamper-resistant record when information needs to be shared among business partners without setting up a costly centralized IT infrastructure. Let’s look at how supply chains benefit when data is shared through a private blockchain.

Next Generation IoT Technologies Using The Block Chain (Samsung Developer Connection)

Published on Dec 1, 2014

Talk by Gurvinder Ahluwalia

An overview of how Samsung and IBM are thinking about the next generation of IoT infrastructure, and why they are using Blockchain. This presents an approach to address the problems of cost, privacy and longevity of smart devices on the Internet Things.

Key Use Cases Intersecting Blockchain and IoT (IBM Internet of Things)

Published on Dec 9, 2016

Jerry Cuomo (Vice President Blockchain Technologies, IBM), Mika Lammi (Head of IoT Business Development, Kouvola Innovation), and James Murphy (Offering Manager, IBM Watson IoT Platform – Risk Management & Security) talk about the trends in business that are driving the pairing of Blockchain and IoT.

Blockchains for the Internet of Things – Solving the IoTs Most Critical Problems (

Published on Jul 7, 2016

Blockchains are poised to revolutionize the Industrial Internet of Things (IIoT) by providing security, peer-to-peer device communications and new functionality via smart properties. Andre De Castro CEO of Blockchain of Things, Inc. discusses why blockchains make sense for the IIoT



Animated Humor: Okay Rodney on the IoT, Cloud and Security

These are part of a set of cartoon animations that I created several years ago to enliven discussion of these subjects. All are standard YouTube license, and were created very simply using text to voice and PowerPoint graphics, exclusively.

Okay Rodney: The Internet of Things (IoT)

Okay Rodney: The Mobile Cloud

Okay Rodney: Security


Verizon Grabs Skyward to Help Feed the Drones

Verizon has just acquired drone operations management company Skyward to add to its growing concentration in the IoT. Drones have enormous potential, and management will be of increasing importance in expanding deployment opportunities.

Drones have been operating at the edges of the technological horizon for several years now, but recent developments have made them more interesting and more important for business. Today’s drones come in innumerable sizes and shapes, and miniaturization of technology has meant that they are able to carry more sensors and provide more useful information about areas over which they pass. They have also been provided with new ways for interaction, such as capability to manipulate controls or pick up and release objects. This is creating innumerable opportunities, particularly in areas such as agriculture, aerial photography, mapping, surveying, and other such operations. Regulation issues are also being gradually adjusted to accommodate new concepts of drone usage.

As the importance of drones has continues to grow, concerns have been raised regarding air safety and interference with other devices. Regulations have limited operations to line-of-sight and there is a range of rules which need to be followed. Additionally, operation of drones can be complex, particularly if individuals with piloting experience are not available or the mission demands extensive coordination. As drones continue to develop, we are beginning to see hive operations involving innumerable drones, spectacularly illustrated recently in Intel’s Super Bowl lighting display; we are also beginning to see various levels of autonomy and precise coordination issues.

Within this mix, huge opportunities exist, and this is the objective of Verizon Skyward acquisition. Verizon itself has been developing an LTE based network to allow operation of unmanned airborne vehicles (UAVs) in operations beyond visual line-of-sight (BVLOS). As line-of-sight restrictions are lifted, remote operation of drones will become significantly more common, and drone fleets will need to be managed.

Verizon began work on its in-flight LTE operations in 2014 and expanded operations in 2016 by engaging American Aerospace (AATI) to test connectivity between aerial platforms and Verizon’s 4G LTE network. Verizon’s airborne LTE operations (ALO) initiative has now undergone technical trials across the country in a combination of unmanned and manned aircraft using the company’s 4G LTE network.

Entering this area at an early stage permits Verizon to build the network connections and experience that it will require to offer services for complex drone operations involving many devices. But, operating and managing such networks demands that devices remain within FAA parameters, meet regulations, and function in accordance with agreements such as insurance that may be related to specific task environments. Skyward is an operations management solution for commercial drone businesses. It is a cloud-based platform that integrates a drone airspace map with flight planning tools, fleet and equipment management, and a digital system of record. This makes it possible to coordinate a complex drone operation with confidence, leading to greater accessibility to the technology and the possibility of expanding the opportunity horizon for drone usage.

Verizon is taking a leadership role in this area and is betting on the further development of drones and IOT devices as an additional service opportunity for the company. Already, it has over $1 billion of revenue from the IOT space. Development of drones is a natural outgrowth of this concentration.

According to Verizon’s news release, Mike Lanman, senior vice president Enterprise Products and IOT said:

“Last quarter we announced our strategy to drive innovation and widespread adoption for in-flight wireless connectivity through our Airborne LTE Operations (ALO) initiative, a new service to simplify certification and connectivity of wireless drones. This acquisition is a natural progression of our core focus on operating in innovative, high-growth markets, leveraging our network, scale, fleet management, device management, data analytics and security enablement capabilities and services to simplify the drone industry and help support the adoption of IoT.”

Undoubtedly AI will play a part in creating more autonomous drones systems; but these operations, given their complexity, must remain within the bounds of regulation and human legal constraints.

In a greater sense operation of a centralized solution that provides management of a regulatory environment is very similar to what is occurring within the GRC space. As new technology evolves and needs to be fitted into human society and legal and economic boundaries, regulatory management platforms need to be put into place to ensure that behavior, no matter how autonomous or operator-controlled, remains within parameters that ensure harmonious operation of all components of our increasingly complex technological world.

Digital Assistants Coming of Age with Alexa

Digital Assistants are bringing AI to a new level, and this will have consequences for how the IoT develops. At the recent CES show, there was one stand-out performer: Alexa. Amazon’s voice-actuated helper was available on a wide range of technology, demonstrating the possibilities of interactive AI-driven control. The popularity of Alexa highlights the increasing role of intelligent digital assistants and leads us to consider what we might expect of this area in future.

Smart digital assistants have been explored by major software and IT companies and include Watson-based assistants from IBM; Apple’s Siri digital assistant; Microsoft’s Cortana; Google’s Now and Amazon’s Alexa. This category has been developing for several years. But 2017 seems like it could be a watershed year, and part of the reason is Alexa.

One of the reasons that Alexa is popular among developers is that it can control many different home systems and is relatively device-agnostic. The Alexa platform is easily integrated into an array of products, offering different types of services, and wireless control of home devices. The net result is that it can function as an easily configured hub for home services, as well as providing immediate information from the network. Alexa can be used for home security, for home information, to turn off lighting, to adjust the thermostat, and to provide general purpose information–or, indeed, any function to which a third-party provider might like to add a wireless connection.

This strategy has propelled Alexa forward, and at CES, Alexa was the voice of robots, the mouthpiece of smart systems, the driver of speakers and audio systems, the recipient and enabler of voice commands, and the dominant digital assistant at the show.

With digital assistants, we can see a development across the years from first ventures in natural language processing (NLP). Siri on Apple was the first critical development through the marketing prowess of Apple. It brought AI assistance into the mainstream and legitimized it for everyone. Capability has grown with improvements in NLP, better user interfaces, and greater integration with products and services.

Use of digital assistants will certainly grow as the capabilities continue to expand. But it will be device agnostic platforms that will be of greatest importance. In the early days of personal computing, it was only when the IBM PC became available that the computer explosion really took off. Other companies were able to build equipment and software for a commonly available platform. This led to a huge spurt of creativity around the globe, enabling developments which eventually led to the software world of today. Alexa is perhaps another watershed, popularizing the opening of digital assistants to a wider range of possibilities.

There are several keys to importance of digital assistant functions.

  • Configuration must be easy, simple, and performable by individual consumers.
  • The assistant must be able to understand voice commands out-of-the-box; a technology which has reached a visible turning point recently.
  • The digital assistant must be capable of performing useful functions; this means integration with home devices and excludes strictly proprietary system.
  • The digital assistant must integrate with existing smart technology in the home, in the automobile, and even in the workplace
  • The digital assistant must be unobtrusive and available as needed. Personal robots in varying forms could be available throughout the location, for example.
  • The digital assistant must be capable of performing necessary tasks. Perhaps the biggest limitation today is that smart homes are hard to find and people are difficult to convince that and expensive addition will help them in their daily lives.
  • Digital assistants must and will become inexpensive and ubiquitous; this means that they must coexist with humans and other devices
  • Digital assistants must be capable of integrating with other digital assistants to provide a complete response. This will demand new software as these systems spread and become more relevant.

For the present, digital assistants are in an early stage of acceptance. In time, however, the technology pioneered to some extent by Alexa is likely to become commonplace. It will influence the home environment as well as the office. As with much new technology, initial benefit will be found in the home. It is certain that it will also move into the enterprise environment. Capability to interact with other systems will be of critical importance as these devices proliferate. This will create a market for AI software designed to meet the needs of digital assistants and will bolster the move toward cognitive computing.

Companies need to understand that this wave of development is about to become important, Software markets will develop, new communications requirements will emerge, and the burgeoning Internet of things will become smarter and smarter and smarter.

Image Processing with Machine Learning: The Video

Here is a mixed collection of sales and academic presentations on the topic of machine learning for embedded vision. Where available, descriptions are provided from the web source.

Real time image processing is absolutely critical for most autonomous systems, and understanding current capabilities is important in developing new applications for business and in consumer technology. Deep learning is proving essential to making these systems work.

Visual Intelligence in Computers – Fei Fei Li (Stanford Vision Lab)

As director of one of the top Artificial Intelligence (AI) labs in the world, Dr. Fei-Fei Li is leading the next wave of AI that is rapidly being integrated into companies, governments and the lives of individual consumers. The way we work, drive, entertain and live our lives will never be the same. Dr. Li heads a team of the world’s top scientists and students who enable computers and robots to see and think, as well as conduct cognitive and neuroimaging experiments to understand how our brains function. She is a world-renowned expert on computer vision, machine learning, artificial intelligence, cognitive neuroscience and big data analytics. She directs both the Stanford Artificial Intelligence Lab (SAIL) and the Stanford Vision Lab, as well as teaching Computer Science at Stanford University.

CEVA’s Jeff VanWashenova interviewed at AutoSens 2016 (AutoSens)

CEVA is a leader in developing DSP technologies for image recognition. Here, Alex Lawrence-Berkeley interviews CEVA’s Jeff VanWashenova at AutoSens 2016, held at AutoWorld in Brussels, Belgium.

Computer Vision System Toolbox Overview (MATLAB)

Design and simulate computer vision and video processing systems using Computer Vision System Toolbox™. The Toolbox provides algorithms, functions, and apps for designing and simulating computer vision and video processing systems. You can perform feature detection, extraction, and matching; object detection and tracking; motion estimation; and video processing. For 3-D computer vision, the system toolbox supports camera calibration, stereo vision, 3-D reconstruction, and 3-D point cloud processing. With machine learning based frameworks, you can train object detection, object recognition, and image retrieval systems.

Movidius Demonstration of Its Machine Intelligence Technology (Embedded Vision Alliance)

Jack Dashwood, Marketing Communications Director at Movidius, demonstrates the company’s latest embedded vision technologies and products at the May 2016 Embedded Vision Summit. Specifically, Dashwood demonstrates the Fathom neural compute framework, running an image classifier in an ultra-low power embedded environment (under 1W), and enabling a whole new class of miniature robot overlords of which to be fearful.

The Internet of Intelligent Things and Your Security

The recent Dyn server DDoS attack using a botnet of cameras and other devices points to a growing security problem with the IoT. The attack exploited known vulnerabilities in common internet-connected devices such as cameras, watches, and lighting systems. Taking down a few popular websites was an important warning on network and device vulnerabilities. But this also draws attention to a much more significant issue: What happens when devices become intelligent and autonomous? Without adequate attention to security beginning in the earliest stages of device development, and without security standards, security issues could easily develop beyond our capacity to control them.

Currently, devices tend to have less processing capability and lack adequate security measures.Consumers tend to leave them unprotected. These defects make it possible to create attack systems using thousands of devices. But these attacks are part of a larger problem:

  • The growing availability of attack software. Shortly after the Mirai software online attack on Dyn, Mirai was released to the public domain, making it possible for less sophisticated hackers to create effective DDoS attacks using devices. Mirai itself incorporates concepts from predecessors such as Lizardstresser which used a botnet of home routers. These and similar programs are now evolving to create new threats.
  • The increasing sophistication of DDoS attacks, and ability to project damaging outcomes using relatively unsophisticated software. Large scale DDoS attacks are growing with available software and bandwidth. According to Akami’s most recent State of the Internet Security Report, the first quarter of 2016 marked an all time high in the number of attacks peaking at more than 100 Gbps.
  • The proliferation of devices for popular use, with no effective standards in place for security. Device makers need to respond rapidly to a growing market for consumer devices and security is often an afterthought. Gartner forecasts that 6.4 billion connected things will be in use worldwide in 2016, reaching 11.4 billion by 2018.
  • The failure of consumers, businesses, and device manufacturers to emphasize security in the release and installation of internet-connected devices. Devices using default passwords continue to offer the greatest vulnerability to attack.

IoT DDoS attacks are only the beginning of a new and more complex cyber threat environment. Symantec’s 2016 Internet Security Threat Report describes multiple vulnerabilities in 50 commercially available devices, including a ‘smart’ door lock that could be opened remotely online without a password; vulnerabilities in medical devices such as insulin pumps, x-ray systems, CT-scanners, medical refrigerators, and implantable defibrillators; vulnerabilities in Internet-connected TVs and connection vulnerabilities in thousands of everyday devices, including routers, webcams, and Internet phones due to networking issues. This is before even considering industrial controls and devices that can be used in complex compound attacks.

At the moment, the chief concerns are with potential for havoc with networks. The Spiceworks 2016 IoT Trends survey of 440 IT professionals found one of the top security concerns to be the fact that IoT devices create more entry points into the network (84%) and that IoT manufacturers aren’t implementing sufficient security measures (about 75%). Proliferation of network entry points creates potential for dangerous backdoors, and unsecured devices provide unlimited opportunity for intrusion. While a DDoS attack requires control of thousands of devices; a network break-in requires only one.

As devices get smarter, the dynamics are likely to change and issues will become more serious. Potential for more botnet attacks using strategies well beyond DDoS certainly exists. The range of vulnerabilities will inevitably involve Industrial Control Systems (ICS), as well as an increasing range of autonomous vehicles and robots.

At the present time, the security issues are manageable. Reviewing vulnerabilities in this area and ensuring adequate measures are in place is the best place to start. It is important to consider security usage, device access, and network isolation of devices that might be easily compromised. Device management policies need to be in effect, and need to be constantly adjusted to growing threats. We are beginning to see use of big data and machine learning as a part of the solution, but this needs to be incorporated in a comprehensive program that also focuses upon security awareness and vigilance.

The IoT remains loosely defined, and is growing to include an increasing variety of things, both for consumers and for industry. This can increase confusion regarding possible security issues. Devices include the personal items of which we are all aware; but they also include routers and modems, network equipment, autonomous systems, and the equipment and Industrial Control Systems. Each type of device brings its own security threat, and each unsecured device is a potential recruit to a botnet device brigade. It is important to consider carefully how devices will interact with your business and personal life.

Device makers need to consider security issues from the earliest stages of development, and better standards need to be put in place to ensure that more complex devices do not become a new problem for our increasingly complex networks of connected things.

The latest attacks point to the growing threat from connected devices. But DDoS attacks are the tip of the iceberg. There are direct physical threats to automobile and home systems developing as well. The IoT must be secured; this is of growing importance to business, government, and the individual consumer.

China Files a Million Patents: Implications for AI and the IoT

The World Intellectual Property Organization (WIPO) reports that China has become the first country ever to file 1 million patent applications in a single year. While this was in the context of growing patent applications around the world, the size of the difference is remarkable. Chinese patent applications totaled as many applications as the next three offices combined: the U.S. (589,410), Japan (318,721) and the Republic of Korea (213,694). Chinese applications were led by electrical engineering and telecom, followed by computer technology and semiconductors. Growing innovation in these areas shines a spotlight on the potential for new opportunities in the burgeoning Internet of Things (IoT). It also points to an important shift in innovation toward Asia at a critical point where AI technologies, Industry 4.0, and consumer devices are starting to move forward.
The Chinese patent applications were largely for home consumption rather than for international patent protection. China’s applications flow from a highly competitive economy in which domestic protection is important and current government policies have a focused upon developing innovation in all sectors—but particularly in electronics and IT. The huge number of Chinese patent applications reflects this policy, which applies pressure to municipal governments, educational institutions, and companies to produce patent as a demonstration of support.

The recent DDoS attack on the Dyn server in the US used a botnet largely comprised of inexpensive Internet-connected Chinese cameras. The desire to innovate in these types of devices and the growing market is creating the elements of a security problem. As firms continue to release devices at a faster and faster rate at lower cost and without security procedures, security issues will proliferate. Chinese ability to innovate, manufacture, and market in this area is likely to lead to a tsunami of unprotected devices. Rapid innovation will lead to new opportunities as well as new threats as devices offered at extreme low cost are adopted before their impact can be fully understood.

Chinese innovation is also of importance in considering the growth of new product ideas within Asia. It is interesting to note that three of the top five patent producers are in Asia (China, Japan, and Korea). Asian firms have focused upon industrial automation and ability to produce inexpensive devices. This makes it likely to create a hub of innovation in IoT within Asia generally, adding to existing concentrations in robotics and mobility. As frictions continue to develop between China and the West, it is possible that new concentrations of innovation will emerge. The IoT presents a wide range of possibilities for growing markets in a stagnant economy. The connected consumer device territory is still a greenfield area. Low cost in this market is an absolute advantage, since it reduces consumer resistance in experimenting with new technology. By boosting innovation and also controlling low cost manufacture the Chinese are in an ideal position to pursue these types of developments.

For the rest of the world, patent applications continue to grow, particularly in information technology. Computer technology (7.9% of total) leads global applications, followed by electrical machinery (7.3%) and digital communication (4.9%). While the Chinese have focused upon internal patents, most other countries pursue international protection. Offsetting the million internal patents, Chinese international patents remain a relatively tiny number (42,154). While this is growing somewhat, it also demonstrates the country’s ability to isolate itself; or, perhaps, in the words of Deng, “hide your strength, bide your time.”