IoT Application Testing - Complexities & Challenges
                     IoT Application Testing - Complexities & Challenges
Satya K Vivek
Writes for Gadgeon.com, an IT outsourcing services
, and has expertise in IoT software development.
 
The Internet of Things (IoT) is a network of physical objects (devices, 
vehicles, buildings, etc.) embedded with electronics, software, sensors & 
network connectivity that are designed to collect & exchange data. 
Enterprises are adopting the IoT for the benefits offered, e.g., operations 
optimization, cost reduction & improved efficiency. 
IoT development & adoption are driven by multiple factors, including 
readily available low-cost sensors, increased bandwidth & processing 
power, wide-spread usage of smartphones, availability of big data analysis 
tools & scalability of IPV6.  Organizations are also focusing on external 
benefits, such as: generating revenues from IoT-enabled products, 
services, & customer experiences. The diagram to the left illustrates the 
interconnection among various subsystems in a typical IoT application. 
Unique Characteristics & Requirements of IoT Systems
Quality Assurance (QA) organizations are wise to view software testing 
beyond devices & sensors.  Huge volumes of data generated across a 
smart ecosystem add great technical complexity, demanding a holistic 
approach. IoT applications have several unique factors:
• Combination of hardware, sensors, connectors, gateways, & application 
software in a single system
• Real-time streaming analytics / complex event processing
• Support for data volume, velocity, variety, & veracity
• Visualization of large-scale data
Testing Challenges for IoT Applications
Primary challenges:
• Dynamic environment: Unlike application testing in a defined 
environment, IoT solutions have a very dynamic environment with 
millions of sensors & different devices used in conjunction with intelligent 
software.
• Real-time complexity: IoT applications have multiple, real-time scenarios 
& its use cases are extremely complex.
• Scalability of system: Creating a test environment to assess functionality, 
scalability & reliability is challenging.
Other operational challenges:
• Related subsystems & components owned by third-party units
• Complex set of uses cases to create test cases & data
• Hardware quality & accuracy
• Security & privacy issues
• Safety concerns
Addressing IoT Application Testing Complexities
A comprehensive test plan strategy requires various types of testing, test 
lab setup, tools & simulators to be deployed.Considering the difficulties in 
generating big data from the thing in a testing environment, it is crucial to 
evaluate data simulation & virtualization techniques. Stubs can be 
considered as options during early stages while data recorders can serve 
as alternatives at later stages. Beyond test planning & data simulation, 
metrics-driven, exhaustive test execution is performed to achieve a stable 
system. QA organizations can split IoT test areas into the two layers 
described below.
The Device Interaction Layer
This layer is where the software & the hardware components of a real-
time IoT environment interact. One typical example is a BLE device 
transmitting real-time data to a mobile device app. There is often a lot of 
interaction testing occurring on the functional side of QA. However, other 
types of testing could also be required in addition to typical software 
testing:
• Conformance with standards: These are mostly device performance 
attributes specific to devices & sensors. These attributes must be 
validated against the standards of the device & its communications 
protocol. Hardware vendors perform most of these tests, but there could 
be certain domain or use-case requirements for an environment that was 
not tested.
• Interoperability: The ability of different devices to support the required 
functionality among themselves, other external devices & 
implementations.
• Security: With billions of sensors being deployed, it’s crucial to tackle 
data privacy & security concerns across the IoT ecosystem. The following 
are the different types of security testing requirements:
• Identity & authentication
• Data protection
• Data encryption
• Storage data security in local machines & in the cloud
The User Interaction Layer
This layer is the touch point between the thing & user. The success of the 
overall system depends on user receiving a seamless experience. Key tests 
in this layer include:
• Network capability & device level tests: The specific aspects of network 
communication & connectivity are validated by simulating different 
network modes, as well as device level validation.
• Usability & user experience: Usability & user experience are important 
in terms of the real-time usability; it involves human / machine interaction 
& the real-time experience the IoT system provides in a specific 
interaction.
• The IoT services & back-end IoT environment: While integration testing 
of the interfaces is key, there is a complex data layer that comes into play.
Creating a QA environment to enable validation of such an interface 
means addressing the growing data volume challenges of the IoT 
deployment. The front-end validation environment can be realized by 
assembling data recorders & simulators. The service & data layer 
validations will involve complex simulation services, such as the 
generation of millions of sensor hits, machine learning algorithms & the 
ability to generate time-boxed traffic.
There are methods to create such an ecosystem, e.g., leveraging 
sandboxes of development services or creating mock environments using 
virtualization tools. However, numerous implementation synergies are 
required to establish a working set of environments for a thorough 
services & back-end validation platform.
About Gadgeon
Gadgeon is known for its expertise in Industrial IoT and engineering 
excellence. We connect devices, operations, and processes to create 
business value, and revolutionize enterprises with the power of data. As 
an end-to-end technology services company, we successfully enabled the 
digital journey of customers with critical digital services ranging from 
embedded systems, cloud app development, mobile app development, 
data & analytics, application modernization, emerging technology based 
solutions, and testing & test automation across the industries such as 
connected factory, telecom & datacom, digital healthcare, CSPs, and 
home & building automation.
Thank you for time in reading this article!