How it works

Challenges of testing tv and middleware apps

Problems

  • Test results are not clear

    Meaning that when a test fails the source of the error is not easily identified; this forces the developer to debug (when this possibility is available) to find the reason of the test failure

  • Test coverage

    IPTV applications are often large and take a lot of time to cover with testing. In addition it is easy to break something elsewhere while fixing issues.

  • Limited resources

    Middleware devices often lack resources such as memory, thus discovering memory leaks is essential for best performance.

  • Accuracy and efficiency

    Manual testing is time-consuming and not 100% accurate whereas various video (OCR) based tools require setup, training and are oftentimes very sensitive to UI changes.

Solution

Simote is easy test automation of infrared devices & applications that enables regression, performance and stability testing of TV applications, platforms and hardware.

Simote is able to record infrared based usage scenarios and playback them on N devices (simulating an end user). During playbacks Simote captures defined technical regression, performance and stability related key performance indicators.

Product specs

  • Sends IR signals to a TV or set-top box
  • Test multiple parallel test devices
  • Add multiple recordings to a queue in a specific order that can be played during a test for a chosen time period
  • Monitor the status of devices tested via Simote Portal web interface
  • Export test results in PDF format
  • Integrate Simote with external systems via scriptable API

How it works

Issues captured by simote

Device restarts/crashes

  • State transitions that led to restart
  • Timestamp
  • Associated device

Application errors

  • State transitions that led to restart
  • Timestamp
  • Associated device
  • Error message - associated file and line number

Application performance metrics

  • Minimum, Maximum, Average and count of GUI view's covered by test run

Anything else through scriptable API