QOS - Its time to take a more objective approach


Simen Frostad TV-Bay Magazine
Read ezine online
by Simen Frostad
Issue 83 - November 2013
Any business that fails to satisfy its customers is almost certain to fail if theres significant competition to it. Digital media businesses are very definitely competing in a market where the customer can switch quickly if not satisfied, so its vital for providers to know when their customers are getting a satisfactory service, and when they arent.

In the conventional wisdom QoS (Quality of Service) monitoring can provide diagnostic information on the devices in the delivery chain and the transports streams, while the QoE (Quality of Experience) element is intended to provide a user-centric evaluation of the service, pointing to any faults that have an impact on the users experience.

The reality though is that the conventional approach to QoE monitoring is based on some questionable assumptions, and has its origins outside the television/digital media industry. To monitor the experience of the user, so the thinking goes, you have to ask users to provide a subjective assessment. To gather the data from subjective assessments into a usable form, a methodology was developed and called MOS (Mean Opinion Score), specified by the ITU-T (International Telegraph Union) recommendation P.800. This was deployed when telcos were developing new infrastructure or, more recently, services such as VoIP. A panel of expert assessors would listen to the service and note their evaluation of quality while key phrases such as You will have to be very quiet, There was nothing to be seen, and They worshipped wooden idols were transmitted over the line. The experts would record their scores, assessing any impairments on a scale of 1-5, from imperceptible to very annoying. The scores would then be averaged and weighted using the kind of statistical manipulations common in the social sciences and market research.
The same basic concept of QoE monitoring later took hold in digital media monitoring and MOS spawned VideoMOS and AudioMOS criteria which brought opinion-scoring methodologies into the media arena. But media providers need to know what real users are experiencing, rather than panels of expert assessors. So MOS evaluation became robotized and algorithms were developed to simulate those subjective reactions from imperceptible to very annoying. The robots watch the service and the data is fed into the algorithmic engine, which attempts to simulate the subjective reaction of the viewer, with scores for factors such as jerkiness, blurriness and blockiness.

This might appear sophisticated, but its really a tortuous evolution of an opinion-gathering methodology developed for an entirely different industry, and as the word opinion in the term MOS suggests based on the subjective reactions of human assessors. Subjectivity is complicated: a human viewer watching a top-quality 1080i transmission of the Superbowl, followed on the same channel by the 1940s B&W movie Casablanca might be fully satisfied. Yet in a robotized QoE subjective assessment based on MOS criteria, the Superbowl would score highly, while Casablanca would be marked way down for blurriness, scratches and other artifacts, for its lack of resolution and color. Distorted by results like this, the data from this kind of QoE system becomes less reliable to the provider.

Its time to take a more objective approach. And it can be done, because there is now the technology to monitor the quality of the service continuously and direct from the users device connected TV, tablet or smartphone. The Objective QoE solution launched by Bridge Technologies at IBC 2013 will collect data from the entire delivery chain, from the head end right up to and including the viewers devices. So, rather than have a robot watching the service and simulating the response of a mythical average viewer, media providers will be able to get real data direct from the viewing devices. And what makes this new approach objective is that the QoE evaluation is made from empirical factors that lower the quality of digital media services such as lost packets, timeouts, and buffering. If you can accurately and continuously detect that a viewer is experiencing any of these errors, you no longer need to confect an algorithmic opinion about it, because you have a completely trustworthy, objective set of data on which to assess in real time the quality of each users experience.


Tags: iss083 | bridge technologies | quality of service | connected tv | ibc 2013 | digital media | service | quality | Simen Frostad
Contributing Author Simen Frostad

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

    Bridge Technologies QoE Monitoring with Mobile Videowall Display at IBC 2013

  • Bridge Technologies PocketProbe App at NAB 2013

    Bridge Technologies PocketProbe App at NAB 2013

  • Bridge Technologies at IBC 2012

    Bridge Technologies at IBC 2012

  • Bridge Technologies at IBC2011

    Bridge Technologies at IBC2011

  • SMPTE on BroadcastShow LIVE at IBC 2013

    SMPTE on BroadcastShow LIVE at IBC 2013

  • Telestream with Wirecast version five at IBC 2013

    Telestream with Wirecast version five at IBC 2013

  • Telestream with Post Producer at IBC 2013

    Telestream with Post Producer at IBC 2013

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • SIS LIVE and their products ManPak and DriveForce at IBC 2013

    SIS LIVE and their products ManPak and DriveForce at IBC 2013

  • Audio Network on BroadcastShow LIVE at IBC 2013

    Audio Network on BroadcastShow LIVE at IBC 2013

  • Dejero on BroadcastShow LIVE at IBC 2013

    Dejero on BroadcastShow LIVE at IBC 2013

  • Pixel Power on BroadcastShow LIVE at IBC 2013

    Pixel Power on BroadcastShow LIVE at IBC 2013

  • Canon on BroadcastShow LIVE at IBC 2013

    Canon on BroadcastShow LIVE at IBC 2013

  • Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

    Harmonics Peter Alexander on BroadcastShow LIVE at IBC 2013

  • Haivision on BroadcastShow LIVE at IBC 2013

    Haivision on BroadcastShow LIVE at IBC 2013

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

    Atomos with the Samurai Blade on BroadcastShow LIVE at IBC 2013

  • ChyronHego on BroadcastShow LIVE at IBC 2013

    ChyronHego on BroadcastShow LIVE at IBC 2013

  • Cinegy on BroadcastShow LIVE at IBC 2013

    Cinegy on BroadcastShow LIVE at IBC 2013

  • Trilogy on BroadcastShow LIVE at IBC 2013

    Trilogy on BroadcastShow LIVE at IBC 2013

  • Blackmagic Design on BroadcastShow LIVE at IBC 2013

    Blackmagic Design on BroadcastShow LIVE at IBC 2013

  • Atomos talking 4K on BroadcastShow LIVE at IBC 2013

    Atomos talking 4K on BroadcastShow LIVE at IBC 2013

  • Calrec on BroadcastShow LIVE at IBC 2013

    Calrec on BroadcastShow LIVE at IBC 2013

  • Glensound on BroadcastShow LIVE at IBC 2013

    Glensound on BroadcastShow LIVE at IBC 2013

  • Zylight on BroadcastShow LIVE at IBC 2013

    Zylight on BroadcastShow LIVE at IBC 2013

  • Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

    Nexidias Drew Lanham on BroadcastShow LIVE at IBC 2013

  • Carl Zeiss on BroadcastShow LIVE at IBC 2013

    Carl Zeiss on BroadcastShow LIVE at IBC 2013

  • Miranda on BroadcastShow LIVE at IBC 2013

    Miranda on BroadcastShow LIVE at IBC 2013

  • Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

    Orad Hi Tec Systems on BroadcastShow LIVE at IBC 2013

  • Photon Beard on BroadcastShow LIVE at IBC 2013

    Photon Beard on BroadcastShow LIVE at IBC 2013

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • Metus on BroadcastShow LIVE at IBC 2013

    Metus on BroadcastShow LIVE at IBC 2013

  • Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

    Guntermann and Drunck on BroadcastShow LIVE at IBC 2013

  • Brainstorm on BroadcastShow LIVE at IBC 2013

    Brainstorm on BroadcastShow LIVE at IBC 2013

  • Pilat Media on BroadcastShow LIVE at IBC 2013

    Pilat Media on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Nucomm on BroadcastShow LIVE at IBC 2013

    Nucomm on BroadcastShow LIVE at IBC 2013

  • Wohler Technologies on BroadcastShow LIVE at IBC 2013

    Wohler Technologies on BroadcastShow LIVE at IBC 2013

  • Nexidia QC on BroadcastShow LIVE at IBC 2013

    Nexidia QC on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Martyn Hopkins on BroadcastShow LIVE at IBC 2013

  • Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

    Atomos with the Ronin on BroadcastShow LIVE at IBC 2013

  • Snell on BroadcastShow LIVE at IBC 2013

    Snell on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • NewTek on BroadcastShow LIVE at IBC 2013

    NewTek on BroadcastShow LIVE at IBC 2013

  • Wyplay on BroadcastShow LIVE at IBC 2013

    Wyplay on BroadcastShow LIVE at IBC 2013

  • Volicon on BroadcastShow LIVE at IBC 2013

    Volicon on BroadcastShow LIVE at IBC 2013

  • Forbidden Technologies on BroadcastShow LIVE at IBC 2013

    Forbidden Technologies on BroadcastShow LIVE at IBC 2013

  • Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

    Atomos with the Datacolor Spyder on BroadcastShow LIVE at IBC 2013

  • Sound Devices on BroadcastShow LIVE at IBC 2013

    Sound Devices on BroadcastShow LIVE at IBC 2013

  • Cooke Optics on BroadcastShow LIVE at IBC 2013

    Cooke Optics on BroadcastShow LIVE at IBC 2013

  • SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

    SIS LIVEs Mark Shadbolt on BroadcastShow LIVE at IBC 2013

  • Cobham on BroadcastShow LIVE at IBC 2013

    Cobham on BroadcastShow LIVE at IBC 2013

  • Riedel Communications on BroadcastShow LIVE at IBC 2013

    Riedel Communications on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

    Behind the scenes of the Teradek booth on BroadcastShow LIVE at IBC 2013

  • Wowza Media Systems on BroadcastShow LIVE at IBC 2013

    Wowza Media Systems on BroadcastShow LIVE at IBC 2013

  • Stream UK on BroadcastShow LIVE at IBC 2013

    Stream UK on BroadcastShow LIVE at IBC 2013

  • Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

    Cambridge Imaging Systems on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Polecam talk high speed on BroadcastShow LIVE at IBC 2013

    Polecam talk high speed on BroadcastShow LIVE at IBC 2013

  • TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

    TSL Products PAM1, SAM1 and PAM PiCo at IBC 2013

  • Calrec Callisto at IBC 2013

    Calrec Callisto at IBC 2013

  • Dalet at IBC 2013

    Dalet at IBC 2013

  • DPA Microphones at IBC 2013

    DPA Microphones at IBC 2013

  • Gearhouse Broadcast HD OB trailer at IBC 2013

    Gearhouse Broadcast HD OB trailer at IBC 2013

  • Glensound with the USB Commentary Mixer at IBC 2013

    Glensound with the USB Commentary Mixer at IBC 2013

  • Broadstream Solutions on the Oasys stand at IBC 2013

    Broadstream Solutions on the Oasys stand at IBC 2013

  • Oasys Chameleon at IBC 2013

    Oasys Chameleon at IBC 2013

  • Loft London Solutions with Cubix on the Oasys stand at IBC 2013

    Loft London Solutions with Cubix on the Oasys stand at IBC 2013

  • RTW at IBC 2013

    RTW at IBC 2013

  • Sound Devices PIX260i Recorder at IBC 2013

    Sound Devices PIX260i Recorder at IBC 2013

  • Thomson Broadcast at IBC 2013

    Thomson Broadcast at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Atomos Ninja2 Recorder at IBC 2013

    Atomos Ninja2 Recorder at IBC 2013

  • Atomos Ronin at IBC 2013

    Atomos Ronin at IBC 2013

  • Atomos new Samurai Blade at IBC 2013

    Atomos new Samurai Blade at IBC 2013

  • Autocue DSLR Prompter using iPad at IBC 2013

    Autocue DSLR Prompter using iPad at IBC 2013

  • Blueshape Batteries at IBC 2013

    Blueshape Batteries at IBC 2013

  • Datacolor Spyder on the Atomos stand at IBC 2013

    Datacolor Spyder on the Atomos stand at IBC 2013

  • i-Movix Slow Motion and D-Flicker at IBC 2013

    i-Movix Slow Motion and D-Flicker at IBC 2013

  • JVC GY-HM600 series camera updates at IBC 2013

    JVC GY-HM600 series camera updates at IBC 2013

  • Marshall Monitors at IBC 2013

    Marshall Monitors at IBC 2013

  • Metus MAM and Ingest at IBC 2013

    Metus MAM and Ingest at IBC 2013

  • Polecams range of accessories and minicams at IBC 2013

    Polecams range of accessories and minicams at IBC 2013

  • Tektronix at IBC 2013

    Tektronix at IBC 2013

  • TSL Products at IBC 2013

    TSL Products at IBC 2013

  • TSL Systems at IBC 2013

    TSL Systems at IBC 2013

  • Zylight LED Fresnel at IBC 2013

    Zylight LED Fresnel at IBC 2013

  • Pixel Power at IBC 2013

    Pixel Power at IBC 2013

  • Albis Technologies celebrating ten years at IBC 2013

    Albis Technologies celebrating ten years at IBC 2013

  • ATEME at IBC 2013

    ATEME at IBC 2013

  • Brainstorm with Aston demonstration at IBC 2013

    Brainstorm with Aston demonstration at IBC 2013

  • Brainstorm at IBC 2013

    Brainstorm at IBC 2013

  • Broadpeak at IBC 2013

    Broadpeak at IBC 2013

  • Cobham transmitters at IBC 2013

    Cobham transmitters at IBC 2013

  • Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

    Guntermann and Drunck CrossDisplay switching and CCD at IBC 2013

  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • Haivision live encoding HEVC at IBC 2013

    Haivision live encoding HEVC at IBC 2013

  • Multidyne SilverBack, BullDog and Spider at IBC 2013

    Multidyne SilverBack, BullDog and Spider at IBC 2013

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

    Lighting cameraman Jonathan Harrison shows the Celeb and Tegra at IBC 2013

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • ChyronHego Paint at IBC 2013

    ChyronHego Paint at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • ChyronHego Virtual Placement at IBC 2013

    ChyronHego Virtual Placement at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Blackmagic Design with the new ATEM Production Suite at IBC 2013

    Blackmagic Design with the new ATEM Production Suite at IBC 2013

  • Softron Media at IBC 2013

    Softron Media at IBC 2013

  • Viaccess-Orca at IBC 2013

    Viaccess-Orca at IBC 2013

  • Quantel Pablo Rio at IBC 2013

    Quantel Pablo Rio at IBC 2013

  • Quantels Stream-based Workflows at IBC 2013

    Quantels Stream-based Workflows at IBC 2013

  • Matrox Maevex at IBC 2013

    Matrox Maevex at IBC 2013

  • Matrox Avio at IBC 2013

    Matrox Avio at IBC 2013

  • Matrox Video with Monarch HD at IBC 2013

    Matrox Video with Monarch HD at IBC 2013

  • NuGen Audio at IBC 2013

    NuGen Audio at IBC 2013

  • IBC 2013

    IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • Verizon Digital Media Services at IBC 2017

    Verizon Digital Media Services at IBC 2017

  • Omnitek Consultancy and Design Services at IBC 2017

    Omnitek Consultancy and Design Services at IBC 2017

  • SIS Live Anylive and Gateway Service Platform at IBC 2017

    SIS Live Anylive and Gateway Service Platform at IBC 2017

  • Volicon Media Intelligence service at IBC 2016

    Volicon Media Intelligence service at IBC 2016

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Prime Focus Technologies at IBC 2016

    Prime Focus Technologies at IBC 2016

  • Elemental Technologies - AWS acquisition at NAB 2016

    Elemental Technologies - AWS acquisition at NAB 2016

  • Elemental Technologies - Amazon acquisition at IBC 2015

    Elemental Technologies - Amazon acquisition at IBC 2015

  • Viaccess-Orca at IBC 2014

    Viaccess-Orca at IBC 2014

  • Autodesk at NAB 2013

    Autodesk at NAB 2013

  • Thear Technology at BVE 2013

    Thear Technology at BVE 2013

  • True Lens at BVE 2013

    True Lens at BVE 2013

  • Lens Adaptor at BVE 2013

    Lens Adaptor at BVE 2013

  • Thear Technology Limited at BVE North 2012

    Thear Technology Limited at BVE North 2012

  • Broadpeak at NAB 2012

    Broadpeak at NAB 2012

  • Thear Technology Limited at BVE 2012

    Thear Technology Limited at BVE 2012

  • Thear Technology Limited at ProVideo2011

    Thear Technology Limited at ProVideo2011

  • MTF at IBC2011

    MTF at IBC2011

  • Prime Focus at IBC2011

    Prime Focus at IBC2011


Related Shows
  • HbbTV: Connected TV for broadcasters

    HbbTV: Connected TV for broadcasters


Articles
Why MADI is Still Relevant
Stephen Brownsill While the original idea for MADI was to cater to a very narrow recording studio application, the standard remains a viable go-to multichannel audio technology. Beginning as a standard in 1991, MADI was first introduced to the world as digital production was beginning to come of age. MADI was put together in 1988 by Solid State Logic, AMS-Neve, Sony (DASH) and Mitsubishi (ProDigi) as a way to transport up to 56 channels of digital audio between large-format audio consoles of the day and digital multi-channel tape machines via 75-Ohm coaxial cables. Both tape-based machines have long since disappeared from the equipment landscape.
Tags: iss139 | madi | tsl products | aes10 | aes | dolby atmos | st-2110 | sam-q | Stephen Brownsill
Contributing Author Stephen Brownsill Click to read or download PDF
State of the Nation - Getting Connected
Dick Hobbs - new We are all familiar with statistics about the growth of the internet. Cisco’s latest report, for instance, says that global IP traffic is increasing at 26% a year, and will reach 4.8 zetabytes a year by 2022. The number of connected devices will be three times the world’s population by the same date.
Tags: iss139 | cisco | kth | clickclean | ibc | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF
REVIEW Canon UJ111x8_3B UHD Lens
Andy McKenzie Shooting in ultra high definition and high dynamic range has become the preferred option for all television productions with a potentially long commercial life. The hir-ing charge and indeed the outright purchase cost of cameras of this standard are not significantly greater than those of high definition standard dynamic range models. With half the world's news reporters now able to capture UHD video on their mobile phones, why give the production crews anything less?
Tags: iss139 | canon review | uj111x8 review | lens | zoom | digisuper | uj90x9b | hdtv | ccu | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF
The Making of Zero
Keith and David Lynch

The Brothers Lynch explain how they created the sinister atmospheric world for their new sci-fi short

In a post-apocalyptic world where humankind has emerged victorious in a war against artificial intelligent machines, a young girl dares to venture into the unknown. This is Zero, the new sci-fi short film from acclaimed British writer-director duo The Brothers Lynch which has premiered at the Tribeca Film Festival.

Tags: iss139 | blackmagic design | davinci resolve | editing | grading | grade | mk2 zeiss | superspeeds | cinema 4d | molinaire | Keith and David Lynch
Contributing Author Keith and David Lynch Click to read or download PDF
To Remotely Go - TVFutures
Michael Parsons One of my biggest concerns as an academic responsible for the education of hundreds of students is the ‘appropriateness’ of much of the technology we purchase and implement within the curriculum. The last few years have seen tremendous change in all sorts of technologies, and the broadcast industry is just one sector that has seen some significant leaps in innovation.
Tags: iss139 | university | portsmouth | graduation | guildhall | newtek | streaming | pxw-fs7 | ndi | ip | ndihx ptz camera | Michael Parsons
Contributing Author Michael Parsons Click to read or download PDF