Assessing the quality of player tracking data generated using broadcast footage
In light of growing demand for more cost-efficient match analysis technologies, the launch of a new standard within the FIFA Quality Programme for Electronic Performance and Tracking Systems aims to contribute to the democratisation of football technologies.
This new standard, referred to as “Broadcast EPTS”, assesses systems capable of generating player performance data using only convential broadcast footage.
FIFA has been working on setting standards for Electronic Performance and Tracking Systems since 2015 and progressively introduced first the impact testing for wearable EPTS and then in 2018 the performance standard for commercially available EPTS to be assessed against a scientifically recognised gold standard (VICON).
As technology evolves, the FIFA Quality Programme needs to continuously adapt to the latest innovations. One such trend that has been amplified by the circumstances of the current global pandemic is the demand for player tracking data from broadcast footage – meaning no technology or technician is required on-site for data-generation. As a response to this need, FIFA has developed a bespoke test method to validate the accuracy of such systems to be able to provide guidance to coaching staff, scouting departments, clubs, competition organisers and viewers as to how good the output from these software products truly are.
The advantages and applications of Broadcast EPTS seem obvious: no on-site operations in stadiums, lower budget for data generation from existing resources as well as a potential for generating data from historic footage including limited to short clips. The influence of video quality parametres, stadium environment or missing players in zoomed shots has, however, not been quantified until now making statement on data validity speculative. The new FIFA Basic standard for Broadcast EPTS within the FIFA Quality Programme for EPTS has therefore been put in place to tackle this issue.
Testing of Broadcast EPTS to the FIFA Basic Standard
In order for a provider to be certified under the FIFA Quality Programme as FIFA Basic for Broadcast EPTS, they must follow the procedure outlined in the newly-established test requirements.
The testing consists of making a number of clips available to providers from which the player tracking data is extracted. The videos are from the commonly used world feed and a lateral tactical camera angle (typical of match analysis) tracking all outfield players and at least one goalkeeper at all times. The data provided will then be assessed for concurrent validity with VisionKit – a non-commercial tracking system that has itself been validated against VICON. While it is important to stress that concurrent validity is not a scientific gold standard, it does, however, allow for assessment of whether key metrics including distance and velocity are within a reasonable range of the baseline and, more importantly, does so at a fraction of the cost in-line with the objective of making the technology more broadly available. A scientific peer-reviewed paper on VisionKit is to be published shortly.
Since 18th January 2021, Broadcast EPTS providers have had the opportunity to participate in an ongoing test event in order to assess their systems. Technology providers have until the end of March 2021 to submit test applications with the first results anticipated for publication in June 2021. The next anticipated window for testing is in Q3 2021. In the future, FIFA intends to host these events on an annual basis with future iterations of the testing expected to include a wider range of camera angles.
Any provider wanting to test their system or receive updates about the ongoing test process should contact the FIFA Quality Programme team via firstname.lastname@example.org.