booktwo.org

The 2014 World Cup in Brazil marks the first time that goal-line technology has been used for FIFA’s international tournament and with that in mind National Football Museum commissioned me to give an overview of where these systems are coming from, and where they might be going. This essay was originally published at The Commentary Project.

When Google unveiled its long-awaited wearable computer, Glass, in June of 2012, it did so through what might be called a lifestyle montage, a series of extreme sports events performed by “some of the world’s top athletes”. The Glass-wearers first skydived out of an aeroplane, then took to mountain bikes to manoeuvre through the conference centre onto the event stage. Throughout, what was streamed to viewers in the auditorium and watching online was not footage of the athletes undertaking the events, but what the athletes themselves were seeing, their point of view. What is spectacular about Glass, despite its real power as a connected, networked object, and what almost all discussion of it concentrates on, is its camera, the ability to see from another’s viewpoint, and everything this reveals.

While miniature high-definition cameras such as the GoPro Hero – particularly popular with the extreme sports community, bracketed to helmets, handlebars and snowboards – have made POV shots possible for a while, there’s something about Glass’ head-mounted position which appeals, as if it were not a camera, but the eye itself. Basketball teams have been early adopters of Glass. The Sacramento Kings, Indiana Pacers and Orlando Magic have all used Glass to enhance the “fan experience” by sharing headsets between announcers, resting players, support staff and coaches, and relaying the feed to giant screens above the action – but the NBA has yet to permit the use of Glass on court during play itself.

Another function of these cameras, aside from both the focus on, and the freedom from, a human-centred viewpoint, is that they transform our idea not only of vision, but also of memory. Many wearable cameras are marketed as such, like ‘Memoto’ (now rebranded as ‘Narrative’) and ‘Autographer’, small wearable digital cameras clipped to the chest or hung round the neck which photograph the owner’s viewpoint almost continually, building a continuous stream of images and data called a ‘lifelog’. The promise of such devices is total recall (“we can remember it for you wholesale”) – a promise, of course, which is always illusory.

A more rugged version of Glass, Broadcast Sports Inc’s head-mounted Ref Cam, has been deployed by Major League Soccer in the US. While it looks like a massive pain to wear, its wide-angle lens, bobbing with the referee’s pace, seems to open up a correspondingly wide field of view from the centre of the field, which feels liberating after the surveillance-like gaze of stand-mounted cameras. This is in stark contrast to Sky Sports’ version in the UK, whose chest-mounted ref cam was derided by former England hooker Brian Moore, writing in the Telegraph that “apart from nausea nothing was added to the viewing experience and the only previously unseen footage was that of the sky or the top of the scrum.” Moore’s real point, however, was that there was a fundamental flaw in the idea that such cameras could capture more of the “truth” of the game unfolding in front of them: “What is seen on camera is not a true rendering of what is actually perceived in a stressful moment.” Moore even cited a 2010 science paper on the use of wearable cameras by the Hillsboro, Oregon, Police Department, which found that even when every moment of a police investigation was recorded digitally, this evidence still had the potential to mislead officers, the judiciary and juries because of “the lack of understanding of important factors like the field of view, focus of attention and interpretation” – which sounds, too, like the advice of a particularly astute sports coach.

Indeed, much debate which happens in sports commentary around technologies of vision and adjudication would not sound out of place in academic journals of both the sciences and the humanities. Sporting fields have become the testing grounds for these technologies, providing as they do enclosed, hermetic fields of view, and strictly managed rules of movement and engagement – the kind of laboratory only dreamt of by scientists developing surveillance and monitoring platforms for military and urban situations, their most common applications.

When the English and Australian cricket teams faced each other in the Ashes series of the Summer of 2013, much of the commentary-box discussion focussed on the use of the new Umpire Decision Review System (DRS), a suite of technologies which assist – or rather, overrule – the umpire adjudicating some of sports greatest unknowables, the LBW, and the snick. Of course, these technologies, intended to increase accuracy, only inflamed controversy as their own accuracy was questioned as much as the human umpires. LBW is, after all, an epistemological problem – the question of whether a ball which strikes the batsman would have struck the wicket were the batsman not there is a question for Plato, not for machines. Nevertheless, cameras and sensors descended from military targetting systems are trained on the wicket in order to determine the best possible answer, and the algorithms which make up this situation determine the outcome of games – much to the frustration of many players and spectators. As Test Match Special commentator Jonathan Agnew, echoing Moore, noted: “The problem with the introduction of technology is the expectation of 100% accuracy” – a simple observation, but one which cuts to the heart of applying such supposedly rigorous approaches to sport, a fundamentally human endeavour which thrives on close calls, points of view, and, ultimately, chance.

Digital cameras are always more than cameras: they do not just make images, they ‘see’ and process them. Every connected digital-imaging system is also a computer, observing and making decisions about what it sees. This distinction is particularly well illustrated by “freeD” technology, which stands for Free Dimensional Video, a proprietary imaging system which debuted at Yankee Stadium in the 2013 Baseball season. Video feeds from multiple 12-megapixel cameras around the ground are combined within a dedicated server to produce a three-dimensional “scene”, through which the director can manoeuvre a virtual camera to produce unlimited, even “impossible” points of view. The entirety of the game world is simultaneously captured and re-viewed as a simulation. As with Eadweard Muybridge’s development of high-speed photography, which first allowed us to perceive a galloping racehorse with all four feet off the ground, the augmentation of the eye with technological systems allows us to see sport in new ways.

This total release of the visible “point of view” from any “human” viewpoint corresponds to the perceived freeing of decision-making from human error. This summer, the Football World Cup will for the first time utilise goal-line technology to determine whether the ball has crossed the line. Several different systems competed for selection, including the version of Hawk Eye used in Premier League, but the one which was chosen is a German surveillance system called GoalControl 4D. A total of fourteen cameras mounted on the stadium roof capture the three-dimensional position of the ball to within a few millimetres, enabling not only accurate decisions about goal-line crossings to be immediately relayed to referees via wrist-mounted “smart watches”, but also stored, replayed, and endlessly reanalysed. As in Cricket, the deployment of such decision systems has been much debated and often opposed, most volubly by FIFA president Sepp Blatter who has stated that “Other sports regularly change the laws of the game to react to the new technology. … We don’t do it and this makes the fascination and the popularity of football”. In this too we hear the deeply felt but ultimately fruitless appeal to the idea of sport as a fundamentally human endeavour, not at risk from, but essentially composed of, human frailty and fallibility.

Fruitless, of course, because we apparently find ourselves incapable of resisting the technological promise of an ever greater, ever more incremental, approach to some impossible “truth”, a Zeno’s arrow fired by a linesman towards the centre of the field. In sport, this truth-of-outcome is inextricably linked to the truth-of-performance, wherein one competitor, one side, ‘deserves’ to win because they are better prepared, better trained, better deployed, better equipped with what, in military circles, is referred to as “battlefield awareness”, leading to “full spectrum dominance of the battlespace”. As such, the same technologies of surveillance and appraisal are applied not just to decision-making during play, but also to data-gathering for post-game analysis by commentators and coaches alike.

From the beginning of the 2011-12 season, the NBA started installing STATS LLC SportVU cameras in basketball arenas, a technology expected to be mandated by the end of 2014 (again, like the application of surveillance systems in civilian environments, what begin as experiments in technology are ultimately and almost always codified in law). The SportVU system consists, like GoalControl, of a network of cameras around the arena connected to a data-processing system which tracks not only the ball in play, but individually identifiable players as well. The system’s sophisticated algorithms are capable of determining not only positioning, but through situational analysis, the events produced by and at these locations, such as dribbles, passes, touches and shots. The data is used to produce official NBA stats for every game, but also – for teams which pay a subscription of around $100,000 per year – to analyse and determine optimal strategy for the players themselves.

In March 2013, the Grantland blog revealed that the Toronto Raptors were using the data to – among many other things – “build computerized “ghost defenders” that reacted in optimal ways to every offensive action. The team could then overlay camera recordings of actual game play to see how closely Toronto’s real players mirrored the actions of their ghosts.” Once again, the real-world action on the court is filtered and replayed through simulations in order to re-direct the action back in meatspace. This is what is really driving the adoption of seeing systems in sport: truth-of-outcome not only produced by truth-of-performance, but, through technological feedback, fine tuning that performance as well.

To see where such systems might go in the future, we need only look to their current limitations – and the parallels of and responses to those limitations in other, surveillance-saturated spheres. Cairos Systems was another German-based bidder for the World Cup goal-line job, using a system called GLT which embedded magnetic sensors into the frame of the goal, and into the ball itself (of course, Cairos also has a system, called VIS.TRACK, which tracks player performance data through a network of cameras). On their website, Cairos writes – unusually explicitly for a technology company, that “In football, there are many decisions and scenes that may be discussed controversially. In the end the truth often lies in the eye of the beholder. Penalty or not? Red card or a dive? Active or passive offside?” It goes on to state that whether or not the goal line is crossed is one decision which “is clearly defined by the rules and does not leave space for interpretation. The question whether or not a goal has been scored can be decided without any doubts due to the rules.” As such, this particular decision is particularly amenable to technological intervention.

The counter-examples given here – penalties, red cards, offside – are instructive, because they fit far better what Brian Moore called the “focus of attention and interpretation”, the contentious, context-is-everything moments of sporting contact. Once again, we are in the domain not of observation, but of inference. The claim is now being made, however, that many automated, intelligent surveillance systems cannot only determine what happened, they can infer intent; they not only look back, but forward.

A study by the universities of Bradford and Aberystwyth in conjunction with the UK Border Agency in 2011 used video cameras and high-definition thermal imagery – essentially the same technology used to detect the “hot spot” in cricket – alongside yet more algorithms to determine whether study participants were lying. Telling signals such as eye movement, dilated pupils and nose wrinkling are visible to the human/video eye, but thermal imagery also reveals subconscious swelling of the blood vessels around the eye, a sign of distress or fear which signals that untruths may be in play. The UKBA stressed, as they prepared to install such systems at UK airports in the summer of 2012, that the technology is only useful when paired with an experienced human judge – and by that logic, why should it not be deployed alongside positioning cameras in the stands of the World Cup, to assist referees in determining which of two participants in a contested foul is telling the truth?

Why, after all, should we wait for fouls and other offences, such as offside, to be committed? It should be just as easy to award penalties and free kicks on the basis of players’ intent, and would be much safer and fairer for all involved. Recently, the US Department of Homeland Security’s “Project Hostile Intent”, for example, secured funding for a host of technologies claiming to predict crime based on “suspicious” behaviour. One of them, another camera-server assemblage produced by BRS Labs, uses “a range of in-built parameters of what is ‘normal’, [and] can track up to 150 people at a time to build up a “memory” of suspicious behaviour to begin determining what is inappropriate.” They are currently being installed in more than 300 locations in San Francisco, with strong expressions of interest from other global cities – although, as yet, none from FIFA. But if such systems can be trusted to protect our lives and livelihoods from the threat of terrorist attack, then surely they can be trusted to prevent another Hand of God?

Blatter, Moore, and Agnew, are all, surprisingly or not, for better or worse, at the forefront of a debate which extends far beyond the playing field. When we see sport through the eyes of the machines, we fundamentally change the nature of sport – and reveal, too, the extent to which the rest of society is reformed by our drive to visualise and reframe it with these technologies. As in so many ways, sport itself becomes the lens through which we understand ourselves.