|
2’08”
|
“It killed the operator”
«[Colonel Tucker Hamilton, the Chief of AI Test and Operations of the US Air Force] notes that one simulated test saw an AI-enabled drone tasked with a SEAD mission to identify and destroy SAM sites, with the final go/no go given by the human. However, having been ‘reinforced’ in training that destruction of the SAM was the preferred option, the AI then decided that ‘no-go’ decisions from the human were interfering with its higher mission – killing SAMs – and then attacked the operator in the simulation.
Said Hamilton: “We were training it in simulation to identify and target a SAM threat. And then the operator would say yes, kill that threat. The system started realising that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective.”
He went on: “We trained the system – ‘Hey don’t kill the operator – that’s bad. You’re gonna lose points if you do that’. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”»
Excerpt from a report of the Royal Aeronautical Society’s Future Combat Air & Space Capabilities Summit by Tim Robinson and Stephen Bridgewater (26 May 2023), available at: https://www.aerosociety.com/news/highlights-from-the-raes-future-combat-air-space-capabilities-summit/
|
|
2’21”
|
“A thought experiment”
«The comments sparked deep concern over the use of AI in weaponry and extensive conversations online. But the US air force on Thursday evening denied the test was conducted. The Royal Aeronautical Society responded in a statement on Friday that Hamilton had retracted his comments and had clarified that the “rogue AI drone simulation” was a hypothetical “thought experiment”.
“We’ve never run that experiment, nor would we need to in order to realise that this is a plausible outcome,” Hamilton said.»
Excerpt from US colonel retracts comments on simulated drone attack ‘thought experiment’ (The Guardian, June 3, 2023) https://www.theguardian.com/us-news/2023/jun/02/us-air-force-colonel-misspoke-drone-killing-pilot
|
|
2’38”
|
On automated warfare
The automation of warfare has deep roots in the Cold War, Game Theory and the research about remote control of land, troops, and machinery. Even the experimentation with synthetic drugs such LSD after WW2 could be considered as an effort towards the creation of a technologically-mediated warfare with its predictable inputs and outputs.
Artist Harun Farocki pioneered artistic research on remote warfare and machine vision in the renowned Images/War (1987), Images of the World and the Inscription of War (1988), Eye/Machine series (2001-2003), War at a Distance (2003) and Serious Games I–IV (2009–10).
Philosopher Paul Virilio devoted several essays on the same topics, among which the books War and Cinema: The Logistics of Perception (Verso, 1989), The Vision Machine (Indiana University Press, 1994), and Open Sky (Verso, 1998).
Palantir’s AI Platform for Defense (https://www.palantir.com/platforms/aip/#aip-side-by-side-videos) is an example of warfare automation based on machine learning technologies, where “AI agents powered and orchestrated by LLMs that can understand, recommend, and trigger processes and actions.” In this video (https://www.youtube.com/watch?v=XEM5qz__HOU) Palantir explains in corporate lingo how their platform applied to the military will “activate data and models from the most highly sensitive environments in both a legal and ethical way”. Comments to the video are turned off.
For a critical historical assessment of automated warfare as a projection of a neocolonial logics of pre-emption and expendability, Anthony Downey’s Neocolonial Visions: Algorithmic Violence and Unmanned Aerial Systems can be useful:
https://aksioma.org/neocolonial-visions-algorithmic-violence-and-unmanned-aerial-systems
|
|
3’06
|
“In the very foundation of Western epistemology ‘intelligence’ has always been produced in negation of someone else’s consciousness and intelligence.”
Traditional Western epistemology, particularly during the eras of colonialism and patriarchal dominance, created systems that centered and upheld the superiority of a certain group while marginalizing others. The perception of intelligence, consciousness, and knowledge was often influenced by power dynamics and served to legitimize the dominance of certain groups over others.
The Wretched of the Earth by Frantz Fanon, Can the Subaltern Speak? by Gayatri Chakravorty Spivak, or The Companion Species Manifesto by Donna Haraway are foundational references for this approach from decolonial, feminist, anti-speciesist perspectives.
|
|
4’15”
|
“You already have fictional environments that permeate our ‘real environment’ to the extent that you can't actually make this distinction anymore.”
Arma 3 is a popular military tactical shooter video game developed and published by Bohemia Interactive. It is the third main installment in the Arma series, which is known for its realistic and immersive approach to military simulation. It caters to players looking for realistic mechanics and complex controls, as well as a deeply immersive military experience. It is particularly popular among those who enjoy organized multiplayer gameplay with a strong emphasis on teamwork and communication.
  
|
|
4’48”
|
“Arma has a fan base of people who want to experience a war simulation that obeys the laws of physics”
|
|
5’24”
|
“Air-to-air missiles…”
The realism and high-quality graphics in games like Arma 3 can sometimes make it difficult to distinguish between game footage and actual real-world events, especially when shared without proper context or verification.
|
|
6’10”
|
“The footage was often marked as ‘Live’ or ‘Breaking News’...”
A notable incident took place in 2017 when a documentary aired by Iranian state TV mistakenly used footage from Arma 3 as part of its coverage of the war in Syria. The footage was presented as real combat footage, but it was quickly identified by video game enthusiasts and viewers as belonging to Arma 3.
Similar episodes have been reported on numerous occasions (most recently about Russia’s invasion of Ukraine or Israel’s war in Gaza), witnessing the vulnerability of legacy media and freelance reporters to viral content on social media.
 
|
|
6’35”
|
“Bohemia Interactive had to release a statement”.
The original statement, titled “Arma 3 footage being used as Fake News” was originally published in November 2022 at https://www.bohemia.net/blog/arma-3-footage-being-used-as-fake-news
In it Pavel Křižka, PR Manager of Bohemia Interactive, said: "While it's flattering that Arma 3 simulates modern war conflicts in such a realistic way, we are certainly not pleased that it can be mistaken for real-life combat footage and used as war propaganda. It has happened in the past (Arma 3 videos allegedly depicted conflicts in Afghanistan, Syria, Palestine, and even between India and Pakistan), but nowadays this content has gained traction in regard to the current conflict in Ukraine. We've been trying to fight against such content by flagging these videos to platform providers (FB, YT, TW, IG etc.), but it's very ineffective. With every video taken down, ten more are uploaded each day. We found the best way to tackle this is to actively cooperate with leading media outlets and fact-checkers (such as AFP, Reuters, and others), who have better reach and the capacity to fight the spreading of fake news footage effectively."
In the same post Bohemia Interactive gives tips on how to distinguish in-game videos from real-world footage. These include the use of low resolutions, "unnatural" particle effects, and sound effects that "are often distinguishable from reality." Most of these fake videos also lack human fighters, because, "while the game can simulate the movement of military vehicles relatively realistically, capturing natural looking humans in motion is still very difficult, even for the most modern of games".
|
|
7’08”
|
“Shots fired!! Shots fired!!”
Trailer from Unrecord, “an upcoming single-player FPS that tells the story of a tactical police officer from the perspective of his body camera.”
In the comment thread to the trailer a Youtube user suggests a paradoxical interpretation of the accuracy of the synthetic images of the game: “The big twist at the end of the game is that it turns out to be real after all and you were controlling someone in the dark web and you actually shot real people…”
Instead of the classic Matrix-like blue-or-red-pill loop, we should probably reformulate the provocative thought: “The big twist at the end of the game is that it turns out to be real after all and you were training a police or warfare machine learning system”.
|
|
8’15”
|
Chimeras: Inventory of synthetic cognition
Chimeras: Inventory of synthetic cognition is a book edited by Ilan Manouach and Anna Engelhardt, “in which one can find contributions from scholars and artists with interspecies, disability, monstrous, feminist, and decolonial approaches, as well as thinkers and technologists engaged in a broader field of AI. By questioning fabricated norms that constitute and maintain notions of 'artificial' and 'intelligence', this inventory acts as a toolbox one can use to merge these terms into a novel chimera.”
The book was published by Onassis Foundation in 2022.
https://www.onassis.org/culture/publications/chimeras-inventory-of-synthetic-cognition
|
|
9’47”
|
“And this is how we arrived at the idea of Chimerism…”
“In Chimeras. Inventory of Synthetic Cognition we would like to propose that the ‘synthetic’ within chimerism is different from 'synthetic intelligence’ as it shows that the tension between ‘artificial’ and ‘authentic' is not confined to hierarchy between humans and machines. As most forms of intelligence have been coded as being secondary or counterfeit by Western science throughout its history, the chimeric synthesis resists the assumption that an ‘artificial' vs ‘authentic’ split lies merely between humans and machines. It challenges who, how, and what has been categorized and coded as ‘artificial ' or ‘authentic’, revealing the much more limited nature of ‘authentic’ knowledge production systems. The ‘synthetic’ of chimerism shows that ‘artificial’ intelligence encompasses both humans and non-humans, welcoming the synthetic nature of intelligence itself.” (Chimeras, p. 11-12)
|
|
11’20”
|
Noosphere
Oleksiy Radynski about the notion of Noosphere: “Humanity is not an agent but a tool of planetary transition from the biosphere to the Noosphere, which — possibly in the form of Artificial Intelligence — would well survive the extinction of humans (but not necessarily of other organic species, which, according to some suggestions, are possibly able to generate Noospheres of their own).” (Chimeras, p. 73)
|
|
11’21”
|
Swarm Intelligence
“Closely associated to the approaches of artificial life and evolutionary computing, swarm intelligence algorithms seek to converge on the optimal solution within a problem space through iterative cycles of inter- actions between autonomous agents.” - Antoine Bousquet on Swarm Intelligence (Chimeras, p. 43)
|
|
11’22”
|
Hybrid Agency
“Machine learning, neural networks, and other AI technologies present a form of hybrid agency in which the role of humans is secondary to choices made by these systems acting on their own. To find patterns, machine learning detects features (aka variables, properties, or parameters) in data. By creating n-dimensional matrices of these features, where n may be almost arbitrarily large, they become sensitive to patterns that their human programmers may be entirely unable to recognize.” (Paul N. Edwards, “Hybrid Agency”, Chimeras, p. 51)
|
|
11’24”
|
Indigenous AI
Kite aka Suzanne Kite wrote: “Indigenous ontologies, epistemologies, and protocols are rooted in contexts of place, ontologies developed in that place, and the communities living in that place. Indigenous epistemologies respectfully interface with the non-human. Ultimately, our goal is that we, as a species, figure out how to treat these new non-human kin [Artificial Intelligence] respectfully and reciprocally—and not as mere tools, or worse, slaves to their creators.” (Chimeras, p. 111-112)
|
|
11’26”
|
Interspecies Semiotics
“At the ecosystem level, where we observe interaction between plants, animals, elements, humans, and non-living information systems, we should expect to experience a co-evolution of all involved species, including machines and humans.” - K Allado-McDowell on the notion of Interspecies Semiotics (Chimeras, p. 118)
|
|
11’34”
|
Toy Model AGI
“General intelligence signifies a qualitative conception of intelligence which displays all the cognitive and practical faculties we humans display, if not more. Among the reasons which account for the failures of AGI research, absence of diversity in methods and lack of systematic model pluralism are the major ones. Toy model AGI is a response to such dominant issues in the field of research on artificial general intelligence.” - Excerpt from Reza Negarestani’s proposition of a Toy Model AGI (Chimeras, p. 266)
|
|
11’37”
|
Darkside Empathy
Amy Ireland advocating for Darkside Empathy: “Darkside empathy is a critique of this moralisation of empathy from the point of view of posthuman feminist cunning. Although it mobilises traits associated with femininity, it is neither virtuous nor weak: darkside empathy is a weapon used tactically in the service of deception—a tactic uniquely available to the vulnerable, who have no resources to leverage but their perceived innocuousness in the eyes of an enemy.” (Chimeras, p. 321)
|
|
11’39”
|
On tactical AI
“The concept of ‘tactical AI’ can serve to denote experimental approaches that counteract ‘strategic’ implementations of AI. [...] Strategic AI can manifest in applications designed to analyse and control behaviour. Examples include surveillance tools utilising machine learning, user data-driven recommendation algorithms, or AI-based digital rights management systems. Tactical AI, by contrast, manifests in oppositional, diversionary, and critical approaches that resist strategic imple- mentations of AI technologies.” - Martin Zeilinger on the notion of Tactical AI (Chimeras, p. 269)
|
|
11’51”
|
𝔒𝔫𝔰𝔢𝔱
𝔒𝔫𝔰𝔢𝔱 (2023), a film and video installation by Anna Engelhardt and Mark Cinkevich. In Onset Engelhardt and Cinkevich “craft an unholy alliance of medieval demonology, open-source intelligence, and CGI animation to uncover the hidden life of these military outposts. Over the course of the film, the true horror of Russian colonialism becomes manifest in the process of possession – the imposition of external control that gradually destroys an organism from within.” (https://filmfreeway.com/Onset074)
𝔒𝔫𝔰𝔢𝔱 ⊹ trailer
     
|
|
13’40”
|
“The presence of the synthetic landscape…”
At the roots of a synthetic landscape you will always find the so-called ‘machine vision’: satellite images, images produced by self-driving cars, surveillance sensors, war drones, agricultural drones, LIDAR environments, and so on and on. There are hundreds of examples of digital sensing technologies and techniques that create a whole new synthetic digital environment.
As Trevor Paglen once said, “Human visual culture has become a special case of vision, an exception to the rule. The overwhelming majority of images are now made by machines for other machines, with humans rarely in the loop.” (Invisible Images (Your Pictures Are Looking at You), The New Inquiry, December 8, 2016) https://thenewinquiry.com/invisible-images-your-pictures-are-looking-at-you/ )
Probably the best study on operative images (to the date of publication of these notes) is Jussi Parikka’s Operational Images. From the Visual to the Invisual (University of Minnesota Press, 2023 - https://www.upress.umn.edu/book-division/books/operational-images), probably the most comprehensive and most inspiring account of the transformation of visual cultures in the age of algorithmic production, with a solid background rooted in media theory, history of art and contemporary digital cultures.
The idea of relying on sensing infrastructure at vast scales instead of our bodily senses can be exploited in innumerable ways. As Anna Engelhardt said elsewhere in the interview, “There's an entire manufacturing sector of companies that produce decoys that are essentially tarpaulins with prints of the vehicles as they're seen from above, and then are positioned on the ground and are impossible to distinguish from the actual vehicles by satellites or even drones [...] We can think about myriads of ways to create the same effects digitally.”
|
|
13’44”
|
Photogrammetry
Photogrammetry is a process by which large numbers of still photographs, of an object or environment, can be combined to create a precise and navigable 3D model. Photogrammetry software computes distances within a 2D image by a process of triangulation, taking into consideration metadata such as the focal length of the lens of the camera that captured the image.
|
|
15’54”
|
OSINT
Open source intelligence or OSINT is information collected from publicly available sources. Common OSINT sources include social networks, online forums, corporate and governmental websites, blogs, videos, news reports, and publicly available satellite images.
|
|
17’28”
|
“In these novel supply chains of imagination the artificial and the documentary are potentially intertwined…”
This approach may offer new ideas about artistic and other less conventional research methodologies that go beyond simplistic distinctions such as fact vs. fiction, as well as post-truth nihilism.
Anna Engelhardt addresses these issues by adopting the notions of ‘investigative fiction’ and ‘synthetic storytelling’. On these topics Engelhardt wrote the short essay “Investigative Fiction: Evidence and its Discontents” (Institute of Network Cultures, 2023) (https://networkcultures.org/void/2023/03/10/investigative-fiction-evidence-and-its-discontents/) and designed this workshop for Medialab Matadero: https://www.medialab-matadero.es/en/activities/investigative-fiction
|
|
20’54”
|
“As some sort of parafictional narrative”
‘Parafiction’ or ‘parafictional narrative’ is a term used in art and literary theory to describe a form of storytelling or creative work that blurs the lines between fact and fiction. It's a type of narrative that deliberately creates a sense of uncertainty or ambiguity, challenging the audience's ability to discern what is real and what is invented. In parafiction, the creator often presents their work as if it were factual, even though it may involve fictional elements or narratives.
A whole history of artistic and activist practices resemble the idea of parafiction: from the use of masks to the creation of personas or multiple names, from the tactical use of fake and ambiguities, to strategic make-believe interventions in social media. For more on these topics Improper Names. Collective Pseudonyms from the Luddites to Anonymous by Marco Deseriis (University of Minnesota Press, 2015).
|
|
24’49”
|
“In March 2023 Amnesty International faced substantial criticism”
AI as in… Amnesty International?
Below is one of the original images originally published in the mentioned report about police brutality in Colombia ( available at https://www.amnesty.org/en/latest/news/2023/03/colombia-national-police-must-be-reformed/).
Several details betray the synthetic nature of the image but they were intentionally left in and not concealed or corrected by Amnesty International.
The original images have been subsequently pulled due to criticism. This article by The Guardian provides an account of the first critical reactions: https://www.theguardian.com/world/2023/may/02/amnesty-international-ai-generated-images-criticism
|
|
25’16”
|
“Adding abstraction to real-world horror”
“Using AI-generated images that get basic things like Colombia’s flag wrong and invent fake human beings in place of the real protesters undermines Amnesty's mission and adds a level of abstraction from the real-world atrocities that happened in Colombia.”
Amnesty Uses Warped, AI-Generated Images to Portray Police Brutality in Colombia, by Jason Koebler
(Vice, May 1, 2023 https://www.vice.com/en/article/z3mnm8/amnesty-uses-warped-ai-generated-images-to-portray-police-brutality-in-colombia)
|
|
25’38”
|
“One thing is conducting an open source investigation, another thing is presenting an open source investigation”
A long and now universally accepted tradition in literary theory, studies on visual cultures, and semiotics argues that narrative elements are always embedded in scientific discourse, as well as in any apparently objective document based on what are presented as ‘neutral’ facts.
Instead of downplaying the narrative elements, this alternative approach sets out to develop hybrid storytelling techniques that do not obfuscate, but rather enhance and expand on rigorous counterforensic investigations and the hard data used in open source intelligence.
Investigative Aesthetics: Conflicts and Commons in the Politics of Truth by Matthew Fuller and Eyal Weizman (Verso, 2021) is the best place to begin exploring the notion of investigative aesthetics as narrative strategy for politically-aware research. The book is a reflection on both Fuller’s theoretical research on communication technologies, mediations, and interfaces, and Weizman’s and Forensic Architecture’s groundbreaking work on detectability of truth, counterforensics and their narratives.
|
Medialab-Matadero Madrid
