Category Archives: Software

Exploitation Forensics. Interview with Vladan Joler

Vladan Joler and Kate Crawford, Anatomy of an AI system (detail)

If you find yourself in Ljubljana this week, don’t miss SHARE Lab. Exploitation Forensics at Aksioma.

The exhibition presents maps and documents that SHARE Lab, a research and data investigation lab based in Serbia, has created over the last few years in order to prize open, analyze and make sense of the black boxes that hide behind our most used platforms and devices.

The research presented at Aksioma focuses on two technologies that modern life increasingly relies on: Facebook and Artificial Intelligence.

The map dissecting the most famous social media ‘service’ might be sober and elegant but the reality it uncovers is everything but pretty. A close look at the elaborate graph reveals exploitation of material and immaterial labour and generation of enormous amounts of wealth with not much redistribution in between (to say the least.) As for the map exploring the deep materiality of AI, it dissects the whole supply chain behind the technology deployed by Alexa and any other ‘smart’ device. From mining to transport and with more exploitation of data, labour, resources in the process.

Should you not find yourself in Ljubljana, then you can still discover the impulses, findings and challenges behind the maps in this video recording of the talk that the leader of the SHARE Lab, Prof. Vladan Joler, gave at Aksioma two weeks ago:

Talk by Vladan Joler at the Aksioma Project Space in Ljubljana on 29 November 2017

In the presentation, Joler talks superpowers of social media and AI invisible infrastructures but he also makes fascinating forays into the quantification of nature, the language of neural networks, accelerating inequality gaps, troll-hunting and issues of surveillance capitalism.

I also took the Aksioma exhibition as an excuse to ask Vladan Joler a few questions:

SHARE Lab (Vladan Joler and Andrej Petrovski), Facebook Algorithmic Factory (detail)

Hi Vladan! The Facebook maps and the information that accompanies them on the Share lab website are wonderful but also a bit overwhelming. Is there anything we can do to resist the way our data are used? Is there any way we can still use Facebook while maintaining a bit of privacy, and without being too exploited or targeted by potentially unethical methods? Would you advise us to just cancel our Facebook account? Or is there a kind of medium way?

I have my personal opinion on that, but the issue is that in order to make such a decision each user should be able to understand what happens to their private data, data generated by activity and behaviour and many other types of data that is being collected by such platforms. However, the main problem, and the core reasoning behind our investigations, is that what happens within Facebook for example, i.e. the way it works is something that we can call a black box. The darkness of said boxes is shaped by many different layers of in-transparency. From different forms of invisible infrastructures over the ecosystems of algorithms to many forms of hidden exploitation of human labour, all those dark places are not meant to be seen by us. The only thing that we are allowed to see are the minimalist interfaces and shiny offices where play and leisure meet work. Our investigations are exercises in testing our own capacities as independent researchers to sneak in and put some light on those hidden processes. So the idea is to try and give the users of those platforms more facts so that they are able decide if the price they are paying might be too high in the end. After all, this is a decision that each person should make individually.

Another issue is that, the deeper we were going into those black boxes, the more we became conscious of the fact that our capacities to understand and investigate those systems are extremely limited. Back to your question, personally I don’t believe that there is a middle way, but unfortunately I also don’t believe that there is a simple way out of this. Probably we should try to think about alternative business models and platforms that are not based on surveillance capitalism. We are repeating this mantra about open source, decentralised, community-run platforms, to no real effect.

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Jure Goršič / Aksioma

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Janez Janša

The other depressing thing is that for many people, Facebook IS the internet. They just don’t care about privacy, privacy belongs in the past and being targeted is great because it means that Facebook is extra fun and useful. Do you think that fighting for privacy is a futile battle? That we should just go with the flow and adapt to this ‘new normal’?

It is interesting to think that privacy already belongs to the past since historically speaking privacy as we understand it today is not an old concept. It is questionable whether we ever had a moment in time when we had properly defined our right to privacy and we were able to defend it. So, from my point of view, it is more of a process of exploration and an urge to define in each moment what privacy means in present time. We should accept the decentralised view on the term privacy and accept that for different cultures this word has a different meaning and not just imply, for example, European view on privacy. Currently, with such a fast development of technology, with the lack of transparency-related tools and methodologies, outdated laws and ineffective bureaucracies, we are left behind in understanding what is really going on behind the walls of leading corporations whose business models are based on surveillance capitalism. Without understanding what is going on behind the walls of the five biggest technology firms (Alphabet, Amazon, Apple, Facebook and Microsoft) we cannot rethink and define what privacy in fact is nowadays.

The dynamics of power on the Web have dramatically changed, and Google and Facebook now have a direct influence over 70% of internet traffic. Our previous investigations are saying that 90% of the websites we investigated have some of the Google cookies embedded. So, they are the Internet today and even more, their power is spilling out of the web into many other segments of our life, from our bedrooms, cars, cities to our bodies.

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Janez Janša

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Janez Janša

Could you explain us the title of the show “Exploitation Forensics”?

Oxford dictionary is giving us two main uses of the word exploitation : (1) the action or fact of treating someone unfairly in order to benefit from their work and (2) the action of making use of and benefiting from resources. Basically both versions are essentially related to two maps that are featured in the exhibition. We can read our maps as visualisations of exploitation process regardless whether we speak about exploitation of our immaterial labour (content production and user behaviour as labour) or we go deeper and say that we are not even a workers in that algorithmic factory, but pure, raw material, i.e. a resource (user behavioural data as a resource). Each day users of Facebook provide 300.000.000 hours of unpaid immaterial labour and this labour is transformed into the 18 billion US dollars of revenue each year. We can argue if that is something that we can call exploitation or not, for the simple reason that users use those platforms voluntarily, but for me the key question is do we really have an option to stay out of those systems anymore? For example, our Facebook accounts are checked during visa applications, and the fact that you maybe don’t have a profile can be treated as an anomaly, as a suspicious fact.

Not having profile will place you in a different basket and maybe different price model if you want to get life insurance and for sure, not having Linkedin account if you are applying for a job will lower your chances of getting the job you want. Our options of staying out are more and more limited each day and the social price we are paying to stay out of it is higher and higher.

If our Facebook map is somehow trying to visualise one form of exploitation, the other map that had the unofficial title “networks of metal, sweat and neurons” is visualising basically three crucial forms of exploitation during the birth, life and death of our networked devices. Here we are drawing shapes of exploitation related to different forms of human labour, exploitation of natural resources and exploitation of personal data quantified nature and human made products.

The word forensics is usually used for scientific tests or techniques used in connection with the detection of crime; and we used many different forensic methods in our investigations since my colleague Andrej Petrovski has a degree in cyber forensics. But in this case the use of this word can be treated also as a metaphor. I like to think of black boxes such as Facebook or complex supply chains and hidden exploitations as crime scenes. Crime scenes where different sort of crimes against personal privacy, nature exploitation or let’s say in some broad sense crime against humanity happens.

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Jure Goršič / Aksioma

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Jure Goršič / Aksioma

The maps are incredibly sophisticated and detailed. Surely you can’t have assimilated and processed all this data without the help of data crunching algorithms? Could you briefly describe your methodology? How you managed to absorb all this information and turn it into a visualisation that is both clear and visually-appealing?

In our previous investigations (eg. Metadata Investigation: Inside Hacking Team or Mapping and quantifying political information warfare) we relied mostly on process of data collection and data analysis, trying to apply different methods of metadata analysis similar to ones that organisations such as the NSA or Facebook probably use to analyse our personal data. For that we used different data collection methods and publicly available tools for data analysis (eg. Gephi, Tableau, Raw Graphs). However, the two maps featured in the exhibition are mostly product of long process of diving and digging into publicly available documentation such as 8000 publicly available patents registered by Facebook, their terms or services documentation and some available reports from regulatory bodies. At the beginning, we wanted to use some data analysis methods, but we very quickly realised that the complexity of data collection operations by Facebook and the number of data points they use is so big that any kind of quantitative analysis would be almost impossible. This tells a lot about our limited capacity to investigate such complex systems. By reading and watching hundreds of patents we were able to find some pieces of this vast mosaic of data exploitation map we were making.

So, those maps, even though they look in some way generative and made by algorithms, they are basically almost drawn by hand. Sometimes it takes months to draw such an complex map, but somehow I need to say that I really appreciate slowness of this process. Doing it manually gives you the time to think about each detail. Those are more cognitive maps based on collected information then data visualizations.

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Jure Goršič / Aksioma

In a BBC article, you are quoted as saying “If Facebook were a country, it would be bigger than China.” Which reminded me of several news stories that claim that the way the Chinese use the internet is ‘a guide to the future’ (cf. How China is changing the internet) Would you agree with that? Do you think that Facebook might eventually eat up so many apps that we’ll find ourselves in a similar situation, with our lives almost entirely mediated by Facebook?

The unprecedented concentration of wealth within the top five technology companies allows them to treat the whole world of innovation as their outsourced research and development. Anarcho-Capitalist ecosystem of startups is based on a dream that in one moment one of those top five mega companies will acquire them for millions of dollars.

If you just take a look at one of the graphs from our research on “The human fabric of the Facebook Pyramid” mapping the connections within Facebook top management, you will probably realise that through their board of directors they have their feet in most important segments of technological development in combination with political power circles. This new hyper-aristocracy has a power to eat up any new innovation, any attempt that will potentially endanger their monopoly.

The other work in the Aksioma show is Anatomy of an AI system, a map that guides “visitors through the birth, life and death of one networked device, based on a centralized artificial intelligence system, Amazon Alexa, exposing its extended anatomy and various forms of exploitation.” Could you tell us a few words about this map? Does it continue the Facebook research or is it investigating different issues?

Barcelona-based artist Joana Moll infected me with this obsession about materiality of technology. For years we were investigating networks and data flows, trying to visualise and explore different processes within those invisible infrastructures. But then after working with Joana I realised that each of those individual devices we were investigating, has let’s say another dimension of existence, that is related to the process of their birth, life and death.

We started to investigate what Jussi Parikka described as geology of media. In parallel with that, our previous investigations had a lot to do with issues of digital labour, beautifully explained in works of Christian Fuchs and other authors, and this brought us to investigate the complex supply chains and labour exploitation in the proces.

Finally, together with Kate Crawford from AI Now Institute, we started to develop a map that is a combination of all those aspects in one story. The result is a map of the extended anatomy of one AI based device, in this case Amazon Echo. This anatomy goes really deep, from the process of exploitation of the metals embedded in those devices, over the different layers of production process, hidden labour, fractal supply chains, internet infrastructures, black boxes of neural networks, process of data exploitation to the death of those devices. This map basically combines and visualises three forms of exploitation: exploitation of human labour, exploitation of material resources and exploitation of quantified nature or we can say exploitation of data sets. This map is still in beta version and it is a first step towards something that we are calling in this moment – AI Atlas that should be developed together with AI Now institute during next year.

Do you plan to build up an atlas with more maps over time? By looking at other social media giants? Do you have new targets in view? Other tech companies you’d like to dissect in the way you did Facebook?

The idea of an Atlas as a form is there from the beginnings of our investigations when we explored different forms of networks and invisible infrastructures. The problem is that the deeper our investigations went, those maps became more and more complex and grew in size. For example, maps exhibited at Aksioma are 4×3 m in size and still there are parts of the maps that are on the edge of readability. Complexity, scale and materiality of those maps became somehow a burden itself. For the moment there are two main forms of materialisations of our research. First, the main form are stories on our website and recently those big printed maps are starting to have their life at different gallery spaces around. It is just recently that our work was exhibited in art context and I need to say that I kind of enjoy in that new turn.

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Janez Janša

SHARE Lab. Exploitation Forensics at Aksioma Project Space. Photo: Janez Janša

How are you exhibiting the maps in the gallery? Do you accompany the prints with information texts or videos that contextualize the maps?

Yes. As You mentioned before, those maps are somewhat overwhelming, complex and not so easy to understand. On the website we have stories, narratives that guide the readers through the complexities of those black boxes. But at the exhibitions we need to use different methods to help viewers navigate and understand those complex issues. Katarzyna Szymielewicz from Panoptykon Foundation, created video narrative that is accompanying our Facebook map and we are usually exhibiting a pile of printed Facebook patents, so visitors can explore them by themselves.

Thanks Vladan!

SHARE Lab. Exploitation Forensics is at Aksioma | Project Space in Ljubljana until 15 December 2017.

Previously: Critical investigation into the politics of the interface. An interview with Joana Moll and Uberworked and Underpaid: How Workers Are Disrupting the Digital Economy.

Smart guide for connected objects, activism on the dance floor, cooking with phones, a human Alexa. Just another edition of the DocLab conference

The DocLab Interactive Conference closed at De Brakke Grond in Amsterdam on Sunday 19th of November. An integral part of the International Documentary Film Festival Amsterdam (IDFA), DocLab looks at how contemporary artists, designers, filmmakers and other creators use technology to devise and pioneer new forms of documentary storytelling. There’s an exhibition, an immersive network summit, screenings, performances and a conference. The conference is my favourite. Digital pioneers share with the audience their latest experiments and boldest visions of the future. Each year, the same thing happens: the talks start at 10 in the morning, i blink and it’s already 6pm. Their picnic bag is a monstrosity for anyone who’s into eating healthy (more about food later) but that’s just about the only negative thing i can say about the event.

Here are the notes i wrote down during the talks. They are not exhaustive, they only aim to highlight a few ideas and projects i found particularly thought-provoking:

Brett Gaylor at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Superflux and Mozilla, Our Friends Electric

Brett Gaylor is the project lead for Mozilla’s Web Made Movies project. Before working with Mozilla, Brett directed the award wining documentary Rip! A Remix Manifesto (2008), an open source documentary investigating remix culture and copyright in the digital age. Two years ago, he authored Do not Track: an online, interactive documentary series about who’s watching you and who’s profiting from your private data.

In his brief and lively presentation, Gaylor talked about the Internet of Shit and the connected salt shakers, forks and other ‘smart objects’ that are actually stupid, insecure and easily hackable. He also showed us an extract of Our Friends Electric, a short film by Superflux and Mozilla which imagines life in the company of an AI virtual assistant that has its own personality.

But if there’s one link you should click on in this pre-Christmas silly period, it’s this one: privacy not included, a guide for shopping connected gadgets that respect your online privacy and security.

Memo Akten at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Human Nature – Supernormal Stimuli

Memo Akten is a computational artist interested in tools that enable people to express themselves. His work also looks at the tensions between nature, science, technology, ethics, ritual, tradition and religion.

Akten will i’m sure go down in media art history as a brilliant researcher into AI but also as the guy who told us about Australian beetles mating habits. The males of the Julodimorpha bakewelli species like to couple with big, brown-orangey conquests covered in dimples. Which leads them to copulate with discarded brown beer bottles. The behaviour nearly wiped out the whole species until the beer company decided to change the bottle design and save them from extinction.

Humans are not necessarily always more perceptive than beetles. We also project meaning into what we see and Akten’s work explores how this translates when it comes to algorithms and how the way we use AI actually uncovers our own human biases.

He also made a couple of valid points about the rise of AI and the explosion of big data. The artist believes that AI has been around for years but we need it now more than before to make sense of big data. A second reason for this new interest in AI is that the very organizations that push it are the ones that rely the most on big data to make profits.

Bogomir Doringer at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Short documentary which sums up the concept and background of FACELESS

The stand-out talk for me was Bogomir Doringer‘s. The artist and curator contemporary investigates collective and individual dynamics. He introduced us to two of his ongoing areas of research.

The first one is FACELESS which started in 2005, turned into an exhibition at q21 MuseumsQuartier Vienna in 2012 and has now taken the form of a book that should be published next year. FACELESS looks at the topic of hidden faces in society in relationship to the surveillance technologies deployed by government after 9/11. He showed us dozens of examples that demonstrate how widespread hidden and masked faces have become in the media, in the creative arts and in pop culture. From David Bowie album cover to Raf Simons putting balaclavas on the runway in 2012. From Adam Harvey to public protests where people wear masks to express that they are one body. From Burqa fetish to Jill Magid’s performance with CCTV in Liverpool.

The second research Doringer presented is I Dance Alone, a work in progress anchored in the artist’s experience of partying in the club Industria in Belgrade during the 1999 Nato bombings. To mock death or simply to try and forget about it. I Dance Alone places cameras above and on the dance floor in order to understand the rituals taking place when people dance. One of these rituals has urgency. The other is all about entertainment. I found the ‘urgency’ side of the research fascinating. Aside from the Industria nightclub, the artist also mentioned Bassiani, a nightclub in Tbilisi, Georgia. One important dimension of Bassiani is its social activism and in particular the way it encourages clubbers to join street protests, influence drug policy and tolerance towards the LGBTQ community in the country.

Lauren McCarthy at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Lauren McCarthy performance as part of Doclab. Photo Nichon Glerum

Lauren McCarthy discussed her attempts to become a human version of Alexa. The performance involves having access to all the software that runs your house and installing all kinds of gadgets in your home. Once everything’s in place you can ask her anything. From the weather forecast to an honest opinion about your new outfit.

Conference host Ove Rishøj Jensen introducing Jonathan Harris at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Jonathan Harris at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

This edition of IDFA was particularly satisfying for anyone interested in digital creativity because for the first time ever, the guest of honour of the festival was a digital pioneer. Jonathan Harris is an artist, a computer scientist and someone who’s generally concerned with bringing more compassion and human warmth to digital technology.

Harris has gained fame for works that include the interactive I Love Your Work (a portrait of nine women who make lesbian porn), We Feel Fine (a touching visualization of human feelings), The Whale Hunt (a 9 day journey with the Inupiat Eskimos), etc. For the DocLab conference, Harris focused on Cowbird, a website he conceived as a library of life experiences “filled with small moments of human connection.” A kind of instagram but more thoughtful, less complacent and launched years before instagram.

Yasmin Elayat at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

With her visually powerful and socially-engaged work at Scatter, Yasmin Elayat is trying to open up the storytelling and production process to the audience. One way to do that is by sharing the tools that Scatter builds. Their depthkit, for example, aims to put volumetric filmmaking into the hands of everyone.

She also briefly presented the very promising Racial Terror Project (working title) which uses VR to time-travels to sites where Claude Neal was dragged and lynched by a mob of white men in 1934 Florida. The project aims to be a ‘magical realist documentary’ that would reclaim the sites where violence took place and contextualize them.

Micha Wertheim at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Micha Wertheim is a stand-up comedian, writer and satirist. Last year, Wertheim performed an experiment that made theatre history: he never appeared on stage during his performance. Instead, he used a robot, a printer, a stereo and a set of headphones to coax an unaware audience to perform the whole show in his absence. I hope the video of his presentation will be published at some point. It was hilarious and frankly genius.

Jonathan Puckey at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Interactive designer Jonathan Puckey presented one of his ‘older’ projects and i adore it! The Radio Garden website allows you to spin a globe and listen to live radio from anywhere around the world.

He was actually on stage to present the interactive VR music video Dance Tonite which is rather impressive but i’m a fan of radio so i’ll stay with that good old media today.

Panel with Francesca Panetta, the Guardian executive editor for VR, Oscar Raby, Creative Director at virtual Reality studio VRTOV, and filmmaker Zhao Qi

The panel with Francesca Panetta, the Guardian executive editor for VR, Oscar Raby, Creative Director at virtual Reality studio VRTOV, and filmmaker Zhao Qi managed to pack many ideas and reflections in 20 minutes or so. I learnt that:
– The Guardian has its own VR studio and they’ve made 8 VR pieces so far. They recently gave out 100 000 google cardboard goggles.
– Women in VR face much gender discrimination.
– There are some 5000 VR cinema and arcades in China, making it easier for VR creators to reach audiences.
– Research has shown that if someone reads an article, there is 1% chance that that person will look for more information about that topic. But there’s 25% chance if they experience that same topic via VR.
– In case you’re wondering about the graphic screened behind the panelists on the photo above, it’s the hype cycle which represents “the maturity, adoption and social application of specific technologies.”

Emilie Baltz at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Emilie Baltz creates musical licking performances and other food design extravaganzas. For DocLab, she collaborated with chef Matthias Van Der Nagel and Klasien V.D. Zandschulp to make us literally cook using a mobile phone, a box containing ingredients with strange textures and spiritual encouragement from Futurist Filippo Tommaso Marinetti. The result was called Amuse Telebouche.

The public enjoying Amuse Telebouche at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

As you can see in this very flattering photo, I’m not too keen on experimenting with food:

Killjoy me refusing to taste Amuse Telebouche at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

The public cooking Amuse Telebouche at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

The public cooking Amuse Telebouche at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

The public cooking Amuse Telebouche at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Jessica Brillhart and Jason Kottke at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Jessica Brillhart and Jason Kottke sat down on stage to ponder upon the question: The internet is on fire. What would you save? The selection included two of my favourite: Wikipedia and David OReilly’s Octocat.

Erwin Verbruggen at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Erwin Verbruggen from the Netherlands Institute for Sound and Vision presented FREEZE! A manifesto for safeguarding and preserving born-digital heritage.

W/O/R/K performance by Anagram at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

W/O/R/K performance by Anagram at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

W/O/R/K performance by Anagram at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

Caspar Sonnen and the team of at Doclab interactive conference in de Brakke Grond, part of the IDFA. Photo Nichon Glerum

This event was part of the DocLab: Uncharted Rituals program, made possible by the Netherlands Film Fund, Mondriaan Fund, De Brakke Grond and Diversion.

Previously: DocLab exhibition asks “Are robots imitating us or are we imitating robots?”
Image on the homepage: Emilie Baltz.

Critical investigation into the politics of the interface. An interview with Joana Moll

Joana Moll, AZ: move and get shot, 2012-2014

Joana Moll is a young artist and researcher whose work critically explores the way post-capitalist narratives affect the alphabetization of machines, humans and ecosystems. Her main research topics include Internet materiality, surveillance, online tracking, critical interfaces and language.

I first encountered Joana’s work a couple of years ago when i read about her online works such as Texas Border, AZ: Move and Get Shot and Virtual Watchers which look into the crowdsourcing of the surveillance of the US/Mexico border by civilians.

Joana Moll, in collaboration with anthropologist Cédric Parizot, The Virtual Watchers

Joana Moll, AZ: move and get shot, 2012-2014

These projects expose two rising features of contemporary culture: the insidious militarization of civil society but also the dilution of individual responsibility enabled by technology. I would really recommend that you check out the talk Surveillance through social networks along the US-Mexico Border that she gave a couple of years ago at AntiAtlas of Borders conference because today’s interview is not going to focus specifically on these works.

The reason why i got in touch with Joana is that she is the co-founder of the Critical Interface Politics Research Group at HANGAR, a centre for arts production and research in Barcelona.

This ongoing research project investigates the complex physical structure of the Internet and in particular the many actors, (infra)structures, systems and materials that have a direct but often covert impact on every aspect of our daily lives: submarine and underground cables that perpetuate colonialist heritage, companies and countries that have access to our data, ecological costs of online habits, commodification of data, cultural biases within user interface design, etc.

Joana Moll not only probes into these questions in her own artistic works but she has also started to develop a series of workshops, strategies and tools that enable other people, no matter how tech savvy they are, to delve into these issues but also to subvert the material and computational architectures of the internet.

Poetic Destruction of the Interface, a workshop on Critical Interface Politics at HANGAR, Barcelona, 2016

Performing PageRank physically, from Poetic Destruction of the Interface, a workshop on Critical Interface Politics at HANGAR, Barcelona, 2016

Joana will be giving online classes about the power of interfaces and the way we can learn to democratize them in May with the School of Machines, Making & Make-Believe. In the meantime, i had a skype chat with Joana. Here’s what it sounded like:

Hi Joana! The Tracking Forensics workshops, which you organised together with Andrea Noni and Vladan Joler, looked at the material impact of the so-called digital immateriality on the ecosystems. The word ‘forensics’ suggests the collection of criminal evidences. Why did you chose this title for the workshops?

Maybe i should start with the background of the workshop?

You know what? That’s a good idea!

Hangar in Barcelona was at the origin of this workshop. They invited me over a year ago to lead an investigation for IMAGIT, a European project that deals with criticism of interfaces. They asked me to develop some actions that would flesh out some of the more abstract concepts that they explored in the Manifesto for a critical approach to the user interface.

I ended up developing 3 workshops that lasted each for 12 hours. Over the course of these workshops, we explored topics such as the materiality of the internet, code, cognition, power and then interface, intervention governance, bias in the interfaces, etc. We were trying to cover everything that goes beyond the interface.

And then while working with another colleague at Hangar, we started to talk a lot about forensics, tracking forensics, online tracking and surveillance, I have been exploring these topics for many years. So we came up with this idea of doing the same workshops that we had done already but the difference would be that we’d focus much more on tracking.

We invited Vladan to give a talk in the workshop because he was already at Hangar doing a residency i had curated on the topic of tracking forensics and ethical uses of collected data.

The term “forensics” refers to cyber forensics (or computer forensics), the official term used when you follow the path of crime where evidence is stored digitally. You thus approach the online traces as if you were in front of a crime scene.

As for “tracking”, it refers to the action of monitoring people’s activity on the internet. Basically the workshop was about showing how you can understand the dynamics, the mechanisms that corporations, agencies and governments use to collect your data. Share Lab in Serbia did a massive research on that topic.

Interface Hack, from Poetic Destruction of the Interface, a workshop on Critical Interface Politics at HANGAR, Barcelona, 2016

Poetic Destruction of the Interface, a workshop on Critical Interface Politics at HANGAR, Barcelona, 2016

Tracking Forensics Atlas. Map #2 Tracerouting Top 100 domains

How did you proceed to uncover the physical paths of information? What kind of methodology and strategies did you use?

Archaeology! We made a big archive at Hangar with a group called Critical Interface Politics Research Group. If you have a look on the website, you will find tools, encryption, visualisation, research, activism, etc. But there’s still so much more information we should add.

During the workshops, we used various software but the most important thing lays in the tangible approach to these digital infrastructures and issues because the way you acknowledge things is totally different whether you just work with screens or you experience them physically. For example, we used maps to draw out a forensic analysis of the paths of information.

Poetic Destruction of the Interface, a workshop on Critical Interface Politics at HANGAR, Barcelona, 2016

As an individual who didn’t get the chance to participate to the workshop, how can i become better informed about the infrastructures hidden behind our dependency on the digital?

Together with the Share Lab, we are doing some Do It Yourself Tracking Forensics that we hope to publish soon. It’s basically what his residency at Hangar was about. It’s a project that Andrea and I proposed to do and Hangar is helping us develop it with group of Cyber Forensic people. This DIY is going to be for everyone because it has been very important for me right from the start to engage in critical pedagogic strategy. I want to not only help people with no technical skills understand all these things that are actually responsible for sculpting our reality but also i want this DIY to help them intervene autonomously in these systems.

Aren’t there other groups working on the same issues and putting resources out there just like what you’re trying to do? Or do you have to do all that research from scratch?

There are other people working on similar issues but because we do things in a different way, we still have to do all the research. For example, the Share Lab in Serbia is looking at similar issues but they only cover a part of it. Also Tactical Tech Collective, with whom I’ve collaborated on two projects, developed many pedagogical manuals on the issue. And then of course there is Julian Oliver and Danja Vasiliev but they don’t cover the physical part as in depth as we do, they are mostly looking at the architecture of information and that’s something that we, on the other hand, only cover very briefly. Our focus is on internet infrastructure and tracking. The pedagogy aspect is also very important for us. I also discovered a group in Austria that did a massive research in tracking. The output was a great paper that’s almost a book actually. There are other people in Amsterdam also but again, it’s different.



Your work DEFOOOOOOOOOOOOOOOOOOOOOREST explores the tangible and devastating impact of the most mundane habit: the use of The project visualises the amount of trees needed to absorb the amount of CO2 generated by the global visits to the search engine every second. The website is very simple yet so powerful that it makes me very anxious. I close DEFOOOOOOOOOOOOOOOOOOOOOREST almost as soon as i’ve opened it. It makes me feel helpless. Once we are more aware of the consequences of our daily internet gestures, is there anything we can do apart from despair?

It’s an ongoing debate. Because of course it’s easy to put all the weight on the shoulders of the end user and make them feel guilty for everything. However, i think it’s very important that we visualize the physical and ecological impact of our online actions. It needs to be embedded in the social imagination because it is quite unbelievable. Data generates C02, it pollutes. There are a few things we can do to help with the problem but they are very minimal. If you are a web designer, for example, you can try and put less images or just work in a more efficient way. Companies bear an even larger share of responsibility.

And in this case, policies have to be enforced from above. Change has to come from a political level and we need to take responsibility collectively if we want things to change dramatically.

The Institute for the Advancement of Popular Automatisms, Embrace Stupidity

You are a co-founder of The Institute for the Advancement of Popular Automatisms. I read the about page, clicked around but i must confess that the more i thought i understood, the less i understood. So could you explain me in layman’s terms the activities of IAPA?

A lots of people tell me exactly that! They are not sure whether there are artists behind the project or if it’s just an algorithm doing all the work.

It’s actually very simple. I did this project together with Mexican artist Eugenio Tisselli. The Institute for the Advancement of Popular Automatisms is a platform that enables us to experiment in a very fast way with code, with language, with algorithms, to talk about poetry and the absurd and how machines communicate with humans. With this project we can do all that in a very unorthodox way, by using more the instinct and the irony. The projects that Eugenio and I do aside from this one are research-based and involve long processes. So IFAPA allows us to play a bit. It’s still serious but the approach is more laid-back, more simple. It allows us to play with our ideas and implement things that are important to our work. It’s kind of escape bubble too!

Any upcoming project, field of research or event you could share with us?

I’m working on another project that talks about how different agents that exploit data. I call that ‘data slavery’, there is a lot of dating sites that sell profiles to each other in a crazy way. You can by thousand or even one million profiles for a hundred dollars….

You mean real profiles?

Some of them are real, some are fake. But that doesn’t even matter because the pictures they use are pictures of real people.

I’m about to buy massive amounts of profiles and then try and understand where else these profile, these pictures, these names, or emails can be found. And from there, i want to explore the data of these slavery markets. In a previous research I did on the topic I’ve seen that one single profile was being exploited by more than 50 online services.

Together with Vladan we are writing a text that explores and exposes the ecological footprint of surveillance capitalism and we hope to release these before the summer.

Besides, and that’s very recent news, the next phase of the Critical Interface Politics Research Group will focus on deeply analizying the environmental impacts of internet infrastructures, data flows and interfaces through different interdisciplinary initatives. The plan is to gather a transdisicplinary resesarch team and design serveral interventions that will be able to both, expose the termendous material impact of communication technologies, create mechanisms and tools to reduce such footprint and make them available to the general public. We are in the process of writing the project and looking for partners right now.

Thanks Joana!

Joana Moll will be running a Tracking Forensics workshop at the Resonate Festival in Belgrade on 21 and 22 April. And if you can’t make it to Serbia, Joana will be giving online classes about the power of interfaces and the way we can learn to democratize them. The online program is organized by the School of Machines, Making & Make-Believe in May. I’ll also be giving online classes but on the topic of socially engaged creative practices, same month, only that Joana gets the Tuesdays and i get Mondays.)

Dataghost 2. The kabbalistic computational machine

RYBN, Dataghost 2, 2016. Installation view at STUK in Leuven for the Artefact festival. Photo © Kristof Vrancken

As early as the 1st century, Jews believed that the Torah and other key religious texts contained encoded truths and hidden meanings. They used a system called Gematria to uncover them. According to this numerological system, each Hebrew letter also corresponds to a number (for example: 1 is Aleph, 2 is Bet, 3 is Gimel, 4 is Daleth, etc.) Kabbalists extended the method to other texts and, by converting letters to numbers, they looked for a hidden meaning in each word. Other hermeneutic techniques used by the Kabbalah are Temurah, which rearrange words and sentences to deduce deeper spiritual meanings, and Notarikon which creates words from letters taken from the beginning, middle, or end of words.

RYBN, Dataghost 2, 2016. Installation view at STUK in Leuven for the Artefact festival. Photo © Kristof Vrancken

RYBN, Dataghost 2, 2016. Installation view at STUK in Leuven for the Artefact festival. Photo © Kristof Vrancken

The French collective has applied this numerological system of transformations, associations and substitutions to computing. Their Dataghost 2 installation is a kabbalistic computational machine that seeks to reveal the hidden messages buried within the data traffic.

A daemon, installed on a server, catches all incoming and outgoing digital communications, and dumps their content using network interception tools. All the encapsulated data are then submitted to several decyphering algorithms, reproducing the hermeneutic techniques of Kabbalah. The raw data are decomposed and recomposed according to the substitution principles that govern the Kabbalah, in order to unveil the mystic of network communications.

By following the kabbalistic alpha-numerical system, the fragments of codes generate in the process millions of shell commands, most of them incoherent or nonfunctional. However, from time to time, the commands will ‘make sense’ to the computer. The machine will interpret them as tasks that needs to be executed. At this precise moment, the machine achieves the invocation ritual of a digital Golem.

However, there is no way to predict where the ritual might lead the machine: the executed commands might saturate the memory capacity of the machine, provoke a definitive stop within the software layer, or overpass several critical limits that results in an overheating of certain electronic components, or lead to the destruction of parts of its physical layers. Over the course of its life, the system constantly publishes its self-destructive activity in the form of a print out of all the different commands.

I discovered the work two days ago at the Artefact festival at STUK in Leuven (a mere 15 minute away from Brussels so take the train now if you’re in Belgium because the show is as enchanting as its theme suggests) and Dataghost 2 was dead. The demise came quite early. The email exchange in which the artists and STUK tried to understand what had happened was printed out and added to the exhibition space. The emails reveal that the system probably erased a critical file which brought the whole process to its term.

The installation is currently running in dead mode. Both the printing machine and the screen remain frozen. 

RYBN, Dataghost 2, 2016. Installation view at STUK in Leuven for the Artefact festival. Photo © Kristof Vrancken

RYBN, Dataghost 2, 2016. Installation view at STUK in Leuven for the Artefact festival. Photo © Kristof Vrancken

I found the work brilliant. On the one hand, it is super complex and perplexing, just like most esoteric practices. On the other hand, it demonstrates with great efficiency and simplicity that algorithms (and by extension any technology system) are only as rational (or irrational) as the humans who program them.

Dataghost 2 is exhibited at the artefact festival in Leuven, Belgium. The exhibition, curated by Karen Verschooren from STUK & Ils Huygens from Z33 continues until 9 March 2017

If you find yourself in Paris, the RYBN collective will be discussing Dataghost 2 tomorrow 3 March at the Ecole Nationale Supérieure d’Arts de Paris Cergy.

Related stories: The Occult, Witchcraft & Magic. An Illustrated History. Commission: “Kill Box” by Joseph DeLappe, et al Commission: Kill Box: an international collaboration between U.S. based artist/activist, Joseph DeLappe and Scotland-based artists and game developers, Malath Abbas, Tom Demajo and Albert Elwin. “Kill Box” is the Military term used to describe an area on a grid map that a mission planner designates a target to be destroyed. Kill Box involves audiences in a fictionalized virtual environment based on documented drone strikes in Northern Pakistan.

Modern warfare technology disguises the lethal nature of weapons as they become surgical precision instruments producing ‘clean’ destruction within acceptable limits of “collateral damage.” – Jill Berke, “War on Words: How Language Obscures Violence.”

Kill Box is an online interactive game that critically explores the nature of drone (UAV) warfare, its complexities and consequences. It is an experience that explores the use of technology to transform and extend political and military power, and the abstraction of killing through virtualization. To “play” the game, download the application to your desktop, and make sure your speakers are on.

Kill Box is a 2015 commission of New Radio and Performing Arts, Inc. for its website. It was made possible with funds from the National Endowment for the Arts (USA). Additional funding has been provided by The Phoenix Theatre, Leicester, UK; and The Cutting Room, UK.


Joseph DeLappe is an artist/activist with a substantial body of work on the subject of geopolitics and drones and is considered a pioneer in the nascent field of computer games and art. Professor of the Department of Art at the University of Nevada where he directs the Digital Media program. Joseph is lead artist on Kill Box UAV; dealing with concept/content development, theoretical and historical research into drone warfare and primary lead on installation, development of publicity materials and archiving surrounding the project. Working with electronic and new media since 1983, his work in online gaming performance and electromechanical installation have been shown throughout the United States and abroad. In 2006 he began dead-in-iraq, typing consecutively, all names of America’s military casualties from the war in Iraq into the America’s Army first person shooter online recruiting game. DeLappe created and directs the crowdsourced memorial project,

Malath Abbas is an independent game designer, artist and producer working on experimental and meaningful games. Since co-founding the award winning studio Quartic Llama, Malath is establishing Scotland’s first game collective and co-working space in order to support a community of independent game makers. His current work includes Kill Box, an online game and interactive installation that critically explores the nature of drone warfare, its complexities and consequences.

Tom deMajo is a digital artist, electronic musician and sound designer, and lead designer for project drone. Tom is responsible for unifying the conceptual, experiential, visual and audio aspects of the project, driving the aesthetics and sound in the game. Tom’s work has covered film, animation, games, sound installations and music. He has toured globally as part of electronic music duo Warp Technique, and is a co-founder of Quartic Llama; independent games company. He was designer, sound designer, composer and artist on the award- winning game “other” made with Malath Abbas and in partnership with the National Theatre Scotland. He has collaborated extensively with artists, practitioners and institutions in Scotland and locally such as National Theatre Scotland, Museum of Scotland, Sink, and recently Hot Chocolate and Scottish Dance Theatre. Tom has been regularly invited to contribute to NEoN Digital Arts Festival, and is Artist in Residence at Fleet Collective in Dundee.

Albert Elwin is an artist and programmer, responsible for developing the underlying code for the PD, networking and implementation of all art objects into the project. Originally from New Zealand, Albert now lives and works in Scotland. He studied Computer Games Technology at the University of Abertay Dundee and his career began when he took part in Abertay’s 2012 Dare To Be Digital competition. Albert co-founded Space Budgie, an independent games studio in 2013 where he lead the development of Glitchspace, a visual programming game, well known for its aesthetic and game design. Albert was invited to talk about Glitchspace at various international game festivals, most notably the Game Developer Conference in San Francisco in 2014. For the last 6 months Albert has been working on a wide range of projects and collaborations; developing digital experiments for testing human depth perception at St. Andrews University, an audio/visual digital instrument based on Harmonographs.

Related Works:

Commission Control by Andy Deck and Joe Dellinger (1997)
Here and Now by David Crawford (1999)
CONtext by Jo-Anne Green (2004)

“Like” us on Facebook:

Follow us on Twitter:

Computational Fashion: Topics in fashion and wearable technology

The first Computational Fashion publication is now available. Buy it now on Amazon for just $16 or download a free PDF.

Project Created: 
December 2014

read more

Turbulence Commission: INTERP by Jeff Thompson

Turbulence Commission: INTERP by Jeff Thompson:

INTERP is a series of digital sculptures generated by blending 100 unrelated photographs, placing them into simulated three-dimensional space, and importing them into photogrammetry software, tricking it into thinking that the photographs were of a single object. Thompson is interested in “useless” and culturally-derived data sets, so rather than use an arbitrary archive of photographs (a Google image search for a particular term, for example), it seemed more natural to use a finite set that he had generated himself (approximately 12,000 images when he began the project in 2012). Every photograph was used.

INTERP is a 2014 commission of New Radio and Performing Arts, Inc. for its Turbulence website. It was made possible with funding from the National Endowment for the Arts.


Jeff Thompson received his BFA from the Minneapolis College of Art and Design and his MFA from Rutgers University. He is currently Assistant Professor and Program Director of Visual Art & Technology at the Stevens Institute of Technology in Hoboken, New Jersey. Thompson has exhibited and performed his work internationally at venues including the Museum of the Moving Image, Sheldon Museum of Art, the Taubman Museum of Art, SITE Santa Fe, Bemis Center for Contemporary Art, the Jersey City Museum, and the Weisman Art Museum. Thompson’s visual and written projects have been published by Ugly Duckling Presse, the Parsons Journal for Information Mapping, and Leonardo Electronic Almanac (MIT Press), among others. In addition to his studio practice, Thompson curates exhibitions through Drift Station, a curatorial collaboration that mounts international, experimental exhibitions.

“Like” us on Facebook:

Follow us on Twitter: