Category Archives: DocLab

IDFA DocLab: my favourite interactive documentaries

Over the course of its 10-ish years of existence, the IDFA DocLab festival has been gaily exploring the narrative potentials of augmented reality, virtual reality, interactive documentaries and artificial intelligence. Their program, which is part of the International Documentary Film Festival Amsterdam, features an exciting mix of hi-tech entertainment, innovative ideas and socially-engaged conversations that i haven’t experienced elsewhere.


DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

As usual, this year’s program was packed with dramatic commissions and entertaining debates but it was also anchored in today’s most pressing concerns: the plight of refugees in Europe, the legacies of colonialism, the plague of fake news, violation of human rights, climate change, etc. Topics that are everywhere in newspapers and on television already but with their more immersive, more in-depth treatments, the new digital art forms presented at IDFA DocLab seem to pick up from where traditional media coverage left off.

This year’s edition closed at De Brakke Grond in Amsterdam on Sunday 19th of November. There was an exhibition, an immersive network summit, screenings, performances, a conference and more. I covered some of it last week but in this last article, i’d like to focus on the interactive and/or VR documentaries that you can experience online (with one exception at the end of the list.)

Here’s my short list:


Tessa Louise Pope, Echoes of IS, 2017


Tessa Louise Pope, Echoes of IS, 2017

Tessa Louise Pope’s Echoes of IS is one of the most moving interactive documentaries i’ve ever seen.

The documentary maker brought together 12 people who have been deeply affected by IS and radicalization. Each of these individuals get a chance to explain their own experience. First, there are the people whose lives have been turned upside down by the arrival of IS in their town: a young mother who was forced to flee a life and city she loved in Syria and who continues to be afraid even though she now lives in the Netherlands; a young man who has always fought for a more democratic and fair regime in Syria; a woman whose house was turned into a torture facility after she had been forced to leave her country, etc. Then there are people who were born in the Netherlands: the father whose 14-year-old child was taken away and who died waging jihad. A young man who converted to Islam and suddenly found himself surrounded by incomprehension and intolerance.

In one of the short films, they all get to talk together and explore how their lives have been impacted by the IS.

The films are heart-breaking. Instead of the anonymous refugees who ‘invade’ Europe and the worrisome Muslims, you get to know individuals who have dreams and values similar to ours. The saddest thing about Echoes of IS is that people who should watch it in order to get a more balanced view about the ‘refugee crisis’ will probably never see the film.

You can watch it online too.


Robert Knoth & Antoinette de Jong, Poppy Interactive


Robert Knoth & Antoinette de Jong, Poppy Interactive, Poppy field, Afghanistan


Robert Knoth & Antoinette de Jong, Poppy Interactive

Robert Knoth and Antoinette de Jong have condensed 20 years of investigation into the trails of Afghan heroin into Poppy Interactive.

Afghanistan is by far the largest producer of opium and this online documentary unravels the global network of insurgents, terrorists and criminal organizations that use drug money to fuel conflicts in various areas of the world.

The work combines in-depth analysis, historical facts and global perspective with personal stories. You hear the point of view of opium farmers, soldiers, smugglers, bankers, border guards but also people who would have lived a perfectly normal life in our own neighbourhoods had heroin not destroyed their life, directly or indirectly.

Poppy Interactive efficiently connects the dots between distant locations, global issues and personal dramas that would otherwise seem completely separate.

Shehani Fernando with Francesca Panetta, Nicole Jackson, The Guardian, Limbo ( virtual reality trailer), 2017

The Guardian is now producing VR journalism. To ensure that their audience is able to experience these new forms of reporting, the newspaper has recently given away 100.000 Google Cardboard headsets. The VR works the team developed allow you to get a more immersive outlook on issues such as autism, flaws in forensic investigations or solitary confinement. But it’s their research on the experience of asylum seeking that was presented at IDFA DocLab.

Limbo attempts to recreate the stress and misery of being one of the tens of thousands of people who are waiting for their asylum application to be accepted or rejected by the UK government.

While asylum seekers wait for their Home Office interview and the subsequent decision, they live on £5 a day and are unable to work or choose where they live. In Limbo you step into their shoes and experience their state of mind while you wait for the decision that will determine the rest of your life.

I thought the black and white sketch-like aesthetic would be cold and dull. However, the visual design competently reflects the daily sense of boredom, isolation, humiliation and prejudices experienced by these people during the long months they spend waiting for the Home Office to determine whether or not they can hope for a normal life in a new country. Limbo is very poignant. The warmth of the voices of asylum seekers, immigration lawyers and barristers gives the work a very intimate and distressing dimension.

Next time please distribute the free goggles to Daily Mail readers?


Gina Kim, Bloodless, 2017


Gina Kim, Bloodless, 2017

Gina Kim, Bloodless (Teaser Reel), 2017

You can’t experience Bloodless online but i liked it so much, i had to mention it:

Filmmaker Gina Kim was still in college when a sex worker was brutally murdered by a US soldier stationed in South Korea. On the 28th of October 1992, the body of 26-year-old Yun Keum Yi was found at the Dongducheon camp town. Two beer bottles and one cola bottle were found inside her uterus, and an umbrella penetrated 11 inches into her rectum. Her body had been covered in detergent powder to dispose of evidence.

For years, Kim looked for a way to tell the tragic story without exploiting the images of the victim. And then she found VR. “With VR,” she writes, “the viewer is no longer a passive spectator, who can take voyeuristic pleasure from a spectacle in front of them (and at a distance).”

The 12 minute VR film Bloodless not only traces the last living moments of Yun Keum Yi but it also explores the issue of the comfort women exploited by US army troops stationed in South Korea since the 1950s.

Bloodless was shot on location where the crime took place. It is as visually stunning as it is creepy. The work won the Best VR story award for linear content at the Venice Film Festival this Summer.

More images from the IDFA DocLab exhibition and a couple of trailers:


DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

Everything, a game by David OReilly, explores and meditates on tiny living things, on big systems, on how they are all connected. But it’s about far more than that too.

David O’Reilly, Everything, 2017

Catherine Upin, Julia Cort, Nonny de la Peña and Raney Aronson-Rath, Greenland Melting (excerpt), 2017

Catherine Upin, Julia Cort, Nonny de la Peña and Raney Aronson-Rath, Greenland Melting (360 video produced as a companion to the VR work), 2017

The Last Chair doesn’t have the sexiest topic (old men living alone in the countryside) but i liked it a lot. It was peaceful and moving.


Jessie van Vreden, Anke Teunissen, The Last Chair, 2017


Duncan Speakman, It Must Have Been Dark by Then at DocLab Expo: Uncharted Rituals, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Duncan Speakman, It Must Have Been Dark by Then at DocLab Expo: Uncharted Rituals, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Duncan Speakman, It Must Have Been Dark by Then at DocLab Expo: Uncharted Rituals, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Jonathan Harris exhibition in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Jonathan Harris exhibition in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Jonathan Harris in his church outside of Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

Previous stories about this year’s edition of DocLab: DocLab exhibition asks “Are robots imitating us or are we imitating robots?” and Smart guide for connected objects, activism on the dance floor, cooking with phones, a human Alexa. Just another edition of the DocLab conference.

DocLab exhibition asks “Are robots imitating us or are we imitating robots?”


DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum


Jonathan Harris exhibition in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

The 11th edition of IDFA DocLab closed on Sunday at De Brakke Grond in Amsterdam. An integral part of the International Documentary Film Festival Amsterdam (IDFA), DocLab looks at how contemporary artists, designers, filmmakers and other creators use technology to devise and pioneer new forms of documentary storytelling. It’s a space for debates, conversations, VR experiences, interactive experiments and workshops.

For some reason, i thought that this year’s programme was even more intense than in previous years and i’m going to need 3 blog posts to cover all the ideas and projects i found particularly interesting. There will be one story summing up the notes i took during the DocLab: Interactive Conference. Another post will briefly comment on some of the interactive documentaries i saw in Amsterdam and back home. And today, i’d like to look at a couple of installations that explore the main theme of the festival: Uncharted Rituals or how we have to constantly, subtly and often unknowingly adjust our behaviour and mindset to technology. Instead of the other way round.

Robots and computers are acting more and more like people. They’re driving around in cars, hooking us up with new lovers and talking to us out of the blue. But is the opposite also true— are people acting more and more like robots?

The computers may think so: addicted to our phones, caught in virtual filter bubbles and dependent on just a handful of tech companies, people are acting more and more predictably. The breakthrough of artificial intelligence and immersive media doesn’t just pose the question what technology does to us, but also what we do with this technology.

I have only 3 works to submit to you today but each of them makes valuable comments about the way we might one day have to dance with and around technology in order to coexist with it:


Max Pinckers and Dries Depoorter, Trophy Camera v0.9, 2017


Max Pinckers and Dries Depoorter, Trophy Camera v0.9, 2017


Max Pinckers and Dries Depoorter, Trophy Camera v0.9 at DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

A photographic image is never objective. It is always framed by human aesthetic choices, agendas and conscious or unconscious bias. The Trophy Camera v0.9 aggregates this element of human subjectivity into a photo camera that can only make award-winning pictures.

The AI-powered camera, developed by photographer Max Pinckers and media artist (and DocLab Academy alumnus) Dries Depoorter, has been trained by all the photos that have won an award at the World Press Photo competition, from 1955 to the present.

Based on the identification of labeled patterns, the experimental device is programmed to identify, shoot and save only images that it predicts have at least a 90% chance of winning the competition. These photos are then automatically uploaded to a dedicated website: trophy.camera. I tried several times but my photos were never deemed award-worthy by the camera.


Burhan Ozbilici, WPP of the year 2017


and its trophy.camera version?

The work reminded me of the World Press Photo awards of 2011 when Michael Wolf won an honorary mention in Contemporary Issues with a photo he made by placing a camera on a tripod in front of a computer screen running Google Street View. The award raised a heated debate among photographers. For some of them, Wolf didn’t take the pictures, the cameras on Google street car automatically did it. This is therefore not photojournalism. And yet, who would have paid attention to these scenes if Wolf hadn’t recognized and framed them?

Trophy Camera v0.9 is tongue-in-cheek and irreverent but it points to a future when algorithms will win prizes that have traditionally recognized human creativity and vision.


Sander Veenhof, Patent Alert

Sander Veenhof, Patent Alert

Google, Microsoft and other tech companies are fighting over patents for the smart glasses that scan the environment and layer information over it.

One company owns the rights to scanning common hand gestures, while another holds a patent on helping you to cross the road. Patent Alert exposes the patenting obstacles that will intrude on our experiences with augmented reality headsets once the technology becomes mainstream.

Sander Veenhof created a HoloLens app that uses a cloud-based Computer Vision library to analyse your surrounding and warn you about gestures and behaviours that are not allowed because they are covered by a patent that’s not owned by the supplier of the device you are wearing.

Memo Akten, Learning to see: Hello World! [WIP R&D 3]


Memo Akten, Learning to See: Hello World! at DocLab Expo: Uncharted Rituals. Exhibition view in de Brakke Grond, part of the International Documentary Film festival Amsterdam. Photo Nichon Glerum

Memo Akten‘s Learning to See series of works uses Machine Learning algorithms to reflect on how we make sense of the world and consequently distort it, influenced by our expectations.

One of the investigations in the series, Hello, World!, explores the process of learning and understanding developed by a deep neural network “opening its eyes for the first time.”

The neural network starts off completely blank. It will learn by looking for patterns in what it’s seeing. Over time, the system will built up a database of similarities and connections and use it to make predictions of the future.

Interestingly, Akten’s description of the learning process holds a mirror back to us: But the network is training in realtime, it’s constantly learning, and updating its ‘filters’ and ‘weights’, to try and improve its compressor, to find more optimal and compact internal representations, to build a more ‘universal world-view’ upon which it can hope to reconstruct future experiences. Unfortunately though, the network also ‘forgets’. When too much new information comes in, and it doesn’t re-encounter past experiences, it slowly loses those filters and representations required to reconstruct those past experiences.

How far can we go when we draw parallels between the way a computer trains itself and the way we learn? Are humans the only one who are capable of turning learning into understanding? Or will computers beat us at that too one day? But perhaps more crucially, can computer help us see and oppose our own cognitive biases?

Famous Deaths. Step inside a mortuary chest and experience of the final moments of JFK. Or Gaddafi

23283776236_5a7cd4ef56_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Would you like to know or even experience what John F. Kennedy felt right before he was shot in Dallas, Texas on 22 November 1963? What Whitney Houston experienced moments before she accidentally drowned in her bathtub in 2012? Or what the last moments of Muammar Gaddafi or Lady Diana were like?

Famous Deaths JFK
Famous Deaths, JFK. Image courtesy of the artists

With their project Famous Deaths, Frederik Duerinck and Marcel Brakel are using the power of scent to evoke these final minutes. JFK, for example, might have smelt the autumn wind, the leather of the limousine seats, the perfume of his wife sitting next to him, the exhaust fumes of the vehicles…. Until the scent of blood, brains and gunpowder overtook his senses and life.

The installation takes the form of mortuary chests. Lay down inside one of them and immerse yourself in a fragrance documentary of the last minutes of some of the most famous, most tragic deaths in recent history.

These might seem like macabre experiences to revive and share but judging from the success the project had when i discovered it at the DocLab Expo: Seamless Reality (an exhibition organized by at DocLab and De Brakke Grond in the framework of IDFA in Amsterdam), the project has a lot of potential: as a form of immersive entertainment of course but also as an innovative field of research. In fact, the project started out of a fascination for the strength of smell in communication. Smell, as we know, is a powerful catalyst of memories and emotions. Yet, its potential is under-exploited in communication, design and story-telling.

I never managed to get inside a mortuary chest while i was at DocLab back in November. All spots seemed to have been booked in a matter of minutes that day. So i decided to go for the next best thing: an interview with Marcel and Frederick:

22681526134_e8946ed92d_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Hi Marcel and Frederik! How did you document and recreate the smells and sounds of the last moments of these people’s death? Did you work with scientists? Archivists?

We did all of the research to recreate the events ourselves via internet and film research. For instance with the Whitney Huston scenario we got access to police reports and police photos of the death scene. Out of this picture(s) we could extract a lot of information on the kind of smells that where present in her room, the exact kind of cigarettes she smoked, the brand of perfume she used. Interesting details. She used olive oil as a bathing cream. In the pictures you see a gravy bowl used for the olive oil standing in the bathtub. We tried to be as precise and as close to reality as possible but in the end we had to take some poetic freedom to recreate the scenarios. And of course sometimes we compressed the chain of event in time to speed up things a little.

For the smells we worked with specialists and smell designers. Some smells and perfumes we could just buy from the shelf, other smells where custom created.

Famous Deaths Whitny Huston
Famous Deaths (Whitney Huston.) Image courtesy of the artists

Famous Deaths Khadaffi
Famous Deaths (Muammar Khadaffi.) Image courtesy of the artists

0ense of smell TEC022_B18 copy
Sense of Smell, view inside the book. Image courtesy of the artists

sense-of-smell
Sense of Smell

Neither of you appears to be ancient. So why did you decide to investigate death, instead of any other and more cheerful smell manifestations?

Writing the Sense of Smell book we worked with 5 different themes to get more focus in the research. One of the theme subjects was Taboo. For instance people have a troubling relationship with their body odor. It confronts us with the fact that we are a biochemical systems, a system that is in constant change and in a way is already in a state of decay. Fruit produces smell caused by the amount of fermentation within the fruit. The level of decay determines the level of sweetness and the level of smell. We smell a fish to determine the level of freshness but in fact we are checking the level of decay because we don’t want to get sick eating it. In that way smell confronts us with (our own) mortality and is closely connection with death.

One of the other themes we investigated was Time. Smell is closely connected to our memory system. We can use smell as a time traveling tool to access hidden memories. At the same time, historic events are always kind of abstract to us, they are vague and unclear to us. You can envision a picture of a bread in your head but it is hard to access the smell of that bread. But if you think of a bread and at the same time smell a bread, the smell kind of projects this virtual image of a bread into the here and now. In this way Smell is a very powerful tool to create embodiment, a kind of analogue virtual reality time travel machine. So within Famous Deaths all of these ideas came together

Geur-Machine-1.1
Famous Deaths. Image courtesy of the artists

Why did you chose to work with smell and sound and not images? Why not the 3 of them together? How do you think using the 3 together would have altered the experience?

The idea to use smell for storytelling is not new and there are numerous experiments with it especially in combination with images, but we wanted to do something different. Within the project we choose historic media events that are already present in your mind. The sounds and smells are giving access to these very personalized memories. In that way, we create a dreamlike film experience within your own head. I find it much more interesting to make use of this internal in-brain projection (screen). The experience becomes much more intimate and personal in this way. For a few minutes you become the historic character in first person perspective and you get confronted with your own mortality. I think using images would have created more distance to the story.

23309877025_49d83ae3f0_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

I was reading an interview with you in which you mentioned that “A “scent printer” developed within the project will be used to create the sound/scent experience of “Famous Deaths.” Have you developed this scent printer?

Actually designing a printer was our first step into the project. To do a smell project you first have to have a device to control it. We made lots of prototypes and in the end we created a modular machine to control 32 different smells in a very fast and precise way. But when we finished it we didn’t have a clue about what kind of project we should create with it. The printer has a lot of potential to create all kinds of projects with. We are planning on using the machine to do scientific research with Universities in Holland, but we would also love to redesign the printer into a wearable version.

You’ve been presenting the installation for over a year now, what has the reaction of the public been like? Is the way they react to the death of Gaddafi very different from the way they react to the one of Whitney Houston?

All scenarios are very different from each other. Whitney Huston’s story is very sad and vulnerable. Gaddafi is very violent and action-packed. JFK has more contrast in smell design. Before people enter the installation they all know: in the end, everybody will die. But they don’t know the exact moment. That moment always comes as a surprise.

People sometimes also experience strange side effects. At the Gaddafi scenario we use the smell of a metal car burning. Some people get the impression the installation is heated at such a moment witch is not the case. Some people experience their sense of space changing or have the feeling to float within the installation.

22681524994_5c6f222a01_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

23014089750_e145d8fd47_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

You are a filmmaker and Marcel van Brakel is a theater director. How does this work feed into your own career?

As a theatre and filmmaker we are interested in creating immersive experiences. With smell we discovered a very powerful extra language tool. You can use it to enhance an experience or story, or to give a story a poetic context or contrast.

As a theatre maker I’m used to create experiences in a 360 context. But because of Famous deaths i feel we navigate more and more towards designing intimate embodied and individual media experiences. Virtual reality techniques and smell are excellent tools for that.

What’s next for Famous Deaths and Sense of Smell?

For the moment Famous Deaths is still getting lots of attention and is booked for more and more international shows. In January we will attend the Smart Fipa Festival in France and we will go to SXSW in Texas. After that we will take the installation with us to tour North and South America.

We are also working on a side project: “Synaptic Theatre” to make use of the brain as a kind of media player. In the project we want to create storytelling within the brain by direct stimulation of the brain.

In the project we will create theatrical experiences in combination with real time hormonal or electromagnetic neuro simulation of the audience or user(s). Within the project we will work together with scientists and are going to combine scientific research with design research in several smaller and bigger projects.

We are also setting up a 360 degrees Lab to do Virtual Reality projects to experiment with new ways of immersive interactive storytelling. In some of these we will also work with smell, touch and taste.

22681526624_a072c266f1_c
Famous Deaths, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Thanks Marcel and Frederik!

Famous Deaths is part of Sense of Smell, an international co-creation and research project designed by Communication and Multimedia Design (CMD) Breda at the AVANS University of Applied science, the Netherlands, and part of the European network VIVID.

Previously: ‘DocLab Expo: Seamless Reality’ explores the future of storytelling.
Also part of the IDFA DocLab events: Swatting, vintage VR and virtual museum for stolen art. My notes from DocLab: Interactive Conference 2015 and Sheriff Software: the games that allow you to play traffic cop for real.

I posted my photos of the DocLab exhibition and conference on flickr.

‘DocLab Expo: Seamless Reality’ explores the future of storytelling

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Opening night Doclab. Photo Nichon Glerum
DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

The DocLab Expo: Seamless Reality exhibition takes place each year in Amsterdam in the framework of IDFA, the International Documentary Film Festival Amsterdam. The event showcases thirty installations, virtual reality environments, experiments with artificial intelligence, and other interactive projects that explore the future of documentary storytelling.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Karim Ben Khelifa, The Enemy. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

The show is part of the Seamless Reality program set up by IDFA DocLab, a festival program for ‘undefined art and unexpected experiences’. Organized by DocLab in collaboration with Flemish art center De Brakke Grond, the DocLab Expo: Seamless Reality is an exhibition that shouldn’t interest me (i’m normally not into VR.) Except that it does. Mostly because:

1. Most of the works exhibited challenge the concept of ‘interactivity’ far more efficiently than many of the projects i see at some of the media art festivals i visit each year. Because their focus is on story-telling (rather than, say, subverting or expanding the limits of technology), they tend to me more compelling too. On the other hand, these experimental works make an innovative use of technology and thus tend to feel constrained by the label ‘documentary’. They fall somewhere between media art and documentary making. And that’s what makes them cutting-edge and engaging.

2. DocLab Expo is a free exhibition and it offers a great mix of socially-engaged works that deal with issues ranging from factory labour to activism in Pakistan and experiences that see visitors engage in unconventional activities such as eating meat ice cream, entering a mortuary chest to experience the death of JFK, or being offended by a talking CCTV camera.

3. I saw that DocLab provided visitors with cleaning cloths to sanitize the VR goggles. I don’t want to sound like a hygiene freak (that’s called mysophobia says wikipedia) but try and stay poised when the person who was watching the documentary before you hands you foundation-covered goggles to slip on your head. So they might not be super eco-friendly these little cleaning cloths but they come in handy in such settings!

Here’s a subjective tour of the exhibition and i’ll try to keep it short-ish to make up for boring you stiff with my interminable notes from the DocLab: Interactive Conference the other day:

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Kyle McDonald, Exhausting a Crowd. Netherlands. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

0awherearetyou0
Kyle McDonald, Exhausting a Crowd. Netherlands

In 1974, George Perec sat in Saint-Sulpice Square in Paris and started to write down the activity around him. Not the remarkable and the curious, but all the mundane things that usually pass unnoticed. The experimental literary work, called An Attempt at Exhausting a Place, has inspired one of Kyle McDonald‘s latest artworks.

Nothing truly notable happens in the video of Exhausting a Crowd. You see people going about their daily life in a public place. What gets your attention though is that the work allows observers to attach a “tags” to the people in the crowd, either as speech bubbles with imaginary conversations or thoughts or annotations that comment on the surrounding scene. Some of the notes are witty, others are touching or just plain silly. Although the faces of passerby are too small and blurry to allow you to identify anyone in the crowd, Exhausting a Crowd shows the potential for a new type of hyper-detailed surveillance that combines automation and the participation of citizens.

The version commissioned by DocLab showed images from different locations in Amsterdam, Rotterdam and elsewhere in The Netherlands.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Opening night Doclab. Photo Nichon Glerum
Koert van Mensvoort, Bistro in Vitro. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 22-11-2015, IDFA International Documentary Filmfestival Amsterdam. Doclab Conference in de Brakke Grond. Photo Nichon Glerum
Koert van Mensvoort, Bistro in Vitro. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Koert van Mensvoort, Bistro in Vitro. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Bistro in Vitro is a fictional restaurant specializing in lab-grown meat dishes and inviting us to reflect upon the many ways we might consume it in the future: knitted, see through, tickling your throat, sourced from celebs, etc. Or in the form of an ice cream.

At DocLab, an In Vitro van was offering visitors flavours of speculative meat ice creams that ranged from ‘The Ice Queen’ containing DNA from the Dutch royal family to ‘The Dragon’, a fire-breathing blend of meat that tastes of mythological beast.

Ultimately, philosopher and scientist Koert van Mensvoort and his organization Next Nature Network are using Bistro In Vitro to explore the blurring of boundaries between nature and technology, between what is “born” and what is “made.” If you’re curious about their research, check out The In Vitro Meat Cookbook.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Bert Hana, Rebuild Fukushima. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

HUIS_2
Bert Hana, Rebuild Fukushima

Dagmar van Wersch and Bert Hana have used Google Street View to explore the Fukushima nuclear disaster zone and scan some of its buildings. Using the images they found on the platform, they created construction kits of the houses and invited visitors of DocLab to reconstruct the Japanese houses in Amsterdam by folding the paper kits into 3D buildings. Rebuild Fukishima combines these 3D paper kits with audio interviews of former inhabitants of the area who share their memory of the disaster and give us an idea of what it means to be unable to return home.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ziv Schneider & Laura Chen, RecoVR Mosul: A Collective Reconstruction. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ziv Schneider & Laura Chen, RecoVR Mosul: A Collective Reconstruction. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ziv Schneider & Laura Chen, RecoVR Mosul: A Collective Reconstruction. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

The unfortunate Mosul Museum has not only been badly looted during the 2003 Iraq War, it was further ransacked last year by ISIL. The video of the militants using sledgehammers to destroy historic sites and artifacts toured and shocked the world. RecoVR: Mosul, a Collective Reconstruction crowd sources images from people who had previously visited the museum to digitally reconstruct these artifacts in a virtual reality installation. The DocLab exhibition showed both a series of small 3D printed reproductions of the statues (such as the Lion of Mosul) and the VR environment. The work was created at Economist Media Labs in New York, and in collaboration with Project Mosul, a volunteer group that crowdsourced images of the destroyed Mosul artifacts.

23283768456_d44082bb7c_b
View of the exhibition DocLab Expo: Seamless Reality. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum


Anghad Bhalla & Ted Biggs, The Deeper They Bury Me

The Deeper They Bury Me brings you closer to Herman Wallace, a political prisoner in Louisiana State Penitentiary, a.k.a. Angola Prison, since 1972. One of the Angola Three men, Wallace has spent more than 41 years in prison and was released October 1, 2013, 3 days before he died of cancer.

He was sent to Angola prison in 1971 for armed robbery and convicted a year later of stabbing a prison guard together with Albert Woodfox. It is very likely that they didn’t commit the crime but were framed because of their political convictions.

The very moving documentary plays a bit like a video game and allows you to look around the solitary confinement cell where Wallace spent 4 decades of his life but also to discover the imaginary home he was asked to design by artist Jackie Sumell. Animations depict his dreams. Archive footage allow us to hear him talk about his strategies for survival, the emotional strains of spending 23 hours per day in a cell, the lack of privacy, the infrastructure of U.S. prison buildings, etc. There is a lot to explore in this work but you’re allowed to wander around Wallace’s world for no longer than 20 minutes, the maximum amount of time a prisoner can spend on the phone each day.

The Deeper They Bury Me also illustrates the American penal system, with its cruel practice of extended solitary confinement, the high incarceration rate and the disproportionate number of coloured people sent behind bars.


Brett Gaylor, Do Not Track

Do Not Track combines short video interviews with privacy activists and interactive elements to reveal who is tracking us online and how private information extrapolated from our Internet activities is being collected, used and monetized.

Spending time on the platform watching the videos quickly gets pretty upsetting. The narrator’s identity as well as the content and language of the videos are determined by your own data: your IP address, your Facebook activity, your browsing habits, etc. Our post-Snowden age suddenly gets tangible.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ross Goodwin, word.camera. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ross Goodwin, word.camera. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Ross Goodwin, word.camera. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

The relentlessly amusing word.camera is a talking surveillance camera that can zoom on a person’s face in the crowd, ‘read’ it and say out loud what it “sees.”

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Karim Ben Khelifa, The Enemy. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

The Enemy is an immersive installation which brings face to face combatants from opposite sides. Wearing the VR goggles, you stand in the same room as them and are a witness to their dreams, fears and reasons to fight. With this work, war photographer Karim Ben Khelifa wanted people to feel closer to conflicts and its protagonists but also to try and understand what motivates human beings to engage in violence and to show that, very often, people fighting each other have more in common than they would admit.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Jan Rothuizen, Drawing Room. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Jan Rothuizen, Drawing Room. DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum


Jan Rothuizen, Drawing Room

Jan Rothuizen, who gave a fantastically interesting talk last year at the DocLab conference, created what is probably the first ever “drawn reality.” Having spent some time in the tower room on the roof of De Bijenkorf, the oldest department store in Amsterdam, the artist translated his experience into a 360º virtual reality drawing complete with the sharp witty annotations that characterize his work.


Loïc Suty, The Unknown Photographer

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Opening night Doclab. Photo Nichon Glerum
DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

2imperi9223e_b
Eefje Blankevoort & Dana Lixenberg, Imperial Courts at DocLab Expo: Seamless Reality, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum

Also part of the IDFA DocLab events: Swatting, vintage VR and virtual museum for stolen art. My notes from DocLab: Interactive Conference 2015 and Sheriff Software: the games that allow you to play traffic cop for real.

I posted my photos of the DocLab exhibition and conference on flickr.

Sheriff Software: the games that allow you to play traffic cop for real

Over the past few years, artist Dries Depoorter has been exploring issues of privacy in ways that are at times thought-provoking and at times almost comical (often both.) He started by looking into his own privacy, either through a program that was taking and sharing online one screenshot a day of his computer screen at a random time or through a website that used Google Streetview to disclose in real time the artist’s exact location and direction.

His recent works explore how other people are willing to surrender their privacy for the sake of entertainment, safety or just the prospect of a one night stand. One of the outcomes of this approach is the recently released and muchdiscussed Tinder In which puts side by side and to often surprising outcomes the profile pictures that an individual selects to represent himself or herself on two platforms that are at opposite ends of the social spectrum: LinkedIn & Tinder.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Installation view of Sheriff Software (JayWalking) at the DocLab: Seamless Reality exhibition, part of IDFA International Documentary Filmfestival Amsterdam. Photo credit: Nichon Glerum for IDFA DoLab

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Installation view of Sheriff Software (JayWalking) at the DocLab: Seamless Reality exhibition, part of IDFA International Documentary Filmfestival Amsterdam. Photo credit: Nichon Glerum for IDFA DoLab

Depoorter’s new investigation into privacy will premiere this week at the DocLab: Seamless Reality exhibition in Amsterdam (more practical info below.) The set of works, grouped under the name Sheriff Software, invites people to not just be the object of the attentions of the CCTV cameras that relentlessly gaze upon us but also to use them, turn the scrutiny back at the police and even play traffic cop.

The first piece in the series is JayWalking, a software that scans traffic lights at intersections in different countries, check whether the light is red or green and spots anyone braving the red light and jaywalking. Visitors of DocLab are given the opportunity to witness any infraction and decide whether or not to send to the police a screenshot that proves the transgression. The consequence of the decision of the public is made even more tangible by a counter at the bottom of the screen that shows how much the fines are for the offense in the country where it’s being committed.

Will we report the unsuspecting jaywalker? Will we click on the button that can send a screenshot of the violation to the nearest police station?

Are we going to empathize with our fellow pedestrians? Or are we going to point the finger and divulge their minor crimes? I doubt that people at DocLab will be willing to snitch on jaywalkers when there is a group of people around. But how different would it be if they were alone at the moment of taking the decision? JayWalking reminded me of an experiment that took place in 2006 when Shoreditch residents were given access to live CCTV footage of their neighbourhood on their own tv sets. People were invited to tune in the “community safety channel” and report any suspicious behaviour by text to the local police. Apparently, local CCTV cameras attracted viewing figures with an “equivalent reach of prime time, week-day broadcast programming”.

Then of course, there’s the no so minor detail of face recognition systems. I guess the JayWalking screenshots will only be showing the blurry silhouette of the offender. But what will happen if one day/when surveillance cameras are equipped with automated facial recognition technology?

005vc15970While Jaywalking enables people to spy on other people, the second work in the Sheriff Software series lets people watch the watchers. Called Seattle Crime Cams, the piece relies on the Seattle Area Traffic and Cameras system which monitors city traffic.

Seattle not only shares the live stream of its CCTV network, it also share the dispatch from the Seattle Police Department radio. Depoorter’s Seattle Crime Cams will connect police calls with the live stream of the nearest webcam. The public will be able to witness incoming calls that report a traffic incident or a robbery and see how long it takes for the police to arrive at the scene.

Seattle Crime Cams turns us into ultimate long-distance disaster tourists, virtually present at the scene of the crime in Seattle. In this city, which is filled to the brim with traffic cameras, the police make the calls they receive available online. Using the latest calls, the closest live webcams are constantly zooming in on the very latest violations.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Installation view of Sheriff Software (Seattle Crime Cams) at the DocLab: Seamless Reality exhibition, part of IDFA International Documentary Filmfestival Amsterdam. Photo credit: Nichon Glerum for IDFA DoLab

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Installation view of Sheriff Software (Seattle Crime Cams) at the DocLab: Seamless Reality exhibition. Photo credit: Nichon Glerum for IDFA DoLab

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
Installation view of Sheriff Software (Seattle Crime Cams) at the DocLab: Seamless Reality exhibition, part of IDFA International Documentary Filmfestival Amsterdam. Photo credit: Nichon Glerum for IDFA DoLab

Sheriff Software is premiered at IDFA DocLab, a ridiculously interesting program of screenings, performances, talks, exhibition and other events that explore the future of documentary storytelling. Think augmented reality, artificial intelligence, live cinema and interactive experiments. The installation is part of the DocLab: Seamless Reality exhibition (19-29 November) and will be also be one of the highlights of DocLab Live: The Art of Artificial Intelligence (23 November.) The program is organized by IDFA (the International Documentary Festival in Amsterdam) and the Flemish Arts Centre De Brakke Grond.

Amsterdam, 19-11-2015, IDFA International Documentary Filmfestival Amsterdam. Photo Nichon Glerum
DocLab: Seamless Reality exhibition, part of IDFA International Documentary Filmfestival Amsterdam. Photo credit: Nichon Glerum for IDFA DoLab