As we move into 2020, we thought we would take a moment to reflect on both our struggles and success of 2019. From our new staff additions Ziv Schneider and Alex Calderwood, to project launches (below), to new partnerships, Brown has had a big year!
Premiers and Launches
This year we saw yet another batch of Magic Grant funded groups pull off a stunning breadth of innovative media projects. The style of their launch has always varied from project to project, sometimes with a public event, and other times with the release of a story or the publication of a journal article.
At MIT, we experienced the launch of Charlotte Brathwaite and June Cross’ theatrical performance of Casting the Vote, an American Dinner Party which met an audience for a meal at MIT to discuss the suppression of black votes, and reminded us that not all reporting has to find an audience on the page or screen alone. The experience was then replicated in New York City at La MaMa in the East Village.
Casting the Vote premiers in Boston.
In October at Casa Refugio Citlaltépetl in Mexico City, Alejandra Ibarra Chaoul’s project Democracy Fighters was launched and has been an incredible success story about how to use the web to exhibit intricate reporting. You can interact with the archive at ddld.mx.
Sheets representing the work of journalists killed in Mexico for the launch of Democracy Fighters.
When Deportation is a Death Sentence (now The Global Migration Project), led by Sarah Stillman and Giorgia Lupi, spent the year transforming strenuous reporting into a massive database that tells the tragic stories of US deportees who face violence and death at the other end of their deportation. With this effort, they are set to launch a visualization accompanied by stories in The New Yorker this spring.
Research in Action
“Text-based Editing of Talking-head Video,” was published in ACM Transactions on Graphics to widespread publicity, demonstrating the researchers’ ability to change what a speaker appears to be saying in a video, and replace the dialogue with text of their choosing. The video accompanying the paper garnered over 300,000 views and the research was mentioned in dozens of publications including The Washington Post, The Verge, Fox News, Vice, Observer, and Business Insider. The team hopes the technology can be used for storytelling purposes: streamlining film edits and post-production, or adapting content for different audiences. On the ethical considerations of this technology, Fried writes,
We hope that publication of the technical details of such systems can spread awareness and knowledge regarding their inner workings, sparking and enabling associated research into the aforementioned forgery detection, watermarking and verification systems.
The research was conducted by a research group that included Brown Fellow Ohad Fried, supervised by Stanford Director Maneesh Agrawala.
An image from Scene Graph Prediction with Limited Labels, describing the system architecture.
In addition to this research, we saw major publications from a variety of our projects in journals such as ACM, AAAI, and IEEE. These included Learning to Engage in Conversations with AI Systems, Scene Graph Prediction with Limited Labels, Visual Relationships as Functions, and AI-based Request Augmentation to Increase Crowdsourcing Participation all from the project Learning to Engage in Conversations with AI Systems.
Brown in the Press
Mark Hansen and Bernat Ivancsics published an incredibly nuanced piece in CJR, Actually it’s about Ethics, AI, and Journalism, which tells a detailed story about the role–past, present, and future–of machines in journalism, and nearly does it in under 10,000 words. On top of that, The Follower Factory, an investigation into Twitter-followers-for-hire was a Pulitzer Prize finalist.
Democracy Fighters has been featured in at least a dozen Mexican news outlets, including in an interview with Chaoul in IMER, which you can listen to here, and an article in the Knight Center for Journalism in the Americas Blog.
With the relaunch of 1000Cut, there has been a flurry of press, including a mention in Patricia Marx’ piece in the New Yorker that explored the myriad ways that VR is being used to transport people into other places and into other bodies, a technique 1000Cut employs with the goal of enacting measurable social change.
The project Decoding Differences in Forensic Software demonstrated how computer scientists can play the role of journalist by auditing the code of forensic DNA software. They demonstrated that the program they analyzed skewed towards false positives in nearly a quarter of the samples they tested, meaning that in the 1300 criminal cases in which it has been employed, it is likely to have helped convict innocent people. Clarkson Professor Jeanna Matthews was interviewed on Spit, a DNA podcast sponsored by 23andMe, and Jessica Goldthwaite, a forensic-DNA legal expert on the team, appeared in the Netflix documentary series Exhibit A.
Jessica Goldthwaite appears on Exhibit A
Mackenzie Leake’s work for the project RoughCut was featured in a WIRED article How AI Will Turn Us All into Filmmakers, detailing how their software is able to understand the different styles of film shots, in order to speed up the laborious work of editing footage. The article makes the prediction that “an AI-propelled boom is about to transform the way we wield the moving image.”
Lectures at Columbia and Stanford
Just this past semester, we welcomed Alberto Cairo to Columbia, author of How Charts Lie: Becoming Smarter about Visual Information to speak to journalists about ‘graphicacy’ and being mindful as we interpret charts and all forms of visual data (read our interview here). At Columbia, we also hosted Stephanie Hankey, co-founder and Executive Director of Tactical Tech, to speak about the group’s recent work. They are a Berlin-based NGO focused on technology and human-rights, including their investigation of over 300 companies who sell insights from personal user data to political organizations around the world. Finally, we continued our Distinguished Lectures in Computational Innovation series, with speakers such as Yi-Min Chee on emerging use-cases of blockchain technology, Runa Sandvik, the former Senior Director of Information Security at the New York Times, who spoke about empowering security teams to support their high-risk users, and Dr. Fernando Perez, one of the inventors of the Jupyter notebook.
Maneesh Agrawala speaks with tech journalist Kara Swisher at Stanford.
At Stanford, we hosted a number of journalists and technologists for our Media Innovation Lectures. These included Kara Swisher, co-founder of Recode, who gave her thoughts on media trends and the state of the technology industry, (watch it here). We hosted Ceci Stallsmith, Slack’s Director of Platform Marketing, who spoke about predicting platform shifts and technological trends. We also welcomed people like Jeremy Bowers, Director of Engineering at The Washington Post, Neil Chase, CEO of the nonprofit CalMatters.org, and Wired Magazine Editor in Chief Nicholas Thomson to speak about leadership in media technology. Thomson talked about the ‘tech-lash’, the role of trust in technology, and how the magazine built an early following in the tech world reporting on the good and bad of technology’s impact on our lives. Read more about his thoughts in Ann Grimes’s CJR write up.
We also held a handful of workshops on the Columbia and Stanford campuses this year, bringing outside experts from Propublica, The New York Times, and MIT to teach a range of topics such as reporting on Twitter, data visualization, drone photography, and visually stunning science photography.
Designer, technologist, and social entrepreneur Stephanie Hankey speaks at Columbia.
New Partnerships
NewsCounts is a new project that aims to create a national network of publishers, journalists, and academic researchers to tackle problems related to covering the 2020 census. Kicking things off, Mark Hansen, Mike Ananny, Ben Livingston, and Angela Woodall are taking a trip from Detroit to Oakland, stopping along the way to pair local newsrooms with academic researchers such as social scientists, demographers, computer scientists and data scientists.
The Brown Institute has partnered with SIPA for Mapping Data Flows, a project that takes aim at understanding how data is collected and used by the major technology companies in the United States. Under the direction of John Batelle, Adjunct Professor and Senior Research Scholar at SIPA, the team poured over end user licensing agreements and Terms of Services of Amazon, Apple, Facebook, and Google, transforming textual descriptions of data usage into discrete data points. Juan Francisco Saldarriaga, senior data and design researcher at the Brown Institute, then built a multi-layer interactive visualization of the flow of user data from its collection to the point that it is used by the tech companies. Interact with Mapping Data Flows here.
Mapping Data Flows visualization
Get Connected
If you are interested in the latest from the Brown Institute, sign up for our newsletter or follow us on our site at brown.columbia.edu. Our annual Magic Grant Call for Proposals will launch in late January and you can get in touch with us directly at browninstitute@columbia.edu or brown_institute@stanford.edu.