SMART Data Sprint 2023


In the 2023 #SMARTDataSprint, participants will learn how to apply quali-quantitative methods to map and understand the different schemes of image classification of multiple vision APIs, using networks and manual coding.

Artificial intelligence (AI) is ubiquitous in everyday life and popular imagination. From voice recognition and machine vision to chatbots capable of writing poems, AI systems are trained to do what humans can or cannot do. These systems offer solutions for digital platforms, the public sector, transportation, manufacturing and beyond. The 7th edition of the SMART Data Sprint invites participants to empirically explore AI outputs and pinpoint what schemes of image representation are currently available in the vision API market. As a field of AI, computer vision revolves around interpreting visual information from images and videos. In short, it refers to developing algorithms and associated systems to learn behaviour strategies within specific environments through instructions and training data sets. Also, computer vision systems stipulate representation schemes that vary according to the platforms that use them. A well-known example of machine vision is the detection and extraction of information about entities in images, e.i. “human face”, “fashion accessory”, “photo shoot”, which, in turn, facilitate the analysis of image collections.

Private big tech companies and open-source projects have made computer vision methods available through application programming interfaces (APIs). However, vision APIs are still relatively new web technologies. For example, Amazon Rekognition and Clarifai vision APIs were launched in late 2016, whereas Google Vision API was in early 2017. From a methodological perspective, it is challenging to keep updated with the opportunities and controversies of vision AI solutions, particularly for scholars aiming to repurpose these computational mediums for research. In vision API recent studies, researchers point out that the lack of documentation or “proper” training for these models has raised social concerns and debates about race and gender discrimination [1]. While they have questioned the performance of vision APIs [2]. On the other hand, vision APIs have facilitated tracking image circulation [3] and the making of new methods and methodologies [4]. But little attention has been paid to developing a good understanding of the AI technique in use.

We move beyond single-vision API studies in this pocket version of the SMART Data Sprint. Participants will map what entities characterise different vision APIs (knowledge representations). Also, ask whether and when these are good enough for analysing image collections (methodological inquiry). Finally, they will learn to make sense of Vision AI taxonomies through networks of images and computer vision outputs (and following digital methods recipes!).

[1] Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdsija, S., Schoonvelde, M., & Lockhart, J. W. (2020). Diagnosing Gender Bias in Image Recognition Systems. Socius, 6, 2378023120967171. https://doi.org/10.1177/2378023120967171

[2] Mintz, Silva et al. (2019). Interrogating Vision APIs. 2019 SMART Data Sprint. NOVA University Lisbon. https://www.researchgate.net/publication/332910402_Interrogating_Vision_APIs

[3] d’Andrea, C., & Mintz, A. (2019). Studying the Live Cross-Platform Circulation of Images With Computer Vision API: An Experiment Based on a Sports Media Event. International Journal Of Communication, 13, 21. Retrieved from https://ijoc.org/index.php/ijoc/article/view/10423/2627

[4] Omena, J. J., Elena , P., Gobbo, B. ., & Jason , C. (2021). The Potentials of Google Vision API-based Networks to Study Natively Digital Images. Diseña, (19), Article.1. https://doi.org/10.7764/disena.19.Article.1

………Organizers: Janna Joceli Omena, Jason Chao, Ana Marta Flores, Rita Sepúlveda & Elias Bitencourt.

Venue, Applications and Tuition Fee

  • Venue:

In a pocket version, the 7th edition of the SMART Data Sprint is taking place at the University of Amsterdam, being held jointly with the Digital Methods Winter School. 

Opening Time: 9 January at 9.15am Amsterdam time. 

Opening Day Location: University of Amsterdam, Roeterseilandcampus. Building C Room C0.02 Nieuwe Achtergracht 166-1018 WV Amsterdam. Spill-over rooms (when C0.02 is full): REC B2.05, REC B2.06, REC B2.07, REC B3.03

Everyday Winter School location: Media Studies, University of Amsterdam. Turfdraagsterpad 9. 1012 XT Amsterdam

  • Applications deadline and tuition fee:

To apply please send a letter of motivation, your CV, a headshot photo, 100-word bio as well as a copy of your passport (details page only) to smart.inovamedialab [at] fcsh.unl.pt, with a copy to winterschool [at] digitalmethods.net. The full program and schedule of the joint Winter Schools are available by 19 December 2022, at digitalmethods.net.

Applications open at

Deadline for Applications

Tuition Fee **

14 December 2022

5 January 2023

EUR 347,- [all participants]

Accommodation

#SMARTDataSprint pocket version will be held jointly with the Digital Methods Winter School, Amsterdam. Here are some suggestions for accommodation:

The Social Hub Amsterdam (Amsterdam West)

Jan van Galenstraat 335 1061 AZ Amsterdam, The Netherlands Tel: +31 20 760 4000 (Arrival: 7 or 8 January 2022; Departure: 14 January 2023) https://www.thestudenthotel.com/amsterdam-west Reservations [at] thesocialhub.co or tel. +31 20 760 7575. Here is the promo code (with instructions) for online booking.

The Volkshotel

Wibautstraat 150 1091 GR Amsterdam +31 20 261 2100 https://www.volkshotel.nl/

Preparation 

If you plan to join us in Amsterdam, please bring your computer and everything you need to support your work. Also, check below some methodological recipes, tutorials and research tools we recommend.

→ Digital methods recipes by #SMARTDataSprint collaborators and the Public Data Lab 

→ Research software by Jason Chao and the SMART Team

→ Data collection playlist, Working with spreadsheets by iNOVA Media Lab

→ The computer vision network approach to analysing image collections by JJO

→ Practical Labs playlist by SMART Data Sprint and collaborators 

We also love the following sources and research tools: Research software by Bernhard Rieder, RawGraphs tutorials, Facepager tutorials, Node XL by Social Media Research Foundation, Gephi tutorials by Mathieu Jacomy, YouTube Channel Data Pitman by Fábio Gouveia (in Portuguese) and DMI tools. 

Organizers

Social Media Research Techniques Group (SMART), iNOVA Media Lab-ICNOVA, NOVA University Lisbon https://metodosdigitais.fcsh.unl.pt/ (deprecated domain: http://smart.inovamedialab.org/)

Digital Methodologies Hub, www.digitalmethodologies.org 

  • About SMART Data Sprint 

The SMART Data Sprint is an intensive hands-on work week driven by digital methods projects about and with online data, media methods, web technologies, and software. The data sprint is directed by Janna Joceli Omena and constituted by Jason Chao, Ana Marta Flores, Rita Sepúlveda, Elena Pilipets, and Elias Bitencourt. Together they form the 2022 SMART team.

˚ ˚ SMART Data Sprint history in gifs 🤓✨

Below is a list of selected research outputs, projects and research software:

  • Book: Métodos Digitais: teoria-prática-crítica (2019)
  • Special issue: The Data Sprint Approach to Research: Experiments, Protocols and Knowledge (2022)
  • Project: MyGender (2020-2024)
  • Research software:
  • Chao, T. H. J. (2021). Memespector GUI: Graphical User Interface Client for Computer Vision APIs (Version 0.2) [Software]. Available from https://github.com/jason-chao/memespector-gui.
  • Chao, T. H. J. & Omena, J. J. (2021). Offline Image Query and Extraction Tool (Version 0.1) [Software]. Available from https://github.com/jason-chao/offline-image-query.
  • Chao, Jason (2021). Domain Name Extractor [Software]. Available from https://colab.research.google.com/drive/1NE35PpE05U2TngM5P1wdNI3i-TCqm89-?

˚ ˚ Social media:

An initiative to develop digital methodologies in the context of interdisciplinary projects. https://digitalmethodologies.org/