Digital Media Winter Institute 2019
SMART Data Sprint: Beyond Visible Engagement
28 January – 1 February 2019
9:30 – 17:30 I #SMARTdatasprint I Research Blog I
Facebook Group: SMART Data Sprint I @iNOVAmedialab
Universidade Nova de Lisboa I NOVA FCSH I iNOVA Media Lab
˚ ˚ Project Title
Project title: How is health being depicted in online debates? The cases of ‘detox practices’ and ‘vaccines’
Project leader: Elaine Teixeira Rabello
Team Work (alphabetical order): Bia Carneiro | Daniel Ribeiro | Fábio Gouveia | Jan Alyne Prado | Marit Brademann | Tania França.
Project pitch slides and final presentation slides
https://drive.google.com/file/d/1y1JQxl9jhbrr0RY8Kqb6Iy78qRs3lMmO/view?usp=sharing (pitch)
https://drive.google.com/open?id=1M6bNng8fUNP8vDT6djREIOLQzKLR7oJbftKl6mD-Y-Q (final presentation)
Key Findings
Introduction
Research Questions
Research Design Visual Protocol
Methodology
Findings
Discussion/Conclusion
References
- On google and websites, the controversy of detox is debated in terms of drink and food, beauty, health and marketing of products on Google. Debunkers of both detox therapies and anti-vaxxer movements are more prominent on this platform.
- On Instagram, the discourse of anti-vaccine predominates over the pro-vaxxers. Relating to detox, the separation of themes is less visible.
- Youtube seems to play an essential role in comedy related to the anti-vaxxer movement.
- It is noticeable the existence/agency of platform-concentrated actors – namely anti-vaxxers on social media vs pro vaxxers on top ranked websites.
- The Facebook pages case, where we see the overlap between detoxers and anti-vaxxers, shows that investigating the concentration of actors per platform and cross-platform actors (eg, the Vaxopedia as a concentrated platform and the Dr Oz as a cross-platform) can give us an idea of how some subjects can be captured by other issues, like the interfaces between detox practices and vaccine controversy.
The use of web platforms for social interaction and for generating, accessing, and building information is a characteristic of contemporary society. There is indeed a plethora of information circulating through digital media; however, it does not mean that the population is better informed. On the contrary, the overabundance of information comes with the price of being difficult, at least for the average user, to sort the wheat from the chaff when it comes to internet-mediated communication.
Bad health advice and misinformation of all kinds, for instance, can be easily found on the internet. Considering these representations are formed and circulated among people in any societal group, a critical part of any study on social, sensitive must take into account the role of digital arenas on their sociological perspectives to understand the notions of science, innovation, health and its process, which forms the basis for people’s attitudes. A way of conducting studies with digital methods in this direction is to start from the information that circulates and, from native digital objects and methods, to track this information on the web, seeking for not only actors but mottos of engagement and interests of different people.
This considered, in this project, we aimed to use digital methods to discuss the central role of the users/general public, having a more in-depth look on recent health-related topics recently or currently presented on the internet: the detox therapies and the anti-vaccination movement. We are specifically looking for understanding how these facts circulate and impact differently on the engagement, depending on the arguments, speech and interests mobilised for the dominant voices. To do so, we adopted the framework of Critical Analytics (Rogers, 2018), trying to address in each case:
Who is debating? What are their discourses? [dominant voices]
Who is there and who is not there? [concern]
To whom is it a matter of concern and for how long? [commitment]
What are the program-neutral-anti program arguments [positioning]
Who is joining the pro-neutral-anti positions? [alignment]
The case of detox therapies – the movement and the spread of “detoxifying” food practices – illustrates how health-related knowledge goes beyond the biomedical fields of expertise, and is (re)appropriated for commercial interests of all kinds, with a high appeal to the society, merging aesthetic and health discourses. Several narratives about “toxins” and various processes of removing them from the body circulate on the Internet, especially on social networks and on blogs claiming for the ultimate expertise in the subject, in a tense and contradictory relationship with biomedical science. We hypothesise that these narratives constitute a peculiar style of thought and epistemology built embedded on the digital platforms.
Previous research has worked on scrapping media items on Instagram and crawling expert blogs and websites. There are the well-known dominant voices on detox:
Rebeldietician
Mercola
Naturalnews
doctorOz
Foodbabe
From the mapping of “Do It Yourself” narratives dwelling on the universe of detox practices, we have identified a folk conception of what a “toxin” is, how they act on the body, and how to get rid of them in other words. We also may notice the existence of one or more therapeutic system (s), with its pathophysiology, nosology, diagnosis and therapy digitally “created”, finding no resemblance on information derived from health services or academic circles. Such “health” practices range from the innocuous to the dangerous, with economic importance, given the potential for consumption systems and highlighting the importance of investigating the appropriation of the topic for the ordinary people. This has resulted in a public health concern; INSERM has recently published a communication stating that detox therapies and the spread of misinformation on the internet are a significant concern to French science How people give accreditation some discourses, recipes and expertise that deny scientific facts, turning (or aiming to do so) them into daily practice in their lives?
Another case is the anti-vaccination movement (whose people engaged in it call themselves “anti-vaxxers”) fastly spread around the world and especially on the internet. Despite being a consensus amongst scientists and public health agents, the efficacy and safety of vaccines started to be questioned by militants of the so-called anti-vaccine movement. These group of people have mobilised, mainly through internet and web platforms, a large number of people with their discourse of risk on immunising children. The speech, aiming primarily parents, has spread amongst Americans and Europeans and has had a significant impact on immunisation programs (Camargo Jr, Grant, 2015).
This topic depicts a controversy with direct impacts on health systems and public engagement in science and technology debates, playing with popular (non)information of health care, risk and protection. Why and how people start to deny a scientific, effective, well-proven and well-established knowledge/practice such as vaccination? Why and how it gains popularity among ordinary people? From previous monitoring, we see a kind of “counter expertise” without scientific basis, but that it resonates in some communities, many highly educated, and has been a barrier to effective immunization (since vaccination, to be effective, needs to be done on a large scale) and the anti-vaccination campaign on the internet have, therefore, directly impacted the health of an entire population.
From some preliminary research, we know that the starting point of most health controversies may rely on different dominant voices and, probably, circulates amongst ordinary people in different ways. We also know that these three facts have mobilised different interests, actors, communication strategies and discourses. We identified a list of the leading “voices” or “actors” – the dominant voices – who incites the online debate regarding each of these issues. During the sprint, the goals were:
1- Identify where, on the web, these debates have happened;
2- Map the issues on different platforms, aiming for critical analytics;
3-Identify the ideas, arguments and representations of health underlying publications and posts;
4- Infer possible outcomes, referrals and social impacts of what circulates in the network concerning health behaviour regarding the issues chosen as cases.
How is health being depicted on online controversies in the cases of ‘detox practices’ and ‘vaccines’?
During the sprint, we focused on:
What does “Detox” and “(anti)vaccine” mean in different online platforms?
Are these arenas dominated by misinformation or not?
Are the contents cross-platforms or they differ from each other?
In which spaces were misinformations not taken over?
Research design visual protocol
The scheme below shows the study design of our project. Notice that we designed the research per platform, considering the idea of their different affordances. However, the overall objectives and questions remained the same (the previous topic), despite the different ways of collecting, analysing and visualising data.
In the next session, we further explore the data collection proceedings per platform.
Using the DMI tool Lipmmanian Service (Google Scraper), we scraped the 100 top ranked URLs for each theme, according to the parameters: English (Language), anytime (time), anywhere (Location), anywhere on the article (where the query appears on the page). Then, we extracted the host domains from URLs and filtering unique domains using the DMI tool Harvester. We also proceeded to the visual analysis of these URLs. Using the same protocol mentioned for Lipmmanian Service, we used Google Image Scraper to retrieve the images of the top-ranked URLs.
Some visualisations displayed in the results section regarding URL´s images were made using ImageSorter tool to organise this image dataset.
For Detox, the query was “detox”. After the scraping, we manually coded the URLs regarding their positioning as pro-detox, anti-detox or neutral. Social media platforms (Facebook and Instagram) were considered neutral, as the scientific directories. URL or host pages that could not be retrieved or analysed, due to access restrictions, were excluded from the dataset.
Dataset: URLs and images [query: detox];
Extraction Tools:
Google Scraper to find the 100 top ranked URLs. Manual coding for positioning [anti vs pro].
Google Image Scraper to get the 400 top ranked 400 images per position;
Google Vision API to ML codification and labelling of images;
Visualisation Software:
Gephi [ForceAtlas2 – Giant Component – Modularity Class colours – Nodes as Images or Labels]; VOS Viewer; Google Spreadsheet for tables; TagCrowd for word clouds, ImageSorter to plot images.
For Vaccination, we used eight queries, considering the spelling variations. After scraping, we manually coded the URLs regarding their positioning as pro-vaccine, anti-vaccine or neutral. Social media platforms (Facebook and Instagram) were considered neutral, as the scientific directories. URL or host pages that could not be retrieved or analysed, due to access restrictions, were excluded from the dataset.
Dataset: URLs and images [queries: anti-vax, antivaxx, anti-vaccination, anti-vaxxer, anti vaxx, antivax, antivaccine, anti vaccine];
Extraction Tools: Google Scraper to find the 100 top ranked URL. Manual coding for positioning [anti vs pro]. Google Image Scraper to get the 400 top ranked images per position; Google Vision API to ML codification and labelling of images;
Visualisation Software: Gephi [ForceAtlas2 – Giant Component – Modularity Class colours – Nodes as Images or Labels] and Google Spreadsheet for tables, ImageSorter to plot images.
The manual coding, for both cases, we did by verifying in each website (through internal search) if they display content either in favour or against vaccination. We proceeded to verify these framing trends with the first 50 websites listed by the result of Google Scraper.
The objective of the Facebook analysis was to assess the types of information disseminated by selected prominent pages, and the extent to which they were connected, both with regards to sharing similar content – or content with similar messaging – and with regards to their institutional relationship to other pages.
For that, three groups of pages were defined:
- anti-vaccination advocacy
- detox therapies
- evidence-based medicine/nutrition
The latter was intended to show the counter-narrative to the first two.
As shown in the table below, each group is composed of five pages with a strong presence and following. The two most popular pages for the anti-vaccination and detox samples are the same, as they disseminate content about both issues.
|
Group 1) Anti-vaccination advocacy |
Group 2) Detox therapies |
Group 3) Science-based medicine/nutrition |
|
NaturalNews 2.9M likes Dr Joseph Mercola 1.8M likes GreenMedInfo.com 517k likes Dr. Tenpenny on Vaccines and Current Events 229k likes Autism Mothers 217k likes |
NaturalNews 2.9M likes Dr Joseph Mercola 1.8M likes Dr Mehmet Oz 6M likes @droz Rebeldietitian 1.9M likes Foodbabe 1.1M likes |
I fucking love science 25M likes WebMD 1.8M likes The Skeptics Guide to the Universe 1M likes Refutations to Anti-Vaccine Meme 254k likes Healthline 200k likes |
First, data from the pages were extracted with NodeXL Pro’s feature “Facebook Fan Page Network”. The extraction parameters were restricted to posts by the page admins and did not include comments or posts by other users. The timeframe was four years based on the most recent page (Rebel Dietitian, created in 2014): 01/01/2014 – 31/12/2018
Then we built five word clouds with the top 500 liked posts by pages of dominant voices for detox therapies (5), anti-vaccine advocacy (5), and evidence-based medicine/nutrition (5).
A network of the pages liked by the sample pages was also extracted with NodeXL Pro’s feature “Facebook Fan Pages Network” with 1.5 network level. Data was imported to Gephi, and the graph was created with a ForceAtlas layout. Nodes were coloured according to Modularity Class, with manual adjustment to colour-code according to the four groups – Antivaxx (green), Detox (blue), Antivaxx&Detox (purple), Evidence-based medicine/nutrition (orange). Edges are coloured by source node.
(see the results in the next section; the network presented does not include the page “Rebel Dietitian” and as such is intended for visualisation of preliminary results only. A final network with all sample pages will be further published).
For both cases – detox and vaccines – the dataset as composed by images and hashtags collected until 22/01/19 with the DMI tool Instaloader. The query for detox as “detox” and for vaccines we searched for “vaxxed”.
Data processing and filtering were made merging individual files and converting them to a CSV file. Hashtags with non-western characters were not considered in this sample. Only hashtags that were cited at least 1% of the sample were considered. Afterwards, the query hashtag was removed to better visualisation and cluster formation. VOSviewer was used for visualisation [text data – association strength – attraction = 4].
YouTube
For both cases, we used the Video List Module of DMI Youtube data tools. We set the iterations at 10 (resulting in the max of 500 items per query) and ranked the results by view count (results were sorted from highest to lowest number of views).
Spreadsheets were generated for each query with information from the videos that came in the search for the YouTube Data Tools tool, and videos are categorised by YouTube into 14 categories.
The visualisations were generated with the tool Raw Graphs.
DETOX CASE
From the top-ranked pages, we classified them according to their positioning. We saw that most of them were “pro-detox therapies”, which means, they were disseminating ideas, techniques and recipes to detoxify your body, mind and life. Only four of them were debunkers of the detox therapies, and we considered Wikipedia and the only neutral since the page “detox” exhibited information about both sides of the controversy.
|
PRO (“Detoxers”) |
ANTI (“Debunkers”) |
NEUTRAL (sources of info) |
Trying to understand the visual depiction of the pro-detox discourses on these websites, we scraped the images and built image-terms network, ending with 8 main clusters: cluster 1 is about arts and music; the group 2 concerns on person, coaching, beauty and body shaping; cluster 3 is about environment; cluster 4 depicts business and marketing strategies; cluster 5 shows products and talks about toxins, substances and chemistry; group 6 exhibits fruits and juices; cluster 7 shows vegetables and recipes; cluster 8 concerns on mushrooms and possible poisoning food; cluster 9 plants and flowers.
Platform: Google Images
Extraction Tools: Google Image Scraper
Terms: Google Vision API
Selection: pro-detox
Nodes: Images or Terms
Edges: images x terms
Software: Gephi
Settings: Giant Component – ForceAtlas 2
The same strategy was adopted to the detox debunker websites, ending with 6 main clusters: The cluster 1 depicts mainly bananas and other yellow images; cluster 2 concerns business and marketing strategies; the group 3 shows healthy foods, mostly fruits and vegetables; the cluster 4 exhibits drinks and juices; the cluster 5 shows plants; cluster 6 concerns beauty products.
Platform: Google Images
Extraction Tools: Google Image Scraper
Terms: Google Vision API
Selection: anti-detox
Nodes: Images or Terms
Edges: images x terms
Software: Gephi
Settings: Giant Component – ForceAtlas 2
When taking the two networks, we can see that the clusters relate to the same topics, which is expected as the debunkers mention the same issues detoxers discuss in their websites. The same result is seen when plotting the images of pro and anti-detox altogether. Both positions use Colorful fruits, vegetables, juices, smoothies, with background supplements and beauty products.
Detox images
Tool: Google Scraper
Visualisation: Image sorter
Overall, the controversy is debated in terms of drink and food, beauty, health and marketing of products on Google.
Considering the content analysis of 500 top liked posts on the main pro-detox Facebook Pages, we came up with the following. Results related to images will be addressed further, in connection with Vaccine results, in this section.
Rebel Dietician FB Page – Tool: Node XL + Tagcrownd
Dr Oz FB Page – Tool: Node XL + Tagcrownd
Food Babe FB Page – Tool: Node XL + Tagcrownd
Dr Joseph Mercola FB Page – Tool: Node XL + Tagcrownd
Natural News FB Page – Tool: Node XL + Tagcrownd
From previous research, we knew that the detox case on Instagram was closely connected to the discussion of superfood, so we opted for using both queries. In experimental networking, we noticed that the co-hashtagging process seemed to cluster by language, as the cluster yellow may suggest. Anyway, the discourse on Instagram is firmly connected to body shaping, fitness and lose of weight concerns.
Instagram co-hashtag network of Detox.
Nodes: hashtags
Edges: co-occurrences of
Software: VOSviewer
Cut: Presence > 1% of the sample
Youtube
The ranking of the visualisations shows us that videos, related to Education, Music, How to & Style, People & Blogs and Entertainment, were the main categories of the detox videos visualisation.
Videos “Detox” (most views) by category: Heatmap, data from Jan 31 2019 – Tools: Youtube Data Tools + Raw Graph
Regarding the “Like” attribute, the videos related to How to & Style, People & Blogs were the ones which received most likes.
Videos “Detox” (most likes) by category: Heatmap, data from Jan 31 2019 – Tools: Youtube Data Tools + Raw Graph
Another attribute analysed was the one referring to comments. It indicated that the categories How to & Style, People & Blogs and Entertainment were the ones who were the most commented.
Videos “Detox” (most comments) by category: Heatmap, data from Jan 31 2019 – Tools: Youtube Data Tools + Raw Graph
VACCINE CASE
From the top-ranked pages, we classified them according to their positioning. We saw that the majority were “pro-vaccine”, which means, they were defending the vaccine as a core public health strategy and debunking the anti-vax program. The only anti-vax website listed on the top-ranked vaccine websites on Google was “Vaxopedia.org”, as follows:
|
PRO-Vaccine |
ANTI-Vaccine |
NEUTRAL (source of info) |
Trying to understand the visual depiction of the pro-vaccine discourses on these websites, we scraped the images and built a network image-terms, we ended up with a network, which allowed us to pinpoint 7 some clusterized topics: cluster 1 is about hair, sleep, skin, smile and other human features related to wellness; group 2 depicts medical equipment, babies and healthcare facilities; the cluster 3 is about art, performance and events, probably vax campaigns; bunch 4 depicts the pharma industry; group 5 shows institutions and governmental buildings; group 6 relates to wildlife, sports and environment; finally the cluster 7 shows scientific-related information.
Platform: Google Images
Extraction Tools: Google Image Scraper
Terms: Google Vision API
Selection: pro-vaccine
Nodes: Images or Terms
Edges: images x terms
Software: Gephi
Settings: Giant Component – ForceAtlas 2
For the anti-vaccine program, we followed the same strategy for the vaxopedia.org, ending more specific themes, all of them somehow related to marketing, propaganda and events for disseminating anti-vax campaigns. Even not clustering in details, we can see that, aside from the bottom left group related to body part depiction, all the others refer to media, propaganda material, graphs, illustration and informative diagrams, advertisement, events and festivals with spokespersons.
Platform: Google Images
Extraction Tools: Google Image Scraper
Terms: Google Vision API
Selection: anti-vaccine (vaxopedia)
Nodes: Images or Terms
Edges: images x terms
Software: Gephi
Settings: Giant Component – ForceAtlas 2
When plotting the images (we did it only for the pro-vaccine group, since there was only one anti-vax), we can the discourse stays towards syringe and variations around them (drawings, kids, arms etc.), logos, twitter printscreens and images of protests.
Pro-vaccine images
Tool: Google Scraper
Visualisation: Image sorter
From the previous exploration, we opted for using the queries antivaccine, antivaxmemes, antivaxxer & vaxxed. After exploring further, we understood that the query vaxxer was the best one in terms of examining both positionings regarding vaccines.
Looking at Vaxxed co-hashtag network , we see that the discourse on Instagram is strongly against vaccines, being the clusters red and orange clearly anti-vaxxer and other connected (e.g. clue, yellow, green) related to the controversy (already debunked) of the vaccines hashtag debate being the cause of autism and different harmful outcomes, being substitute for other practices that are supposed to be less dangerous or, even, better to the health.
Instagram co-hashtag network for Vaxxed.
Nodes: hashtags
Edges: co-hashtags
Software: VOSviewer
Cut: Presence > 1% of the sample
YouTube
The ranking of videos on “Antivaxxer” show that the top five categories refer to Comedy, Science & Technology, Educations, People & Blogs and News & Politics.
The videos categorised as Comedy were the most watched, followed by Science & Technology. Those labelled as People & Blogs, News & Politics and Education have more or less equal access.
Videos “Antivaxxer” (most views) by category: Heatmap based on data from Jan 31 2019 Tools: Youtube Data Tools + Raw Graph
Proportionality, among those who have received likes, also remains in that grouping.
Videos “Antivaxxer” (most likes) by category: Heatmap based on data from Jan 31 2019. Tools: Youtube Data Tools + Raw Graph
The categories “Comedy” and “Science & Technology” have most comments.
Videos “Antivaxxer” (most comments) by category: Heatmap based on data from Jan 31 2019. Tools: Youtube Data Tools + Raw Graph
According to the rankings most viewed, likes and comments, indicate that the less viewed videos of Science & Technology were, on the other hand, the most commented ones. Those of “Science & Technology” and “News & Politics” also received the most likes.
The Facebook case: are the vaccine and detox movements somehow connected?
A network of the pages liked by the sample pages was also extracted with NodeXL Pro’s feature “Facebook Fan Pages Network” with 1.5 network level. Data was imported to Gephi, and the graph was created with a ForceAtlas layout.
Nodes were coloured according t Modularity Class, with manual adjustment to colour-code according to the four groups – Antivaxx (green), Detox (blue), Antivaxx&Detox (purple), Evidence-based medicine/nutrition (orange). Edges are coloured by source node.
Regarding their Facebook pages network, Detox and Antivaxx pages have connections at the “institutional recognition level” (i.e. pages liking each other), but – as expected – have no connection to evidence-based medicine pages. The exception is Dr Oz, which is only connected to this third group, despite belonging in the Detoxer category. This raises the question about how celebrity “doctors” positions themselves to others in their respective fields and their concern with legitimacy.
Also, the Detox and Antivaxx pages predominantly disseminate their content. All top URLs and domains tended to be from the page admins’ websites, and as such, there is no content overlap and/or central sources these pages all go to for content.
See on the table below the main connections identified for each of the dominant voices.
|
Rebel Dietitian |
Dr Mehmet Oz |
Food Babe |
Natural News |
Dr Joseph Mercola |
Green Med News |
Autism Mothers |
Dr Tenpenny |
Health Line |
I fucking love science |
|
Top domains |
|||||||||
The main point of this project was to address digital platforms under distinct approaches, according to their affordances. E.g. On Youtube, the channels may provide possibilities to view in depth quali + quanti perspectives on Detox, according to distinct categories and platform affordances. Subcategories facilitate to observe the proportions and patterns of actions towards the channels. For instance, it was possible to see that the Detox videos belonging to comedy were the most accessed videos, but the ones categorized under Science and Technology were proportionally the most commented (although not the most accessed ones).
Specialized and news websites, on the other hand, may provide perspectives under which it is possible to observe subframes related to a “bigger” issue (Vaccine), “ideological” and critical affiliations among them (eg. through Google Scraper, Issue Crawler etc).
It was interesting to observe patterns of circulation/relevance among actors/platforms, such as Amazon as a key actor which appears on Google, Youtube etc.
Also Vaxopedia (anti-vaccine) is very active across platforms regarding thematization.
On Instagram, regarding vaccine issue, it was possible to see the composition of thematic groups, on gluten diets, maternity, autism, conspiration theory on farmaco cartels, and non western biomedical activities, such as alternative medicine and therapies.
In case of Detox, we found more difficult to analyse in depth, due to language segmentation. Nevertheless, some concepts, such as fitness, health, diet, beauty and vegan were found as prominent subcategories. This analysis would need a better query design, as Detox is a worldwide used term.
Another interesting finding was the overlap between Detox and Antivaxx page likes; Dr. Oz only page is connected to evidence-based medicine/nutrition pages.
Overall, this project addressed logics of the circulation of scientific information on fake news, conspiracy theories and misinformation regarding health-related issues (Camargo Jr & Grant, 2015), aiming to understand the mobilisation of expertise on health debates (Rabello & Camargo Jr, 2019), focusing on the role of the public perspective of science and technology.
This study sought to identify aspects related to the circulation of knowledge in social media regarding Detox and Vaccine issues. The data and analysis resulted on a list with further questions that may support the reformulation of public policies, specifically the public health ones.
- What is the implication (consequences) for health services regarding the practice of therapies stimulated and oriented by social media without technical support?
- Should the Primary Care protocols, for example, give special attention to families that are guided by the anti-vaccination movement?
- What is the impact on the health for the population due to the use of supplements, medicines which are sold as a solution to aesthetic problems, diabetes development, kidney and heart diseases?
- How should the services be organized, and professionals be qualified for these issues? The analysis performed from the data collected in this study brings important elements which require more in-depth studies, in order to answer how health management can benefit from the use of interactive data on social networking sites for a more appropriate health care.
Camargo K; Grant R. (2015) Public health, science, and policy debate: being right is not enough. Am J Public Health; 105(2):232-235.
Collins H, Evans R. (2007) Rethinking Expertise. Chicago: University of Chicago Press.
Metze-Burghouts, T et al. (2018). Visual Analysis and Controversies. Digital Methods Initiative Summer School Wiki, University of Amsterdam. Amsterdam, 2018.
Meyer ET; Schroeder R (2015) Knowledge Machines: Digital Transformations of the Sciences and Humanities. Cambridge, MA: MIT Press.
Pearce W et al. (2018). Visual cross-platform analysis: digital methods to research social media images. Information Communication & Society, p 1-20, 2018.
Rabello, E T et. al. (2018). Visualising engagement on Zika epidemic. Smart Data Sprint Blog, Inova Media Lab, Universidade Nova de Lisboa. Lisboa, 2018. https://metodosdigitais.fcsh.unl.pt/smart-2018/project-reports/visualising-engagement-on-zika-epidemic/
Rogers, R (2018). Otherwise Engaged: Social Media from Vanity Metrics to Critical Analytics. International Journal of Communication : IJoC, 12, 450-472.
Rogers, R (2015). Digital methods for web research. In Scott E; Kosslyn, S. (ed). Emerging social and behavioral sciences.,. Emerging trends in the social and behavioral sciences. John Wiley & Sons.
Rogers, R. (2013). Digital Methods. London: MIT Press.
Turnhout, E., M. Stuiver, J. Klostermann, B. Harms and C. Leeuwis (2013) ‘New roles of science in society: Different repertoires of knowledge brokering’ Science and Public Policy 40 (3): 354-365.