Resources & Publications
All members of the network can share their recent work on media bias here.
Resources
Most recent models are published on Huggingface
[Benchmark, GitHub] MBIB – the first Media Bias Identification Benchmark Task and Dataset Collection
[Dataset, GitHub] BABE – Bias Annotations By Experts
[Scale/Questionnaire to measure bias perception] Do You Think It’s Biased? How To Ask For The Perception Of Media Bias (A set of tested questions to assess media bias perception to be used in any bias-related research)
[Dataset, Zenodo] MBIC -A Media Bias Annotation Dataset Including Annotator Characteristics
Publications
2025
Hinterreiter, Smi; Wessel, Martin; Schliski, Fabian; Echizen, Isao; Latoschik, Marc Erich; Spinde, Timo
NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback Proceedings Article Forthcoming
In: Proceedings of the International AAAI Conference on Web and Social Media (ICWSM'25), AAAI, Copenhagen, Denmark, Forthcoming, (Conditionally accepted for publication).
Abstract | Links | BibTeX | Tags: crowdsourcing, HITL, linguistic bias, media bias, news bias
@inproceedings{Hinterreiter2025NewsUnfold,
title = {NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback},
author = {Smi Hinterreiter and Martin Wessel and Fabian Schliski and Isao Echizen and Marc Erich Latoschik and Timo Spinde},
url = {https://media-bias-research.org/wp-content/uploads/2024/07/Preprint_ICWSM_25_NewsUnfold.pdf},
year = {2025},
date = {2025-06-01},
urldate = {2025-06-01},
booktitle = {Proceedings of the International AAAI Conference on Web and Social Media (ICWSM'25)},
volume = {19},
publisher = {AAAI},
address = {Copenhagen, Denmark},
abstract = {Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31},
note = {Conditionally accepted for publication},
keywords = {crowdsourcing, HITL, linguistic bias, media bias, news bias},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31
2024
Hinterreiter, Smi; Spinde, Timo; Oberdörfer, Sebastian; Echizen, Isao; Latoschik, Marc Erich
News Ninja: Gamified Annotation of Linguistic Bias in Online News Journal Article Forthcoming
In: Proc. ACM Hum.-Comput. Interact., vol. 8, no. CHI PLAY, Forthcoming, (Publisher: Association for Computing Machinery. Conditionally accepted for publication).
Abstract | Links | BibTeX | Tags: crowdsourcing, Game With A Purpose, linguistic bias, media bias, news bias
@article{Hinterreiter2024News,
title = {News Ninja: Gamified Annotation of Linguistic Bias in Online News},
author = {Smi Hinterreiter and Timo Spinde and Sebastian Oberdörfer and Isao Echizen and Marc Erich Latoschik},
url = {https://media-bias-research.org/wp-content/uploads/2024/07/Preprint_News_Ninja.pdf},
doi = {10.1145/3677092},
year = {2024},
date = {2024-10-14},
urldate = {2024-10-14},
journal = {Proc. ACM Hum.-Comput. Interact.},
volume = {8},
number = {CHI PLAY},
abstract = {Recent research shows that visualizing linguistic bias mitigates its negative effects. However, reliable automatic detection methods to generate such visualizations require costly, knowledge-intensive training data. To facilitate data collection for media bias datasets, we present News Ninja, a game employing data-collecting game mechanics to generate a crowdsourced dataset. Before annotating sentences, players are educated on media bias via a tutorial. Our findings show that datasets gathered with crowdsourced workers trained on News Ninja can reach significantly higher inter-annotator agreements than expert and crowdsourced datasets with similar data quality. As News Ninja encourages continuous play, it allows datasets to adapt to the reception and contextualization of news over time, presenting a promising strategy to reduce data collection expenses, educate players, and promote long-term bias mitigation.},
note = {Publisher: Association for Computing Machinery.
Conditionally accepted for publication},
keywords = {crowdsourcing, Game With A Purpose, linguistic bias, media bias, news bias},
pubstate = {forthcoming},
tppubtype = {article}
}
Recent research shows that visualizing linguistic bias mitigates its negative effects. However, reliable automatic detection methods to generate such visualizations require costly, knowledge-intensive training data. To facilitate data collection for media bias datasets, we present News Ninja, a game employing data-collecting game mechanics to generate a crowdsourced dataset. Before annotating sentences, players are educated on media bias via a tutorial. Our findings show that datasets gathered with crowdsourced workers trained on News Ninja can reach significantly higher inter-annotator agreements than expert and crowdsourced datasets with similar data quality. As News Ninja encourages continuous play, it allows datasets to adapt to the reception and contextualization of news over time, presenting a promising strategy to reduce data collection expenses, educate players, and promote long-term bias mitigation.
Wessel, Martin; Horych, Tomas
Beyond the Surface: Spurious Cues in Automatic Media Bias Detection Proceedings Article
In: Bharathi B Bharathi Raja Chakravarthi, Paul Buitelaar (Ed.): Proceedings of the Fourth Workshop on Language Technology for Equality, Diversity, Inclusion, pp. 21–30, Association for Computational Linguistics, 2024.
Abstract | Links | BibTeX | Tags:
@inproceedings{nokey,
title = {Beyond the Surface: Spurious Cues in Automatic Media Bias Detection},
author = {Martin Wessel and Tomas Horych},
editor = {Bharathi Raja Chakravarthi, Bharathi B, Paul Buitelaar, Thenmozhi Durairaj, György Kovács, Miguel Ángel García Cumbreras},
url = {https://aclanthology.org/2024.ltedi-1.3},
year = {2024},
date = {2024-03-21},
booktitle = {Proceedings of the Fourth Workshop on Language Technology for Equality, Diversity, Inclusion},
pages = {21–30},
publisher = {Association for Computational Linguistics},
abstract = {This study investigates the robustness and generalization of transformer-based models for automatic media bias detection. We explore the behavior of current bias classifiers by analyzing feature attributions and stress-testing with adversarial datasets. The findings reveal a disproportionate focus on rare but strongly connotated words, suggesting a rather superficial understanding of linguistic bias and challenges in contextual interpretation. This problem is further highlighted by inconsistent bias assessment when stress-tested with different entities and minorities. Enhancing automatic media bias detection models is critical to improving inclusivity in media, ensuring balanced and fair representation of diverse perspectives.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This study investigates the robustness and generalization of transformer-based models for automatic media bias detection. We explore the behavior of current bias classifiers by analyzing feature attributions and stress-testing with adversarial datasets. The findings reveal a disproportionate focus on rare but strongly connotated words, suggesting a rather superficial understanding of linguistic bias and challenges in contextual interpretation. This problem is further highlighted by inconsistent bias assessment when stress-tested with different entities and minorities. Enhancing automatic media bias detection models is critical to improving inclusivity in media, ensuring balanced and fair representation of diverse perspectives.
Horych, Tomas; Wessel, Martin; Wahle, Jan Philip; Ruas, Terry; Wassmuth, Jerome; Greiner-Petter, Andre; Aizawa, Akiko; Gipp, Bela; Spinde, Timo
MAGPIE: Multi-Task Analysis of Media-Bias Generalization with Pre-Trained Identification of Expressions Proceedings Article
In: "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation", 2024.
Abstract | Links | BibTeX | Tags: dataset, multi-task learning, Transfer learning
@inproceedings{Horych2024a,
title = {MAGPIE: Multi-Task Analysis of Media-Bias Generalization with Pre-Trained Identification of Expressions},
author = {Tomas Horych and Martin Wessel and Jan Philip Wahle and Terry Ruas and Jerome Wassmuth and Andre Greiner-Petter and Akiko Aizawa and Bela Gipp and Timo Spinde},
url = {https://aclanthology.org/2024.lrec-main.952},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation"},
abstract = {Media bias detection poses a complex, multifaceted problem traditionally tackled using single-task models and small in-domain datasets, consequently lacking generalizability. To address this, we introduce MAGPIE, a large-scale multi-task pre-training approach explicitly tailored for media bias detection. To enable large-scale pre-training, we construct Large Bias Mixture (LBM), a compilation of 59 bias-related tasks. MAGPIE outperforms previous approaches in media bias detection on the Bias Annotation By Experts (BABE) dataset, with a relative improvement of 3.3% F1-score. Furthermore, using a RoBERTa encoder, we show that MAGPIE needs only 15% of fine-tuning steps compared to single-task approaches. We provide insight into task learning interference and show that sentiment analysis and emotion detection help learning of all other tasks, and scaling the number of tasks leads to the best results. MAGPIE confirms that MTL is a promising approach for addressing media bias detection, enhancing the accuracy and efficiency of existing models. Furthermore, LBM is the first available resource collection focused on media bias MTL.},
keywords = {dataset, multi-task learning, Transfer learning},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias detection poses a complex, multifaceted problem traditionally tackled using single-task models and small in-domain datasets, consequently lacking generalizability. To address this, we introduce MAGPIE, a large-scale multi-task pre-training approach explicitly tailored for media bias detection. To enable large-scale pre-training, we construct Large Bias Mixture (LBM), a compilation of 59 bias-related tasks. MAGPIE outperforms previous approaches in media bias detection on the Bias Annotation By Experts (BABE) dataset, with a relative improvement of 3.3% F1-score. Furthermore, using a RoBERTa encoder, we show that MAGPIE needs only 15% of fine-tuning steps compared to single-task approaches. We provide insight into task learning interference and show that sentiment analysis and emotion detection help learning of all other tasks, and scaling the number of tasks leads to the best results. MAGPIE confirms that MTL is a promising approach for addressing media bias detection, enhancing the accuracy and efficiency of existing models. Furthermore, LBM is the first available resource collection focused on media bias MTL.
2023
Wessel, Martin; Horych, Tomas; Ruas, Terry; Aizawa, Akiko; Gipp, Bela; Spinde, Timo
Introducing MBIB - the first Media Bias Identification Benchmark Task and Dataset Collection Proceedings Article
In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’23), ACM, New York, NY, USA, 2023, ISBN: 978-1-4503-9408-6/23/07.
Abstract | Links | BibTeX | Tags:
@inproceedings{Wessel2023,
title = {Introducing MBIB - the first Media Bias Identification Benchmark Task and Dataset Collection},
author = {Martin Wessel and Tomas Horych and Terry Ruas and Akiko Aizawa and Bela Gipp and Timo Spinde},
url = {https://media-bias-research.org/wp-content/uploads/2023/04/Wessel2023Preprint.pdf
},
doi = {https://doi.org/10.1145/3539618.3591882},
isbn = {978-1-4503-9408-6/23/07},
year = {2023},
date = {2023-07-01},
urldate = {2023-07-01},
booktitle = {Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’23)},
publisher = {ACM},
address = {New York, NY, USA},
abstract = {Although media bias detection is a complex multi-task problem, there is, to date, no unified benchmark grouping these evaluation tasks. We introduce the Media Bias Identification Benchmark (MBIB), a comprehensive benchmark that groups different types of media bias (e.g., linguistic, cognitive, political) under a common framework to test how prospective detection techniques generalize. After reviewing 115 datasets, we select nine tasks and carefully propose 22 associated datasets for evaluating media bias detection techniques. We evaluate MBIB using state-of-the-art Transformer techniques (e.g., T5, BART). Our results suggest that while hate speech, racial bias, and gender bias are easier to detect, models struggle to handle certain bias types, e.g., cognitive and political bias. However, our results show that no single technique can outperform all the others significantly.We also find an uneven distribution of research interest and resource allocation to the individual tasks in media bias. A unified benchmark encourages the development of more robust systems and shifts the current paradigm in media bias detection evaluation towards solutions that tackle not one but multiple media bias types simultaneously.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Although media bias detection is a complex multi-task problem, there is, to date, no unified benchmark grouping these evaluation tasks. We introduce the Media Bias Identification Benchmark (MBIB), a comprehensive benchmark that groups different types of media bias (e.g., linguistic, cognitive, political) under a common framework to test how prospective detection techniques generalize. After reviewing 115 datasets, we select nine tasks and carefully propose 22 associated datasets for evaluating media bias detection techniques. We evaluate MBIB using state-of-the-art Transformer techniques (e.g., T5, BART). Our results suggest that while hate speech, racial bias, and gender bias are easier to detect, models struggle to handle certain bias types, e.g., cognitive and political bias. However, our results show that no single technique can outperform all the others significantly.We also find an uneven distribution of research interest and resource allocation to the individual tasks in media bias. A unified benchmark encourages the development of more robust systems and shifts the current paradigm in media bias detection evaluation towards solutions that tackle not one but multiple media bias types simultaneously.
Spinde, Timo; Richter, Elisabeth; Wessel, Martin; Kulshrestha, Juhi; Donnay, Karsten
What do Twitter comments tell about news article bias? Assessing the impact of news article bias on its perception on Twitter Journal Article
In: Online Social Networks and Media, vol. 37-38, pp. 100264, 2023, ISSN: 2468-6964.
Abstract | Links | BibTeX | Tags: Hate speech detection, media bias, Sentiment analysis, Transfer learning
@article{SPINDE2023100264,
title = {What do Twitter comments tell about news article bias? Assessing the impact of news article bias on its perception on Twitter},
author = {Timo Spinde and Elisabeth Richter and Martin Wessel and Juhi Kulshrestha and Karsten Donnay},
url = {https://www.sciencedirect.com/science/article/pii/S246869642300023X},
doi = {https://doi.org/10.1016/j.osnem.2023.100264},
issn = {2468-6964},
year = {2023},
date = {2023-01-01},
journal = {Online Social Networks and Media},
volume = {37-38},
pages = {100264},
abstract = {News stories circulating online, especially on social media platforms, are nowadays a primary source of information. Given the nature of social media, news no longer are just news, but they are embedded in the conversations of users interacting with them. This is particularly relevant for inaccurate information or even outright misinformation because user interaction has a crucial impact on whether information is uncritically disseminated or not. Biased coverage has been shown to affect personal decision-making. Still, it remains an open question whether users are aware of the biased reporting they encounter and how they react to it. The latter is particularly relevant given that user reactions help contextualize reporting for other users and can thus help mitigate but may also exacerbate the impact of biased media coverage. This paper approaches the question from a measurement point of view, examining whether reactions to news articles on Twitter can serve as bias indicators, i.e., whether how users comment on a given article relates to its actual level of bias. We first give an overview of research on media bias before discussing key concepts related to how individuals engage with online content, focusing on the sentiment (or valance) of comments and on outright hate speech. We then present the first dataset connecting reliable human-made media bias classifications of news articles with the reactions these articles received on Twitter. We call our dataset BAT - Bias And Twitter. BAT covers 2,800 (bias-rated) news articles from 255 English-speaking news outlets. Additionally, BAT includes 175,807 comments and retweets referring to the articles. Based on BAT, we conduct a multi-feature analysis to identify comment characteristics and analyze whether Twitter reactions correlate with an article’s bias. First, we fine-tune and apply two XLNet-based classifiers for hate speech detection and sentiment analysis. Second, we relate the results of the classifiers to the article bias annotations within a multi-level regression. The results show that Twitter reactions to an article indicate its bias, and vice-versa. With a regression coefficient of 0.703 (p<0.01), we specifically present evidence that Twitter reactions to biased articles are significantly more hateful. Our analysis shows that the news outlet’s individual stance reinforces the hate-bias relationship. In future work, we will extend the dataset and analysis, including additional concepts related to media bias.},
keywords = {Hate speech detection, media bias, Sentiment analysis, Transfer learning},
pubstate = {published},
tppubtype = {article}
}
News stories circulating online, especially on social media platforms, are nowadays a primary source of information. Given the nature of social media, news no longer are just news, but they are embedded in the conversations of users interacting with them. This is particularly relevant for inaccurate information or even outright misinformation because user interaction has a crucial impact on whether information is uncritically disseminated or not. Biased coverage has been shown to affect personal decision-making. Still, it remains an open question whether users are aware of the biased reporting they encounter and how they react to it. The latter is particularly relevant given that user reactions help contextualize reporting for other users and can thus help mitigate but may also exacerbate the impact of biased media coverage. This paper approaches the question from a measurement point of view, examining whether reactions to news articles on Twitter can serve as bias indicators, i.e., whether how users comment on a given article relates to its actual level of bias. We first give an overview of research on media bias before discussing key concepts related to how individuals engage with online content, focusing on the sentiment (or valance) of comments and on outright hate speech. We then present the first dataset connecting reliable human-made media bias classifications of news articles with the reactions these articles received on Twitter. We call our dataset BAT - Bias And Twitter. BAT covers 2,800 (bias-rated) news articles from 255 English-speaking news outlets. Additionally, BAT includes 175,807 comments and retweets referring to the articles. Based on BAT, we conduct a multi-feature analysis to identify comment characteristics and analyze whether Twitter reactions correlate with an article’s bias. First, we fine-tune and apply two XLNet-based classifiers for hate speech detection and sentiment analysis. Second, we relate the results of the classifiers to the article bias annotations within a multi-level regression. The results show that Twitter reactions to an article indicate its bias, and vice-versa. With a regression coefficient of 0.703 (p<0.01), we specifically present evidence that Twitter reactions to biased articles are significantly more hateful. Our analysis shows that the news outlet’s individual stance reinforces the hate-bias relationship. In future work, we will extend the dataset and analysis, including additional concepts related to media bias.
Spinde, Timo; Hinterreiter, Smi; Haak, Fabian; Ruas, Terry; Giese, Helge; Meuschke, Norman; Gipp, Bela
The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias Journal Article
In: arXiv preprint, 2023.
@article{Spinde2023MediaBias,
title = {The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias},
author = {Timo Spinde and Smi Hinterreiter and Fabian Haak and Terry Ruas and Helge Giese and Norman Meuschke and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2023/12/spinde2023.pdf},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {arXiv preprint},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2022
Krieger, David; Spinde, Timo; Ruas, Terry; Kulshrestha, Juhi; Gipp, Bela
A Domain-adaptive Pre-training Approach for Language Bias Detection in News Proceedings Article
In: 2022 ACM/IEEE Joint Conference on Digital Libraries (JCDL), Cologne, Germany, 2022.
@inproceedings{Krieger2022,
title = {A Domain-adaptive Pre-training Approach for Language Bias Detection in News},
author = {David Krieger and Timo Spinde and Terry Ruas and Juhi Kulshrestha and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2022/06/Krieger2022_mbg.pdf},
doi = {10.1145/3529372.3530932},
year = {2022},
date = {2022-06-01},
urldate = {2022-06-01},
booktitle = {2022 ACM/IEEE Joint Conference on Digital Libraries (JCDL)},
address = {Cologne, Germany},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Zhukova, Anastasia; Hamborg, Felix; Gipp, Bela
Towards Evaluation of Cross-document Coreference Resolution Models Using Datasets with Diverse Annotation Schemes Proceedings Article
In: Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 4884–4893, European Language Resources Association, Marseille, France, 2022.
Abstract | Links | BibTeX | Tags:
@inproceedings{zhukova2022a,
title = {Towards Evaluation of Cross-document Coreference Resolution Models Using Datasets with Diverse Annotation Schemes},
author = {Anastasia Zhukova and Felix Hamborg and Bela Gipp},
url = {https://aclanthology.org/2022.lrec-1.522},
year = {2022},
date = {2022-06-01},
booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
pages = {4884--4893},
publisher = {European Language Resources Association},
address = {Marseille, France},
abstract = {Established cross-document coreference resolution (CDCR) datasets contain event-centric coreference chains of events and entities with identity relations. These datasets establish strict definitions of the coreference relations across related tests but typically ignore anaphora with more vague context-dependent loose coreference relations. In this paper, we qualitatively and quantitatively compare the annotation schemes of ECB+, a CDCR dataset with identity coreference relations, and NewsWCL50, a CDCR dataset with a mix of loose context-dependent and strict coreference relations. We propose a phrasing diversity metric (PD) that encounters for the diversity of full phrases unlike the previously proposed metrics and allows to evaluate lexical diversity of the CDCR datasets in a higher precision. The analysis shows that coreference chains of NewsWCL50 are more lexically diverse than those of ECB+ but annotating of NewsWCL50 leads to the lower inter-coder reliability. We discuss the different tasks that both CDCR datasets create for the CDCR models, i.e., lexical disambiguation and lexical diversity. Finally, to ensure generalizability of the CDCR models, we propose a direction for CDCR evaluation that combines CDCR datasets with multiple annotation schemes that focus of various properties of the coreference chains.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Established cross-document coreference resolution (CDCR) datasets contain event-centric coreference chains of events and entities with identity relations. These datasets establish strict definitions of the coreference relations across related tests but typically ignore anaphora with more vague context-dependent loose coreference relations. In this paper, we qualitatively and quantitatively compare the annotation schemes of ECB+, a CDCR dataset with identity coreference relations, and NewsWCL50, a CDCR dataset with a mix of loose context-dependent and strict coreference relations. We propose a phrasing diversity metric (PD) that encounters for the diversity of full phrases unlike the previously proposed metrics and allows to evaluate lexical diversity of the CDCR datasets in a higher precision. The analysis shows that coreference chains of NewsWCL50 are more lexically diverse than those of ECB+ but annotating of NewsWCL50 leads to the lower inter-coder reliability. We discuss the different tasks that both CDCR datasets create for the CDCR models, i.e., lexical disambiguation and lexical diversity. Finally, to ensure generalizability of the CDCR models, we propose a direction for CDCR evaluation that combines CDCR datasets with multiple annotation schemes that focus of various properties of the coreference chains.
Spinde, Timo; Krieger, Jan-David; Ruas, Terry; Mitrović, Jelena; Götz-Hahn, Franz; Aizawa, Akiko; Gipp, Bela
Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles Proceedings Article
In: Proceedings of the iConference 2022, Virtual event, 2022.
@inproceedings{Spinde2022a,
title = {Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles},
author = {Timo Spinde and Jan-David Krieger and Terry Ruas and Jelena Mitrović and Franz Götz-Hahn and Akiko Aizawa and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2022/03/Spinde2022a_mbg.pdf},
doi = {https://doi.org/10.1007/978-3-030-96957-8_20},
year = {2022},
date = {2022-03-04},
urldate = {2022-03-04},
booktitle = {Proceedings of the iConference 2022},
address = {Virtual event},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Jeggle, Christin; Haupt, Magdalena; Gaissmaier, Wolfgang; Giese, Helge
How do we raise media bias awareness effectively? Effects of visualizations to communicate bias Journal Article
In: PLOS ONE, vol. 17, no. 4, pp. 1-14, 2022.
Abstract | Links | BibTeX | Tags:
@article{10.1371/journal.pone.0266204,
title = {How do we raise media bias awareness effectively? Effects of visualizations to communicate bias},
author = {Timo Spinde and Christin Jeggle and Magdalena Haupt and Wolfgang Gaissmaier and Helge Giese},
url = {https://doi.org/10.1371/journal.pone.0266204},
doi = {10.1371/journal.pone.0266204},
year = {2022},
date = {2022-01-01},
journal = {PLOS ONE},
volume = {17},
number = {4},
pages = {1-14},
publisher = {Public Library of Science},
abstract = {Media bias has a substantial impact on individual and collective perception of news. Effective communication that may counteract its potential negative effects still needs to be developed. In this article, we analyze how to facilitate the detection of media bias with visual and textual aids in the form of (a) a forewarning message, (b) text annotations, and (c) political classifiers. In an online experiment, we randomized 985 participants to receive a biased liberal or conservative news article in any combination of the three aids. Meanwhile, their subjective perception of media bias in this article, attitude change, and political ideology were assessed. Both the forewarning message and the annotations increased media bias awareness, whereas the political classification showed no effect. Incongruence between an articles’ political position and individual political orientation also increased media bias awareness. Visual aids did not mitigate this effect. Likewise, attitudes remained unaltered.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Media bias has a substantial impact on individual and collective perception of news. Effective communication that may counteract its potential negative effects still needs to be developed. In this article, we analyze how to facilitate the detection of media bias with visual and textual aids in the form of (a) a forewarning message, (b) text annotations, and (c) political classifiers. In an online experiment, we randomized 985 participants to receive a biased liberal or conservative news article in any combination of the three aids. Meanwhile, their subjective perception of media bias in this article, attitude change, and political ideology were assessed. Both the forewarning message and the annotations increased media bias awareness, whereas the political classification showed no effect. Incongruence between an articles’ political position and individual political orientation also increased media bias awareness. Visual aids did not mitigate this effect. Likewise, attitudes remained unaltered.
Haak, Fabian; Schaer, Philipp
Auditing Search Query Suggestion Bias Through Recursive Algorithm Interrogation Proceedings Article
In: WebSci '22: 14th ACM Web Science Conference 2022, ACM, 2022.
BibTeX | Tags: bias esupol haak myown schaer
@inproceedings{haak2022auditing,
title = {Auditing Search Query Suggestion Bias Through Recursive
Algorithm Interrogation},
author = {Fabian Haak and Philipp Schaer},
year = {2022},
date = {2022-01-01},
booktitle = {WebSci '22: 14th ACM Web Science Conference 2022},
publisher = {ACM},
keywords = {bias esupol haak myown schaer},
pubstate = {published},
tppubtype = {inproceedings}
}
Zhukova, Anastasia; Hamborg, Felix; Donnay, Karsten; Gipp, Bela
XCoref: Cross-Document Coreference Resolution in the Wild Proceedings Article
In: Information for a Better World: Shaping the Global Future: 17th International Conference, IConference 2022, Virtual Event, February 28 – March 4, 2022, Proceedings, Part I, pp. 272–291, Springer-Verlag, Berlin, Heidelberg, 2022, ISBN: 978-3-030-96956-1.
Abstract | Links | BibTeX | Tags: Cross-document coreference resolution, media bias, news analysis
@inproceedings{zhukova2022,
title = {XCoref: Cross-Document Coreference Resolution in the Wild},
author = {Anastasia Zhukova and Felix Hamborg and Karsten Donnay and Bela Gipp},
url = {https://doi.org/10.1007/978-3-030-96957-8_25},
doi = {10.1007/978-3-030-96957-8_25},
isbn = {978-3-030-96956-1},
year = {2022},
date = {2022-01-01},
booktitle = {Information for a Better World: Shaping the Global Future: 17th International Conference, IConference 2022, Virtual Event, February 28 – March 4, 2022, Proceedings, Part I},
pages = {272–291},
publisher = {Springer-Verlag},
address = {Berlin, Heidelberg},
abstract = {Datasets and methods for cross-document coreference resolution (CDCR) focus on events or entities with strict coreference relations. They lack, however, annotating and resolving coreference mentions with more abstract or loose relations that may occur when news articles report about controversial and polarized events. Bridging and loose coreference relations trigger associations that may expose news readers to bias by word choice and labeling. For example, coreferential mentions of “direct talks between U.S. President Donald Trump and Kim” such as “an extraordinary meeting following months of heated rhetoric” or “great chance to solve a world problem” form a more positive perception of this event. A step towards bringing awareness of bias by word choice and labeling is the reliable resolution of coreferences with high lexical diversity. We propose an unsupervised method named XCoref, which is a CDCR method that capably resolves not only previously prevalent entities, such as persons, e.g., “Donald Trump,” but also abstractly defined concepts, such as groups of persons, “caravan of immigrants,” events and actions, e.g., “marching to the U.S. border.” In an extensive evaluation, we compare the proposed XCoref to a state-of-the-art CDCR method and a previous method TCA that resolves such complex coreference relations and find that XCoref outperforms these methods. Outperforming an established CDCR model shows that the new CDCR models need to be evaluated on semantically complex mentions with more loose coreference relations to indicate their applicability of models to resolve mentions in the “wild” of political news articles.},
keywords = {Cross-document coreference resolution, media bias, news analysis},
pubstate = {published},
tppubtype = {inproceedings}
}
Datasets and methods for cross-document coreference resolution (CDCR) focus on events or entities with strict coreference relations. They lack, however, annotating and resolving coreference mentions with more abstract or loose relations that may occur when news articles report about controversial and polarized events. Bridging and loose coreference relations trigger associations that may expose news readers to bias by word choice and labeling. For example, coreferential mentions of “direct talks between U.S. President Donald Trump and Kim” such as “an extraordinary meeting following months of heated rhetoric” or “great chance to solve a world problem” form a more positive perception of this event. A step towards bringing awareness of bias by word choice and labeling is the reliable resolution of coreferences with high lexical diversity. We propose an unsupervised method named XCoref, which is a CDCR method that capably resolves not only previously prevalent entities, such as persons, e.g., “Donald Trump,” but also abstractly defined concepts, such as groups of persons, “caravan of immigrants,” events and actions, e.g., “marching to the U.S. border.” In an extensive evaluation, we compare the proposed XCoref to a state-of-the-art CDCR method and a previous method TCA that resolves such complex coreference relations and find that XCoref outperforms these methods. Outperforming an established CDCR model shows that the new CDCR models need to be evaluated on semantically complex mentions with more loose coreference relations to indicate their applicability of models to resolve mentions in the “wild” of political news articles.
2021
Spinde, Timo; Plank, Manuel; Krieger, Jan-David; Ruas, Terry; Gipp, Bela; Aizawa, Akiko
Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By Experts Proceedings Article
In: Findings of the Association for Computational Linguistics: EMNLP 2021, Dominican Republic, 2021.
@inproceedings{Spinde2021f,
title = {Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By Experts},
author = {Timo Spinde and Manuel Plank and Jan-David Krieger and Terry Ruas and Bela Gipp and Akiko Aizawa},
url = {https://media-bias-research.org/wp-content/uploads/2022/01/Neural_Media_Bias_Detection_Using_Distant_Supervision_With_BABE___Bias_Annotations_By_Experts_MBG.pdf},
doi = {10.18653/v1/2021.findings-emnlp.101},
year = {2021},
date = {2021-11-01},
urldate = {2021-11-01},
booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
address = {Dominican Republic},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Hinterreiter, Smi
A Gamified Approach To Automatically Detect Biased Wording And Train Critical Reading Proceedings Article
In: 2021 IEEE International Conference on Data Mining Workshops (ICDMW), 2021.
@inproceedings{hinterreiter2021a,
title = {A Gamified Approach To Automatically Detect Biased Wording And Train Critical Reading},
author = {Smi Hinterreiter},
url = {https://media-bias-research.org/wp-content/uploads/2021/10/hinterreiter2021a.pdf},
doi = {10.1109/ICDMW53433.2021.00141},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
booktitle = {2021 IEEE International Conference on Data Mining Workshops (ICDMW)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo
An Interdisciplinary Approach for the Automated Detection and Visualization of Media Bias in News Articles Proceedings Article
In: 2021 IEEE International Conference on Data Mining Workshops (ICDMW), 2021.
Links | BibTeX | Tags: media bias, news analysis, slanted coverage, text retrieval
@inproceedings{spinde2021g,
title = {An Interdisciplinary Approach for the Automated Detection and Visualization of Media Bias in News Articles},
author = {Timo Spinde},
url = {https://media-bias-research.org/wp-content/uploads/2021/09/Spinde2021g.pdf},
doi = {10.1109/ICDMW53433.2021.00144},
year = {2021},
date = {2021-09-30},
urldate = {2021-09-30},
booktitle = {2021 IEEE International Conference on Data Mining Workshops (ICDMW)},
keywords = {media bias, news analysis, slanted coverage, text retrieval},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Sinha, Kanishka; Meuschke, Norman; Gipp, Bela
TASSY - A Text Annotation Survey System Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL), 2021.
@inproceedings{Spinde2021c,
title = {TASSY - A Text Annotation Survey System},
author = {Timo Spinde and Kanishka Sinha and Norman Meuschke and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2022/01/Spinde2021c.pdf},
doi = {10.1109/JCDL52503.2021.00052},
year = {2021},
date = {2021-09-01},
urldate = {2021-09-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Kreuter, Christina; Gaissmaier, Wolfgang; Hamborg, Felix; Gipp, Bela; Giese, Helge
Do You Think It’s Biased? How To Ask For The Perception Of Media Bias Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL), 2021.
@inproceedings{Spinde2021e,
title = {Do You Think It’s Biased? How To Ask For The Perception Of Media Bias},
author = {Timo Spinde and Christina Kreuter and Wolfgang Gaissmaier and Felix Hamborg and Bela Gipp and Helge Giese},
url = {https://media-bias-research.org/wp-content/uploads/2022/01/Spinde2021e.pdf},
doi = {10.1109/JCDL52503.2021.00018},
year = {2021},
date = {2021-09-01},
urldate = {2021-09-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Krieger, David; Plank, Manu; Gipp, Bela
Towards A Reliable Ground-Truth For Biased Language Detection Proceedings Article
In: Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), Virtual Event, 2021.
@inproceedings{Spinde2021d,
title = {Towards A Reliable Ground-Truth For Biased Language Detection},
author = {Timo Spinde and David Krieger and Manu Plank and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2022/01/Spinde2021d.pdf},
doi = {10.1109/JCDL52503.2021.00053},
year = {2021},
date = {2021-09-01},
urldate = {2021-09-01},
booktitle = {Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL)},
address = {Virtual Event},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Haak, Fabian; Engelmann, Björn
IRCologne at GermEval 2021: Toxicity Classification Proceedings Article
In: Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments, pp. 47–53, Association for Computational Linguistics, Duesseldorf, Germany, 2021.
Abstract | Links | BibTeX | Tags: 2021 bias classification data engelmann haak nlp programming snorkel toxic
@inproceedings{haak-engelmann-2021-ircologne,
title = {IRCologne at GermEval 2021: Toxicity Classification},
author = {Fabian Haak and Björn Engelmann},
url = {https://aclanthology.org/2021.germeval-1.7},
year = {2021},
date = {2021-09-01},
booktitle = {Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments},
pages = {47--53},
publisher = {Association for Computational Linguistics},
address = {Duesseldorf, Germany},
abstract = {In this paper, we describe the TH Köln's submission for the Shared Task on the Identification of Toxic Comments at GermEval 2021. Toxicity is a severe and latent problem in comments in online discussions. Complex language model based methods have shown the most success in identifying toxicity. However, these approaches lack explainability and might be insensitive to domain-specific renditions of toxicity. In the scope of the GermEval 2021 toxic comment classification task (Risch et al., 2021), we employed a simple but promising combination of term-frequency-based classification and rule-based labeling to produce effective but to no lesser degree explainable toxicity predictions.},
keywords = {2021 bias classification data engelmann haak nlp programming snorkel toxic},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we describe the TH Köln's submission for the Shared Task on the Identification of Toxic Comments at GermEval 2021. Toxicity is a severe and latent problem in comments in online discussions. Complex language model based methods have shown the most success in identifying toxicity. However, these approaches lack explainability and might be insensitive to domain-specific renditions of toxicity. In the scope of the GermEval 2021 toxic comment classification task (Risch et al., 2021), we employed a simple but promising combination of term-frequency-based classification and rule-based labeling to produce effective but to no lesser degree explainable toxicity predictions.
Hamborg, F.; Heinser, K.; Zhukova, A.; Donnay, K.; Gipp, B.
Newsalyze: Effective Communication of Person-Targeting Biases in News Articles Proceedings Article
In: 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 130-139, IEEE Computer Society, Los Alamitos, CA, USA, 2021.
Abstract | Links | BibTeX | Tags: visualization;costs;atmospheric measurements;voting;natural languages;manuals;particle measurements
@inproceedings{hamborg2021a,
title = {Newsalyze: Effective Communication of Person-Targeting Biases in News Articles},
author = {F. Hamborg and K. Heinser and A. Zhukova and K. Donnay and B. Gipp},
url = {https://doi.ieeecomputersociety.org/10.1109/JCDL52503.2021.00025},
doi = {10.1109/JCDL52503.2021.00025},
year = {2021},
date = {2021-09-01},
booktitle = {2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL)},
pages = {130-139},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Media bias and its extreme form, fake news, can decisively affect public opinion. Especially when reporting on policy issues, slanted news coverage may strongly influence societal decisions, e.g., in democratic elections. Our paper makes three contributions to address this issue. First, we present a system for bias identification, which combines state-of-the-art methods from natural language understanding. Second, we devise bias-sensitive visualizations to communicate bias in news articles to non-expert news consumers. Third, our main contribution is a large-scale user study that measures bias-awareness in a setting that approximates daily news consumption, e.g., we present respondents with a news overview and individual articles. We not only measure the visualizations' effect on respondents' bias-awareness, but we can also pinpoint the effects on individual components of the visualizations by employing a conjoint design. Our bias-sensitive overviews strongly and significantly increase bias-awareness in respondents. Our study further suggests that our content-driven identification method detects groups of similarly slanted news articles due to substantial biases present in individual news articles. In contrast, the reviewed prior work rather only facilitates the visibility of biases, e.g., by distinguishing left- and right-wing outlets.},
keywords = {visualization;costs;atmospheric measurements;voting;natural languages;manuals;particle measurements},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias and its extreme form, fake news, can decisively affect public opinion. Especially when reporting on policy issues, slanted news coverage may strongly influence societal decisions, e.g., in democratic elections. Our paper makes three contributions to address this issue. First, we present a system for bias identification, which combines state-of-the-art methods from natural language understanding. Second, we devise bias-sensitive visualizations to communicate bias in news articles to non-expert news consumers. Third, our main contribution is a large-scale user study that measures bias-awareness in a setting that approximates daily news consumption, e.g., we present respondents with a news overview and individual articles. We not only measure the visualizations&#x0027; effect on respondents&#x0027; bias-awareness, but we can also pinpoint the effects on individual components of the visualizations by employing a conjoint design. Our bias-sensitive overviews strongly and significantly increase bias-awareness in respondents. Our study further suggests that our content-driven identification method detects groups of similarly slanted news articles due to substantial biases present in individual news articles. In contrast, the reviewed prior work rather only facilitates the visibility of biases, e.g., by distinguishing left- and right-wing outlets.
Cabot, Pere-Lluís Huguet; Abadi, David; Fischer, Agneta; Shutova, Ekaterina
Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions Proceedings Article
In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 1921–1945, Association for Computational Linguistics, Online, 2021.
Abstract | Links | BibTeX | Tags:
@inproceedings{huguet-cabot-etal-2021-us,
title = {Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions},
author = {Pere-Lluís Huguet Cabot and David Abadi and Agneta Fischer and Ekaterina Shutova},
url = {https://aclanthology.org/2021.eacl-main.165},
doi = {10.18653/v1/2021.eacl-main.165},
year = {2021},
date = {2021-04-01},
booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
pages = {1921–1945},
publisher = {Association for Computational Linguistics},
address = {Online},
abstract = {Computational modelling of political discourse tasks has become an increasingly important area of research in the field of natural language processing. Populist rhetoric has risen across the political sphere in recent years; however, due to its complex nature, computational approaches to it have been scarce. In this paper, we present the new Us vs. Them dataset, consisting of 6861 Reddit comments annotated for populist attitudes and the first large-scale computational models of this phenomenon. We investigate the relationship between populist mindsets and social groups, as well as a range of emotions typically associated with these. We set a baseline for two tasks associated with populist attitudes and present a set of multi-task learning models that leverage and demonstrate the importance of emotion and group identification as auxiliary tasks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Computational modelling of political discourse tasks has become an increasingly important area of research in the field of natural language processing. Populist rhetoric has risen across the political sphere in recent years; however, due to its complex nature, computational approaches to it have been scarce. In this paper, we present the new Us vs. Them dataset, consisting of 6861 Reddit comments annotated for populist attitudes and the first large-scale computational models of this phenomenon. We investigate the relationship between populist mindsets and social groups, as well as a range of emotions typically associated with these. We set a baseline for two tasks associated with populist attitudes and present a set of multi-task learning models that leverage and demonstrate the importance of emotion and group identification as auxiliary tasks.
Spinde, Timo; Rudnitckaia, Lada; Hamborg, Felix; Bela,; Gipp,
Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings Proceedings Article
In: Proceedings of the iConference 2021, Beijing, China (Virtual Event), 2021.
@inproceedings{Spinde2021Embeddings,
title = {Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings},
author = {Timo Spinde and Lada Rudnitckaia and Felix Hamborg and Bela and Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2021/01/Spinde2021.pdf},
doi = {10.1007/978-3-030-71305-8_17},
year = {2021},
date = {2021-03-01},
urldate = {2021-03-01},
booktitle = {Proceedings of the iConference 2021},
address = {Beijing, China (Virtual Event)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Rudnitckaia, Lada; Kanishka, Sinha; Hamborg, Felix; Bela,; Gipp,; Donnay, Karsten
MBIC – A Media Bias Annotation Dataset Including Annotator Characteristics Proceedings Article
In: Proceedings of the iConference 2021, Beijing, China (Virtual Event), 2021.
@inproceedings{Spinde2021MBIC,
title = {MBIC – A Media Bias Annotation Dataset Including Annotator Characteristics},
author = {Timo Spinde and Lada Rudnitckaia and Sinha Kanishka and Felix Hamborg and Bela and Gipp and Karsten Donnay},
url = {https://media-bias-research.org/wp-content/uploads/2021/01/Spinde2021a.pdf},
doi = {10.6084/m9.figshare.17192924},
year = {2021},
date = {2021-03-01},
urldate = {2021-03-01},
booktitle = {Proceedings of the iConference 2021},
address = {Beijing, China (Virtual Event)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Rudnitckaia, Lada; Mitrović, Jelena; Hamborg, Felix; Granitzer, Michael; Gipp, Bela; Donnay, Karsten
Automated identification of bias inducing words in news articles using linguistic and context-oriented features Journal Article
In: Information Processing & Management, vol. 58, no. 3, pp. 102505, 2021, ISSN: 0306-4573.
Abstract | Links | BibTeX | Tags: bias data set, context analysis, feature engineering, media bias, news analysis, text analysis
@article{SPINDE2021102505,
title = {Automated identification of bias inducing words in news articles using linguistic and context-oriented features},
author = {Timo Spinde and Lada Rudnitckaia and Jelena Mitrović and Felix Hamborg and Michael Granitzer and Bela Gipp and Karsten Donnay},
url = {https://www.sciencedirect.com/science/article/pii/S0306457321000157/pdfft?md5=64e81212b3bfa861d01a6fe3d5b979c3&pid=1-s2.0-S0306457321000157-main.pdf},
doi = {https://doi.org/10.1016/j.ipm.2021.102505},
issn = {0306-4573},
year = {2021},
date = {2021-01-01},
journal = {Information Processing & Management},
volume = {58},
number = {3},
pages = {102505},
abstract = {Media has a substantial impact on public perception of events, and, accordingly, the way media presents events can potentially alter the beliefs and views of the public. One of the ways in which bias in news articles can be introduced is by altering word choice. Such a form of bias is very challenging to identify automatically due to the high context-dependence and the lack of a large-scale gold-standard data set. In this paper, we present a prototypical yet robust and diverse data set for media bias research. It consists of 1,700 statements representing various media bias instances and contains labels for media bias identification on the word and sentence level. In contrast to existing research, our data incorporate background information on the participants’ demographics, political ideology, and their opinion about media in general. Based on our data, we also present a way to detect bias-inducing words in news articles automatically. Our approach is feature-oriented, which provides a strong descriptive and explanatory power compared to deep learning techniques. We identify and engineer various linguistic, lexical, and syntactic features that can potentially be media bias indicators. Our resource collection is the most complete within the media bias research area to the best of our knowledge. We evaluate all of our features in various combinations and retrieve their possible importance both for future research and for the task in general. We also evaluate various possible Machine Learning approaches with all of our features. XGBoost, a decision tree implementation, yields the best results. Our approach achieves an F1-score of 0.43, a precision of 0.29, a recall of 0.77, and a ROC AUC of 0.79, which outperforms current media bias detection methods based on features. We propose future improvements, discuss the perspectives of the feature-based approach and a combination of neural networks and deep learning with our current system.},
keywords = {bias data set, context analysis, feature engineering, media bias, news analysis, text analysis},
pubstate = {published},
tppubtype = {article}
}
Media has a substantial impact on public perception of events, and, accordingly, the way media presents events can potentially alter the beliefs and views of the public. One of the ways in which bias in news articles can be introduced is by altering word choice. Such a form of bias is very challenging to identify automatically due to the high context-dependence and the lack of a large-scale gold-standard data set. In this paper, we present a prototypical yet robust and diverse data set for media bias research. It consists of 1,700 statements representing various media bias instances and contains labels for media bias identification on the word and sentence level. In contrast to existing research, our data incorporate background information on the participants’ demographics, political ideology, and their opinion about media in general. Based on our data, we also present a way to detect bias-inducing words in news articles automatically. Our approach is feature-oriented, which provides a strong descriptive and explanatory power compared to deep learning techniques. We identify and engineer various linguistic, lexical, and syntactic features that can potentially be media bias indicators. Our resource collection is the most complete within the media bias research area to the best of our knowledge. We evaluate all of our features in various combinations and retrieve their possible importance both for future research and for the task in general. We also evaluate various possible Machine Learning approaches with all of our features. XGBoost, a decision tree implementation, yields the best results. Our approach achieves an F1-score of 0.43, a precision of 0.29, a recall of 0.77, and a ROC AUC of 0.79, which outperforms current media bias detection methods based on features. We propose future improvements, discuss the perspectives of the feature-based approach and a combination of neural networks and deep learning with our current system.
Ehrhardt, Jonas; Spinde, Timo; Vardasbi, Ali; Hamborg, Felix
Omission of Information: Identifying Political Slant via an Analysis of Co-occurring Entities Book Section
In: Information between Data and Knowledge, vol. 74, pp. 80–93, Werner Hülsbusch, Glückstadt, 2021, (Session 2: Information Behavior and Information Literacy 2).
Abstract | Links | BibTeX | Tags: media bias; bias by omission; news articles; co-occurrences
@incollection{epub44939,
title = {Omission of Information: Identifying Political Slant via an Analysis of Co-occurring Entities},
author = {Jonas Ehrhardt and Timo Spinde and Ali Vardasbi and Felix Hamborg},
url = {https://epub.uni-regensburg.de/44939/},
year = {2021},
date = {2021-01-01},
booktitle = {Information between Data and Knowledge},
volume = {74},
pages = {80--93},
publisher = {Werner Hülsbusch},
address = {Glückstadt},
series = {Schriften zur Informationswissenschaft},
abstract = {Due to the strong impact the news has on society, the detection and analysis of bias within the media are important topics. Most approaches to bias detection focus on linguistic forms of bias or the evaluation and tracing of sources. In this paper, we present an approach that analyzes co-occurrences of entities across articles of different news outlets to indicate a strong but difficult to detect form of bias: bias by omission of information. Specifically, we present and evaluate different methods of identifying entity co-occurrences and then use the best performing method, reference entity detection, to analyze the coverage of nine major US news outlets over one year. We set a low performing but transparent baseline, which is able to identify a news outlet?s affiliation towards a political orientation. Our approach employing reference entity selection, i. e., analyzing how often one entity co-occurs with others across a set of documents, yields an F1-score of F1 = 0.51 compared to F1 = 0.20 of the TF-IDF baseline.},
note = {Session 2: Information Behavior and Information Literacy 2},
keywords = {media bias; bias by omission; news articles; co-occurrences},
pubstate = {published},
tppubtype = {incollection}
}
Due to the strong impact the news has on society, the detection and analysis of bias within the media are important topics. Most approaches to bias detection focus on linguistic forms of bias or the evaluation and tracing of sources. In this paper, we present an approach that analyzes co-occurrences of entities across articles of different news outlets to indicate a strong but difficult to detect form of bias: bias by omission of information. Specifically, we present and evaluate different methods of identifying entity co-occurrences and then use the best performing method, reference entity detection, to analyze the coverage of nine major US news outlets over one year. We set a low performing but transparent baseline, which is able to identify a news outlet?s affiliation towards a political orientation. Our approach employing reference entity selection, i. e., analyzing how often one entity co-occurs with others across a set of documents, yields an F1-score of F1 = 0.51 compared to F1 = 0.20 of the TF-IDF baseline.
Garz, Marcel; Martin, Gregory J.
Media Influence on Vote Choices: Unemployment News and Incumbents' Electoral Prospects Journal Article
In: American Journal of Political Science, vol. 65, no. 2, pp. 278-293, 2021.
Abstract | Links | BibTeX | Tags:
@article{https://doi.org/10.1111/ajps.12539,
title = {Media Influence on Vote Choices: Unemployment News and Incumbents' Electoral Prospects},
author = {Marcel Garz and Gregory J. Martin},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/ajps.12539},
doi = {https://doi.org/10.1111/ajps.12539},
year = {2021},
date = {2021-01-01},
journal = {American Journal of Political Science},
volume = {65},
number = {2},
pages = {278-293},
abstract = {Abstract How does news about the economy influence voting decisions? We isolate the effect of the information environment from the effect of change in the underlying economic conditions themselves by taking advantage of left-digit bias. We show that unemployment figures crossing a round-number “milestone” cause a discontinuous increase in the amount of media coverage devoted to unemployment conditions, and we use this discontinuity to estimate the effect of attention to unemployment news on voting, holding constant the actual economic conditions on the ground. Milestone effects on incumbent U.S. governor vote shares are large and notably asymmetric: Bad milestone events hurt roughly twice as much as good milestone events help.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract How does news about the economy influence voting decisions? We isolate the effect of the information environment from the effect of change in the underlying economic conditions themselves by taking advantage of left-digit bias. We show that unemployment figures crossing a round-number “milestone” cause a discontinuous increase in the amount of media coverage devoted to unemployment conditions, and we use this discontinuity to estimate the effect of attention to unemployment news on voting, holding constant the actual economic conditions on the ground. Milestone effects on incumbent U.S. governor vote shares are large and notably asymmetric: Bad milestone events hurt roughly twice as much as good milestone events help.
Babaei, Mahmoudreza; Kulshrestha, Juhi; Chakraborty, Abhijnan; Redmiles, Elissa M.; Cha, Meeyoung; Gummadi, Krishna P.
Analyzing Biases in Perception of Truth in News Stories and Their Implications for Fact Checking Journal Article
In: IEEE Transactions on Computational Social Systems, 2021.
@article{Babaei2021Analy-56086,
title = {Analyzing Biases in Perception of Truth in News Stories and Their Implications for Fact Checking},
author = {Mahmoudreza Babaei and Juhi Kulshrestha and Abhijnan Chakraborty and Elissa M. Redmiles and Meeyoung Cha and Krishna P. Gummadi},
doi = {10.1109/TCSS.2021.3096038},
year = {2021},
date = {2021-01-01},
journal = {IEEE Transactions on Computational Social Systems},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Haak, Fabian; Schaer, Philipp
Perception-Aware Bias Detection for Query Suggestions Proceedings Article
In: Boratto, Ludovico; Faralli, Stefano; Marras, Mirko; Stilo, Giovanni (Ed.): Advances in Bias and Fairness in Information Retrieval - Second International Workshop on Algorithmic Bias in Search and Recommendation, BIAS 2021, Lucca, Italy, April 1, 2021, Proceedings, Springer Nature, Switzerland, 2021, ISBN: 978-3-030-78817-9.
Links | BibTeX | Tags: 2021 bias esupol haak myown schaer
@inproceedings{haak2021perceptionaware,
title = {Perception-Aware Bias Detection for Query Suggestions},
author = {Fabian Haak and Philipp Schaer},
editor = {Ludovico Boratto and Stefano Faralli and Mirko Marras and Giovanni Stilo},
doi = {10.1007/978-3-030-78818-6_12},
isbn = {978-3-030-78817-9},
year = {2021},
date = {2021-01-01},
booktitle = {Advances in Bias and Fairness in Information Retrieval - Second International Workshop on Algorithmic Bias in Search and Recommendation, BIAS 2021, Lucca, Italy, April 1, 2021, Proceedings},
volume = {1418},
publisher = {Springer Nature},
address = {Switzerland},
series = {Communications in Computer and Information Science},
keywords = {2021 bias esupol haak myown schaer},
pubstate = {published},
tppubtype = {inproceedings}
}
Zhukova, Anastasia; Hamborg, Felix; Donnay, Karsten; Gipp, Bela
Concept Identification of Directly and Indirectly Related Mentions Referring to Groups of Persons Proceedings Article
In: Diversity, Divergence, Dialogue: 16th International Conference, IConference 2021, Beijing, China, March 17–31, 2021, Proceedings, Part I, pp. 514–526, Springer-Verlag, Beijing, China, 2021, ISBN: 978-3-030-71291-4.
Abstract | Links | BibTeX | Tags: Coreference resolution, media bias, news analysis
@inproceedings{zhukova2021,
title = {Concept Identification of Directly and Indirectly Related Mentions Referring to Groups of Persons},
author = {Anastasia Zhukova and Felix Hamborg and Karsten Donnay and Bela Gipp},
url = {https://doi.org/10.1007/978-3-030-71292-1_40},
doi = {10.1007/978-3-030-71292-1_40},
isbn = {978-3-030-71291-4},
year = {2021},
date = {2021-01-01},
booktitle = {Diversity, Divergence, Dialogue: 16th International Conference, IConference 2021, Beijing, China, March 17–31, 2021, Proceedings, Part I},
pages = {514–526},
publisher = {Springer-Verlag},
address = {Beijing, China},
abstract = {Unsupervised concept identification through clustering, i.e., identification of semantically related words and phrases, is a common approach to identify contextual primitives employed in various use cases, e.g., text dimension reduction, i.e., replace words with the concepts to reduce the vocabulary size, summarization, and named entity resolution. We demonstrate the first results of an unsupervised approach for the identification of groups of persons as actors extracted from a set of related articles. Specifically, the approach clusters mentions of groups of persons that act as non-named entity actors in the texts, e.g., “migrant families” = “asylum-seekers.” Compared to our baseline, the approach keeps the mentions of the geopolitical entities separated, e.g., “Iran leaders” ≠ “European leaders,” and clusters (in)directly related mentions with diverse wording, e.g., “American officials” = “Trump Administration.”},
keywords = {Coreference resolution, media bias, news analysis},
pubstate = {published},
tppubtype = {inproceedings}
}
Unsupervised concept identification through clustering, i.e., identification of semantically related words and phrases, is a common approach to identify contextual primitives employed in various use cases, e.g., text dimension reduction, i.e., replace words with the concepts to reduce the vocabulary size, summarization, and named entity resolution. We demonstrate the first results of an unsupervised approach for the identification of groups of persons as actors extracted from a set of related articles. Specifically, the approach clusters mentions of groups of persons that act as non-named entity actors in the texts, e.g., “migrant families” = “asylum-seekers.” Compared to our baseline, the approach keeps the mentions of the geopolitical entities separated, e.g., “Iran leaders” ≠ “European leaders,” and clusters (in)directly related mentions with diverse wording, e.g., “American officials” = “Trump Administration.”
2020
Cabot, Pere-Lluís Huguet; Dankers, Verna; Abadi, David; Fischer, Agneta; Shutova, Ekaterina
The Pragmatics behind Politics: Modelling Metaphor, Framing and Emotion in Political Discourse Proceedings Article
In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4479–4488, Association for Computational Linguistics, Online, 2020.
Abstract | Links | BibTeX | Tags:
@inproceedings{huguet-cabot-etal-2020-pragmatics,
title = {The Pragmatics behind Politics: Modelling Metaphor, Framing and Emotion in Political Discourse},
author = {Pere-Lluís Huguet Cabot and Verna Dankers and David Abadi and Agneta Fischer and Ekaterina Shutova},
url = {https://aclanthology.org/2020.findings-emnlp.402},
doi = {10.18653/v1/2020.findings-emnlp.402},
year = {2020},
date = {2020-11-01},
booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
pages = {4479–4488},
publisher = {Association for Computational Linguistics},
address = {Online},
abstract = {There has been an increased interest in modelling political discourse within the natural language processing (NLP) community, in tasks such as political bias and misinformation detection, among others. Metaphor-rich and emotion-eliciting communication strategies are ubiquitous in political rhetoric, according to social science research. Yet, none of the existing computational models of political discourse has incorporated these phenomena. In this paper, we present the first joint models of metaphor, emotion and political rhetoric, and demonstrate that they advance performance in three tasks: predicting political perspective of news articles, party affiliation of politicians and framing of policy issues.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
There has been an increased interest in modelling political discourse within the natural language processing (NLP) community, in tasks such as political bias and misinformation detection, among others. Metaphor-rich and emotion-eliciting communication strategies are ubiquitous in political rhetoric, according to social science research. Yet, none of the existing computational models of political discourse has incorporated these phenomena. In this paper, we present the first joint models of metaphor, emotion and political rhetoric, and demonstrate that they advance performance in three tasks: predicting political perspective of news articles, party affiliation of politicians and framing of policy issues.
Spinde, Timo; Hamborg, Felix; Gipp, Bela
Media Bias in German News Articles : A Combined Approach Proceedings Article
In: Proceedings of the 8th International Workshop on News Recommendation and Analytics ( INRA 2020), Virtual event, 2020.
@inproceedings{Spinde2020,
title = {Media Bias in German News Articles : A Combined Approach},
author = {Timo Spinde and Felix Hamborg and Bela Gipp},
url = {https://media-bias-research.org/wp-content/uploads/2021/01/Media-Bias-in-German-News-Articles-A-Combined-Approach.pdf},
doi = {10.1007/978-3-030-65965-3_41},
year = {2020},
date = {2020-09-01},
urldate = {2020-09-01},
booktitle = {Proceedings of the 8th International Workshop
on News Recommendation and Analytics ( INRA 2020)},
address = {Virtual event},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Ganguly, Soumen; Kulshrestha, Juhi; An, Jisun; Kwak, Haewoon
Empirical Evaluation of Three Common Assumptions in Building Political Media Bias Datasets Proceedings Article
In: pp. 939-943, 2020.
@inproceedings{Ganguly_Kulshrestha_An_Kwak_2020,
title = {Empirical Evaluation of Three Common Assumptions in Building Political Media Bias Datasets},
author = {Soumen Ganguly and Juhi Kulshrestha and Jisun An and Haewoon Kwak},
url = {https://ojs.aaai.org/index.php/ICWSM/article/view/7362},
year = {2020},
date = {2020-05-01},
urldate = {2020-05-01},
journal = {Proceedings of the International AAAI Conference on Web and Social Media},
volume = {14},
number = {1},
pages = {939-943},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Spinde, Timo; Hamborg, Felix; Gipp, Bela
An Integrated Approach to Detect Media Bias in German News Articles Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, pp. 505–506, Association for Computing Machinery, Virtual Event, China, 2020, ISBN: 9781450375856.
Abstract | Links | BibTeX | Tags: content analysis, frame analysis, media bias, news bias, news slant
@inproceedings{10.1145/3383583.3398585,
title = {An Integrated Approach to Detect Media Bias in German News Articles},
author = {Timo Spinde and Felix Hamborg and Bela Gipp},
url = {https://doi.org/10.1145/3383583.3398585},
doi = {10.1145/3383583.3398585},
isbn = {9781450375856},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020},
pages = {505–506},
publisher = {Association for Computing Machinery},
address = {Virtual Event, China},
series = {JCDL '20},
abstract = {Media bias may often affect individuals' opinions on reported topics. Many existing methods that aim to identify such bias forms employ individual, specialized techniques and focus only on English texts. We propose to combine the state-of-the-art in order to further improve the performance in bias identification. Our prototype consists of three analysis components to identify media bias words in German news articles. We use an IDF-based component, a component utilizing a topic-dependent bias dictionary created using word embeddings, and an extensive dictionary of German emotional terms compiled from multiple sources. Finally, we discuss two not yet implemented analysis components that use machine learning and network analysis to identify media bias. All dictionary-based analysis components are experimentally extended with the use of general word embeddings. We also show the results of a user study.},
keywords = {content analysis, frame analysis, media bias, news bias, news slant},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias may often affect individuals' opinions on reported topics. Many existing methods that aim to identify such bias forms employ individual, specialized techniques and focus only on English texts. We propose to combine the state-of-the-art in order to further improve the performance in bias identification. Our prototype consists of three analysis components to identify media bias words in German news articles. We use an IDF-based component, a component utilizing a topic-dependent bias dictionary created using word embeddings, and an extensive dictionary of German emotional terms compiled from multiple sources. Finally, we discuss two not yet implemented analysis components that use machine learning and network analysis to identify media bias. All dictionary-based analysis components are experimentally extended with the use of general word embeddings. We also show the results of a user study.
Spinde, Timo; Hamborg, Felix; Donnay, Karsten; Becerra, Angelica; Gipp, Bela
Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, pp. 389–392, Association for Computing Machinery, Virtual Event, China, 2020, ISBN: 9781450375856.
Abstract | Links | BibTeX | Tags: bias visualization, news bias, news slant, perception of news
@inproceedings{10.1145/3383583.3398619,
title = {Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias},
author = {Timo Spinde and Felix Hamborg and Karsten Donnay and Angelica Becerra and Bela Gipp},
url = {https://doi.org/10.1145/3383583.3398619},
doi = {10.1145/3383583.3398619},
isbn = {9781450375856},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020},
pages = {389–392},
publisher = {Association for Computing Machinery},
address = {Virtual Event, China},
series = {JCDL '20},
abstract = {Traditional media outlets are known to report political news in a biased way, potentially affecting the political beliefs of the audience and even altering their voting behaviors. Many researchers focus on automatically detecting and identifying media bias in the news, but only very few studies exist that systematically analyze how theses biases can be best visualized and communicated. We create three manually annotated datasets and test varying visualization strategies. The results show no strong effects of becoming aware of the bias of the treatment groups compared to the control group, although a visualization of hand-annotated bias communicated bias in-stances more effectively than a framing visualization. Showing participants an overview page, which opposes different viewpoints on the same topic, does not yield differences in respondents' bias perception. Using a multilevel model, we find that perceived journalist bias is significantly related to perceived political extremeness and impartiality of the article.},
keywords = {bias visualization, news bias, news slant, perception of news},
pubstate = {published},
tppubtype = {inproceedings}
}
Traditional media outlets are known to report political news in a biased way, potentially affecting the political beliefs of the audience and even altering their voting behaviors. Many researchers focus on automatically detecting and identifying media bias in the news, but only very few studies exist that systematically analyze how theses biases can be best visualized and communicated. We create three manually annotated datasets and test varying visualization strategies. The results show no strong effects of becoming aware of the bias of the treatment groups compared to the control group, although a visualization of hand-annotated bias communicated bias in-stances more effectively than a framing visualization. Showing participants an overview page, which opposes different viewpoints on the same topic, does not yield differences in respondents' bias perception. Using a multilevel model, we find that perceived journalist bias is significantly related to perceived political extremeness and impartiality of the article.
Garz, Marcel; Sörensen, Jil; Stone, Daniel F.
Partisan selective engagement: Evidence from Facebook Journal Article
In: Journal of Economic Behavior & Organization, vol. 177, pp. 91-108, 2020, ISSN: 0167-2681.
Abstract | Links | BibTeX | Tags: Filter bubble, media bias, Polarization, Political immunity, Social media
@article{GARZ202091,
title = {Partisan selective engagement: Evidence from Facebook},
author = {Marcel Garz and Jil Sörensen and Daniel F. Stone},
url = {https://www.sciencedirect.com/science/article/pii/S0167268120302079},
doi = {https://doi.org/10.1016/j.jebo.2020.06.016},
issn = {0167-2681},
year = {2020},
date = {2020-01-01},
journal = {Journal of Economic Behavior & Organization},
volume = {177},
pages = {91-108},
abstract = {This study investigates the effects of variation in “congeniality” of news on Facebook user engagement (likes, shares, and comments). We compile an original data set of Facebook posts by 84 German news outlets on politicians that were investigated for criminal offenses from January 2012 to June 2017. We also construct an index of each outlet's media slant by comparing the language of the outlet with that of the main political parties, which allows us to measure the congeniality of the posts. We find that user engagement with congenial posts is higher than with uncongenial ones, especially in terms of likes. The within-outlet, within-topic design allows us to infer that the greater engagement with congenial news is likely driven by psychological and social factors, rather than a desire for accurate or otherwise instrumental information.},
keywords = {Filter bubble, media bias, Polarization, Political immunity, Social media},
pubstate = {published},
tppubtype = {article}
}
This study investigates the effects of variation in “congeniality” of news on Facebook user engagement (likes, shares, and comments). We compile an original data set of Facebook posts by 84 German news outlets on politicians that were investigated for criminal offenses from January 2012 to June 2017. We also construct an index of each outlet's media slant by comparing the language of the outlet with that of the main political parties, which allows us to measure the congeniality of the posts. We find that user engagement with congenial posts is higher than with uncongenial ones, especially in terms of likes. The within-outlet, within-topic design allows us to infer that the greater engagement with congenial news is likely driven by psychological and social factors, rather than a desire for accurate or otherwise instrumental information.
Garz, Marcel; Sood, Gaurav; Stone, Daniel F.; Wallace, Justin
The supply of media slant across outlets and demand for slant within outlets: Evidence from US presidential campaign news Journal Article
In: European Journal of Political Economy, vol. 63, pp. 101877, 2020, ISSN: 0176-2680.
Abstract | Links | BibTeX | Tags: Horse race news, media bias, Media slant, Motivated beliefs, Polarization, Selective exposure
@article{GARZ2020101877,
title = {The supply of media slant across outlets and demand for slant within outlets: Evidence from US presidential campaign news},
author = {Marcel Garz and Gaurav Sood and Daniel F. Stone and Justin Wallace},
url = {https://www.sciencedirect.com/science/article/pii/S0176268020300252},
doi = {https://doi.org/10.1016/j.ejpoleco.2020.101877},
issn = {0176-2680},
year = {2020},
date = {2020-01-01},
journal = {European Journal of Political Economy},
volume = {63},
pages = {101877},
abstract = {We conduct across-outlet and within-outlet (and within-topic) analyses of “congenially” slanted news. We study “horse race” news (news on candidates' chances in an upcoming election) from six major online outlets for the 2012 and 2016 US presidential campaigns. We find robust evidence that horse race headlines were slanted congenially with respect to the preferences of the outlets' typical readers. However, evidence of congenial slant in the timing and frequency of horse race stories is weaker. We also find limited evidence of greater within-outlet demand for headlines most congenial to outlets' typical readers, and somewhat stronger evidence of greater demand for relatively uncongenial headlines. We discuss how various aspects of our results are consistent with each of the major mechanisms driving slant studied in the theoretical literature, and may help explain when each mechanism is more likely to come into play. In particular, readers may be more likely to click on uncongenial headlines due to inferring that these stories are particularly informative when they stand in contrast to an outlet's typically congenial slant.},
keywords = {Horse race news, media bias, Media slant, Motivated beliefs, Polarization, Selective exposure},
pubstate = {published},
tppubtype = {article}
}
We conduct across-outlet and within-outlet (and within-topic) analyses of “congenially” slanted news. We study “horse race” news (news on candidates' chances in an upcoming election) from six major online outlets for the 2012 and 2016 US presidential campaigns. We find robust evidence that horse race headlines were slanted congenially with respect to the preferences of the outlets' typical readers. However, evidence of congenial slant in the timing and frequency of horse race stories is weaker. We also find limited evidence of greater within-outlet demand for headlines most congenial to outlets' typical readers, and somewhat stronger evidence of greater demand for relatively uncongenial headlines. We discuss how various aspects of our results are consistent with each of the major mechanisms driving slant studied in the theoretical literature, and may help explain when each mechanism is more likely to come into play. In particular, readers may be more likely to click on uncongenial headlines due to inferring that these stories are particularly informative when they stand in contrast to an outlet's typically congenial slant.
Hamborg, Felix; Zhukova, Anastasia; Gipp, Bela
Automated Identification of Media Bias by Word Choice and Labeling in News Articles Proceedings Article
In: Proceedings of the 18th Joint Conference on Digital Libraries, pp. 196–205, IEEE Press, Champaign, Illinois, 2020, ISBN: 9781728115474.
Abstract | Links | BibTeX | Tags: automated content analysis, automated frame analysis, CAQDAS, CAS, emotions, entity perception, news bias, news slant, NLP
@inproceedings{hamborg2019a,
title = {Automated Identification of Media Bias by Word Choice and Labeling in News Articles},
author = {Felix Hamborg and Anastasia Zhukova and Bela Gipp},
url = {https://doi.org/10.1109/JCDL.2019.00036},
doi = {10.1109/JCDL.2019.00036},
isbn = {9781728115474},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the 18th Joint Conference on Digital Libraries},
pages = {196–205},
publisher = {IEEE Press},
address = {Champaign, Illinois},
series = {JCDL '19},
abstract = {Media bias can strongly impact the individual and public perception of news events. One difficult-to-detect, yet powerful form of slanted news coverage is bias by word choice and labeling (WCL). Bias by WCL can occur when journalists refer to the same concept, yet use different terms, which results in different sentiments being sparked in the readers, such as the terms "economic migrants" vs. "refugees." We present an automated approach to identify bias by WCL that employs models and manual analysis approaches from the social sciences, a research domain in which media bias has been studied for decades. This paper makes three contributions. First, we present NewsWCL50, the first open evaluation dataset for the identification of bias by WCL consisting of 8,656 manual annotations in 50 news articles. Second, we propose a method capable of extracting instances of bias by WCL while outperforming state-of-the-art methods, such as coreference resolution, which currently cannot resolve very broadly defined or abstract coreferences used by journalists. We evaluate our method on the NewsWCL50 dataset, achieving an F1=45.7% compared to F1=29.8% achieved by the best performing state-of-the-art technique. Lastly, we present a prototype demonstrating the effectiveness of our approach in finding frames caused by bias by WCL.},
keywords = {automated content analysis, automated frame analysis, CAQDAS, CAS, emotions, entity perception, news bias, news slant, NLP},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias can strongly impact the individual and public perception of news events. One difficult-to-detect, yet powerful form of slanted news coverage is bias by word choice and labeling (WCL). Bias by WCL can occur when journalists refer to the same concept, yet use different terms, which results in different sentiments being sparked in the readers, such as the terms "economic migrants" vs. "refugees." We present an automated approach to identify bias by WCL that employs models and manual analysis approaches from the social sciences, a research domain in which media bias has been studied for decades. This paper makes three contributions. First, we present NewsWCL50, the first open evaluation dataset for the identification of bias by WCL consisting of 8,656 manual annotations in 50 news articles. Second, we propose a method capable of extracting instances of bias by WCL while outperforming state-of-the-art methods, such as coreference resolution, which currently cannot resolve very broadly defined or abstract coreferences used by journalists. We evaluate our method on the NewsWCL50 dataset, achieving an F1=45.7% compared to F1=29.8% achieved by the best performing state-of-the-art technique. Lastly, we present a prototype demonstrating the effectiveness of our approach in finding frames caused by bias by WCL.
2019
Bonart, Malte; Samokhina, Anastasiia; Heisenberg, Gernot; Schaer, Philipp
An investigation of biases in web search engine query suggestions Journal Article
In: Online Information Review, vol. 44, no. 2, pp. 365-381, 2019, ISSN: 1468-4527.
Abstract | Links | BibTeX | Tags: bias bonart esupol myown schaer
@article{bonart_investigation_2019,
title = {An investigation of biases in web search engine query suggestions},
author = {Malte Bonart and Anastasiia Samokhina and Gernot Heisenberg and Philipp Schaer},
url = {https://www.emerald.com/insight/content/doi/10.1108/OIR-11-2018-0341/full/html},
doi = {10.1108/OIR-11-2018-0341},
issn = {1468-4527},
year = {2019},
date = {2019-12-01},
urldate = {2020-01-02},
journal = {Online Information Review},
volume = {44},
number = {2},
pages = {365-381},
abstract = {Purpose
Survey-based studies suggest that search engines are trusted more than social media or even traditional news, although cases of false information or defamation are known. The purpose of this paper is to analyze query suggestion features of three search engines to see if these features introduce some bias into the query and search process that might compromise this trust. The authors test the approach on person-related search suggestions by querying the names of politicians from the German Bundestag before the German federal election of 2017.
Design/methodology/approach
This study introduces a framework to systematically examine and automatically analyze the varieties in different query suggestions for person names offered by major search engines. To test the framework, the authors collected data from the Google, Bing and DuckDuckGo query suggestion APIs over a period of four months for 629 different names of German politicians. The suggestions were clustered and statistically analyzed with regards to different biases, like gender, party or age and with regards to the stability of the suggestions over time.
Findings
By using the framework, the authors located three semantic clusters within the data set: suggestions related to politics and economics, location information and personal and other miscellaneous topics. Among other effects, the results of the analysis show a small bias in the form that male politicians receive slightly fewer suggestions on “personal and misc” topics. The stability analysis of the suggested terms over time shows that some suggestions are prevalent most of the time, while other suggestions fluctuate more often.
Originality/value
This study proposes a novel framework to automatically identify biases in web search engine query suggestions for person-related searches. Applying this framework on a set of person-related query suggestions shows first insights into the influence search engines can have on the query process of users that seek out information on politicians.},
keywords = {bias bonart esupol myown schaer},
pubstate = {published},
tppubtype = {article}
}
Purpose
Survey-based studies suggest that search engines are trusted more than social media or even traditional news, although cases of false information or defamation are known. The purpose of this paper is to analyze query suggestion features of three search engines to see if these features introduce some bias into the query and search process that might compromise this trust. The authors test the approach on person-related search suggestions by querying the names of politicians from the German Bundestag before the German federal election of 2017.
Design/methodology/approach
This study introduces a framework to systematically examine and automatically analyze the varieties in different query suggestions for person names offered by major search engines. To test the framework, the authors collected data from the Google, Bing and DuckDuckGo query suggestion APIs over a period of four months for 629 different names of German politicians. The suggestions were clustered and statistically analyzed with regards to different biases, like gender, party or age and with regards to the stability of the suggestions over time.
Findings
By using the framework, the authors located three semantic clusters within the data set: suggestions related to politics and economics, location information and personal and other miscellaneous topics. Among other effects, the results of the analysis show a small bias in the form that male politicians receive slightly fewer suggestions on “personal and misc” topics. The stability analysis of the suggested terms over time shows that some suggestions are prevalent most of the time, while other suggestions fluctuate more often.
Originality/value
This study proposes a novel framework to automatically identify biases in web search engine query suggestions for person-related searches. Applying this framework on a set of person-related query suggestions shows first insights into the influence search engines can have on the query process of users that seek out information on politicians.
Survey-based studies suggest that search engines are trusted more than social media or even traditional news, although cases of false information or defamation are known. The purpose of this paper is to analyze query suggestion features of three search engines to see if these features introduce some bias into the query and search process that might compromise this trust. The authors test the approach on person-related search suggestions by querying the names of politicians from the German Bundestag before the German federal election of 2017.
Design/methodology/approach
This study introduces a framework to systematically examine and automatically analyze the varieties in different query suggestions for person names offered by major search engines. To test the framework, the authors collected data from the Google, Bing and DuckDuckGo query suggestion APIs over a period of four months for 629 different names of German politicians. The suggestions were clustered and statistically analyzed with regards to different biases, like gender, party or age and with regards to the stability of the suggestions over time.
Findings
By using the framework, the authors located three semantic clusters within the data set: suggestions related to politics and economics, location information and personal and other miscellaneous topics. Among other effects, the results of the analysis show a small bias in the form that male politicians receive slightly fewer suggestions on “personal and misc” topics. The stability analysis of the suggested terms over time shows that some suggestions are prevalent most of the time, while other suggestions fluctuate more often.
Originality/value
This study proposes a novel framework to automatically identify biases in web search engine query suggestions for person-related searches. Applying this framework on a set of person-related query suggestions shows first insights into the influence search engines can have on the query process of users that seek out information on politicians.
Kulshrestha, Juhi; Eslami, Motahhare; Messias, Johnnatan; Zafar, Muhammad Bilal; Ghosh, Saptarshi; Gummadi, Krishna P.; Karahalios, Karrie
Search bias quantification : investigating political bias in social media and web search Journal Article
In: Information Retrieval Journal, vol. 22, no. 1-2, pp. 188–227, 2019, ISSN: 1386-4564.
@article{Kulshrestha2019-04Searc-53924,
title = {Search bias quantification : investigating political bias in social media and web search},
author = {Juhi Kulshrestha and Motahhare Eslami and Johnnatan Messias and Muhammad Bilal Zafar and Saptarshi Ghosh and Krishna P. Gummadi and Karrie Karahalios},
doi = {10.1007/s10791-018-9341-2},
issn = {1386-4564},
year = {2019},
date = {2019-01-01},
journal = {Information Retrieval Journal},
volume = {22},
number = {1-2},
pages = {188--227},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Hamborg, Felix; Zhukova, Anastasia; Gipp, Bela
Illegal Aliens or Undocumented Immigrants? Towards the Automated Identification of Bias by Word Choice and Labeling Proceedings Article
In: Taylor, Natalie Greene; Christian-Lamb, Caitlin; Martin, Michelle H.; Nardi, Bonnie (Ed.): Information in Contemporary Society, pp. 179–187, Springer International Publishing, Cham, 2019, ISBN: 978-3-030-15742-5.
Abstract | Links | BibTeX | Tags:
@inproceedings{hamborg2019,
title = {Illegal Aliens or Undocumented Immigrants? Towards the Automated Identification of Bias by Word Choice and Labeling},
author = {Felix Hamborg and Anastasia Zhukova and Bela Gipp},
editor = {Natalie Greene Taylor and Caitlin Christian-Lamb and Michelle H. Martin and Bonnie Nardi},
url = {https://media-bias-research.org/wp-content/uploads/2023/05/hamborg2019.pdf},
isbn = {978-3-030-15742-5},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {Information in Contemporary Society},
pages = {179--187},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Media bias, i.e., slanted news coverage, can strongly impact the public perception of topics reported in the news. While the analysis of media bias has recently gained attention in computer science, the automated methods and results tend to be simple when compared to approaches and results in the social sciences, where researchers have studied media bias for decades. We propose Newsalyze, a work-in-progress prototype that imitates a manual analysis concept for media bias established in the social sciences. Newsalyze aims to find instances of bias by word choice and labeling in a set of news articles reporting on the same event. Bias by word choice and labeling (WCL) occurs when journalists use different phrases to refer to the same semantic concept, e.g., actors or actions. This way, instances of bias by WCL can induce strongly divergent emotional responses from readers, such as the terms ``illegal aliens'' vs. ``undocumented immigrants.'' We describe two critical tasks of the analysis workflow, finding and mapping such phrases, and estimating their effects on readers. For both tasks, we also present first results, which indicate the effectiveness of exploiting methods and models from the social sciences in an automated approach.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias, i.e., slanted news coverage, can strongly impact the public perception of topics reported in the news. While the analysis of media bias has recently gained attention in computer science, the automated methods and results tend to be simple when compared to approaches and results in the social sciences, where researchers have studied media bias for decades. We propose Newsalyze, a work-in-progress prototype that imitates a manual analysis concept for media bias established in the social sciences. Newsalyze aims to find instances of bias by word choice and labeling in a set of news articles reporting on the same event. Bias by word choice and labeling (WCL) occurs when journalists use different phrases to refer to the same semantic concept, e.g., actors or actions. This way, instances of bias by WCL can induce strongly divergent emotional responses from readers, such as the terms ``illegal aliens'' vs. ``undocumented immigrants.'' We describe two critical tasks of the analysis workflow, finding and mapping such phrases, and estimating their effects on readers. For both tasks, we also present first results, which indicate the effectiveness of exploiting methods and models from the social sciences in an automated approach.
2018
Babaei, Mahmoudreza; Kulshrestha, Juhi; Chakraborty, Abhijnan; Benevenuto, Fabrício; Gummadi, Krishna P.; Weller, Adrian
Purple Feed: Identifying High Consensus News Posts on Social Media Proceedings Article
In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 10–16, Association for Computing Machinery, New Orleans, LA, USA, 2018, ISBN: 9781450360128.
Abstract | Links | BibTeX | Tags: audience leaning based features, consensus, news consumption in social media, Polarization, purple feed
@inproceedings{10.1145/3278721.3278761,
title = {Purple Feed: Identifying High Consensus News Posts on Social Media},
author = {Mahmoudreza Babaei and Juhi Kulshrestha and Abhijnan Chakraborty and Fabrício Benevenuto and Krishna P. Gummadi and Adrian Weller},
url = {https://doi.org/10.1145/3278721.3278761},
doi = {10.1145/3278721.3278761},
isbn = {9781450360128},
year = {2018},
date = {2018-01-01},
booktitle = {Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society},
pages = {10–16},
publisher = {Association for Computing Machinery},
address = {New Orleans, LA, USA},
series = {AIES '18},
abstract = {Although diverse news stories are actively posted on social media, readers often focus on the news which reinforces their pre-existing views, leading to 'filter bubble' effects. To combat this, some recent systems expose and nudge readers toward stories with different points of view. One example is the Wall Street Journal's 'Blue Feed, Red Feed' system, which presents posts from biased publishers on each side of a topic. However, these systems have had limited success. We present a complementary approach which identifies high consensus 'purple' posts that generate similar reactions from both 'blue' and 'red' readers. We define and operationalize consensus for news posts on Twitter in the context of US politics. We show that high consensus posts can be identified and discuss their empirical properties. We present a method for automatically identifying high and low consensus news posts on Twitter, which can work at scale across many publishers. To do this, we propose a novel category of audience leaning based features, which we show are well suited to this task. Finally, we present our 'Purple Feed' system which highlights high consensus posts from publishers on both sides of the political spectrum.},
keywords = {audience leaning based features, consensus, news consumption in social media, Polarization, purple feed},
pubstate = {published},
tppubtype = {inproceedings}
}
Although diverse news stories are actively posted on social media, readers often focus on the news which reinforces their pre-existing views, leading to 'filter bubble' effects. To combat this, some recent systems expose and nudge readers toward stories with different points of view. One example is the Wall Street Journal's 'Blue Feed, Red Feed' system, which presents posts from biased publishers on each side of a topic. However, these systems have had limited success. We present a complementary approach which identifies high consensus 'purple' posts that generate similar reactions from both 'blue' and 'red' readers. We define and operationalize consensus for news posts on Twitter in the context of US politics. We show that high consensus posts can be identified and discuss their empirical properties. We present a method for automatically identifying high and low consensus news posts on Twitter, which can work at scale across many publishers. To do this, we propose a novel category of audience leaning based features, which we show are well suited to this task. Finally, we present our 'Purple Feed' system which highlights high consensus posts from publishers on both sides of the political spectrum.
Ribeiro, Filipe N.; Henrique, Lucas; Benevenuto, Fabricio; Chakraborty, Abhijnan; Kulshrestha, Juhi; Babaei, Mahmoudreza; Gummadi, Krishna P.
Media Bias Monitor : Quantifying Biases of Social Media News Outlets at Large-Scale Proceedings Article
In: Twelfth International AAAI Conference on Web and Social Media, pp. 290–299, AAAI Press, Palo Alto, California, 2018, ISBN: 978-1-57735-798-8.
@inproceedings{Ribeiro2018-06-15Media-53950,
title = {Media Bias Monitor : Quantifying Biases of Social Media News Outlets at Large-Scale},
author = {Filipe N. Ribeiro and Lucas Henrique and Fabricio Benevenuto and Abhijnan Chakraborty and Juhi Kulshrestha and Mahmoudreza Babaei and Krishna P. Gummadi},
url = {https://aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/view/17878},
isbn = {978-1-57735-798-8},
year = {2018},
date = {2018-01-01},
booktitle = {Twelfth International AAAI Conference on Web and Social Media},
pages = {290--299},
publisher = {AAAI Press},
address = {Palo Alto, California},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Bonart, Malte; Schaer, Philipp
Intertemporal Connections Between Query Suggestions and Search Engine Results for Politics Related Queries Proceedings Article
In: EuroCSS 2018 Dataset Challenge, Cologne, 2018.
Links | BibTeX | Tags: bias bonart esupol myown schaer
@inproceedings{bonart2018intertemporal,
title = {Intertemporal Connections Between Query Suggestions and Search Engine Results for Politics Related Queries},
author = {Malte Bonart and Philipp Schaer},
url = {https://arxiv.org/abs/1812.08585},
year = {2018},
date = {2018-01-01},
booktitle = {EuroCSS 2018 Dataset Challenge},
address = {Cologne},
keywords = {bias bonart esupol myown schaer},
pubstate = {published},
tppubtype = {inproceedings}
}
2014
Garz, Marcel
Good news and bad news: evidence of media bias in unemployment reports Journal Article
In: Public Choice, vol. 161, no. 3/4, pp. 499–515, 2014, ISSN: 00485829, 15737101.
Abstract | Links | BibTeX | Tags:
@article{10.2307/24507505,
title = {Good news and bad news: evidence of media bias in unemployment reports},
author = {Marcel Garz},
url = {http://www.jstor.org/stable/24507505},
issn = {00485829, 15737101},
year = {2014},
date = {2014-01-01},
journal = {Public Choice},
volume = {161},
number = {3/4},
pages = {499--515},
publisher = {Springer},
abstract = {This study employs information obtained from media content analyses, as well as economic and political data, to investigate negativity in unemployment news between 2001 and 2010 in Germany. The data indicate a substantial bias in terms of the amounts of negative and positive reports, compared with the actual development of unemployment. Moreover, the media tend to place negative unemployment reports more prominently than positive ones. The estimates suggest that the bias is not the consequence of journalists asymmetrically interpreting the official unemployment numbers. Instead, it is associated with the exploitation of often non-economic information and structural influences in the process of news production.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This study employs information obtained from media content analyses, as well as economic and political data, to investigate negativity in unemployment news between 2001 and 2010 in Germany. The data indicate a substantial bias in terms of the amounts of negative and positive reports, compared with the actual development of unemployment. Moreover, the media tend to place negative unemployment reports more prominently than positive ones. The estimates suggest that the bias is not the consequence of journalists asymmetrically interpreting the official unemployment numbers. Instead, it is associated with the exploitation of often non-economic information and structural influences in the process of news production.
