Can LLMs Beat Humans on Discerning Human-written and LLM-generated Science News?
The paper introduces SANews, a manually annotated dataset of paired human-written and GPT-3.5-generated science news articles from ScienceAlert, and shows that a Guided Few-shot prompting template with a single example boosts LLMs (including open-weight LLaMA-3 70b) to match or exceed graduate-student performance in discerning their origin.- doi
- 10.1145/3720553.3746674
- isbn
- 979-8-4007-1534-1
- name
- Can LLMs Beat Humans on Discerning Human-written and LLM-generated Science News?
- source
- BITS XML (with PDF figure crops)
- acm_url
- https://dl.acm.org/doi/10.1145/3720553.3746674
- authors
- Dominik Soós, Meng Jiang, Jian Wu
- doi_url
- https://doi.org/10.1145/3720553.3746674
- license
- CC BY 4.0
- summary
- The paper introduces SANews, a manually annotated dataset of paired human-written and GPT-3.5-generated science news articles from ScienceAlert, and shows that a Guided Few-shot prompting template with a single example boosts LLMs (including open-weight LLaMA-3 70b) to match or exceed graduate-student performance in discerning their origin.
- keywords
- science news detection, machine generated text evaluation, large language model
- published
- 2025-09-15
- conference
- HT '25: 36th ACM Conference on Hypertext and Social Media, Chicago, IL, USA, September 15-19, 2025
- open_access
- true
- acm_html_url
- https://dl.acm.org/doi/full/10.1145/3720553.3746674
- ccs_concepts
- Computing methodologies → Parallel computing methodologies; Natural language generation; Natural language processing.
- displayAuthor
- Dominik Soós, Meng Jiang, Jian Wu
- proceedings_url
- https://dl.acm.org/doi/proceedings/10.1145/3720553
- displayPublishTime
- 2025-09-15
- acm_reference_format
- Dominik Soós, Meng Jiang, and Jian Wu. 2025. Can LLMs Beat Humans on Discerning Human-written and LLM-generated Science News? In Proceedings of the 36th ACM Conference on Hypertext and Social Media (HT ’25), September 15–19, 2025, Chicago, IL, USA. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3720553.3746674
Powered by Seed HypermediaOpen App