Online social media platforms are turning into the prime source of news and narratives about worldwide events. However, a systematic summarization-based narrative extraction that can facilitate communicating the main underlying events is lacking. To address this issue, we propose a novel event-based narrative summary extraction framework. Our proposed framework is designed as a probabilistic topic model, with categorical time distribution, followed by extractive text summarization. Our topic model identifies topics’ recurrence over time with a varying time resolution. This framework not only captures the topic distributions from the data, but also approximates the user activity fluctuations over time. Furthermore, we define significance-dispersity trade-off (SDT) as a comparison measure to identify the topic with the highest lifetime attractiveness in a timestamped corpus. We evaluate our model on a large corpus of Twitter data, including more than one million tweets in the domain of the disinformation campaigns conducted against the White Helmets of Syria. Our results indicate that the proposed framework is effective in identifying topical trends, as well as extracting narrative summaries from text corpus with timestamped data.

Topic Modeling; Graphical Models; Narrative Extraction; Topic Detection and Tracking; Extractive Text Summarization; Online Social Media

• Information systems → Data mining; • Computing methodologies → Information extraction; Topic modeling; • Mathematics of computing → Gibbs sampling; • Networks → Online social networks.

Social media and microblogging platforms, such as Twitter and Facebook, are becoming the primary sources of real-time content regarding the ongoing socio-political events, including the United States Presidential Election in 2016 [11], and natural and manmade emergencies, such as the COVID-19 pandemic in 2020 [9]. However, without the appropriate tools, the massive textual data from these platforms makes it extremely challenging to obtain relevant information on significant events or the flow of information [14], distinguish between high-quality content and disinformation [18, 24, 30], or identify the opinions within a polarized domain [13]. The challenges mentioned above have been studied from different aspects related to topic detection and tracking within the field of Natural Language Processing (NLP). Researchers have developed automatic document summarization tools and techniques, which intend to provide concise and fluent summaries over a large corpus of textual data [21]. Preserving the key information in the summary and producing summaries that are comparable to human-created narratives are the primary goals of the extractive and abstractive approaches for automatic text summarization [2]. News websites are a prime example of such techniques, where automatic text summarization algorithms are applied to generate news headlines and titles from the news content [34]. The shortage of labeled data for text analysis has encouraged researchers to develop novel unsupervised algorithms that consider co-occurrence of words in documents as well as emerging new techniques such as exploiting an additional source of information similar to Wikipedia knowledge-based topic

models [40, 41]. Additionally, unsupervised learning enables training general-purpose systems that can be used for a variety of tasks and applications as strong classifiers [7]. In this regard, statistical models of co-occurrence such as Latent Dirichlet Allocation (LDA) [6], discover the relevant structure and co-occurrence dependencies of words within a collection of documents to capture the distribution of topic latent variable from the data. Although abundant timestamped textual data, particularly from social media platforms and news reports are available for analysis, the changes in the distribution of data over time have been neglected in most of the topic mining algorithms proposed in the literature [38]. For instance, time-series analysis on datasets over the events relative to 2012 US presidential election suggests that modeling topics and extracting summaries without considering the text-time relationship lead to missing the rise and fall of topics over time, the changes in terms of correlations, and the emergence of new topics [12]. Although continuous-time topic models such as [38] have been proposed in the literature, topical models with continuous-time distribution cannot model many modes in time, which leads to deficiency in modeling the fluctuations. Additionally, continuoustime models suffer from instability problems in the case of analyzing a multimodal dataset that is sparse in time. As studying datasets related to time-series activities on online social media platforms necessitates resolving multi-modality and sparsity, continuous-time topic models cannot accurately model the rises and falls in the distribution of topics for these platforms. Despite the existence of basic statistical methods to inspect the number of active individuals in a topic over time, recognizing the attractiveness of a topic within a time duration based on the distribution of the topic has not been investigated in literature. The significance of such measurement is evident when speculating the number of involved individuals in a topic is not feasible as the dataset lack information regarding the number of individuals in the conversations. In this paper, we propose a probabilistic model of topics over time with categorical time distribution to detect topical recurrence, designed as an LDA-based generative model. To achieve probabilistic modeling of narratives over topical trends, we incorporate the components of narratives including named-entities and temporal-causal coherence between events into our topical model. We believe that what differentiates a narrative model2 from topic analysis and summarization approaches is the ability to extract relevant sequences of text relative to the corresponding series of events associated with the same topic over time. Accordingly, our proposed narrative framework integrates unsupervised topic mining with extractive text summarization for narrative identification and summary extraction. We compare the identified narratives by our model with the topics identified by Latent Dirichlet Allocation (LDA) [6] and Topics over Time (TOT) [38]. This comparison includes presenting numerical results and analysis for a large corpus of more than one million tweets in the domain of disinformation campaigns conducted against the White Helmets of Syria. The collected dataset contains tweets spanning 13 months within the years 2018 and 2019. Our results provide evidence that our proposed method is effective in identifying topical trends within a timestamped data.

Furthermore, we define a novel metric called significance-dispersity trade-off (SDT) in order to compare and identify topics with higher lifetime attractiveness in timestamped data. Finally, we demonstrate that our proposed model discovers time localized topics over events that approximate the distribution of user activities on social media platforms. The remaining of this paper is organized as follows: First, an overview of the related works is provided in Section 2. In Section 3, we provide a detailed explanation of our proposed method followed by the experimental setup and results. Finally, in Section 5 we conclude the paper and discuss future directions.

In this section, we first provide a background on narrative analysis and how literature has investigated stories in social media. Then, we present an overview of topic modeling and text summarization.

2.1 Narrative Analysis

2.2 Topic Mining and Text Summarization

In this section we explain our narrative framework. The framework comprises of 2 steps: I. Narrative modeling based on topic identification over time; and II. extractive summarization from the

identified narratives. To discover the narratives over topical events, first, we use our discrete-time generative narrative model as an unsupervised learning algorithm to learn the distribution of textual contents from daily conversation cascades. Then, we extract narrative summaries over topical events from sentences in the time categories. This is achieved by sampling from the identified distribution of narratives and performing sentence ranking. Narrative modeling and summarization steps are explained below in separate subsections.

3.1 Narrative Modeling

To model narratives, we design our topic model such that the discovered topics present a series of timely ordered topical events. Accordingly, the topical events deliver a narrative covering distinct events over the same topic. In this regard, we present Narratives Over Categorical time (NOC)3, a novel probabilistic topic model that discovers topics based on both word co-occurrence and temporal information to present a narrative of events. According to the topic-time relationship explained above, we refer to the topics over time as narratives, topical events as events, and the extracted timely ordered sentences of documents with a high probability of belonging to each event as the extracted narrative summary. To fully comply with the definition of narrative, we assume a causality relation between the conversation cascades in social media. However, we do not investigate the causality relation across the conversation cascades or named-entities. The differences between our narrative model with dynamic topic models [5], topic models with continuous-time distribution [38], and hierarchical topic models [17, 28] include: not filtering the data for a specific event, imposing sharp transition for topic-time changes with time slicing, discovering topical events without scalability and sparsity issues, allowing multimodal topic distribution in time as a result of categorical time distribution, and selecting an appropriate slicing size such that distinct topical events be recognizable. Additionally, categorical time distribution enables discovering topical events with varying time resolution, for instance, weekly, biweekly, and monthly. Time discretization brings the question of selecting the appropriate slicing size or the number of categories that depend on the characteristics of the dataset under study. On the contrary, topical models with continuous-time distribution cannot model many modes in time. Additionally, continuous-time models such as [38] suffer from instability problems if the dataset is multimodal and sparse in time. Furthermore, categorical time enables discovering topic recurrence which results in identifying topical events related to distinct narrative activities, which is of our interest in this paper. Narrative activities in social media refer to the amount of textual content that is circulating in online platforms over time, corresponding to a specific topic. The generative process in NOC, models timestamps, and words per documents using Gibbs sampling which is a Markov Chain Monte Carlo (MCMC) algorithm. The graphical model of NOC is illustrated in Figure 1. As can be seen from the graphical model, the

Table 1: Symbols and definitions used in this paper.

Figure 1: The graphical model for NOC with Gibbs sampling.

posterior distribution of topics is dependent on both text and time modalities. This generative procedure can be described as follows:

I. For each topic 𝑧, draw 𝑇multinomials 𝜙𝑧from a Dirichlet prior 𝛽; II. For each document 𝑑, draw a multinomial 𝜃𝑑from a Dirichlet prior 𝛼; III. For each word 𝑤𝑑𝑖in 𝑑: (a) draw a topic 𝑧𝑑𝑖from multinomial 𝜃𝑑; (b) draw a word 𝑤𝑑𝑖from multinomial 𝜙𝑧𝑑𝑖; (c) draw a timestamp 𝑡𝑑𝑖from categorical 𝜓𝑧𝑑𝑖;

where the timestamps 𝑡𝑑𝑖for words 𝑤𝑑𝑖in each document 𝑑are identical. The list of symbols and their descriptions can be found in table 1. The model parameterization is as below:

𝜃𝑑|𝛼∼Discrete(𝛼)

𝜙𝑧|𝛽∼Discrete(𝛽)

𝑧𝑑𝑖|𝜃𝑑∼Multinomial(𝜃𝑑)

(1)

𝑤𝑑𝑖|𝜙𝑧𝑑𝑖∼Multinomial(𝜙𝑧𝑑𝑖)

𝑡𝑑𝑖|𝜓𝑧𝑑𝑖∼Categorical(𝜓𝑧𝑑𝑖)

In this model, Gibbs sampling provides an approximate inference instead of exact inference. To calculate the probability of topic assignment to word 𝑤𝑑𝑖, we first need to calculate the joint probability of the dataset as P(𝒛𝒅𝒊,𝒘𝒅𝒊, 𝒕𝒅𝒊|𝑤−𝑑𝑖,𝑡−𝑑𝑖,𝑧−𝑑𝑖, 𝛼, 𝛽,𝜓) and use chain rule to derive the probability of P(𝒛𝒅𝒊|𝒘, 𝒕, 𝒛−𝒅𝒊, 𝛼, 𝛽,𝜓) as below, where −𝑑𝑖subscript refers to all tokens except 𝑤𝑑𝑖:

P(𝑧𝑑𝑖|𝒘, 𝒕, 𝒛−𝒅𝒊, 𝛼, 𝛽,𝜓) ∝(𝑚𝑑𝑧𝑑𝑖+ 𝛼𝑧𝑑𝑖−1)

× 𝑛𝑧𝑑𝑖𝑤𝑑𝑖+ 𝛽𝑤𝑑𝑖−1

Í𝑉 𝑣=1(𝑛𝑧𝑑𝑖𝑣+ 𝛽𝑣) −1 𝑝(𝑡𝑧𝑑𝑖∈𝑏𝑘) (2)

where 𝑛𝑧𝑣refers to the number of words 𝑣assigned to topic 𝑧, 𝑚𝑑𝑧 refers to the number of word tokens in document𝑑that are assigned to topic 𝑧, and 𝑏𝑘represents the 𝑘th time slice. The details on the Gibbs sampling derivation can be found in the Appendix section. After each iteration of Gibbs sampling, we update the probability of 𝑝(𝑡𝑧𝑑𝑖∈𝑏𝑘) as follows:

𝐾 Õ

𝑝(𝑡𝑧𝑑𝑖∈𝑏𝑘) = 1

𝑘=1 I(𝑡𝑧𝑑𝑖∈𝑏𝑘) (3)

𝐾

where I(.) is equal to 1 when 𝑡𝑧𝑑𝑖∈𝑏𝑘, and 0 otherwise. In this paper, we report results with bi-weekly categorical time resolution. To determine the values for hyper-parameters 𝛼and 𝛽 and to investigate the sensitivity of the model to these values, we repeated our experiment with symmetric Dirichlet distributions using values 𝛼∈[0.1, 0.5, 1], 𝛽∈[0.01, 0.1, 0.5, 0.8, 1]. We observed that the model did not show significant sensitivity to the values of these hyper-parameters. Thus, we fix 𝛼= 1 and 𝛽= 0.5, both as symmetric Dirichlet distributions. We initialize the hyperparameter 𝜓in 2 ways for comparison: I. random initialization (model referred as 𝑁𝑂𝐶𝑅); and II. based on the probability of user activity per time category, illustrated in Figure 3c, (model referred as 𝑁𝑂𝐶𝐴). To estimate the number of topics for our experiments, we first visualize the tweets’ hashtag co-occurrence graph. We measure the graph modularity to examine the structure of the communities in this graph. We observe the highest modularity score of 0.41 using modularity resolution equal to 0.85. Figure 2 illustrates a downsampled version of this graph, where each color represents a modularity class. The edges of the graph are weighted according to the number of hashtags’ co-occurrence in the document collection. Our modularity analysis suggests that few distinct hashtag communities exist. Additionally, the dataset under study contains tweets associated with a single domain. As a result, we assume the number of topics to be relatively low. To choose an appropriate number of topics, we repeated LDA with the number of topics as 𝑇∈[4, . . . , 20] with increments of size 1. We evaluated the 𝑐𝑣coherence of topics identified by LDA and observed the highest coherence score for 𝑇= 5 and 𝑇= 6, respectively. Thus, we report our experimental results using these values.

Figure 2: The hashtag co-occurrence graph for twitter dataset on the domain of White Helmets of Syria for a period of 13 months, from 2018 to 2019. This graph represents a down-sampled version of the hashtag co-occurrence for this data for the sake of visualization.

3.2 Narrative Summary Extraction

As mentioned earlier, the discovered topics by NOC present a series of timely ordered topical events. Thus, the topical events deliver a narrative covering distinct social media events over the same topic. Figure 3 demonstrates the generated narrative distributions with NOC, where the hyperparameter 𝜓was randomly initialized (referred to as NOCR). In this figure, the vertical axis represents

the empirical probabilities of the topics. This figure represents that the identified narratives by our model are distinct from each other and the collapsed distribution of all narratives approximates the distribution of social media user activity over time. The identified narratives can be evaluated using effective evaluation metrics for topic models. Accordingly, we calculate pointwise mutual information [26] to measure the coherence of a topic 𝑧as follows:

Õ

𝐶𝑜ℎ𝑧= 2

𝑗<𝑘⩽𝐾 𝑙𝑜𝑔𝑝(𝑤𝑗,𝑤𝑘)

𝑝(𝑤𝑗)𝑝(𝑤𝑘) (4)

𝐾(𝐾−1)

where 𝐾is the number of most probable words for each narrative, 𝑝(𝑤𝑗) and 𝑝(𝑤𝑘) refer to the probabilities of occurrence for words 𝑤𝑗and 𝑤𝑘, and 𝑝(𝑤𝑗,𝑤𝑘) represents the probability of cooccurrence for the two words in the collection of documents. We compare our model with LDA and TOT[38], where TOT is a probabilistic topic model over time with Beta distribution for time. Table 2 displays the average coherence score measured across the discovered topics by LDA, TOT, and NOC. For NOC, we investigate initializing the parameter 𝜓with random and user activity-based initialization, referred to as NOCR and NOCA, respectively. We consider 𝐾= 500 most probable words from each topic. This comparison suggests that the narratives identified by NOC are more coherent than the identified topics by LDA, with an improvement in coherence of about 35%. The observed improvement comparing

with TOT was about 27%. Additionally, initializing the hyperparameter 𝜓in NOC using the distribution of user activity improves the narrative coherence by about 3%.

4.1 Proposed Evaluation Metric

The topic attractiveness to social media users can be investigated by basic statistical methods as a measure of the length of conversation cascades, the number of initiated textual content, and the number of unique users performing an activity relative to the underlying topic. The user activity fluctuations for timestamped data may contain activity bursts that are illustrative of significant events. Similarly, the generation and propagation of textual content within an online platform can illustrate the narrative activity relative to the events over time, where a burst represents a significant narrative activity. Additionally, the recurrence of a topic can be considered as an attractiveness measure for the associated topic. In this regard, we propose the significance-dispersity trade-off (SDT) metric to compare the identified narratives against each other. SDT measures the lifetime attractiveness of the identified narratives based on the distribution of narratives over topical events. The significance of such measurement is evident when speculating the number of involved individuals in a topic is not feasible as the dataset lack information regarding the number of individuals in the conversations. The proposed metric quantifies the significance of the narrative activities and recurrence of a topic via employing the Shannon entropy for the discovered narrative distributions. The intuition behind the SDT score is that the value of the entropy is maximum when the probability distribution is uniform. On the contrary, this value is minimum if the distribution is a delta function. This is visualized in Figure 4 in the Appendix section. We define dispersity of a categorical time topic distribution as a measure of the dispersion of the time categories. Based on this definition, SDT score of topic 𝑧can be obtained as:

𝑆𝐷𝑇𝑧= 𝐻𝛾(𝐻𝑚𝑎𝑥−𝐻)1−𝛾, (5)

where H is the Shannon entropy for the categorical distribution of time for topic 𝑧:

𝐾 Õ

𝑘 𝑝𝑧𝑙𝑜𝑔2𝑝𝑧, (6)

𝐻𝑧= −

𝐻𝑚𝑎𝑥= 𝑙𝑜𝑔2(𝐾),

and 𝐾refers to the number of time slices in the distribution. We assume that social media topics with high lifetime attractiveness are significant and recurrent. However, the probability distribution imposes a trade-off on the two. The parameter𝛾provides a weighted geometric mean of 𝐻and 𝐻𝑚𝑎𝑥−𝐻that enables promoting either

Table 3: The comparison of SDT scores for five topic narratives.

significance or recurrence, dependent on the application under study. A larger value of parameter 𝛾promotes dispersity for SDT score, and a smaller amount of this parameter promotes mode significance. The bounds for the SDT score are:



0 if 𝐻= 0 & 𝛾! = 0 0 if 𝐻= 𝐻𝑚𝑎𝑥& 𝛾! = 1 𝛾𝛾(1 −𝛾)1−𝛾𝐻𝑚𝑎𝑥 if 𝐻= 𝛾𝐻𝑚𝑎𝑥& 0 < 𝛾< 1

(7)

𝑆𝐷𝑇𝑖=



where 𝐻= 0 occurs when the distribution under study is uniform, and 𝐻= 𝐻𝑚𝑎𝑥relates to delta distribution. Since the time categorical distribution of our narrative model allows many modes in time, recurrent narratives can be identified. Additionally, the narrative activity fluctuations can be modeled using categorical time distribution in topic analysis. Table 3 provides a comparison for the SDT scores measured for the 5 identified narratives, using varying values of 𝛾. The illustration of the distribution of the extracted narratives can be seen in Figure 3a. We can clearly see in this figure that narratives 1 and 3 have the highest dispersity. On the contrary, narratives 4 and 2 have the highest significance. We compare 𝑆𝐷𝑇𝑖for narrative 𝑖with the number of user activity associated with narrative 𝑧. The results suggest that SDT score can be used to identify the narrative with higher lifetime attractiveness in a timestamped dataset. In our experiments, this is achieved for topic 1 when the value of 𝛾is greater than or equal to 0.7. As it can be seen, this topic is associated with the highest user activity count, reported in the same table.

4.2 Dataset Description and Pre-processing

In this paper, we addressed the problem of narrative modeling and narrative summary extraction for social media content. We presented a narrative framework consisting of I. Narratives over topic Categories (NOC), a probabilistic topic model with categorical time distribution; and II. extractive text summarization. The proposed narrative framework identifies narrative activities associated with social media events. Identifying topics’ recurrence and significance over time categories with our model allowed us to propose significance-dispersity trade-off (SDT) metric. SDT can be employed as a comparison measure to identify the topic with the highest lifetime attractiveness in a timestamped corpus. Results on real-world timestamped data suggest that the narrative framework is effective in identifying distinct and coherent topics from the data. Additionally, the results illustrate that the identified narrative distributions approximate the user activity fluctuations over time. moreover, informative, and concise narrative summaries for timestamped data are produced. Further improvement of the narrative framework can be achieved via incorporating the causality relation cross the social media conversation cascades and social media events into account. Other future directions include identifying topical hierarchies and extract summaries associated with each hierarchy.

This work was supported by the Defense Advanced Research Projects Agency (DARPA) under grant number FA8650-18-C-7823. The views and opinions expressed in this article are the authors’ own and should not be construed as official or as reflecting the views of the University of Central Florida, DARPA, or the U.S. Department of Defense.

[1] Rubayyi Alghamdi and Khalid Alfalqi. 2015. A survey of topic modeling in text mining. Int. J. Adv. Comput. Sci. Appl.(IJACSA) 6, 1 (2015). [2] Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saeid Safaei, Elizabeth D Trippe, Juan B Gutierrez, and Krys Kochut. 2017. Text summarization techniques: a brief survey. arXiv preprint arXiv:1707.02268 (2017). [3] Jeffery Ansah, Lin Liu, Wei Kang, Jixue Liu, and Jiuyong Li. 2020. Leveraging burst in twitter network communities for event detection. World Wide Web (2020), 1–26. [4] Regina Barzilay, Noemie Elhadad, and Kathleen R McKeown. 2001. Sentence ordering in multidocument summarization. In Proceedings of the first international

conference on Human language technology research. Association for Computational Linguistics, 1–7. [5] David M Blei and John D Lafferty. 2006. Dynamic topic models. In Proceedings of the 23rd international conference on Machine learning. 113–120. [6] David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993–1022. [7] Sophie Burkhardt and Stefan Kramer. 2019. A Survey of Multi-Label Topic Models. ACM SIGKDD Explorations Newsletter 21, 2 (2019), 61–79. [8] Freddy Chong Tat Chua and Sitaram Asur. 2013. Automatic summarization of events from social media. In Seventh international AAAI conference on weblogs and social media. [9] Matteo Cinelli, Walter Quattrociocchi, Alessandro Galeazzi, Carlo Michele Valensise, Emanuele Brugnoli, Ana Lucia Schmidt, Paola Zola, Fabiana Zollo, and Antonio Scala. 2020. The covid-19 social media infodemic. arXiv preprint arXiv:2003.05004 (2020). [10] Soumi Dutta, Vibhash Chandra, Kanav Mehra, Sujata Ghatak, Asit Kumar Das, and Saptarshi Ghosh. 2019. Summarizing microblogs during emergency events: A comparison of extractive summarization algorithms. In Emerging Technologies in Data Mining and Information Security. Springer, 859–872. [11] Gunn Enli. 2017. Twitter as arena for the authentic outsider: exploring the social media campaigns of Trump and Clinton in the 2016 US presidential election. European journal of communication 32, 1 (2017), 50–61. [12] Samah Gad, Waqas Javed, Sohaib Ghani, Niklas Elmqvist, Tom Ewing, Keith N Hampton, and Naren Ramakrishnan. 2015. ThemeDelta: Dynamic segmentations over temporal topic models. IEEE transactions on visualization and computer graphics 21, 5 (2015), 672–685. [13] Ivan Garibay, Alexander V Mantzaris, Amirarsalan Rajabi, and Cameron E Taylor. 2019. Polarization in social media assists influencers to become more influential: analysis and two inoculation strategies. Scientific Reports 9, 1 (2019), 1–9. [14] Ivan Garibay, Toktam A Oghaz, Niloofar Yousefi, Ece C Mutlu, Madeline Schiappa, Steven Scheinert, Georgios C Anagnostopoulos, Christina Bouwens, Stephen M Fiore, Alexander Mantzaris, et al. 2020. Deep Agent: Studying the Dynamics of Information Spread and Evolution in Social Networks. arXiv preprint arXiv:2003.11611 (2020). [15] Alexandra Georgakopoulou. 2017. 17 Small Stories Research: A Narrative Paradigm for the Analysis of Social Media. The Sage Handbook of social media research methods (2017). [16] Goran Glavaš, Jan Šnajder, Parisa Kordjamshidi, and Marie-Francine Moens. 2014. HiEve: A corpus for extracting event hierarchies from news stories. In Proceedings of 9th language resources and evaluation conference. ELRA, 3678–3683. [17] Thomas L Griffiths, Michael I Jordan, Joshua B Tenenbaum, and David M Blei. 2004. Hierarchical topic models and the nested chinese restaurant process. In Advances in neural information processing systems. 17–24. [18] Naeemul Hassan, Gensheng Zhang, Fatma Arslan, Josue Caraballo, Damian Jimenez, Siddhant Gawsane, Shohedul Hasan, Minumol Joseph, Aaditya Kulkarni, Anil Kumar Nayak, et al. 2017. ClaimBuster: the first-ever end-to-end factchecking system. Proceedings of the VLDB Endowment 10, 12 (2017), 1945–1948. [19] Bram Jans, Steven Bethard, Ivan Vulić, and Marie Francine Moens. 2012. Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 336–344. [20] Hamed Jelodar, Yongli Wang, Chi Yuan, Xia Feng, Xiahui Jiang, Yanchao Li, and Liang Zhao. 2019. Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a survey. Multimedia Tools and Applications 78, 11 (2019), 15169–15211. [21] Amrah Maryam and Rashid Ali. 2018. Twitter Based Event Summarization. In International Conference on Advances in Computing and Data Sciences. Springer, 165–174. [22] Andrew J McMinn and Joemon M Jose. 2015. Real-time entity-based event detection for twitter. In International conference of the cross-language evaluation forum for european languages. Springer, 65–77. [23] Elliot G Mishler. 1995. Models of narrative analysis: A typology. Journal of narrative and life history 5, 2 (1995), 87–123. [24] Ece Çiğdem Mutlu, Toktam Oghaz, Ege Tütüncüler, Jasser Jasser, and Ivan Garibay. 2020. Quantifying Latent Moral Foundations in Twitter Narratives: The Case of the Syrian White Helmets Misinformation. arXiv preprint arXiv:2004.13142 (2020). [25] Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Ranking sentences for extractive summarization with reinforcement learning. arXiv preprint arXiv:1802.08636 (2018). [26] David Newman, Jey Han Lau, Karl Grieser, and Timothy Baldwin. 2010. Automatic evaluation of topic coherence. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, 100–108. [27] Ruth Page. 2013. Seriality and storytelling in social media. StoryWorlds: A Journal of Narrative Studies 5 (2013), 31–54.

[28] Jay Pujara and Peter Skomoroch. 2012. Large-scale hierarchical topic models. In NIPS Workshop on Big Learning, Vol. 128. [29] Xiaojun Quan, Chunyu Kit, Yong Ge, and Sinno Jialin Pan. 2015. Short and sparse text topic modeling via self-aggregation. In Twenty-Fourth International Joint Conference on Artificial Intelligence. [30] Amirarsalan Rajabi, Chathika Gunaratne, Alexander V Mantzaris, and Ivan Garibay. 2020. Modeling Disinformation and the Effort to Counter It: A Cautionary Tale of When the Treatment Can Be Worse Than the Disease. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems. 1975–1977. [31] Pengjie Ren, Zhumin Chen, Zhaochun Ren, Furu Wei, Jun Ma, and Maarten de Rijke. 2017. Leveraging contextual sentence relations for extractive summarization using a neural attention model. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 95–104. [32] Lidan Shou, Zhenhua Wang, Ke Chen, and Gang Chen. 2013. Sumblr: continuous summarization of evolving tweet streams. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 533– 542. [33] PK Srijith, Mark Hepple, Kalina Bontcheva, and Daniel Preotiuc-Pietro. 2017. Sub-story detection in Twitter with hierarchical Dirichlet processes. Information Processing & Management 53, 4 (2017), 989–1003. [34] Jiwei Tan, Xiaojun Wan, and Jianguo Xiao. 2017. From Neural Sentence Summarization to Headline Generation: A Coarse-to-Fine Approach.. In IJCAI. 4109– 4115. [35] Johnny Torres, Carmen Vaca, Luis Terán, and Cristina L Abad. 2020. Seq2Seq models for recommending short text conversations. Expert Systems with Applications 150 (2020), 113270. [36] Josep Valls Vargas. 2017. Narrative information extraction with non-linear natural language processing pipelines. Drexel University. [37] Kristin Veel. 2018. Make data sing: The automation of storytelling. Big Data & Society 5, 1 (2018), 2053951718756686. [38] Xuerui Wang and Andrew McCallum. 2006. Topics over time: a non-Markov continuous-time model of topical trends. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 424–433. [39] Jiacheng Xu and Greg Durrett. 2019. Neural extractive text summarization with syntactic compression. arXiv preprint arXiv:1902.00863 (2019). [40] Kang Xu, Guilin Qi, Junheng Huang, and Tianxing Wu. 2017. Incorporating Wikipedia concepts and categories as prior knowledge into topic models. Intelligent Data Analysis 21, 2 (2017), 443–461. [41] Liang Yao, Yin Zhang, Baogang Wei, Lei Li, Fei Wu, Peng Zhang, and Yali Bian. 2016. Concept over time: the combination of probabilistic topic model with wikipedia knowledge. Expert Systems with Applications 60 (2016), 27–38.

Starting with the joint distribution $P(mathbf{w}, mathbf{t}, mathbf{z} mid alpha, beta, psi)$, we can use conjugate priors to simplify the equations as below:

where P and p refer to the probability mass function (PMF) and probability density function (PDF), respectively. The conditional probability $P(z{di} mid w,t,z{-di}, alpha, beta,psi)$ can be found using the chain rule as:

to the probability mass function (PMF) and probability density function (PDF), respectively. The conditional probability P(𝑧𝑑𝑖|𝑤,𝑡,𝑧−𝑑𝑖, 𝛼, 𝛽,𝜓) can be found using the chain rule as:

P(𝑧𝑑𝑖|𝑤,𝑡,𝑧−𝑑𝑖, 𝛼, 𝛽,𝜓) = P(𝑧𝑑𝑖,𝑤𝑑𝑖,𝑡𝑑𝑖|𝑤−𝑑𝑖,𝑡−𝑑𝑖,𝑧−𝑑𝑖, 𝛼, 𝛽,𝜓)

P(𝑤𝑑𝑖,𝑡𝑑𝑖|𝑤−𝑑𝑖,𝑡−𝑑𝑖,𝑧−𝑑𝑖, 𝛼, 𝛽,𝜓)

∝ P(𝑤,𝑡,𝑧|𝛼, 𝛽,𝜓)

P(𝑤−𝑑𝑖,𝑡−𝑑𝑖,𝑧−𝑑𝑖|𝛼, 𝛽,𝜓)

∝ 𝑛𝑧𝑑𝑖𝑤𝑑𝑖+ 𝛽𝑤𝑑𝑖−1

Í𝑉 𝑣=1(𝑛𝑧𝑑𝑖𝑣+ 𝛽𝑣) −1 (𝑚𝑑𝑧𝑑𝑖+ 𝛼𝑧𝑑𝑖−1) 𝑝(𝑡𝑑𝑖|𝜓𝑧𝑑𝑖)

∝(𝑚𝑑𝑧𝑑𝑖+ 𝛼𝑧𝑑𝑖−1) 𝑛𝑧𝑑𝑖𝑤𝑑𝑖+ 𝛽𝑤𝑑𝑖−1

Í𝑉 𝑣=1(𝑛𝑧𝑑𝑖𝑣+ 𝛽𝑣) −1 𝑝(𝑡𝑧𝑑𝑖∈𝑏𝑘)

(9) The probability of 𝑝(𝑡𝑑𝑖∈𝑏𝑘) can be measured as follows:

𝐾 Õ

𝑝(𝑡𝑧𝑑𝑖∈𝑏𝑘) = 1

𝑘=1 I(𝑡𝑧𝑑𝑖∈𝑏𝑘), (10)

𝐾

where I(.) is equal to 1 when 𝑡𝑧𝑑𝑖∈𝑏𝑘, and 0 otherwise.

The summaries provided here are the results for a sample run of the proposed narrative framework and do not reflect authors’ personal opinions.

In this study, the terms narrative and story are used interchangeably.

In this paper, topic modeling over topical trends is referred to as narrative modeling.

Code for NOC: https://github.com/toktammm/TwitterTopicsoverTime

Implementation used for TOT: https://github.com/ahmaurya/topicsovertime

Dataset and named-entity-recognition sources: https://www.leidos.com/ ; https://www.darpa.mil/program/computational-simulation-of-online-social-behavior ; https://nlp.stanford.edu/software/CRF-NER.html