This poster paper explores semiotics and rhetoric as narrative in social media visual culture, specifically with issues of identity and social change on social media platforms such as YouTube. Under the umbrella of semiotics, postmodernism, and poststructuralism, the paper builds upon the work of Roland Barthes, Stuart Hall, and Safiya Umoja Noble by expanding the concepts of visual semiotics, visual rhetoric, postcolonialism, critical race theory, and algorithms to examine the narrative of the image.

• Human-centered computing → Human computer interaction (HCI) → Interaction paradigms → Hypertext / hypermedia; • Human-centered computing → Collaborative and social computing → Collaborative and social computing theory, concepts and paradigms → Social media

social media; YouTube; visual semiotics; visual rhetoric; visual narrative; digital storytelling; visual culture

ACM Reference format:

Christopher C. Odom. 2020. The Narrative of the Image. In 31st ACM Conference on Hypertext and Social Media (HT’20), July 13–15, 2020, Virtual Event, USA. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3372923.3404806

The expanded cultural context of a connoted image can present an expansive narrative which surrounds the historical cultural narrative of the semiology [1:35]. Fully embracing the work of Stuart Hall, through the lens of the Birmingham School, critical race theory, and postcolonialism, the paper explores social media as an emergent technology built upon the computer metamedium [2:43]; a medium already steeped in white male heteronormativity [3:190]. The messages delivered in social media, whether it be

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. HT '20, July 13–15, 2020, Virtual Event, USA © 2020 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-7098-1/20/07. https://doi.org/10.1145/3372923.3404806

memes, breaking news, or a personal narrative, influence culture at-large. Building upon the work of Safia Umoja Noble in the area of algorithms, what appears on the screen in a social media platform, such as a platform like YouTube, is inescapable from what lies underneath the screen in the code [4:1]. The algorithm that determines which visual narrative gets suggested to the user is impactfully persuasive because the message encoded on the screen may differ from the rhetorical suggestions of the algorithm. The combined research of Barthes, Hall, and Noble form the theoretical framework for the narrative of the image.

The narrative of the image is rooted in the base concepts of semiotics and visual rhetoric. The expanded cultural context of a connoted image can present a vast narrative that encompasses the historical cultural narrative of the semiology. As a demonstration for linguistic academic researchers, Roland Barthes engages in a critical analysis of a Panzani Italian food advertisement image to illustrate how an image may contain visual semiotics that create a lexicon of meaning [1:33]. Barthes focuses upon the rhetoric of the image in terms of connotation and denotation, as well as the relationship of the sign, signifier, and the signified. The signified contains both a denotational and connotational meaning. The denotational definition is the literal everyday meaning of the sign or word, while the connotation is the meaning of the sign in a larger cultural context [1:37]. In terms of visual rhetoric, Barthes's framework for meaning is not ideal for analyzing a casual photograph, whereas a casual photograph does not have clear purpose and intent by the creator to forge meaning and craft a persuasive appeal. However, as artfully stated, Barthes is only looking at the rhetoric of the image, and a casual photograph is not an attempt at a persuasive argument to bestow a larger symbolic, coded, connotated message.

Stuart Hall concludes his essay "Encoding/Decoding" referencing the "politics of signification" which could have easily been a subtitle for his essay [5:144]. Hall is looking specifically at the audio-visual semiotics of aural-visual discourse. Because symbols are polysemic throughout cross-cultural interpretation within a greater society at-large, the combined denoted and connoted message is ultimately an interpretive connotation. The creator of a

message encodes the message with meaning which can be influenced by the hegemonic-dominant culture. That message is then decoded by the recipient but is not necessarily decoded from a point-of-view of the same cultural context as the hegemonic- dominant position. Hall proposes three positions for the hypothetical decoding of televisual discourse construction: dominant-hegemonic position, negotiated code, and oppositional code. Before a message can have an effect, fulfill a need, or be engaged in a use, it must be encoded in meaningful discourse and then dutifully decoded. The message is encoded at a determinate moment, influenced by culture and societal perceptions, and then decoded by the recipient which may or may not be using some of the same code to decode the message [5:138].

Distortions and misunderstandings in messaging result from a lack of equivalence of meaning in the communicated exchange between the creator of the encoded message and the receiver of the decoded message [5:139]. A televisual sign is holistically a connotative message, as both the aspects of the aural-visual denoted and connoted images are ideologically influenced by their associative meanings and polysemic values of both the encoder and the decoder [5:140–141].

Hall's argument should be modified to cover encoding and decoding as either objective or subjective on the account of the encoder or decoder. Whether conscious of subconscious, society at-large has already been influenced by polysemic semiotic signs [5:142]. Hall provides a compelling theoretical framework which explains how two people could both look at and listen to the same televisual content yet see and hear two entirely different things. The hegemonic-dominant viewpoint assumes dominance and naturalism of meaning, yet in reality intercultural meaning can be diverse, even polar opposites [5:143–144].

Imagery that appears on the screen in a social media platform, such as the video sharing social media platform YouTube, is directly related to what lies below in the critical code. In some respects, the algorithm that determines which visual narrative gets suggested to the user is more influentially persuasive because the message encoded on the screen may differ from the rhetorical suggestions of the algorithm underneath the screen.

Noble argues that racist and sexist algorithmic bias marginalizes and oppresses people through digital media platforms [4:1]. The overwhelming embracement of the Google search engine resulted in a total diversion of public goods and knowledge in libraries, in favor of what knowledge and information was made available through a corporate, privately held company. The public generally trusts information they find in a search engine query but it is often rife with misrepresentations resulting in broader social consequences [4:25]. Algorithms disproportionately result in negative biases against culturally marginalized groups, such as women of color. These biased search engine results influence the perception of the general public which help to propagate negative societal actions such as economic redlining, gender profiling, and misrepresentation [4:27].

By default, search engines can become inherently biased and racist because they are a reflection of the values and biases of the people who create and influence the algorithmic results. An algorithm is a mathematical formulation that makes automated decisions which amplify certain voices and silence other voices, moderated by capital and advertising dollars as dominant influences. In a broader social, historical, and economic context, this phenomenon is problematic because it further marginalizes people who are living oppressed in the margins from systemic oppression. In spite of cultural bias and misrepresentations, search engines present algorithmic automated search results as if the results represent neutral, fair, objective, and credible information [4:1].

An important facet of Noble's work is that she does not relegate herself to only identifying the flaws and digital social inequity in the algorithmic oppressive search results of Google. Noble stresses the need for search engine alternatives whose primary motivation is public interest, as opposed to private capitalistic corporate interests [4:179]. Academic scholars should explore additional algorithmic oppression solutions that extend beyond the dissolution of monopolies and focus more upon the government regulation algorithm ethics and social responsibility initiatives similar to those of the EU privacy laws.

This poster paper involves semiotics and rhetoric as narrative—the narrative of the image—in social media visual culture, specifically with issues of identity and social change on social media platforms such as YouTube. The narrative of the image is anchored by a theoretical framework built upon the research of Roland Barthes, Stuart Hall, and Safia Umoja Noble. Through the lens of a transdisciplinary conceptual framework, the narrative of the image traverses and interrelates multimodally with various schools of thought including semiotics, postmodernism, poststructuralism, the Birmingham School, critical race theory, postcolonialism, and algorithms.

ACKNOWLEDGMENTS This work was supported in part by the Delores A. Auzenne Fellowship. Special thanks to Dr. Mel Stanfill for oversight and direction.

[1] Roland Barthes. 1977. Rhetoric of the Image. In Image-Music-Text, translated by Stephen Heath. Hill and Wang, New York, 32–51. [2] Lev Manovich. 2013. Software Takes Command. Bloomsbury Academic, New York; London. [3] Mel Stanfill. 2016. Straighten Up and Fly White: Whiteness, Heteronormativity, and the Representation of Happy Endings for Fans. In Seeing Fans: Representations of Fandom in Media and Popular Culture, Lucy Bennett and Paul Booth (Eds.). Bloomsbury Academic, New York, 187–195. [4] Safiya Umoja Noble. 2018. Algorithms of oppression: how search engines reinforce racism. New York University Press, New York. [5] Stuart Hall. [1980] 2012. Encoding/Decoding. In Media and Cultural Studies: Keyworks, Second Edition, Meenakshi Gigi Durham and Douglas M. Kellner (Eds.). Wiley-Blackwell, Malden, MA, 137–144.