Artificial Intelligence and Multimodality

CRASSH is the University of Cambridge's research centre in the Arts, Social Sciences and Humanities. We host events, conferences, concerts, films, exhibitions and major research projects.

CRASSH

A valuable opportunity for AI researchers and multimodal theorists to connect, and exchange ideas and approaches.

<h2>Artificial Intelligence and Multimodality: From Semiotics to Intelligent Systems</h2>An Online Workshop organised by the University of Cambridge and University College LondonProgramme AI and Multimodality: Learning and Knowledge12:30 – 13:30 Discussion 1: <a href="https://ahc.leeds.ac.uk/languages/staff/538/dr-elisabetta-adami" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer"></a><a href="https://www.liverpool.ac.uk/communication-and-media/staff/kay-ohalloran/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Kay O’Halloran</a> (University of Liverpool), <a href="https://www.imperial.ac.uk/people/l.specia" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Lucia Specia</a> (Imperial College London), <a href="https://ahc.leeds.ac.uk/languages/staff/538/dr-elisabetta-adami" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Elisabetta Adami </a>(University of Leeds) [discussant]14:00 – 1 5:00 Discussion 2: <a href="https://www.nie.edu.sg/profile/LIM_Fei_Victor" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Victor Lim Fei </a>(Nanyang Technological University), <a href="https://uclic.ucl.ac.uk/people/nadia-berthouze" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Nadia Berthouze</a> (University College London)AI and Multimodality: Communication and Agency 16:30 – 17:30 Discussion 3: T<a href="https://multimodalsemiotics.wordpress.com/theo-van-leeuwen/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">heo van Leeuwen</a> (University of Southern Denmark), <a href="https://douwekiela.github.io" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Douwe Kiela</a> (Facebook AI Research), <a href="https://researchportal.hw.ac.uk/en/persons/verena-rieser" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Verena Rieser</a> (Heriot-Watt University)18:00 – 19:00 Discussion 4: <a href="https://ucl.academia.edu/selenanemorin" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Selena Nemorin </a>(University College London), <a href="https://www.ravensbourne.ac.uk/about-us/our-people/carl-smith" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Carl Smith </a>(Ravensbourne University London), <a href="http://www.reading.ac.uk/english-language-and-appliedlinguistics/Aboutus/Staff/r-h-jones.aspx" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer"></a><a href="http://www.reading.ac.uk/english-language-and-appliedlinguistics/Aboutus/Staff/r-h-jones.aspx" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Rodney Jones </a>(University of Reading) [discussant]Please note: even though registration is for the whole programme, attendees are not required to attend all the discussion sessions.<h3></h3>The workshop will be held on Zoom. You can find the link and password in the &#x27;Additional Information&#x27; section of your booking confirmation email. Please also check your Spam folder.This live Zoom event will be recorded and uploaded to CRASSH&#x27;s YouTube channel.<h3></h3><h3>Workshop overview</h3>The workshop brings together the fields of Artificial Intelligence (AI) and Multimodality, with the aim of identifying common ground that connects these two research domains. The speakers, who are all prominent figures with an interest in AI and/or Multimodality, will explore the possibilities and potentialities for interdisciplinary interactions. Such considerations are timely since we live in a world that is increasingly multimodal. The internet, social media, mobile phones, and other digital technologies, have given prominence to forms of discourse in which texts, images, and sounds combine to convey complex messages. Animated emojis, internet memes, the automatic captioning of live shows, and e-literature are all illustrative examples of this trend. Since its emergence in the 1980s, the academic discipline of Multimodality has offered a set of robust analytical frameworks for examining multimodal phenomena. In social semiotic theories, for example, ‘modes’ such as writing, speech, image, music, and gesture, are viewed as socially and culturally determined semiotic resources which can be deployed to create meanings, and they all have different affordances – that is, particular potentialities and constraints that impact the making of signs in specific representations. Each set of modal affordances is determined by the ways in which meaning is created using the particular resources available – and the latter are inevitably characterised by their material, cultural, social, and historical development. A careful consideration of such things enables the form and function of multimodal communication in human social interactions to be analysed in greater detail, and the work of Gunther Kress, Theo Van Leeuven, Carey Jewitt, Jeff Bezemer, and others has been hugely influential.In a parallel development, over the last decade a new generation of Artificial Intelligence (AI) systems has attained unprecedented levels of sophistication when performing generative and analytical tasks which, traditionally, only humans had accomplished convincingly. These advances have been triggered by the resurgence of neural-based machine-learning techniques which coincided with the greater availability of more effective hardware, such as Graphical Processing Units (GPUs), that facilitated parallel computation. The task of building state-of-the-art systems has benefited greatly from the release of free open-source software packages such as PyTorch (2016-present) and Tensorflow (2017-present) – and these advances have made it possible to create intelligent and autonomous systems that are increasingly multimodal. For instance, systems have been developed that can identify racist or sexist internet memes process texts as well as images (e.g., The Facebook Hateful Meme Challenge), while researchers such as Lucia Specia have created Multimodal Machine Translation systems that perform image-guided translation of written or spoken texts. These multimodal tendencies are becoming increasingly prevalent in many modern AI applications.However, despite their many obvious overlapping areas of interest, the two research domains of AI and Multimodality remain largely disjunct. The computer scientists and information engineers who develop multimodal machine learning techniques tend to know very little about dominant theories of Multimodality, while, conversely, the semioticians, linguists, philosophers, sociologists, and social scientists who elaborate theories of Multimodality tend to know comparatively little about the architecture and application of multimodal AI systems. This persistent disconnection is regrettable since more extensive interactions involving researchers from both groups is likely to be mutually beneficial. Specifically, the design of multimodal AI systems could benefit greatly from a deeper awareness of existing theoretical frameworks for assessing human-based multimodal communication, while autonomous and intelligent systems that can create multimodal ensembles as outputs pose fundamental questions for any theories of Multimodality that presuppose human agency in the social and cultural contexts of meaning making.Responding to this undesirable disconnection, this workshop will provide an opportunity for AI researchers and multmodality theorists alike to come together in order to exchange ideas and approaches in order to address questions such as the following: How compatible are these two domains? What are the epistemological underpinnings of these two research traditions? Do they share any common ground? How do prononents of one area view those involved with the other? What would be the benefits of facilitating greater interactions between the two groups? The online workshop is a collaboration between the University of Cambridge and UCL organised by <a href="http://www.crassh.cam.ac.uk/people/profile/marcus-tomalin" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Marcus Tomalin,</a> project manager of the Cambridge-based ‘<a href="http://www.crassh.cam.ac.uk/programmes/giving-voice-to-digital-democracies" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Giving Voice to Digital Democracies</a>’ project, and <a href="https://iris.ucl.ac.uk/iris/browse/profile?upi=SDIAM12" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Sophia Diamantopoulou </a>a member of the <a href="https://www.ucl.ac.uk/ioe/departments-and-centres/centres/centre-multimodal-research" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Centre for Multimodal Research</a> and the facilitator of the <a href="https://multimodalforum.org" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Visual and Multimodal Research Forum</a>.

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Artificial Intelligence and Multimodality

More events from CRASSH

Follow organisers to get events picked for you

Still looking for the right event?

Explore all online events to browse and filter by date, category, and more.

Artificial Intelligence and Multimodality

More events from CRASSH

Follow organisers to get events picked for you

You might also like...

Browse more events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all online events to browse and filter by date, category, and more.