About the Shared Task
The Multimodal AI for Finance Forecasting (Muffin) workshop will host two shared tasks on challenging multimodal financial forecasting problems using artificial intelligence:
Task-1: Financial Prediction from Conference Call Videos
Monetary policy calls (MPC) provide important insights into the actions taken by a country’s central bank on economic goals related to inflation, employment, prices, and interest rates. Investors and analysts critically analyze these video calls to forecast prices of stock market, treasury bonds, gold, and currency exchange rates post the conference call. Prior works in the NLP literature have looked at what is being said during the press conferences although there is a greater need to focus on how it is being said. The use of multimodal (visual+textual+audio) input to answer this question has been largely limited. Non-verbal behavioral cues from conference videos such as eye-movements, facial expressions, postures, gaits, complexity of language, vocal tone and facial expressions of the speakers may reflect emotions that subjects may not express through words and have been found to be strongly correlated with enhanced trading activities in the financial markets. Interpreting and extracting information from financial conference calls reveals difficult challenges such as: (1) Gap in current multimodal AI methods for simultaneously leverage visual, vocal and verbal modalities; (2) Long length of videos (50min to 1 hour) with multi-page text transcripts (3) Need to explore few-shot, semi-supervised, and self-supervised methods due to limited training data; (4) Large variability in conference calls across geographies due to different speakers, demographics, and economic conditions causing unintended bias. To this end, we curated a dataset of video conference calls from 2009 to 2022 released by central banks of 6 major English-speaking economies - USA, Canada, European Union, United Kingdom, New Zealand, and South Africa. The data has been processed to extract video frames, audio recordings, and utterance-aligned text transcripts. The task is to predict the volatility and price movement of stock market indices, gold, currency exchange rates, and bond prices T days after a conference call. We provide a cumulative of 25K data points split across training/development/testing for experimentation.
The train/validation/test corpus can be downloaded from: here .
Relevant research paper: [1] MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues
Task-2: Cryptocurrency Bubble Detection on Social Media
Cryptocurrency trading presents a new investment opportunity for maximizing profits. The rising ubiquity of speculative trading of cryptocurrencies over social media leads to rapid escalation and crash of price in a short period of time, also called bubbles, causing investment losses and bankruptcy. These crypto bubbles are strongly tied to user sentiment and social media usage as opposed to conventional value driven stocks and equities. Such financial bubbles are often a result of social media hype and the intensity of contagion among users, rendering both conventional statistical models and contemporary ML models weak as they are not built to deal with large volumes of unstructured, user-generated text on social media. In order to identify and safeguard against such bubbles, we formulate CryptoBubbles Detection Challenge - a novel multi-span prediction task over future days of time series price date for crypto assets. We have curated a dataset of 50 most traded crypto coins by volume from top 9 crypto exchanges such as Binance, Gateio, etc to obtain a time series of prices for 450+ crypto assets over five years accompanied with over 2.4 million related tweets.
The train/validation/test corpus can be downloaded from: here .
Relevant research paper: [2] Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models
Participation Guidelines
In order to participate, please first fill out this form to register for the shared tasks: here . The team name that is provided in this form will be used in the subsequent submissions and communications.
Use the train / validation / test dataset provided in the github pages to design your systems. Final results should be submitted as a 4-8 page paper to the EasyChair Submission Link by Shared Tasks paper submission deadline (December 15, 2023). All submitted papers will be evaluated based on their (1) novely, (2) performance results, (3) system description, and (4) reproducability. Participants are highly encouraged to submit their code and scripts to enhance reproducability and describe the system parametrs in sufficient details. Accepted submissions will be published as part of workshop proceedings.
Task and Data Citations
If you utilize the any of the above dataset or build solutions for / inspired by the above tasks, please cite the relevant work:
[1] Mathur, Puneet, Atula Tejaswi Neerkaje, Malika Chhibber, Ramit Sawhney, Fuming Guo, Franck Dernoncourt, Sanghamitra Dutta and Dinesh Manocha. “MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues.” Proceedings of the 30th ACM International Conference on Multimedia (2022).
[2] Sawhney, Ramit, Shivam Agarwal, Vivek Mittal, Paolo Rosso, Vikram Nanda and Sudheer Chava. “Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models.” NAACL (2022).