Financial technology is transforming the world of financial services. Using big data analytics to enhance decision-making in areas such as portfolio and risk management has become a top priority for many financial institutions. Thanks to HKUST’s Yi Yang and co-researchers, institutions and investors now have a novel system for predicting financial risk that leverages not just what managers say but also how they say it.

Today’s investors research public companies using data from diverse sources, beyond merely financial statements and accounting numbers. “Massive amounts of unstructured multimedia data recording the verbal and vocal signals of managers,” say the authors, “are publicly available.” Market participants are ready to capitalize on the competitive advantages to be derived from analyzing managers’ verbal and vocal cues.

Although the words executives use in earnings conference calls are known to be associated with firms’ financial outcomes, little research has examined nonverbal vocal cues in such calls. “Due to formidable methodological challenges,” the researchers add, “the promise of business value from unstructured multimodal data has not materialized.”

To fill this gap, they developed DeepVoice, a system for predicting financial risk based on vocal cues in earnings conference calls. DesignVoice integrates verbal communication (words) with nonverbal cues (tone, pitch, and sentiment). At its core is a deep learning technology known as long short-term memory, which is particularly effective in capturing long sequences of input data, such as sentences.

The researchers tested DeepVoice with real earnings conference calls to see how well it predicted financial risks. “DeepVoice is remarkably effective and yields significantly lower forecast errors than the benchmark model,” they report. This improvement translates into economic gains in options trading. “Our DeepVoice design is directly relevant and applicable to financial market participants, including investors, analysts, and regulators,” the researchers conclude.