Logo image
Identifying and Lnderstanding Business Trends using Topic Models with Word Embedding
Conference proceeding

Identifying and Lnderstanding Business Trends using Topic Models with Word Embedding

Yun Ning Pek and Kwan Hui Lim
IEEE International Conference on Big Data, pp.6177-6179
IEEE International Conference on Big Data
01/01/2019

Abstract

Computer Science Computer Science, Artificial Intelligence Computer Science, Information Systems Computer Science, Theory & Methods Science & Technology Technology
Topic modelling and trend analysis are increasingly important in today's digital world, especially for identifying promising business ideas and trends. With the increasing amount. of data being generated daily, a key challenge is to effectively identify emerging business ideas/topics and trends from this large volume of data. Towards this effort, we introduce a framework that allows us to identify promising business ideas from a large stream of academic papers. Academic papers are suitable for this purpose as they study emerging areas and problems in different domains. Our framework comprises three main components, namely: (i) a data collection component that retrieves academic papers and their meta-data; (ii) a topic modelling algorithm that combines traditional topic modelling techniques with recent advances in word embeddings; and (iii) a trend analysis component that allows us to visualize the popularity of different business trends/topics across time. Results on a corpus of 287k academic papers show that our proposed methods outperform the standard baselines based on topic coherence scores and also allows us to understand key temporal trends.

Metrics

1 Record Views

Details

Logo image