Research Article Open Access

Poisson-Gamma Latent Dirichlet Allocation Model for Topics with Word Dependencies

Ibrahim Bakari Bala1 and Mohd Zainuri Saringat1
  • 1 Universiti Tun Hussein Onn Malaysia, Malaysia

Abstract

This paper introduces the Poisson-Gamma Latent Dirichlet Allocation (PGLDA) model for modeling word dependencies in topics modeling. The Poisson document length distribution has been used extensively in the past for modeling topics with the expectation that its effect will fizzle out at the end of the model definition. This procedure often leads to downplaying the effect of word correlation with topics and thus reducing the precision or accuracy of retrieved documents in such a situation. Therefore, we propose a new class of model that relaxes the words independence assumption in the existing Latent Dirichlet Allocation (LDA) model by introducing the Gamma distribution that can capture the correlation between adjacent words in a document. The Poisson document length distribution and Gamma correlation distribution are then convoluted to form a new mixture distribution for modeling word dependencies. Model parameter estimation was achieved via Laplacian approximation of the log-likelihood. The new model was then evaluated using the 20 Newsgroups and AG's News datasets. The applicability of the model was assessed using the F1 score. The results of the evaluation showed appreciable supremacy of PGLDA over LDA.

Journal of Computer Science
Volume 16 No. 12, 2020, 1684-1696

DOI: https://doi.org/10.3844/jcssp.2020.1684.1696

Submitted On: 6 August 2020 Published On: 3 December 2020

How to Cite: Bala, I. B. & Saringat, M. Z. (2020). Poisson-Gamma Latent Dirichlet Allocation Model for Topics with Word Dependencies. Journal of Computer Science, 16(12), 1684-1696. https://doi.org/10.3844/jcssp.2020.1684.1696

  • 2,624 Views
  • 1,027 Downloads
  • 2 Citations

Download

Keywords

  • Poisson Distribution
  • Gamma Distribution
  • Topic Model
  • Latent Dirichlet Allocation