Questions: Topic Modeling and Latent Dirichlet Allocation

Question 1

In LDA, a 'topic' is best described as which of the following?

• A. A human-assigned label (like 'politics' or 'sports') provided during training to guide word grouping
• B. A probability distribution over vocabulary words, where words that co-occur frequently receive high probability
• C. A cluster of documents that discuss the same subject, identified by their TF-IDF vectors
• D. A latent embedding of document meaning in continuous vector space, similar to word2vec

Answer: (B)

In LDA, a topic is a probability distribution over the vocabulary — it assigns a probability to every word in the corpus. The 'politics' topic might assign high probability to 'election,' 'vote,' 'policy,' and low probability to 'touchdown.' The model discovers these distributions purely from word co-occurrence patterns, with no human labels (option A is wrong — LDA is unsupervised). Option C describes document clustering, which is different from topic modeling. Option D describes neural embeddings, a distinct technique.

Question 2

You train an LDA model with k=5 topics on a corpus of academic papers. After examining the top words per topic, topics 3 and 4 appear to cover very similar themes and overlap heavily. What does this suggest?

• A. The model has converged incorrectly and needs to be retrained with better initialization
• B. The number of topics k may be too high for this corpus, causing coherent themes to be split across multiple topics
• C. This always happens with LDA because it cannot separate similar topics without labeled data
• D. Topic overlap means the Gibbs sampler did not run long enough and needs more iterations

Answer: (B)

k (number of topics) is a hyperparameter that must be chosen by the modeler — too few topics leave distinct themes blended together, and too many fragment coherent themes across multiple overlapping topics. Overlapping topics 3 and 4 suggests k is too large for the data's natural structure. This is analogous to choosing k in K-Means clustering. Option A mistakes a modeling choice issue for a convergence failure. Option C is wrong: LDA does distinguish between well-separated themes when k is chosen appropriately. Option D might occasionally be a cause but is not the primary interpretation of persistent topic overlap.

Question 3

LDA requires labeled training data — for example, document categories — in order to discover topics from a text corpus.

• T. True
• F. False

Answer: False

LDA is a fully unsupervised method. It infers latent topic structure from the raw word co-occurrence patterns in a document collection, with no labels provided. This is its key advantage for exploratory analysis: you can apply it to thousands of unlabeled documents and automatically surface recurring themes you didn't know existed. The number of topics k must be chosen, but k is a structural hyperparameter, not a label for any specific topic.

Question 4

In LDA, the number of topics k must be specified by the modeler before training, similar to how k must be chosen in K-Means clustering.

• T. True
• F. False

Answer: True

This is a fundamental limitation of LDA and K-Means alike. The algorithm does not determine the 'correct' number of topics from data alone — it will always find exactly k topics regardless of whether the data actually has k natural themes. Practitioners use heuristics like perplexity on held-out data, coherence scores, or domain knowledge to select k. Setting k too low gives overly broad topics; setting it too high produces fragmented, overlapping topics.

Question 5

Explain the 'dual representation' that LDA produces and why this enables more diverse applications than a model that only classifies documents into categories.

Model answer: LDA produces two complementary outputs: every document gets a topic proportion vector (e.g., 40% healthcare, 35% economics, 25% politics), and every topic gets a word probability distribution (healthcare topic: 'patient,' 'hospital,' 'treatment' have high probability). Classification models only assign a single category per document. The dual representation enables document similarity (compare topic proportion vectors), trend analysis (track how topic prevalence changes over time), content recommendation (find documents with similar topic profiles), and corpus organization — applications that require knowing *how much* of each theme a document contains, not just *which* category it belongs to.

The dual representation is what makes LDA powerful as an exploratory tool rather than just a classifier. A hard classification (this article is 'about politics') loses the nuance that the article is also 30% about economics and 20% about healthcare — nuance that may be exactly what a recommender system or trend analyst needs. The topic proportion vector is a rich, continuous representation of document content, while the word distributions make topics interpretable and auditable by humans.