Aaron Mueller, Mark Dredze
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Neural topic models can augment or replace bag-of-words inputs with the learned representations of deep pre-trained transformer-based word prediction models. One added benefit when using representations from multilingual models is that they facilitate zero-shot polylingual topic modeling. However, while it has been widely observed that pre-trained embeddings should be fine-tuned to a given task, it is not immediately clear what supervision should look like for an unsupervised task such as topic modeling. Thus, we propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual neural topic modeling. We consider fine-tuning on auxiliary tasks, constructing a new topic classification task, integrating the topic classification objective directly into topic model training, and continued pre-training. We find that fine-tuning encoder representations on topic classification and integrating the topic classification task directly into topic modeling improves topic quality, and that fine-tuning encoder representations on any task is the most important factor for facilitating cross-lingual transfer.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.05064")
get_code_for_paper("2104.05064")
have("2104.05064")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2104.05064.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2104.05064)
Connect an agent — have() is free.