Multi-Attribute Multi-Grained Adaptation of Pre-Trained Language Models for Text Understanding from Bayesian Perspective

Dan Xu, Jin Wang, You Zhang, Liang-Chih Yu, Xuejie Zhang

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.

Abstract

Current neural networks often employ multi-domain-learning or attribute-injecting mechanisms to incorporate nonindependent and identically distributed (non-IID) information for text understanding tasks by capturing individual characteristics and the relationships among samples. However, the extent of the impact of non-IID information and how these methods affect pre-trained language models (PLMs) remains unclear. This study revisits the assumption that non-IID information enhances PLMs to achieve performance improvements from a Bayesian perspective, which unearths and integrates non-IID and IID features. Furthermore, we proposed a multi-attribute multi-grained framework for PLM adaptations (M2A), which combines multi-attribute and multi-grained views to mitigate uncertainty in a lightweight manner. We evaluate M2A through prevalent text-understanding datasets and demonstrate its superior performance, mainly when data are implicitly non-IID, and PLMs scale larger.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2503.06085")
get_code_for_paper("2503.06085")
have("2503.06085")

Connect an agent — have() is free.