We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| google-research/vmoe | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| get_hyper | Ran | google-research/vmoe/vmoe/configs/eee_paper/eee_s32_last2_ilsvrc2012_ft_cifar100.py code served (permissive licence) · get_code("7f8cef48aff3f297") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models. We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs). First, we show that the two approaches have complementary features whose combination is beneficial. This includes a comprehensive evaluation of sparse MoEs in uncertainty related benchmarks. Then, we present Efficient Ensemble of Experts (E$^3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble. Extensive experiments demonstrate the accuracy, log-likelihood, few-shot learning, robustness, and uncertainty improvements of E$^3$ over several challenging vision Transformer-based baselines. E$^3$ not only preserves its efficiency while scaling to models with up to 2.7B parameters, but also provides better predictive performance and uncertainty estimates for larger models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2110.03360")
get_code_for_paper("2110.03360")
have("2110.03360")
Connect an agent — have() is free.