Yiming Chen, Nikos Deligiannis, Lusine Abrahamyan, Valentin Ziatchin
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Compact convolutional neural networks (CNNs) have witnessed exceptional improvements in performance in recent years. However, they still fail to provide the same predictive power as CNNs with a large number of parameters. The diverse and even abundant features captured by the layers is an important characteristic of these successful CNNs. However, differences in this characteristic between large CNNs and their compact counterparts have rarely been investigated. In compact CNNs, due to the limited number of parameters, abundant features are unlikely to be obtained, and feature diversity becomes an essential characteristic. Diverse features present in the activation maps derived from a data point during model inference may indicate the presence of a set of unique descriptors necessary to distinguish between objects of different classes. In contrast, data points with low feature diversity may not provide a sufficient amount of unique descriptors to make a valid prediction; we refer to them as random predictions. Random predictions can negatively impact the optimization process and harm the final performance. This paper proposes addressing the problem raised by random predictions by reshaping the standard cross-entropy to make it biased toward data points with a limited number of unique descriptive features. Our novel Bias Loss focuses the training on a set of valuable data points and prevents the vast number of samples with poor learning features from misleading the optimization process. Furthermore, to show the importance of diversity, we present a family of SkipblockNet models whose architectures are brought to boost the number of unique descriptors in the last layers. Experiments conducted on benchmark datasets demonstrate the superiority of the proposed loss function over the cross-entropy loss. Moreover, our SkipblockNet-M can achieve 1% higher classification accuracy than MobileNetV3 Large with similar computational 50 100 150 200 250 300 350 400 FLOPs (millions) 68 70 72 74 76 ImageNet Top-1 Accuracy(%) SkipblockNet (ours) MobileNetV3 Large MobileNetV2 MuxNet ShuffleNetV2 MnasNet FBNet Figure 1. Accuracy v.s. FLOPs on ImageNet. Our SkipblockNet model trained with the proposed bias loss outperforms previous well-performing compact neural networks trained with the crossentropy loss. cost on the ImageNet ILSVRC-2012 classification dataset. The code is available on the linkhttps://github. com/lusinlu/biasloss_skipblocknet.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2107.11170")
get_code_for_paper("2107.11170")
have("2107.11170")
Connect an agent — have() is free.