SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2310.14103 · EMNLP · 2023

Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications

Pierre Colombo, Manuel Faysse, Gautier Viaud, C. Hudelot

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.

Abstract

Instruction Fine-Tuning (IFT) is a powerful paradigm that strengthens the zero-shot capabilities of Large Language Models (LLMs), but in doing so induces new evaluation metric requirements. We show LLM-based metrics to be well adapted to these requirements, and leverage them to conduct an investigation of taskspecialization strategies, quantifying the tradeoffs that emerge in practical industrial settings. Our findings offer practitioners actionable insights for real-world IFT model deployment.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2310.14103")
get_code_for_paper("2310.14103")
have("2310.14103")

Connect an agent — have() is free.