We lifted 7 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| jetbrains-research/pandasplotbench | canonical | 0 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| add_index_to_filename | Not yet run | jetbrains-research/pandasplotbench/plotting_benchmark/vis_generator.py code served (permissive licence) · get_code("c1f9d7fdda583070") |
| dict_of_lists_to_list_of_dicts | Not yet run | jetbrains-research/pandasplotbench/plotting_benchmark/code_plot_generator.py code served (permissive licence) · get_code("260ccf7e5f32f1e6") |
| get_model_by_name | Not yet run | jetbrains-research/pandasplotbench/plotting_benchmark/generation_engines/get_model.py code served (permissive licence) · get_code("a208895e835dd0f2") |
| get_task_changing_single_task | Not yet run | jetbrains-research/pandasplotbench/alter_tasks.py code served (permissive licence) · get_code("aae9fcb300a70dcf") |
| get_task_shanging_task | Not yet run | jetbrains-research/pandasplotbench/alter_tasks.py code served (permissive licence) · get_code("931f899aa47f50a2") |
| read_jsonl | Not yet run | jetbrains-research/pandasplotbench/plotting_benchmark/vis_generator.py code served (permissive licence) · get_code("e661693417b2d5b1") |
| read_responses | Not yet run | jetbrains-research/pandasplotbench/plotting_benchmark/vis_generator.py code served (permissive licence) · get_code("1877a815145d7f0b") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses on generating code for visualizing tabular data - such as a Pandas DataFrame - based on natural language instructions, complementing current evaluation tools and expanding their scope. The dataset includes 175 unique tasks. Our experiments assess several leading Large Language Models (LLMs) across three visualization libraries: Matplotlib, Seaborn, and Plotly. We show that the shortening of tasks has a minimal effect on plotting capabilities, allowing for the user interface that accommodates concise user input without sacrificing functionality or accuracy. Another of our findings reveals that while LLMs perform well with popular libraries like Matplotlib and Seaborn, challenges persist with Plotly, highlighting areas for improvement. We hope that the modular design of our benchmark will broaden the current studies on generating visualizations. Our dataset and benchmark code are available online: https://huggingface.co/datasets/JetBrains-Research/PandasPlotBench; https://github.com/JetBrains-Research/PandasPlotBench.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.02764")
get_code_for_paper("2412.02764")
have("2412.02764")
Connect an agent — have() is free.