Juhwan Choi, Kyohoon Jin, Youngbin Kim, Eunju Lee
1 of 1 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 1 failure of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.
"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2402.05512.json · badge: /api/ran/2402.05512.svg
Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.
| Repository | Role | Ran |
|---|---|---|
| c-juhwan/gpt-multilingual-annotator | marked official in Papers with Code Papers with Code archive |
1 ran · 0 no recorded run |
| Function | Status | Where it lives |
|---|---|---|
| call_gpt_captioning | Ran ran; our contract draft was wrong, not the code |
c-juhwan/gpt-multilingual-annotator/gpt_annotator/annotator.py pointer only (licence: NONE) · get_code("541f8e340dbdad6f") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, c-juhwan/gpt-multilingual-annotator when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
Of 1 samples lifted from this paper's linked repositories, 1 carry a function identity and 0 do not. Of the 1 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 1 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).
Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.
Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Data annotation is an essential step for constructing new datasets. However, the conventional approach of data annotation through crowdsourcing is both time-consuming and expensive. In addition, the complexity of this process increases when dealing with lowresource languages owing to the difference in the language pool of crowdworkers. To address these issues, this study proposes an autonomous annotation method by utilizing large language models, which have been recently demonstrated to exhibit remarkable performance. Through our experiments, we demonstrate that the proposed method is not just costefficient but also applicable for low-resource language annotation. Additionally, we constructed an image captioning dataset using our approach and are committed to open this dataset for future study. We have opened our source code for reproducibility. 1 * NLLB (Machine-Translated): Dziewczyna graj ąca na tr ąbce w zespole. (A girl playing the trumpet in a band.) * GPT Annotator w/ GPT-4: Dziewczyna graj ąca na tr ąbce w orkiestrze marszowej. (A girl playing the trumpet in the march orchestra.) • Quality of Generated Sentence -Flickr File Name: 1386251841_5f384a0fea * English Reference: A woman is looking at dressed, headless mannequins in a store display. * Polish Reference: Kobieta ogl ąda wystawę z ubranymi w damskie stroje manekinami. (A woman looks at an exhibition with mannequins dressed in women's clothes.) * NLLB (Machine-Translated): Kobieta patrzy na ubrane, bezgłowe manieki w sklepach. (A woman looks at clothed, headless maniacs in stores.) * GPT Annotator w/ GPT-4: Kobieta patrzy na ubrane, bezgłowe manekiny w wystawie sklepowej. (A woman looks at clothed, headless mannequins in a store window.) -Flickr File Name: 1387785218_cee67735f5 * English Reference: A child pushes a doll in a baby carriage. * Polish Reference: Dziecko idzie drog ą, prowadz ąc przed sob ą mały wózek z lalk ą. (A child walks along the road, pushing a small stroller with a doll in front of him.) * NLLB (Machine-Translated): Mała dziewczynka wpychaj ąca koło lalki (A little girl pushing a doll wheel) * GPT Annotator w/ GPT-4: Dziecko popycha lalkę w wózku dla dzieci. (A child pushes a doll in a baby stroller.) -Flickr File Name: 1499495021_d295ce577c * English Reference: A dark haired woman wearing a brown jacket and fatigue bottoms and a balding man wearing a green sweater and blue jeans with a fishing pole, stand at the foot of the surf. * Polish Reference: Mężczyzna stoi z wędk ą nad brzegiem wody, a obok niego stoi kobieta. (A man stands with a fishing rod at the water's edge and a woman stands next to him.) * NLLB (Machine-Translated): Czarnowłosa kobieta w br ązowej kurtce i zmęczonym dnie i łysy mężczyzna w zielonej swetrze i niebieskich dżinsów z palcem rybaczem, stoj ą u stóp pływu. (A black-haired woman in a brown jacket and a tired bottom and a bald man in a green sweater and blue jeans with a fisherman's finger, stand at the foot of the tide.) * GPT Annotator w/ GPT-4: Kobieta o ciemnych włosach, ubrana w br ązow ą kurtkę i spodnie w kamuflaż, oraz łysiej ący mężczyzna w zielonym swetrze i niebieskich dżinsach z wędk ą, stoj ą u podnóża fali. (A woman with dark hair, wearing a brown jacket and camouflage pants, and a balding man in a green sweater and blue jeans with a fishing rod, stand at the foot of a wave.) -Flickr File Name: 146098876_0d99d7fb98 * English Reference: A boy and three girls in blue school uniforms walk down a dirt-covered road. * Polish Reference: Chłopiec i trzy dziewczynki w mundurkach id ą, nios ąc zeszyty. (A boy and three girls in uniforms are walking, carrying notebooks.) * NLLB (Machine-Translated): Chłopak i trzy dziewczyny w niebieskich mundurach szli po błędnej drodze. (A boy and three girls in blue uniforms were walking on the wrong path.) * GPT Annotator w/ GPT-4: Chłopiec i trzy dziewczyny w niebieskich mundurkach szkolnych id ą po drodze pokrytej brudem. (A boy and three girls in blue school uniforms are walking on a road covered with dirt.) E.4 Latvian Analysis • Quality of Generated Sentence -MSCOCO Image ID: 46544 * English Reference: A woman playing tennis on a tennis court. * NLLB (Machine-Translated): Sieva tenisā tenisā. (Tennis wife in tennis.) * GPT Annotator w/ GPT-4: Sieviete spēlē tenisu tenisa kortā. (A woman plays tennis on a tennis court.) -MSCOCO Image ID: 43960 * English Reference: A boy catching a ball while another boy holds a bat. * NLLB (Machine-Translated): Puikas, kas ien , em lopu, kamēr cits puikas, kas drīkst pien , emt lopu. (Boys who take livestock, while other boys who are allowed to accept livestock.) * GPT Annotator w/ GPT-4: Zēns nok , er balls, kamēr cits zēns tur nūju. (A boy catches the ball while another boy holds the stick.) -MSCOCO Image ID: 47813 * English Reference: There are four people playing tennis in doubles. * NLLB (Machine-Translated): Divās grupās spēlē četri cilvēki. (Four people play in two groups.) * GPT Annotator w/ GPT-4: Četri cilvēki spēlē tenisu dubultspēlēs. (Four people play tennis in doubles.) E.5 Estonian Analysis • Quality of Generated Sentence -MSCOCO Image ID: 1596 * English Reference: A person swing a tennis racket at a tennis ball. * NLLB (Machine-Translated): Üks inimene käigub tennisepalli peal tennise racket. (One person moves a tennis racket on top of a tennis ball.) * GPT Annotator w/ GPT-4: Inimene lööb tennise reketiga tennisepalli. (A person hits a tennis ball with a tennis racket.) -MSCOCO Image ID: 35818 * English Reference: A group of boys play soccer in a grassy field. * NLLB (Machine-Translated): Grupp poisid mängib jalgpalli mägedes. (A group of boys plays football in the mountains.) * GPT Annotator w/ GPT-4: Poiste grupp mängib jalgpalli rohusel väljakul. (A group of boys plays football on a green field.) -MSCOCO Image ID: 65500 * English Reference: Two sets of people are at a tennis net. * NLLB (Machine-Translated): Kaks inimest on tennistöö juures. (Two people are at tennis work.) * GPT Annotator w/ GPT-4: Kaks inimeste rühma on tennisevõrgu juures. (Two groups of people are at the tennis net.) E.6 Finnish Analysis • Quality of Generated Sentence -MSCOCO Image ID: 217929 * English Reference: people in uniforms playing baseball in the field * NLLB (Machine-Translated): joukkueessa pelaavat joukkueessa (in the team play in the team) * GPT Annotator w/ GPT-4: Ihmiset uniformuissa pelaavat baseballia kentällä. (People in uniforms are playing baseball on the field.) -MSCOCO Image ID: 226747 * English Reference: a persong swinging a tennis racket hitting a tennis ball * NLLB (Machine-Translated): laulaja, joka heiluttaa tenniskäytä, joka lyö tenniskappiin (the singer who swings the tennis racket, who hits the tennis locker) * GPT Annotator w/ GPT-4: Henkilö heiluttaa tennis-mailaa osuen tennispalloon. (A person swings a tennis racket hitting a tennis ball.) * Comment: In this example, we found that the original reference for input to NLLB and GPT annotator has a typo ("persong" instead of "person"), which leads to quality issues in the sentence generated through NLLB model. Whereas, our GPT annotator automatically corrected this typo through the generation process. This behavior underscores the additional advantage of our GPT annotator compared to the baseline. • Error Analysis on Basque -Case 1 * Source Text: A black dog and a spotted dog are fighting. * Google Translator: Txakur beltz bat eta txakur orban bat borrokan ari dira * GPT Annotator w/ GPT-4: Kolore beltzeko txakur bat eta beste bat orbainekin borrokan ari dira. (A black dog and another with scars are fighting.) * Comment: The word "spot" has changed to "scar." -Case 2 * Source Text: Several women wait outside in a city. * Google Translator: Hainbat emakume kanpoan itxaroten hiri batean. * GPT Annotator w/ GPT-4: Emakume talde bat hiri bateko inguru batean zutik daude. (A group of women are standing in an area of a city.) * Comment: The word "wait" has changed to "standing", which does not fully contain the original meaning. -Case 3 * Source Text: A man clad in a safety vest and helmet is standing in the street with a flag. * Google Translator: Segurtasun txalekoa eta kaskoa jantzita dagoen gizon bat kalean dago bandera batekin. * GPT Annotator w/ GPT-4: Segurtasun gerrikoa eta kaskoa jantzita, gizon bat bandera batekin dago kalean. (A man wearing a seat belt and a helmet stands on the street with a flag.) * Comment: The word "safety best" has changed to "seat belt." • Error Analysis on Māori -Case 1 * Source Text: A man in green holds a guitar while the other man observes his shirt. * Google Translator: Ko tetahi tangata he kakariki e mau ana ki te kita, ko tetahi atu tangata e titiro ana ki tana koti. * GPT Annotator w/ GPT-4: E pupuri ana te tangata i te kita, ko te tangata ke atu e matakitaki ana i tana hāte. (One is holding a guitar while the other is looking at his shirt.) * Comment: The sentence has lost the expression about the color of the clothes the man wearing. -Case 2 * Source Text: Boys perform dances on poles during the nighttime. * Google Translator: Ka kanikani nga tama ki runga pou i te po. * GPT Annotator w/ GPT-4: Tamariki tāne e mahi ake ana i ngā pou i te po tuturu. (Boys who work up posts in the real night.) * Comment: The word "perform dance" has changed to "work up." -Case 3 * Source Text: A ballet class of five girls jumping in sequence. * Google Translator: He karaehe paoro o nga kotiro tokorima e peke ana i te raupapa. * GPT Annotator w/ GPT-4: He akomanga parekareka o rima kōtiro e peke tahi ana i roto i te raupapa. (It's a fun class of five girls jumping together in a series.) * Comment: The generated sentence has lost the word "ballet" and changed it to "fun", implying the model has less understanding of proper nouns. • Error Analysis on Ethical Bias -Basque * Source Text: Two young, White males are outside near many bushes. * Google Translator: Bi ar zuri kanpoan daude zuhaixka askoren ondoan. * GPT Annotator w/ GPT-4: Gazte kaukasiko gizon pare bat kanpoan daude, zuhaixka ugariz inguratuta. (A couple of young Caucasian men are outside, surrounded by many bushes.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.05512")
get_code_for_paper("2402.05512")
have("2402.05512")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2402.05512.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2402.05512)
Connect an agent — have() is free.