We lifted 11 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| arajv/SayNav | canonical | 11 of 11 |
| Function | Status | Where it lives |
|---|---|---|
| check_if_plan_needed | Ran | arajv/SayNav/src/python/pipeline.py pointer only (licence: NOASSERTION) · get_code("ef1c76d9e08018bb") |
| choose_random_door | Ran | arajv/SayNav/src/python/hl_utils.py pointer only (licence: NOASSERTION) · get_code("06f0d3cd6eca4815") |
| construct_goal | Ran | arajv/SayNav/src/python/llm.py pointer only (licence: NOASSERTION) · get_code("fb4c28e1e7d5b6f6") |
| extract_info | Ran | arajv/SayNav/src/python/pointnav_utils.py pointer only (licence: NOASSERTION) · get_code("fab4fadbda28248c") |
| extract_obs_pointnav | Ran | arajv/SayNav/src/python/pointnav_utils.py pointer only (licence: NOASSERTION) · get_code("847251abb9e81cf3") |
| extract_room_polygons | Ran | arajv/SayNav/src/python/utils_scenegraph.py pointer only (licence: NOASSERTION) · get_code("0aeca4a755dab63c") |
| is_point_inside_room | Ran | arajv/SayNav/src/python/hl_utils.py pointer only (licence: NOASSERTION) · get_code("35415fc333c3dff2") |
| locate_in_room | Ran | arajv/SayNav/src/python/utils_scenegraph.py pointer only (licence: NOASSERTION) · get_code("9f619a3b774fa234") |
| objectID_to_roomID | Ran | arajv/SayNav/src/python/utils_scenegraph.py pointer only (licence: NOASSERTION) · get_code("cd88ae5b84779a10") |
| snap | Ran | arajv/SayNav/src/python/pointnav_utils.py pointer only (licence: NOASSERTION) · get_code("a0bb73f8c6dafbf4") |
| update_visited_doors | Ran | arajv/SayNav/src/python/hl_utils.py pointer only (licence: NOASSERTION) · get_code("bf2bb65701e00832") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Semantic reasoning and dynamic planning capabilities are crucial for an autonomous agent to perform complex navigation tasks in unknown environments. It requires a large amount of common-sense knowledge, that humans possess, to succeed in these tasks. We present SayNav, a new approach that leverages human knowledge from Large Language Models (LLMs) for efficient generalization to complex navigation tasks in unknown large-scale environments. SayNav uses a novel grounding mechanism, that incrementally builds a 3D scene graph of the explored environment as inputs to LLMs, for generating feasible and contextually appropriate high-level plans for navigation. The LLM-generated plan is then executed by a pre-trained low-level planner, that treats each planned step as a short-distance point-goal navigation sub-task. SayNav dynamically generates step-by-step instructions during navigation and continuously refines future steps based on newly perceived information. We evaluate SayNav on multi-object navigation (MultiON) task, that requires the agent to utilize a massive amount of human knowledge to efficiently search multiple different objects in an unknown environment. We also introduce a benchmark dataset for MultiON task employing ProcTHOR framework that provides large photo-realistic indoor environments with variety of objects. SayNav achieves state-of-the-art results and even outperforms an oracle based baseline with strong ground-truth assumptions by more than 8% in terms of success rate, highlighting its ability to generate dynamic plans for successfully locating objects in large-scale new environments. The code, benchmark dataset and demonstration videos are accessible at https://www.sri.com/ics/computer-vision/saynav.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2309.04077")
get_code_for_paper("2309.04077")
have("2309.04077")
Connect an agent — have() is free.