MLLM-Tool

About MLLM-Tool

MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang, Weixin Luo, Qianyu Chen, Haonan Mai, Jindi Guo, Sixun Dong, Xiaohua (Michael) Xuan, Zhengxin Li, Lin Ma, Shenghua Gao. ShanghaiTech University && Meituan && UniDT This repository hosts the code, data and model weight of MLLM-Tool, the first tool agent MLLM that has the ability to perceive visual- and auditory- input information and recommend appropriate tools for multi-modal instructions. 🎉 News [x] [2024.02.02] 📢📢 We change the permissions for data and checkpoints, no longer need to apply to download them.

Quick Facts

Compatible Skills

These tools work well together with MLLM-Tool for enhanced workflows:

• VisualAgentBench — semantic(0.34)+complementary+same_lang+similar_pop+shared_platform (57%)
• groundingLMM — semantic(0.18)+complementary+rare_topics+same_lang+similar_pop+shared_platform (56%)
• VideoGLaMM — semantic(0.18)+complementary+rare_topics+same_lang+similar_pop+shared_platform (56%)
• luann — semantic(0.18)+complementary+rare_topics+same_lang+similar_pop+shared_platform (56%)
• mooc-cs294-llm-agents — semantic(0.30)+complementary+same_lang+similar_pop+shared_platform (56%)

More AI Tool Tools

Explore other popular ai tool tools:

• stagehand ⭐ 23.4k
• promptfoo ⭐ 23.0k
• gorilla ⭐ 12.8k
• PocketFlow-Tutorial-Codebase-Knowledge ⭐ 12.4k
• llm ⭐ 12.2k
• llm-engineer-toolkit ⭐ 10.5k
• phoenix ⭐ 10.5k
• code2prompt ⭐ 7.4k
• superagent ⭐ 6.5k
• ai-cookbook ⭐ 4.1k

Popular Python Agent Tools

• hermes-agent ⭐ 211.4k · Codex Skill
• AutoGPT ⭐ 185.4k · Agent Tool
• skills ⭐ 157.3k · Claude Skill
• langflow ⭐ 151.4k · Agent Tool
• open-webui ⭐ 143.9k · MCP Server

Frequently Asked Questions