fix bugs and some merged PRs

fix bug

Release Highlights

Data Interpreter: Open Source and Better "Devin"

Data Interpreter is an autonomous agent that uses notebook, browser, shell, stable diffusion, and any custom tool to complete tasks.

Data Interpreter has achieved state-of-the-art scores in machine learning, mathematical reasoning, and open-ended tasks, and can analyze stocks, imitate websites, and train models.

It can debug code by itself, fix failures by itself, and solve a large number of real-life problems by itself.

We open-source our code and provide a wealth of working examples to give everyone access to state-of-the-art AI capabilities.

📝 Paper
🔗 Examples
📖 How to use

RAG

RAG module integrated! Common features such as indexing, retrieval, and rank are supported. Explore this example for a quick start.

New LLMs

New members for the supported LLM family: This time, Claude, QianFan, DashScope, and Yi. Check out here for configuration.

What's Changed

• Add ut mocker for mermaid ink by @shenchucheng in #884
• Unify Interpreter naming by @garylin2099 in #901
• Bump ipykernel to 6.27.1 by @revupst in #938
• Update role.py : Role.set_actions() by @lixiangtiandashen in #922
• Fix action.llm.model in examples/debate_simple by @BaoLiqi in #930
• Feat mock openai embed for document_store and memory UTs by @better629 in #939
• Feat add qianfan and dashscope by @better629 in #940
• Changed requirements.txt by @Abhishek0075 in #933
• Feat simplify provider ut code by @better629 in #942
• Update prompt of ocr example by @mannaandpoem in #944
• Feat: Wrap Openllm, Fireworks and other services into the OpenAILLM class. by @YangQianli92 in #946
• Doc: add docker installation section and fold it by @voidking in #950
• Feat fix missing mistral by @better629 in #953
• Feat add anthropic_api by @better629 in #961
• Update code plan and change and Improve test coverage by @mannaandpoem in #948
• Rename interpreter by @garylin2099 in #952
• Feat: RFC197 by @iorisa in #960
• Feat: +SECURITY.md by @iorisa in #970
• Feat: fix JSONDecodeError error caused by incorrect indentation used … by @moyitech in #963
• Feat: +rfc197 example by @iorisa in #968
• Improvement made : by automating search engine configuration in example search_in_search_engine.py by @Abhishek0075 in #965
• Restructure DataInterpreter by @garylin2099 in #995
• Restore WalmartSalesForecast example. by @orange-crow in #1007
• Feat add offcial yi-34b by @better629 in #1010
• Fix bug#979 : RuntimeError: fail to reduce message length by @azurewtl in #986
• Feat:add user llm config by @liujun3660105 in #997
• Feat add rag by @better629 in #974
• Fixbug: #1031 by @iorisa in #1035
• Update gpt-3.5-turbo-0125 in count_message_tokens by @RuifengFu in #1020
• Fix: typos by @testwill in #1027
• Fix bug: #1018 by @iorisa in #1037
• Fix critical bug: human prior not injected by @garylin2099 in #1043
• Correct bad URL in token_counter by @jinchihe in #1042
• Update readme with data interpreter usage by @garylin2099 in #1047
• Fix text ut error by @shenchucheng in #1049
• Update process for "Fix bug" by @mannaandpoem in #1054
• Fixbug: #1016 by @iorisa in #1058
• Add params to search engine by @geekan in #1065
• Feat: + tree command by @iorisa in #985
• Fix state value extract for issues/1067 by @better629 in #1069
• Fix bug: llm.timeout not working by @iorisa in #1060
• Feat: repo to markdown by @iorisa in #1061
• Fdd custom tool example by @geekan in #1071
• Make the rag module optional by @seehi in #1070
• Fix minecraft typo by @geekan in #1072
• Fixbug: #1076 by @iorisa in #1080
• Replace *.deepwisdom.ai with a local server for scraping by @shenchucheng in #1081
• Fixbug: #958 by @iorisa in #1082
• Updata di example and change the location of the .ipynb locations by @luxiangtaoya in #1090
• Fix issue 868 by @orange-crow in #1098
• Fix provider failed ut by @better629 in #1107
• Skip rebuild seq view by @geekan in #1108
• Fixbug: #1100 by @iorisa in #1105
• Changed concatenation of strings to f-strings to improve readability by @eukub in #648
• Feat rag add more features by @better629 in #1114
• Update paddleocr version to fix its bug by @garylin2099 in #1133

New Contributors

• @revupst made their first contribution in #938
• @lixiangtiandashen made their first contribution in #922
• @BaoLiqi made their first contribution in #930
• @Abhishek0075 made their first contribution in #933
• @YangQianli92 made their first contribution in #946
• @moyitech made their first contribution in #963
• @azurewtl made their first contribution in #986
• @liujun3660105 made their first contribution in #997
• @RuifengFu made their first contribution in #1020
• @testwill made their first contribution in #1027
• @jinchihe made their first contribution in #1042
• @luxiangtaoya made their first contribution in #1090
• @eukub made their first contribution in #648

Kudos to all the contributors for making their first mark!

A patch release for the following issues:

• fixbug: #958 by @iorisa in #1083

A patch release for the following issues:

• feat: gemini + proxy in #971
• fixbug: gbk UnicodeEncodeError in #973

A patch release for the following issues:

• Renamed interpreter, added arxiv link to Data Interpreter paper in #951

A patch release for the following issues:

• Fixed config error in #928

A patch release for the following issues:

• Fixed human interact and execute code bugs in #910
• Fixed tool path bug in #912

A patch release for the following issues:

• Unified Interpreter naming #902
• Fixed numpy dependency error #906

Release Highlights

• Easier and more flexible configuration: Simplified config setup, moving away from a global CONFIG to use Config instances for more granular control. Typical use cases involve assigning different LLMs to different Roles. Please check out here for detailed usage.
• Introduced Context: Context is made up of config, repo, workspace, etc. Each Role can possess its own context, allowing more flexible context management.
• Added metagpt --init-config command: A more friendly setup process for pypi package users.
• Repo restructuring: Consolidated FileRepo and GitRepo responsibilities to simplify code and logic.
• Subscription to tags: Changed the concept of subscription to tags for improved readability and usability.
• Removal of explicit serialize/deserialize functions: Removed these functions from Role, used mixin instead for better clarity.
• Supported initializing ActionNode from pydantic.BaseModel: An easy and structured way to define ActionNode.
• WritePRD Action restructuring: Simplified code, enhanced effectiveness, and fixed buggy logic.
• Env restructuring (experimental): Re-abstracted Env to accommodate various environments more effectively; Uses mark_as_readable mark_as_writable to decorate APIs to uniformly observe and act on the environment through observe and step.
• Embraced Multi-modality: Added a vision agent based on GPT-4V, check out the example here!
• Improved Incremental development: Higher success rate of incremental development compared to previous version.
• Introduced CodeInterpreter: A general agent who solves problems through coding. The agent leverages on plans and tools, capable of tackling a wide range of problems, including data visualization, machine learning modeling, math reasoning, web scraping, webpage imitating, etc. We are adding examples here!

What's Changed

Backwards Incompatible Changes

• IMPORTANT: We removed config.yam, key.yaml and switched to a minimal config2.yaml. Please check the latest setup method here
• We removed Role._init_actions and added Role.set_actions, which allows actions setup at any time besides initialization. For Role initialization specifically, users should also use self.set_actions(...) now

Features

• Easier and more flexible configuration, introduced Context, embraced Multi-modality, improved Incremental development, introduced CodeInterpreter, and more (see release highlights) #852 by @geekan @mannaandpoem @garylin2099 @shenchucheng @voidking @iorisa @better629 @stellaHSR @orange-crow @buttogo @FromCSUZhou @luxiangtaoya
• Supported Generating class views from source code #676 by @iorisa

Bugfixes and improvements

• Improved testing SOP, increased test coverage, and fix bugs during testing, ensuring quality and preparing for more effective development. #673 #678 #685 #688 #689 #690 #694 #696 #698 #702 #703 #707 #709 #710 #711 #713 #715 #718 #727 #729 #863 #867 by @shenchucheng @voidking @iorisa @garylin2099
• Fixed is_human not working at Role initialization #715 by @kkdev163
• Fixed system_prompt error at BaseLLM #732 by @better629
• Constrain the language for the qa_engineer #743 by @arnaudgelas
• Fixed: requirements.txt not written to the disk #744 by @arnaudgelas
• Fixed multiple issues at qa_engineer.py, including crashing if no code_doc, generating tests for non-python files, trying to install dependencies when there are none #765 #775 by @arnaudgelas
• Upgraded Zhipu API #770 #861 by @better629
• Fixed prompt logic when defining to whom the message should be sent. #785 by @arnaudgelas
• Fixed JSONDecodeError when parsing LLM outputs #799 by @HuZixia
• Modified get_choice_delta_text for safer dict access while using ZHIPU API #801 by @pkdog