Commit Graph

718 Commits

Author SHA1 Message Date
Charles Packer
b789549d02
Configurable presets to support easy extension of MemGPT's function set (#420)
* partial

* working schema builder, tested that it matches the hand-written schemas

* correct another schema diff

* refactor

* basic working test

* refactored preset creation to use yaml files

* added docstring-parser

* add code for dynamic function linking in agent loading

* pretty schema diff printer

* support pulling from ~/.memgpt/functions/*.py

* clean

* allow looking for system prompts in ~/.memgpt/system_prompts

* create ~/.memgpt/system_prompts if it doesn't exist

* pull presets from ~/.memgpt/presets in addition to examples folder

* add support for loading agent configs that have additional keys

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2023-11-13 10:43:28 -08:00
Hans Raaf
17c5e3a0bc
I added some json repairs that helped me with malformed messages (#341)
* I added some json repairs that helped me with malformed messages

There are two of them: The first will remove hard line feeds that appear
in the message part because the model added those instead of escaped
line feeds. This happens a lot in my experiments and that actually fixes
them.

The second one is less tested and should handle the case that the model
answers with multiple blocks of strings in quotes or even uses unescaped
quotes. It should grab everything betwenn the message: " and the ending
curly braces, escape them and makes it propper json that way.

Disclaimer: Both function were written with the help of ChatGPT-4 (I
can't write much Python). I think the first one is quite solid but doubt
that the second one is fully working. Maybe somebody with more Python
skills than me (or with more time) has a better idea for this type of
malformed replies.

* Moved the repair output behind the debug flag and removed the "clean" one

* Added even more fixes (out of what I just encountered while testing)

It seems that cut of json can be corrected and sometimes the model is to
lazy to add not just one curly brace but two. I think it does not "cost"
a lot to try them all out. But the expeptions get massive that way :)

* black

* for the final hail mary with extract_first_json, might as well add a double end bracket instead of single

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-09 17:05:42 -08:00
Vivian Fang
e5c0e1276b
Remove AsyncAgent and async from cli (#400)
* Remove AsyncAgent and async from cli

Refactor agent.py memory.py

Refactor interface.py

Refactor main.py

Refactor openai_tools.py

Refactor cli/cli.py

stray asyncs

save

make legacy embeddings not use async

Refactor presets

Remove deleted function from import

* remove stray prints

* typo

* another stray print

* patch test

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-09 14:51:12 -08:00
Sarah Wooders
069780bc05
Use ~/.memgpt/config to set questionary defaults in memgpt configure + update tests to use specific config path (#389) 2023-11-09 14:01:11 -08:00
Charles Packer
055bc805b7
softpass test when keys are missing (#369)
* softpass test when keys are missing

* update to use local model

* both openai and local

* typo

* fix

* Specify model inference and embedding endpoint separately  (#286)

* Fix config tests (#343)

Co-authored-by: Vivian Fang <hi@vivi.sh>

* Avoid throwing error for older `~/.memgpt/config` files due to missing section `archival_storage` (#344)

* avoid error if has old config type

* Dependency management  (#337)

* Divides dependencies into `pip install pymemgpt[legacy,local,postgres,dev]`. 
* Update docs

* Relax verify_first_message_correctness to accept any function call (#340)

* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access

* Update `poetry.lock` (#346)

* mark depricated API section

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* CLI bug fixes for azure

* check azure before running

* Update README.md

* Update README.md

* bug fix with persona loading

* remove print

* make errors for cli flags more clear

* format

* fix imports

* fix imports

* add prints

* update lock

* Add autogen example that lets you chat with docs (#342)

* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access

* Support attach in memgpt autogen agent

* Add docs example

* Add documentation, cleanup

* add gpt-4-turbo (#349)

* add gpt-4-turbo

* add in another place

* change to 3.5 16k

* Revert relaxing verify_first_message_correctness, still add archival_memory_search as an exception (#350)

* Revert "Relax verify_first_message_correctness to accept any function call (#340)"

This reverts commit 30e911057d.

* add archival_memory_search as an exception for verify

* Bump version to 0.1.18 (#351)

* Remove `requirements.txt` and `requirements_local.txt` (#358)

* update requirements to match poetry

* update with extras

* remove requirements

* disable pretty exceptions (#367)

* Updated documentation for users (#365)


---------

Co-authored-by: Vivian Fang <hi@vivi.sh>

* Create pull_request_template.md (#368)

* Create pull_request_template.md

* Add pymemgpt-nightly workflow (#373)

* Add pymemgpt-nightly workflow

* change token name

* Update lmstudio.md (#382)

* Update lmstudio.md

* Update lmstudio.md

* Update lmstudio.md to show the Prompt Formatting Option (#384)

* Update lmstudio.md to show the Prompt Formatting Option

* Update lmstudio.md Update the screenshot

* Swap asset location from #384 (#385)

* Update poetry with `pg8000` and include `pgvector` in docs  (#390)

* Allow overriding config location with `MEMGPT_CONFIG_PATH` (#383)

* Always default to local embeddings if not OpenAI or Azure  (#387)

* Add support for larger archival memory stores (#359)

* Replace `memgpt run` flags error with warning + remove custom embedding endpoint option + add agent create time (#364)

* Update webui.md (#397)

turn emoji warning into markdown warning

* Update webui.md (#398)

* dont hard code embeddings

* formatting

* black

* add full deps

* remove changes

* update poetry

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
Co-authored-by: Vivian Fang <hi@vivi.sh>
Co-authored-by: MSZ-MGS <65172063+MSZ-MGS@users.noreply.github.com>
2023-11-09 12:11:19 -08:00
Sarah Wooders
4d1cb31b97
Fix config tests (#343)
Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-06 18:43:23 -08:00
Hans Raaf
0046ad9606
I made dump showing more messages and added a count (the last x) (#204)
* I made dump showing more message and added a count (the last x)

There seem to be some changes about the implementation so that the
current dump message helper functions do not show a lot of useful info.

I changed it so that you can `dump 5` (last 5 messages) and that it will
print user readable output. This lets you get some more understanding about
what is going on.

As some messages are still not shown I also show the index (reverse) of the
printed message, so one can see what to "pop" to reach a special point
without geting into the drumpraw.

* black

* patch

---------

Co-authored-by: Charles Packer <packercharles@gmail.com>
2023-11-03 20:47:23 -07:00
Charles Packer
cb88449376
try to patch hanging test (#295)
* try to patch hanging test

* add a timeout on the test
2023-11-03 19:11:29 -07:00
cpacker
1b6997ece1 hotfix 2023-11-03 16:25:39 -07:00
Sarah Wooders
2492db6b59
VectorDB support (pgvector) for archival memory (#226) 2023-11-03 16:19:15 -07:00
Charles Packer
270913218a
Add basic tests that are run on PR/main (#228)
* make tests dummy to make sure github workflow is fine

* black test

* strip circular import

* further dummy-fy the test

* use pexpect

* need y

* Update tests.yml

* Update tests.yml

* added prints

* sleep before decode print

* updated test to match legacy flow

* revising test where it fails

* comment out enter your message check for now, pexpect seems to be stuck on only setting the bootup message

* weird now it's not showing Bootup sequence complete?

* added debug

* handle none

* allow more time

* loosen string check

* add enter after commands

* modify saved compontent snippet

* add try again check

* more sendlines

* more excepts

* test passing locally

* Update tests.yml

* dont clearline

* add EOF catch that seems to only happen on github actiosn (ubuntu) but not macos

* more eof

* try flushing

* add strip_ui flag

* fix archival_memory_search and memory print output

* Don't use questionary for input if strip_ui

* Run black

* Always strip UI if TEST is set

* Add another flush

* expect Enter your message

* more debug prints

* one more shot at printing debug info

* stray fore color in stripped ui

* tests pass locally

* cleanup

---------

Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-01 17:01:45 -07:00
Charles Packer
0a075f448b
patch bug (#221) 2023-10-31 11:43:38 -07:00
Sarah Wooders
b7f9560bef
Refactoring CLI to use config file, connect to Llama Index data sources, and allow for multiple agents (#154)
* Migrate to `memgpt run` and `memgpt configure` 
* Add Llama index data sources via `memgpt load` 
* Save config files for defaults and agents
2023-10-30 16:47:54 -07:00
Sarah Wooders
18f14968b5 update poetry 2023-10-26 17:43:37 -07:00
Sarah Wooders
0f251af761 reformat 2023-10-26 16:08:25 -07:00
Sarah Wooders
686bee8a0a add database test 2023-10-26 15:30:31 -07:00
Sarah Wooders
541b2d1403 try to avoid changing current cli logic flow 2023-10-26 14:35:49 -07:00
Sarah Wooders
d3e7fd0517 add archival memory test 2023-10-26 14:25:46 -07:00