Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings
Discussion options

abetlen
Apr 4, 2023
Maintainer

Update (2023-05-19)

  • Imrove documentation for build / installation including available compiler options.
  • Huggingface Transformers API
  • Functional API and code quality improvements
  • Fix performance issues related to memory allocations

Just wanted to get some thoughts down re: short term project goals

  • Add tests or some kind of build process now that presumably more people are going to be using this project. (Add tests #24)
  • Improve the packaging and release process (currently we're using setup.py which I believe is a little outdated).
  • Simplify and test the implementation of stop sequences to ensure correctness in both simple and streaming modes. (Add additional tests for stop sequences #8)
  • Review OpenAI compatibility of the Python API and FastAPI server (Standalone Server #21)
You must be logged in to vote

Replies: 1 comment

Comment options

MillionthOdin16
Apr 5, 2023

Awesome! Until I saw this repo, I hadn't thought about using OpenAI's API spec as a standard to follow. I've seen lots of other servers that have unique implementations for their APIs, but mimicking OpenAI's standard opens up the possibility of using Llama models in tons of projects. Their use is so widespread and people understand it pretty well, so that was an awesome idea!

The the speed and quality might not be the same as GPT, but at least we can run things locally and experiment without worrying about API costs ❤️
You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@abetlen @MillionthOdin16
Morty Proxy This is a proxified and sanitized view of the page, visit original site.