For some use cases, esp. for apps using LLMs only for specific tasks, there might be a known list of few-shot examples with only the last one always changing according to user input. In such cases, caching the completions every time adds little boost to inference speed, and might actually slow down inference when used with slow memory/disk. Therefore, it is ideal to be able to cache those pre-defined of prompts at the start of the app, and then to skip caching in subsequent calls.

This PR adds a parameter (save_cache) to create_completion() and other related functions, as well as the completions API in the server. It is set to be True by default to avoid changing the default behavior, and is only used when caching is enabled.