@jesusch @Null665 @whell

If anyone is also interested in this, I've found out that we need to do two things in order for it to work:

1. Override the env_prefix property of the class' __config__
2. Ask the __config__ instance to prepare fields again

I've put together a hack that works by doing something like this (yes, I know it is fugly):

import pydantic

class MySettings(pydantic.BaseSettings):
    param_a: bool


runtime_env_prefix = "something_set_at_runtime"

# 1. override the original env_prefix
MySettings.__config__.env_prefix = runtime_env_prefix

# 2. prepare fields again with the new env_prefix in place
for field in MySettings.__fields__.values():
    MySettings.__config__.prepare_field(field)

# now we can create our settings and it will look for env variables with the runtime env prefix
settings = MySettings()

I also have a need for this behavior. In the application I'm working on, users will need to configure connections to external data sources, many of which may be various instances of the same type of connection. Think of rclone "remotes", rclone allows users to configure muliple named remotes pointing to various s3 accounts/instances, but all of them share the same underlying configuration (because they are all s3 connections).

Because these connections will necessarily handle secrets, I would like to use BaseSettings for parsing them from the environment/.env files/etc.

Having reviewed the above discussion and related threads, I've also concluded that subclassing is probably the most stable way to implement this feature, but because users will not be interacting with the code directly (they will be using either a CLI or a web app backed by Pydantic models), the subclassing (if that is indeed the best solution) will need to be dynamic.

Here's what I've come up with so far. Very interested to know what others think of this solution, perhaps particularly @dmontagu as IIUC you are the sole Pydantic core dev on this thread (apologies if I've misunderstood that).

I'm particularly concerned with making sure my solution is stable, as this will eventually be deployed to a large user base. Thank you so much in advance for taking a look, and any feedback!

# change-prefix.py

import sys
from typing import Type

from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):

    key: str
    secret: str

    @classmethod
    def dynamically_subclass_with_prefix(cls: Type["Settings"], env_prefix: str) -> "Settings":
        """Dynamically subclass the Settings class, using the user-supplied `env_prefix`."""
        new_cls_name = env_prefix.title() + "Settings"
        config = {"env_prefix": f"{env_prefix}_", "case_sensitive": False}
        _cls = type(
            new_cls_name,
            (cls,),
            {"model_config": SettingsConfigDict(**config)},
        )
        return _cls()
    

if __name__ == "__main__":
    env_prefix = sys.argv[1]
    settings = Settings.dynamically_subclass_with_prefix(env_prefix)
    print(f"{settings = }")
    print(f"{settings.key = }")
    print(f"{settings.secret = }")

➜ FIRST_KEY="abc" FIRST_SECRET="123" python3 change-prefix.py "FIRST"
settings = FirstSettings(key='abc', secret='123')
settings.key = 'abc'
settings.secret = '123'
➜ SECOND_KEY="def" SECOND_SECRET="456" python3 change-prefix.py "SECOND"   
settings = SecondSettings(key='def', secret='456')
settings.key = 'def'
settings.secret = '456'
➜ ANOTHER_KEY="ghi" ANOTHER_SECRET="789" python3 change-prefix.py "ANOTHER"
settings = AnotherSettings(key='ghi', secret='789')
settings.key = 'ghi'
settings.secret = '789'