Description
When WithSubstitution is called multiple times on the same IConfigurationBuilder, it causes recursive accumulation of configuration sources, leading to excessive memory usage.
Root Cause
In SubstitutingConfigurationSource.Build(), when building valueRoot, the code iterates through ALL sources in builder.Sources except itself and creates a new ConfigurationBuilder with copies of those sources. Each subsequent call to WithSubstitution adds a new SubstitutingConfigurationSource that, when built, will copy all previously added sources again.
Demo
To see allocated memory depending on the number of added WithSubstitution calls, you can try:
const int iterations = 5;
// Emulate configurations
var configurations = Enumerable.Range(0, 500)
.Select(key => new KeyValuePair<string, string>(key.ToString(), null));
// Emulate multiple sources
var configuration = new ConfigurationBuilder();
for (var i = 0; i < iterations; i++)
{
configuration.WithSubstitution(c => c.AddInMemoryCollection(configurations));
}
configuration.Build();
// Check memory usage
var currentProcess = Process.GetCurrentProcess();
var usageInMb = currentProcess.WorkingSet64 / 1024 / 1024;On my machine, the results were:
| Sources |
Memory usage in MB |
| 2 |
48 |
| 3 |
49 |
| 4 |
52 |
| 5 |
61 |
| 6 |
139 |
| 7 |
661 |
| 8 |
4947 |
| 9 |
> 32 GB, OOM |
In real projects, there could be many sources, and this behavior may cause memory problems.
Suggestions
- First of all, I would recommend adding a notification in the README to use a single
WithSubstitution if possible.
Important
Avoid the following pattern to prevent memory leaks:
builder
.WithSubstitution(c => c.AddJsonFile("appsettings1.json"))
.WithSubstitution(c => c.AddJsonFile("appsettings2.json"))
.WithSubstitution(c => c.AddJsonFile("appsettings3.json"));Use a single WithSubstitution with multiple sources instead:
builder.WithSubstitution(c =>
{
c.AddJsonFile("appsettings1.json");
c.AddJsonFile("appsettings2.json");
c.AddJsonFile("appsettings3.json");
});Also, I would suggest to add configurable option to prevent multiple registrations:
public class SubstitutingSettings
{
/// <summary>
/// Use single substituting source to prevent multiple registrations, which can cause memory leaks.
/// Default value: false
/// </summary>
public bool SingleSubstitutingSource { get; set; } = false;
}internal class SubstitutingConfigurationSource : IConfigurationSource
{
private readonly SubstitutingSettings _settings;
public SubstitutingConfigurationSource(..., SubstitutingSettings settings)
{
_settings = settings;
}
public IConfigurationProvider Build(IConfigurationBuilder builder)
{
// ...
var valueBuilder = new ConfigurationBuilder();
for (var i = 0; i < builder.Sources.Count; i++)
{
var source = builder.Sources[i];
if (ReferenceEquals(source, this))
{
continue;
}
// Check if SingleSubstitutingSource is enabled
if (_settings.SingleSubstitutingSource && source is SubstitutingConfigurationSource)
{
throw new InvalidOperationException("Multiple substitution sources were detected when SingleSubstitutingSource setting is enabled.");
}
valueBuilder.Add(source);
}
// ...
}
}Usage:
var configuration = new ConfigurationBuilder()
.WithSubstitution(
c => ...,
new SubstitutingSettings { SingleSubstitutingSource = true }
)
.WithSubstitution(
c => ...,
new SubstitutingSettings { SingleSubstitutingSource = true }
)
.Build();SingleSubstitutingSource could be set to true by default to prevent such memory leaks, but this might break existing behavior. A safer approach is to keep the default as false and recommend enabling it in new projects, or introduce this change in a major version update.
Discussion
What do you think? Are there more elegant solutions – for example, adding a CycleDetector at build time to prevent recursive accumulation?
I'm open to suggestions. Would a pull request be welcome?
Description
When
WithSubstitutionis called multiple times on the sameIConfigurationBuilder, it causes recursive accumulation of configuration sources, leading to excessive memory usage.Root Cause
In
SubstitutingConfigurationSource.Build(), when buildingvalueRoot, the code iterates through ALL sources inbuilder.Sourcesexcept itself and creates a newConfigurationBuilderwith copies of those sources. Each subsequent call toWithSubstitutionadds a newSubstitutingConfigurationSourcethat, when built, will copy all previously added sources again.Demo
To see allocated memory depending on the number of added
WithSubstitutioncalls, you can try:On my machine, the results were:
In real projects, there could be many sources, and this behavior may cause memory problems.
Suggestions
WithSubstitutionif possible.Important
Avoid the following pattern to prevent memory leaks:
Use a single
WithSubstitutionwith multiple sources instead:Also, I would suggest to add configurable option to prevent multiple registrations:
Usage:
SingleSubstitutingSourcecould be set totrueby default to prevent such memory leaks, but this might break existing behavior. A safer approach is to keep the default asfalseand recommend enabling it in new projects, or introduce this change in a major version update.Discussion
What do you think? Are there more elegant solutions – for example, adding a
CycleDetectorat build time to prevent recursive accumulation?I'm open to suggestions. Would a pull request be welcome?