How to manage passwords in ASP.NET Core configuration files
Every web application needs to store passwords, ssh keys, ssh certificates or other information that can be classified as sensitive data in order to be able to connect to a database or any other services.
This article is about how to manage passwords in configuration files, to be more specific is how not to store the connection string passwords in appsettings.config files.
In ASP.NET Core application the configuration is based on key-value pairs established by configuration provider. Configuration providers read configuration data into key-value pairs from a variety of configuration source like settings files, environment variables, etc. You can find out more using the official documentation Configuration in ASP.NET Core
This is how a configuration file is added to an application:
...
var host = new HostBuilder()
.ConfigureAppConfiguration((config) => {
config.AddJsonFile(
"appsettings.json", optional: false, reloadOnChange: false);
});
...
and this is how a configuration for a connection string looks like:
{
"ConnectionStrings": {
"DatabaseConnection": "Server=myServerAddress;Database=myDataBase;User Id=myUsername;Password=password;MultipleActiveResultSets=True;"
}
}
In the above example the password is stored in plain text and unencrypted and this is not safe.
The Configuration API is capable of maintaining hierarchical configuration data by flattening the hierarchical data with the use of a delimiter in the configuration keys. For example the following JSON file
{
"section0": {
"key0": "value",
"key1": "value"
},
"section1": {
"key0": "value",
"key1": "value"
}
}
is stored into a ‘Dictionary<string,string>’ with following keys:
section0:key0
section0:key1
section1:key0
section1:key1
According to the documentation if a value for the same key is set by the same or different configuration providers, the last value set on the key is the value used.
When the application is deployed to production a container orchestration tool is used like Docker Swarm, Kubernetes, etc and all of them have methods to handle secrets. This article is about Docker Swarm.
Docker secrets allows you to centrally manage this sensitive data and securely transmit it to only those containers that need access to it. Secrets are encrypted during transit and at rest in a Docker swarm.
For each secret that is created Docker will mount a file inside the container. By default it will mount all the secrets in /run/secrets folder. You can find out more about Docker secrets using the official documentation Manage sensitive data with Docker secrets.
Next, the plan is to create a custom configuration provider that reads the files from the secrets folder and maps them to the application configuration. In this way a configuration key, in our example will be ConnectionStrings:DatabaseConnection, can be moved from appsettings.json into a secret.
You can create custom configuration providers by inherit from ConfigurationProvider abstract class.
public class SwarmSecretsConfigurationProvider : ConfigurationProvider
{
private readonly IEnumerable<SwarmSecretsPath> _secretsPaths;
public SwarmSecretsConfigurationProvider(
IEnumerable<SwarmSecretsPath> secretsPaths)
{
_secretsPaths = secretsPaths;
}
public override void Load()
{
var data = new Dictionary<string, string>
(StringComparer.OrdinalIgnoreCase);
foreach (var secretsPath in _secretsPaths)
{
if (!Directory.Exists(secretsPath.Path) && !secretsPath.Optional)
{
throw new FileNotFoundException(secretsPath.Path);
}
foreach (var filePath in Directory.GetFiles(secretsPath.Path))
{
var configurationKey = Path.GetFileName(filePath);
// if key delimiter is not : the replace
if (secretsPath.KeyDelimiter != ":")
{
configurationKey = configurationKey
.Replace(secretsPath.KeyDelimiter, ":");
}
// read the file content
var configurationValue = File.ReadAllText(filePath);
data.Add(configurationKey, configurationValue);
}
}
Data = data;
}
}
and then add a configuration source
public class SwarmSecretsConfigurationSource : IConfigurationSource
{
public IEnumerable<SwarmSecretsPath> SwarmSecretsPaths { get; private set; }
public SwarmSecretsConfigurationSource()
{
SwarmSecretsPaths = new List<SwarmSecretsPath>()
{
new SwarmSecretsPath()
{
Path = "/run/secrets",
KeyDelimiter = ":",
Optional = false
}
};
}
public SwarmSecretsConfigurationSource(
IEnumerable<SwarmSecretsPath> swarmSecretsPaths)
{
SwarmSecretsPaths = swarmSecretsPaths;
}
public IConfigurationProvider Build(IConfigurationBuilder builder)
{
return new SwarmSecretsConfigurationProvider(SwarmSecretsPaths);
}
}
followed by extension methods to easily add the configuration provider
public static class SwarmSecretsConfigurationExtensions
{
public static IConfigurationBuilder AddSwarmSecrets(
this IConfigurationBuilder builder)
{
builder.Add(new SwarmSecretsConfigurationSource());
return builder;
}
public static IConfigurationBuilder AddSwarmSecrets(
this IConfigurationBuilder builder,
IEnumerable<SwarmSecretsPath> swarmSecretsPaths)
{
builder.Add(new SwarmSecretsConfigurationSource(swarmSecretsPaths));
return builder;
}
public static IConfigurationBuilder AddSwarmSecrets(
this IConfigurationBuilder builder,
Action<SwarmSecretsConfigurationSource> configureSource)
{
return builder.Add(configureSource);
}
}
and the final step is to add the swarm secrets provider to the startup of the application
public static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddSwarmSecrets();
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
Now you can move the database configuration from appsettings.json to docker swarm by creating a secret named ConnectionStrings:DatabaseConnection and you can retrieve this using the ASP.NET Core api IConfiguration.GetConnectionString() in the same way as it was stored in appsettings.config file.
You can find a link to the full source code at the end of this article.
But, this approach has some limitation and drawbacks because you need to store the whole connection string as a secret and if you need to change only the password you are forced to update the whole connection string which add a risk of misconfiguration and break the application.
A more elegant solution will be if the connection string is stored in appsettings.json and only the password to be stored as a secret.
By using custom configuration provider and string interpolation you can achieve this.
Fist the connection string needs to be changed and to add a place holder for the password.
{
"ConnectionStrings": {
"DatabaseConnection": "Server=myServerAddress;Database=myDataBase;User Id=myUsername;Password={{pwd}};MultipleActiveResultSets=True;"
}
}
by convention the password is another configuration value within the same section with the variable name as suffix.
{
"ConnectionStrings": {
"DatabaseConnection_pwd": "thePassword"
}
}
and the key will be ‘ConnectionStrings:DatabaseConnection_pwd’ and this key can be stored as a secret in docker swarm.
To replace the placeholder variable with the actual password in runtime a custom configuration provider must be used. I used a fancy and long name for it InterpolationConfigurationProvider.
public class InterpolationConfigurationProvider : IConfigurationProvider
{
private readonly Regex _variablePattern;
private readonly IConfiguration _innerConfig;
protected IDictionary<string, string> Data { get; set; }
public InterpolationConfigurationProvider(
IConfiguration configuration,
Regex pattern)
{
_innerConfig = configuration;
_variablePattern = pattern;
}
public IEnumerable<string> GetChildKeys(
IEnumerable<string> earlierKeys,
string parentPath)
{
var prefix = parentPath == null ?
string.Empty : parentPath + ConfigurationPath.KeyDelimiter;
return Data
.Where(kv =>
kv.Key.StartsWith(prefix, StringComparison.OrdinalIgnoreCase))
.Select(kv => Segment(kv.Key, prefix.Length))
.Concat(earlierKeys)
.OrderBy(k => k, ConfigurationKeyComparer.Instance);
}
private static string Segment(string key, int prefixLength)
{
var indexOf = key.IndexOf(
ConfigurationPath.KeyDelimiter,
prefixLength,
StringComparison.OrdinalIgnoreCase);
return indexOf < 0 ?
key.Substring(prefixLength) :
key.Substring(prefixLength, indexOf - prefixLength);
}
public IChangeToken GetReloadToken()
{
return _innerConfig.GetReloadToken();
}
public void Load()
{
Data = new Dictionary<string, string>
(StringComparer.OrdinalIgnoreCase);
foreach (var config in _innerConfig.AsEnumerable())
{
var newValue = config.Value;
if (!string.IsNullOrWhiteSpace(newValue))
{
newValue = _variablePattern.Replace(newValue, (m) => {
var replacementValue = m.Value;
var variableName = m.Groups[1].Value;
if (!string.IsNullOrWhiteSpace(variableName))
{
replacementValue = _innerConfig.GetValue<string>
(config.Key + "_" + variableName);
}
return replacementValue;
});
}
Data.Add(config.Key, newValue);
}
}
public void Set(string key, string value)
{
Data[key] = value;
}
public bool TryGet(string key, out string value)
{
return Data.TryGetValue(key, out value);
}
}
The ‘configuration’ parameter in the constructor is a reference to the previous configuration keys added to the application. In this case it will contain the key for the secret password.
Add the configuration source
public class InterpolationConfigurationSource : IConfigurationSource
{
private readonly Regex _pattern = new Regex("(?:{{(.*?)}})");
private readonly IConfigurationRoot _configuration;
public InterpolationConfigurationSource(IConfigurationRoot configuration)
{
_configuration = configuration;
}
public IConfigurationProvider Build(IConfigurationBuilder builder)
{
return new InterpolationConfigurationProvider(_configuration, _pattern);
}
}
and the extension methods
public static class InterpolationConfigurationExtensions
{
public static IConfigurationBuilder AddInterpolation(
this IConfigurationBuilder builder,
IConfigurationRoot config)
{
builder.Add(new InterpolationConfigurationSource(config));
return builder;
}
public static IConfigurationBuilder AddInterpolation(
this IConfigurationBuilder builder,
IConfigurationBuilder config)
{
return AddInterpolation(builder, config.Build());
}
}
and add this new configuration provider to the application startup routine.
public static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddSwarmSecrets();
config.AddInterpolation(config);
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
Now the configuration string including the secret password is available to the application using the ASP.NET Core api IConfiguration.GetConnectionString().
Using SwarmSecretsConfigurationProvider combined with InterpolationConfigurationProvider is flexible enough and provides the following advantages:
- you can handle any application passwords
- you can decide at any time to move a configuration value from appsettings.json to secrets without any code changed, without rebuilding the application.
The full source code of this article is hosted on github.com/gabihodoroaga/blog-app-secrets.