Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[DependencyInjection] added a way to reference dynamic environment variables in service definitions #16403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

fabpot
Copy link
Member

@fabpot fabpot commented Oct 30, 2015

Q A
Bug fix? yesish
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets #10138, #7555
License MIT
Doc PR not yet

When hosting a Symfony app on some PaaS like Heroku, or even if you are just using Docker, you need a way to reference environment variables in the container. That's already possible in Symfony but the values are dumped into the cache. This PR adds a way to reference an env variable that is resolved dynamically even on a compiled container.

ping @dzuelke

@fabpot fabpot force-pushed the env-parameters branch 2 times, most recently from 2fb77e3 to 26acc8f Compare October 30, 2015 23:36
*/
public function __toString()
{
return (string) $this->id;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO is better to cast to a string in the constructor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or dont cast at all. The parameter is already defined as string.

@dzuelke
Copy link
Contributor

dzuelke commented Oct 31, 2015

It looks like this still dumps literal values into the container instead of getenv() calls:

I'm using your branch:

~ $ grep getenv vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php              
        return sprintf("getenv('%s')", $name);

My app/config/config_prod.yml has this (and I moved the base DB config to config_dev.yml):

doctrine:
    dbal:
        url:   "$DATABASE_URL$"
        charset:  UTF8

But the container has a literal value:

~ $ grep postgres app/cache/prod/appProdProjectContainer.php
        return $this->services['doctrine.dbal.default_connection'] = $this->get('doctrine.dbal.connection_factory')->createConnection(array('url' => 'postgres://someuser:somepass@ec2-107-21-221-59.compute-1.amazonaws.com:5432/somedatabase', 'charset' => 'UTF8', 'host' => 'localhost', 'port' => NULL, 'user' => 'root', 'password' => NULL, 'driver' => 'pdo_mysql', 'driverOptions' => array()), new \Doctrine\DBAL\Configuration(), new \Symfony\Bridge\Doctrine\ContainerAwareEventManager($this), array());

If I remember correctly, that's the exact problem I had when I tried to implement this myself. I was looking for a solution where there's a getenv() call in the generated container, and that would have been possible for Symfony's own stuff, but not for third-party bundles with their own dumper, as they or their calling code would have to be aware of this new syntax. There seems to be no central way of injecting this.

Or am I doing something wrong? I used the symfony-demo project with a few dev requirements for all the symfony/polyfill-… repositories.

@fabpot
Copy link
Member Author

fabpot commented Oct 31, 2015

@dzuelke That's intended (or at least, that's the way I see it): a parameter IS evaluated, an env variable is NOT. So, you should use an env variable directly in your definitions instead of assigning the env variable to a parameter first, in which case, it's evaluated. That might be confusing though, so the second commit does what you expected. The only "problem" I can see is that this new behavior introduces some inconsistencies: a parameter storing an env variable is not evaluated only when used as a constructor argument or a method call argument; the same parameter used for a class name, or an alias for instance will be evaluated (as the code expects static values for those elements, and I don't want to support dynamic values here). WDYT?

@sstok
Copy link
Contributor

sstok commented Oct 31, 2015

If you only use it for service definitions you can simple use a custom Expression function.
And that would also be less of a BC break 👍

@fabpot
Copy link
Member Author

fabpot commented Oct 31, 2015

Which BC break?

@linaori
Copy link
Contributor

linaori commented Oct 31, 2015

I think where a parameter is actually named $FOO$, not entirely sure though, never had the use-case myself.

@fabpot
Copy link
Member Author

fabpot commented Oct 31, 2015

Using an expression is indeed a very good idea and should work well. I'll have later.

@dzuelke
Copy link
Contributor

dzuelke commented Oct 31, 2015

So, you should use an env variable directly in your definitions instead of assigning the env variable to a parameter first

How would I do that, @fabpot?

If you only use it for service definitions you can simple use a custom Expression function.

That's what @ruudk suggested in #7555 (comment) and what @stof implemented in https://github.com/Incenteev/DynamicParametersBundle but the problem AFAICT is that every bundle out there would have to be modified to use that.

@wouterj
Copy link
Member

wouterj commented Nov 1, 2015

Can't we change the syntax to $SOMETHING? Env variable names cannot include whitespaces, so it should not cause any problems (and this syntax is equal to how people are used to reference env variables in shell/bash scripts).

@@ -167,12 +168,12 @@ public function resolve()
* @throws ParameterCircularReferenceException if a circular reference if detected
* @throws RuntimeException when a given parameter has a type problem.
*/
public function resolveValue($value, array $resolving = array())
public function resolveValue($value, array $resolving = array(), $protectEnvVariables = false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new protectEnvVariables param is missing from the docblock.

@fabpot fabpot force-pushed the env-parameters branch 2 times, most recently from 9ee8aab to fd16950 Compare November 1, 2015 22:04
@fabpot
Copy link
Member Author

fabpot commented Nov 4, 2015

@dzuelke Can you test the latest version of this patch? It should work fine now.

@dzuelke
Copy link
Contributor

dzuelke commented Nov 4, 2015

Sure @fabpot, will do. I thought you were going to rewrite it to use expressions first, my bad :)

@@ -236,7 +245,7 @@ public function resolveString($value, array $resolving = array())
$resolved = (string) $resolved;
$resolving[$key] = true;

return $self->isResolved() ? $resolved : $self->resolveString($resolved, $resolving);
return $self->isResolved() ? $resolved : $self->resolveString($resolved, $resolving, $protectEnvVariables);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable is missing from the use() statement!

@dzuelke
Copy link
Contributor

dzuelke commented Nov 4, 2015

The preg_replace_callback references $protectEnvVariables but does not use() it, @fabpot:

> post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache



  [Symfony\Component\Debug\Exception\ContextErrorException]  
  Notice: Undefined variable: protectEnvVariables            



Script Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache handling the post-update-cmd event terminated with an exception



  [RuntimeException]                                                         
  An error occurred when executing the "'cache:clear --no-warmup'" command.  



Exception trace:
 () at /Users/dzuelke/Code/heroku/php-test/symfony_env/symfony-demo/vendor/sensio/distribution-bundle/Sensio/Bundle/DistributionBundle/Composer/ScriptHandler.php:439
 Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::executeCommand() at /Users/dzuelke/Code/heroku/php-test/symfony_env/symfony-demo/vendor/sensio/distribution-bundle/Sensio/Bundle/DistributionBundle/Composer/ScriptHandler.php:138
 Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:211
 Composer\EventDispatcher\EventDispatcher->executeEventPhpScript() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:167
 Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:92
 Composer\EventDispatcher\EventDispatcher->dispatchScript() at phar:///usr/local/bin/composer/src/Composer/Installer.php:349
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/UpdateCommand.php:143
 Composer\Command\UpdateCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:838
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:189
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:147
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:120
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:82
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:43
 require() at /usr/local/bin/composer:25

@dzuelke
Copy link
Contributor

dzuelke commented Nov 4, 2015

But even then, still no luck, @fabpot:

~ $ grep -Hn postgres app/cache/prod/appProdProjectContainer.php
app/cache/prod/appProdProjectContainer.php:340:        return $this->services['doctrine.dbal.default_connection'] = $this->get('doctrine.dbal.connection_factory')->createConnection(array('url' => 'postgres://someuser:somepass@ec2-107-21-221-59.compute-1.amazonaws.com:5432/somedatabase', 'charset' => 'UTF8', 'host' => 'localhost', 'port' => NULL, 'user' => 'root', 'password' => NULL, 'driver' => 'pdo_mysql', 'driverOptions' => array()), new \Doctrine\DBAL\Configuration(), new \Symfony\Bridge\Doctrine\ContainerAwareEventManager($this), array());

I'm using a branch based on yours, with my fix (dzuelke/symfony@1cf6e98) applied:

~ $ grep protectEnvVariables vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/ParameterBag/ParameterBag.php | wc -l
8

@dzuelke
Copy link
Contributor

dzuelke commented Nov 4, 2015

Also paging @stof for thoughts. Or am I doing something really wrong here?

@dzuelke
Copy link
Contributor

dzuelke commented Nov 4, 2015

I also played around with various permutations around parameters, and parameters.yml, where I have:

# This file is auto-generated during the composer install
parameters:
    database_driver: pdo_sqlite
    database_host: 127.0.0.1
    database_port: null
    database_name: null
    database_user: root
    database_password: null
    database_path: '%kernel.root_dir%/data/blog.sqlite'
    mailer_transport: smtp
    mailer_host: 127.0.0.1
    mailer_user: null
    mailer_password: null
    locale: en
    secret: secret_value_for_symfony_demo_application
    database_url: $DATABASE_URL$

The value there also gets written out to the cache literally and not as a getenv() call.

@dzuelke
Copy link
Contributor

dzuelke commented Nov 5, 2015

Ping @fabpot / @stof

@@ -1377,6 +1380,8 @@ private function dumpValue($value, $interpolate = true)
// we do this to deal with non string values (Boolean, integer, ...)
// the preg_replace_callback converts them to strings
return $this->dumpParameter(strtolower($match[1]));
} elseif (preg_match('/^\$([^\$]+)\$$/', $value, $match)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is will also match $foo'boom$ making it possible to unwillingly inject PHP code (as the value is not escaped or anything when it's used in dumpEnvVariable ).

And the $$ will not match, but is not normalized to a single $ in the else-block.

@lolautruche
Copy link
Contributor

Hello

Problem with this format is that it clashes with eZ dynamic settings which uses the same.
See https://doc.ez.no/display/EZP/Dynamic+settings+injection

@fabpot
Copy link
Member Author

fabpot commented Nov 12, 2015

Conclusion: This is not possible, and by far. The semantic configuration used by bundle extensions is resolved before being passed to the extension. If we do not resolve it anymore, each extension would have to resolve values on a case-by-case basis AND would need to keep the non-resolved value. That might be a problem when a parameter has influences over the configuration.

So, a generic system is just not possible without rethinking the whole system.

The only thing I can think of is to allow the user to override some service arguments after the container has been compiled, but that would mean relying on conventions sometimes (when the service using the configuration has been dynamically created for instance).

Another possibility is to only add support for some bundles like the DSN for the Doctrine bundle. That might be easier and would probably fix most of the issues.

Closing for now.

@fabpot fabpot closed this Nov 12, 2015
@lolautruche
Copy link
Contributor

More or less in the same field, it's been a while that we, at eZ, think about contributing our siteaccess system to enable one to have context aware settings (e.g.Multisite inside one app).
Would this make sense in Symfony? Or should we better just externalize it and propose it as a lib/bundleBundle?

fabpot added a commit that referenced this pull request Sep 15, 2016
…env(MY_ENV_VAR)% (nicolas-grekas)

This PR was merged into the 3.2-dev branch.

Discussion
----------

[DI] Allow injecting ENV parameters at runtime using %env(MY_ENV_VAR)%

| Q             | A
| ------------- | ---
| Branch?       | master
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets |  #10138, #7555, #16403, #18155
| License       | MIT
| Doc PR        | symfony/symfony-docs#6918

This is an alternative approach to #18155 for injecting env vars into container configurations.

With this PR, anywhere parameters are allowed, one can use `%env(ENV_VAR)%` to inject a dynamic env var. Additionally, if one sets a value to such parameters in e.g. the `parameter.yml` file (`env(ENV_VAR): foo`), this value will be used as a default value when the env var is not defined. If no default value is specified, an `EnvVarNotFoundException` will be thrown at runtime.

Unlike previous attempts that also used parameters (#16403), the implementation is compatible with DI extensions: before being dumped, env vars are resolved to uniquely identifiable string placeholders that can get through DI extensions manipulations. When dumped, these unique placeholders are replaced by dynamic calls to a getEnv method..

ping @magnusnordlander @dzuelke @fabpot

Commits
-------

bac2132 [DI] Allow injecting ENV parameters at runtime using %env(MY_ENV_VAR)% syntax
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.