-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Lock] add mongodb store #31889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Lock] add mongodb store #31889
Conversation
Please target |
55526dc
to
0e77ec6
Compare
rebased from |
I've stripped it from
Added in symfony/symfony-docs#11735 |
This PR was merged into the 4.3 branch. Discussion ---------- [Lock] mongodb store removed from symfony 4.3 MongoDbStore wasn't actually added in symfony 4.3, it will be added in 4.4. See symfony/symfony#31889 Commits ------- 75ad033 #27345 Removed Lock MongoDbStore, it will be added in symfony 4.4
|
||
/** | ||
* MongoDbStore is a StoreInterface implementation using MongoDB as a storage | ||
* engine. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about adding that it requires an external lib?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pattern public static function isSupported(): bool
has been added to MongoDbStore
as well as @requires extension mongodb
on the class. I've also moved the CAUTION:
phpdoc up to the class to be inline with other stores.
753d398
to
f3f8568
Compare
I'm looking into the failing test later today |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice feature !
Status: Needs Work
* | ||
* @internal | ||
*/ | ||
public static function isSupported(): bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imho this is not needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did this because some of Jérémy's stores do it. I thought it might be useful later if a bundle extension needs to check support to auto default/configure something supported, but maybe we could add an interface for this later to all stores if we ever did something like that. Also thinking about it more... you could never auto configure a MongoDbStore since the database name option is mandatory. So I have removed static::isSupported()
.
*/ | ||
public function __construct(Client $mongo, array $options, float $initialTtl = 300.0) | ||
{ | ||
if (!static::isSupported()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can directly check if it is supported here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
* | ||
* @see https://docs.mongodb.com/manual/applications/replication/ | ||
*/ | ||
public function __construct(Client $mongo, array $options, float $initialTtl = 300.0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should not type-hint the mongodb client, what happens if he does not exist ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then the developer wouldn't be able to construct a MongoDB\Client
. If they pass something that isn't a MongoDB\Client
they would get a TypeError:
Uncaught TypeError: Argument 1 passed to Symfony\Component\Lock\Store\MongoDbStore::__construct() must be an instance of MongoDB\Client, x given
This is what I would want to happen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if you don't have MongoDB ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if you don't have mongodb/mongodb
you can't construct a MongoDB\Client
if you don't have ext-mongodb
you can't composer require mongodb/mongodb
if you somehow have installed mongodb/mongodb
and don't have ext-mongodb
you will fail to construct a MongoDB\Client
(this is mongodb/mongodb
's responsibility):
<?php
require __DIR__.'/vendor/autoload.php';
$mongoClient = new MongoDB\Client('mongodb://127.0.0.1');
$store = new Symfony\Component\Lock\Store\MongoDbStore($mongoClient, [
'database' => 'test',
]);
$ docker run --rm -it -v $PWD:$PWD -w $PWD php php ./test.php
Fatal error: Uncaught Error: Class 'MongoDB\Driver\Manager' not found in /home/kralos/src/symfony/vendor/mongodb/mongodb/src/Client.php:87
Stack trace:
#0 /home/kralos/src/symfony/test.php(5): MongoDB\Client->__construct('mongodb://127.0...')
#1 {main}
thrown in /home/kralos/src/symfony/vendor/mongodb/mongodb/src/Client.php on line 87
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MongoDB\Driver\Manager
is provided by ext-mongodb
MongoDB\Client
is provided by mongodb/mongodb
Symfony\Component\Lock\Store\MongoDbStore
depends on mongodb/mongodb
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you not type-hint MongoDB\Client for that reason. maybe i'm wrong cc @jderusse
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the if (!\extension_loaded('mongodb')) {
check in the constructor since it's impossible to hit it. You can't pass a MongoDB\Client
without mongodb/mongodb
which you can't have without ext-mongodb
.
We still have in the class docblock @requires extension mongodb
|
||
if (!isset($this->options['database'])) { | ||
throw new InvalidArgumentException( | ||
'You must provide the "database" option for MongoDBStore' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should inline this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
if ($this->options['gcProbablity'] < 0.0 || $this->options['gcProbablity'] > 1.0) { | ||
throw new InvalidArgumentException(sprintf( | ||
'"%s" gcProbablity must be a float from 0.0 to 1.0, "%f" given.', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should inline this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
6df4973
to
a1fe6d0
Compare
|
This one looks ready for Symfony 4.4/5.0 👍 |
|| (random_int(0, PHP_INT_MAX) / PHP_INT_MAX) <= $this->options['gcProbablity'] | ||
) | ||
) { | ||
if ($this->getDatabaseVersion() < '2.2') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works by accident, version_compare
should be used for these
>>> '2.10' < '2.2'
=> true
instances: 2
But why are we adding support for legacy version anyway? 2.2 was EOL-ed in 2014. We are paying the price not just with added complexity but with extra query on each request as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'd prefer not to support <2.2
. removing support and documenting requirements
{ | ||
$this->getCollection()->deleteMany([ // filter | ||
'expires_at' => [ | ||
'$lte' => $this->createDateTime(microtime(true)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is inconsistent. IMHO parameter should either be non-optional, or not specified here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed, I've opted for not optional. the objective of the function is to create a MongoDb compliant UTCDateTime
to millisecond precision.
if (1 === \count($writeErrors)) { | ||
$code = $writeErrors[0]->getCode(); | ||
} else { | ||
$code = $e->getCode(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor thing, but else branch could be easily avoided here by specifying this as default. This is super cheap call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated
@nicolas-grekas can you let me know if you're happy for this to be merged? I'll rebase/squash etc if everyone is happy. The appveyor CI failure is not Lock component by the way...
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you update the StoreFactory
to add the MongoDB\Client
and mongodb://
in the list of managed DSN.
Need to update the CHANGELOG.md
d2ac16e
to
30d9d2c
Compare
In order to add I have therefore added support for passing a I have also added DSN support to Neither
CI is currently passing (the errors from travis are related to components other then I've left the commit history for review purposes, if we are happy with the current iteration let me know and I can squash (if required). |
@@ -62,7 +65,6 @@ public static function createStore($connection) | ||
|
||
case 0 === strpos($connection, 'flock://'): | ||
return new FlockStore(substr($connection, 8)); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, something funny happened with my rebase
} | ||
|
||
if ($this->options['gcProbablity'] < 0.0 || $this->options['gcProbablity'] > 1.0) { | ||
throw new InvalidArgumentException(sprintf('"%s" gcProbablity must be a float from 0.0 to 1.0, "%f" given.', __METHOD__, $this->options['gcProbablity'])); | ||
throw new InvalidArgumentException(sprintf('%s() gcProbablity must be a float from 0.0 to 1.0, "%f" given.', __METHOD__, $this->options['gcProbablity'])); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove "
in "%f" given
too (for consistency)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated
} | ||
|
||
if (null === $this->collection && !isset($this->options['database'])) { | ||
throw new InvalidArgumentException(sprintf('%s() requires the "database" option when constructing with a %s', __METHOD__, \is_object($mongo) ? \get_class($mongo) : \gettype($mongo))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
database and collection names could be defined in the DSN (similar to PDO). dsn = mongodb://[username:password@]host1[:port1][,host2[:port2:],...]/[database/][collection]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The database you see there is the authentication database, not the database to use for data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't it the purpose of the querystring parameter authSource
? https://symfony.com/doc/master/bundles/DoctrineMongoDBBundle/installation.html#authentication
anyway, if the path
component is a standard to declare the auth Database, we can use custom queryString parameters like mongodb://localhost/?lockDatabase=foo&lockCollection=bar
If we can not create the Store from a DSN, developpers using the FrameworkBundle won't be able to use it from the configuration framework.lock = '%env(LOCK_DNS)%
(see symfony/symfony-docs#12523)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't it the purpose of the querystring parameter
authSource
?
Actually I just tested this (mongodb 4.2.0) and if both /path
and ?authSource
are specified /path
means application database and ?authSource
means authentication database. We are still stuck for getting the collection name.
we can use custom queryString parameters
Seems to be the only way, do you have any objection to us taking ?collection=bar
? I feel like if we ever had a conflict it would be because MongoDB is using this param for the exact same purpose.
I'm going to enforce when $mongo
is a:
MongoDB\Collection
:$options['database']
is ignored$options['collection']
is ignored
MongoDB\Client
:$options['database']
is mandatory$options['collection']
is mandatory
- MongoDB Connection URI string:
/path
is used otherwise$options['database']
, at least one is mandatory?collection=
is used otherwise$options['collection']
, at least one is mandatory
$database = parse_url($mongo, PHP_URL_PATH); | ||
if (null !== $database) { | ||
$this->options['database'] = $database; | ||
} | ||
$query = parse_url($mongo, PHP_URL_QUERY); | ||
$queryParams = []; | ||
parse_str($query ?? '', $queryParams); | ||
$collection = $queryParams['collection'] ?? null; | ||
if (null !== $collection) { | ||
$this->options['collection'] = $collection; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's about something similare to Messenger PDO
if (false === $parsedUrl = parse_url($dsn)) {
throw new InvalidArgumentException(sprintf('The given Mongodb DSN "%s" is invalid.', $dsn));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$this->option = $option + $query;
$this->options['database'] = $this->options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You prefer $options
to take precedence over /path?collection=
?
<?php
declare(strict_types=1);
$mongo = 'mongodb://127.0.0.1/uri-db?collection=uri-col';
$options = [
'database' => 'option-db',
'collection' => 'option-col',
];
if (false === $parsedUrl = parse_url($mongo)) {
throw new \InvalidArgumentException(sprintf('The given MongoDB Connection URI "%s" is invalid.', $mongo));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$options += $query;
$options['database'] = $options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;
print_r($options);
/* Array
(
[database] => option-db
[collection] => option-col
) */
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've implemented this but without polluting $this->options
with querystring params other than collection
.
if (false === $parsedUrl = parse_url($mongo)) {
throw new InvalidArgumentException(sprintf('The given MongoDB Connection URI "%s" is invalid.', $mongo));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$this->options['collection'] = $this->options['collection'] ?? $query['collection'] ?? null;
$this->options['database'] = $this->options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;
@jderusse happy with current implementation? want me to squash? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks @kralos
@nicolas-grekas can we get this merged? do you need a squash first? I don't want to miss |
af7825d
to
a6bfa59
Compare
Thank you @kralos. |
ext-mongodb
andmongodb/mongodb
to test)4.45.1 Doc PRLooks like I messed up
kralos:27345-lock-mongodb
with a force push (trying to fix ci issues) right before it was merged tomaster
(4.3.0
).see #27648
Description
We should support Semaphore Locks with a MongoDB back end to allow those that already use MongoDB as a distributed storage engine.
Symfony already partially supports MongoDB for session storage:
Symfony\Component\HttpFoundation\Session\Storage\Handler\MongoDbSessionHandler
Example