Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[Lock] Added Symfony/Component/Lock/Store/MongoDbStore #27346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
157fcd7
#27345 Added Symfony/Component/Lock/Store/MongoDbStore
May 23, 2018
e73f663
#27345 Fixed typo
May 23, 2018
3838412
#27345 fixed coding standards
May 23, 2018
1a660e6
#27345 Removed dev requirement for mongodb/mongodb
May 23, 2018
d65fddf
#27345 Removed unnessasary reduceLifetime
May 24, 2018
f3cc220
#27345 Improved documentation, exception handling, removed configurab…
May 31, 2018
ec8d217
#27345 fixed typo and @param indentation
May 31, 2018
a1a3be7
#27345 fixed @param indentation
May 31, 2018
1b150ba
#27345 moved clock discrepancy handling to a constructor option 'drift'
May 31, 2018
73bb802
#27345 fixed phpdoc spacing
May 31, 2018
5e86904
#27345 removed drift option in favour or recommending setting TTL higher
May 31, 2018
8572c21
#27345 Added public createTTLIndex method
Jun 5, 2018
dbdad36
#27345 Fixed coding standards
Jun 5, 2018
84c4d15
#27345 Fixed coding standards
Jun 5, 2018
d577191
#27345 Fixed coding standards
Jun 5, 2018
2ce68f5
#27345 Fixed coding standards
Jun 5, 2018
5446b11
#27345 Improved documentation
Jun 5, 2018
a9b85d6
#27345 Fixed spacing for fabbot
Jun 5, 2018
6fdc602
#27345 reordered documentation in order of importance
Jun 5, 2018
9d80a0d
#27345 Fixed spacing for fabbot
Jun 5, 2018
1ac702a
#27345 Fixed spacing for fabbot
Jun 5, 2018
4ba5b41
#27345 Updated return type and exception handling on createTTLIndex
Jun 6, 2018
58ef873
#27345 Updated exception handling on createTTLIndex
Jun 6, 2018
36c3af4
#27345 Fixed spacing for fabbot
Jun 6, 2018
168f194
#27345 Removed initialTtl from array and fixed unit test
Jun 6, 2018
abc72f8
#27345 phpdoc adjusted for fabbot
Jun 6, 2018
e062bf2
#27345 phpdoc adjusted for fabbot
Jun 6, 2018
f2e23b4
#27345 trigger ci retry
Jun 14, 2018
97e5711
#27345 trigger ci retry
Jun 14, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions 1 phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
<env name="LDAP_PORT" value="3389" />
<env name="REDIS_HOST" value="localhost" />
<env name="MEMCACHED_HOST" value="localhost" />
<env name="MONGODB_HOST" value="localhost" />
</php>

<testsuites>
Expand Down
251 changes: 251 additions & 0 deletions 251 src/Symfony/Component/Lock/Store/MongoDbStore.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
<?php

/*
* This file is part of the Symfony package.
*
* (c) Fabien Potencier <fabien@symfony.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace Symfony\Component\Lock\Store;

use Symfony\Component\Lock\Exception\InvalidArgumentException;
use Symfony\Component\Lock\Exception\LockConflictedException;
use Symfony\Component\Lock\Exception\LockExpiredException;
use Symfony\Component\Lock\Exception\LockStorageException;
use Symfony\Component\Lock\Exception\NotSupportedException;
use Symfony\Component\Lock\Key;
use Symfony\Component\Lock\StoreInterface;

/**
* MongoDbStore is a StoreInterface implementation using MongoDB as store engine.
*
* @author Joe Bennett <joe@assimtech.com>
*/
class MongoDbStore implements StoreInterface
{
private $mongo;
private $options;
private $collection;

/**
* @param \MongoDB\Client $mongo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to pass a dns for lazy connection? See PdoTrait from cache components?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A MongoDB\Client instance is already lazy. I can turn off my database and still:

$client = new \MongoDB\Client();
$store = new \Symfony\Component\Lock\Store\MongoDbStore($client, [
    'database' => 'test',
]);

only if I add something like:

$store->createTTLIndex();

does it throw an exception.

* @param array $options
*
* database: The name of the database [required]
* collection: The name of the collection [default: lock]
* initialTtl: The expiration delay of locks in seconds [default: 300.0]
*
* CAUTION: This store relies on all client and server nodes to have
* synchronized clocks for lock expiry to occur at the correct time.
* To ensure locks don't expire prematurely; the ttl's should be set with enough
* extra time to account for any clock drift between nodes.
*
* A TTL index MUST BE used on MongoDB 2.2+ to automatically clean up expired locks.
*
* db.lock.ensureIndex(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a method to createTTLIndex(). There's no need in MongoDB to create the collection, they're created on the fly by the database.

* { "expires_at": 1 },
* { "expireAfterSeconds": 0 }
* )
*
* @see http://docs.mongodb.org/manual/tutorial/expire-data/
*
* writeConcern, readConcern and readPreference are not specified by MongoDbStore
* meaning the collection's settings will take effect.
* @see https://docs.mongodb.com/manual/applications/replication/
*
* Please note, the Symfony\Component\Lock\Key's $resource
* must not exceed 1024 bytes including structural overhead.
* @see https://docs.mongodb.com/manual/reference/limits/#Index-Key-Limit
*/
public function __construct(\MongoDB\Client $mongo, array $options)
{
if (!isset($options['database'])) {
throw new InvalidArgumentException(
'You must provide the "database" option for MongoDBStore'
);
}

$this->mongo = $mongo;

$this->options = array_merge(array(
'collection' => 'lock',
'initialTtl' => 300.0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really wonder if options for lock should be merge with options for database connection?

See PdoTrait from cache components.

What is the opinion of other contributors?

Copy link
Author

@kralos kralos Jun 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had initialTtl as a separate constructor parameter, I brought it into $options when you suggested drift should be.

I personally think it belongs outside of $options.

See also: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Adapter/PdoAdapter.php

), $options);
}

/**
* {@inheritdoc}
*/
public function save(Key $key)
{
$token = $this->getToken($key);
$now = microtime(true);

$filter = array(
'_id' => (string) $key,
'$or' => array(
array(
'token' => $token,
),
array(
'expires_at' => array(
'$lte' => $this->createDateTime($now),
),
),
),
);

$update = array(
'$set' => array(
'_id' => (string) $key,
'token' => $token,
'expires_at' => $this->createDateTime($now + $this->options['initialTtl']),
),
);

$options = array(
'upsert' => true,
);

$key->reduceLifetime($this->options['initialTtl']);
try {
$this->getCollection()->updateOne($filter, $update, $options);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it atomic? on every version of mongodb? What's about réplication propagation?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MongoDB is atomic on a singular statement / document only. Hence the use of a update with filter (1 statement). MongoDB ^4.0 introduces multi statement transactions however this implementation does not rely on that. It should be atomic on all versions. Replication propagation can be controlled with a statement level options called a write concern. I'm adding support for that now. By default, mongo will read and write only to the PRIMARY node in a replica set (this is equivalent of reading and writing only to the master of a replicated RDBMS with replication slaves).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regards to writeConcern, readConcern and readPreference I've opted to leave this to the collection fallback placing the onus on the DBA to setup the collection with alternate settings if desired. The default when left unspecified is to read / write ONLY to the PRIMARY of a replica set. This is safe for the use of locks (it's like only reading / writing to a master in an RDBMS) however an alternate strategy could be left to the DBA depending on the number of nodes they have.

I have added some documentation regarding this decision to symfony-docs

} catch (\MongoDB\Driver\Exception\WriteException $e) {
throw new LockConflictedException('Failed to acquire lock', 0, $e);
} catch (\Exception $e) {
throw new LockStorageException($e->getMessage(), 0, $e);
}

if ($key->isExpired()) {
throw new LockExpiredException(sprintf('Failed to store the "%s" lock.', $key));
}
}

public function waitAndSave(Key $key)
{
throw new NotSupportedException(sprintf(
'The store "%s" does not supports blocking locks.',
__CLASS__
));
}

/**
* {@inheritdoc}
*/
public function putOffExpiration(Key $key, $ttl)
{
$now = microtime(true);

$filter = array(
'_id' => (string) $key,
'token' => $this->getToken($key),
'expires_at' => array(
'$gte' => $this->createDateTime($now),
),
);

$update = array(
'$set' => array(
'_id' => (string) $key,
'expires_at' => $this->createDateTime($now + $ttl),
),
);

$options = array(
'upsert' => true,
);

$key->reduceLifetime($ttl);
try {
$this->getCollection()->updateOne($filter, $update, $options);
} catch (\MongoDB\Driver\Exception\WriteException $e) {
throw new LockConflictedException('Failed to put off the expiration of the lock', 0, $e);
} catch (\Exception $e) {
throw new LockStorageException($e->getMessage(), 0, $e);
}

if ($key->isExpired()) {
throw new LockExpiredException(sprintf(
'Failed to put off the expiration of the "%s" lock within the specified time.',
$key
));
}
}

/**
* {@inheritdoc}
*/
public function delete(Key $key)
{
$filter = array(
'_id' => (string) $key,
'token' => $this->getToken($key),
);

$options = array();

$this->getCollection()->deleteOne($filter, $options);
}

/**
* {@inheritdoc}
*/
public function exists(Key $key)
{
$filter = array(
'_id' => (string) $key,
'token' => $this->getToken($key),
'expires_at' => array(
'$gte' => $this->createDateTime(),
),
);

$doc = $this->getCollection()->findOne($filter);

return null !== $doc;
}

private function getCollection(): \MongoDB\Collection
{
if (null === $this->collection) {
$this->collection = $this->mongo->selectCollection(
$this->options['database'],
$this->options['collection']
);
}

return $this->collection;
}

/**
* @param float $seconds Seconds since 1970-01-01T00:00:00.000Z supporting millisecond precision. Defaults to now.
*
* @return \MongoDB\BSON\UTCDateTime
*/
private function createDateTime(float $seconds = null): \MongoDB\BSON\UTCDateTime
{
if (null === $seconds) {
$seconds = microtime(true);
}

$milliseconds = $seconds * 1000;

return new \MongoDB\BSON\UTCDateTime($milliseconds);
}

/**
* Retrieves an unique token for the given key.
*/
private function getToken(Key $key): string
{
if (!$key->hasState(__CLASS__)) {
$token = base64_encode(random_bytes(32));
$key->setState(__CLASS__, $token);
}

return $key->getState(__CLASS__);
}
}
4 changes: 4 additions & 0 deletions 4 src/Symfony/Component/Lock/StoreInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
namespace Symfony\Component\Lock;

use Symfony\Component\Lock\Exception\LockConflictedException;
use Symfony\Component\Lock\Exception\LockExpiredException;
use Symfony\Component\Lock\Exception\NotSupportedException;

/**
Expand All @@ -25,6 +26,7 @@ interface StoreInterface
* Stores the resource if it's not locked by someone else.
*
* @throws LockConflictedException
* @throws LockExpiredException
*/
public function save(Key $key);

Expand All @@ -34,6 +36,7 @@ public function save(Key $key);
* If the store does not support this feature it should throw a NotSupportedException.
*
* @throws LockConflictedException
* @throws LockExpiredException
* @throws NotSupportedException
*/
public function waitAndSave(Key $key);
Expand All @@ -46,6 +49,7 @@ public function waitAndSave(Key $key);
* @param float $ttl amount of second to keep the lock in the store
*
* @throws LockConflictedException
* @throws LockExpiredException
* @throws NotSupportedException
*/
public function putOffExpiration(Key $key, $ttl);
Expand Down
58 changes: 58 additions & 0 deletions 58 src/Symfony/Component/Lock/Tests/Store/MongoDbClientTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
<?php

/*
* This file is part of the Symfony package.
*
* (c) Fabien Potencier <fabien@symfony.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace Symfony\Component\Lock\Tests\Store;

use Symfony\Component\Lock\Store\MongoDbStore;

/**
* @author Joe Bennett <joe@assimtech.com>
*/
class MongoDbClientTest extends AbstractStoreTest
{
use ExpiringStoreTestTrait;

public static function setupBeforeClass()
{
try {
if (!class_exists(\MongoDB\Client::class)) {
throw new \RuntimeException('The mongodb/mongodb package is required.');
}
$client = self::getMongoConnection();
$client->listDatabases();
} catch (\Exception $e) {
self::markTestSkipped($e->getMessage());
}
}

protected static function getMongoConnection(): \MongoDB\Client
{
return new \MongoDB\Client('mongodb://'.getenv('MONGODB_HOST'));
}

/**
* {@inheritdoc}
*/
protected function getClockDelay()
{
return 250000;
}

/**
* {@inheritdoc}
*/
public function getStore()
{
return new MongoDbStore(self::getMongoConnection(), array(
'database' => 'test',
));
}
}
3 changes: 2 additions & 1 deletion 3 src/Symfony/Component/Lock/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
"psr/log": "~1.0"
},
"require-dev": {
"predis/predis": "~1.0"
"predis/predis": "~1.0",
"mongodb/mongodb": "~1.0"
},
"autoload": {
"psr-4": { "Symfony\\Component\\Lock\\": "" },
Expand Down
1 change: 1 addition & 0 deletions 1 src/Symfony/Component/Lock/phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
<ini name="error_reporting" value="-1" />
<env name="REDIS_HOST" value="localhost" />
<env name="MEMCACHED_HOST" value="localhost" />
<env name="MONGODB_HOST" value="localhost" />
</php>

<testsuites>
Expand Down
Morty Proxy This is a proxified and sanitized view of the page, visit original site.