Change register stream consumer timestamp output format to UNIX per AWS docs by elainearbaugh · Pull Request #2835 · localstack/localstack

elainearbaugh · Aug 11, 2020

Changed register_stream_consumer ConsumerCreationTimestamp output from a formatted string to a UNIX timestamp.

elainearbaugh · Aug 11, 2020

$ env AWS_ACCESS_KEY_ID=accesskey env AWS_SECRET_ACCESS_KEY=secretkey aws kinesis create-stream --endpoint-url https://127.0.0.1:4566 --no-verify-ssl --stream-name test --shard-count 1
/Users/elainearbaugh/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host '127.0.0.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
$ env AWS_ACCESS_KEY_ID=accesskey env AWS_SECRET_ACCESS_KEY=secretkey aws kinesis register-stream-consumer --endpoint-url https://127.0.0.1:4566 --no-verify-ssl --stream-arn "arn:aws:kinesis:us-east-1:000000000000:stream/test" --consumer-name consumer
/Users/elainearbaugh/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host '127.0.0.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
{
    "Consumer": {
        "ConsumerName": "consumer",
        "ConsumerARN": "arn:aws:kinesis:us-east-1:000000000000:stream/test/consumer/consumer",
        "ConsumerStatus": "ACTIVE",
        "ConsumerCreationTimestamp": 1597187722.0
    }
}

compared to running against actual aws:

$ aws kinesis register-stream-consumer --stream-arn <redacted> --consumer-name "test"
{
    "Consumer": {
        "ConsumerName": "test",
        "ConsumerARN": "<redacted>/consumer/test:1597162232",
        "ConsumerStatus": "CREATING",
        "ConsumerCreationTimestamp": 1597162232.0
    }
}

Empirically the timestamp returned from AWS seems to always be a float with .0 at the end, so I copied that

coveralls · Aug 11, 2020

Coverage decreased (-0.09%) to 73.759% when pulling 870470e on elainearbaugh:kinesis-enhanced-fanout into 7bcf985 on localstack:master.

elainearbaugh · Aug 11, 2020

added another test

elainearbaugh · Aug 11, 2020

@whummer not quite sure about the contribution process besides what's in the contribution section of the README, can I get a review please?

whummer

Thanks for this PR @elainearbaugh ! Added two minor comments - can you please take a look? Thanks

whummer · Aug 12, 2020

    return microsecond_time[:-4] + microsecond_time[-1]


+def unix_timestamp(time=None):


I think we can remove this function and use the existing now_utc() in this file. Note that the two have slightly different behavior:

# logic in the new unix_timestamp() function: > int(datetime.datetime.utcnow().timestamp()) 1597214436 # logic in the existing now_utc() function: > calendar.timegm(datetime.datetime.utcnow().timetuple()) 1597221636

At the time of running, the latter seems to be the correct UNIX epoch timestamp (according to epochconverter.com).

whummer · Aug 12, 2020

        elif action == '%s.DescribeStreamConsumer' % ACTION_PREFIX:
            consumer_arn = data.get('ConsumerARN') or data['ConsumerName']
            consumer_name = data.get('ConsumerName') or data['ConsumerARN']
+            creation_timestamp = data.get('ConsumerCreationTimestamp') or data['ConsumerCreationTimestamp']


This can be simplified to:

creation_timestamp = data.get('ConsumerCreationTimestamp')

(Note that in the two lines above, different attribute names are selected as fallbacks if one attribute is not defined.)

ohhh haha yeah I copied that because I wanted to be consistent but didn't notice there were multiple attributes, thanks

elainearbaugh · Aug 12, 2020

Made the changes and double checked output with awscli:

$ env AWS_ACCESS_KEY_ID=accesskey env AWS_SECRET_ACCESS_KEY=secretkey aws kinesis create-stream --endpoint-url https://127.0.0.1:4566 --no-verify-ssl --stream-name test --shard-count 1
/Users/elainearbaugh/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host '127.0.0.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
$ env AWS_ACCESS_KEY_ID=accesskey env AWS_SECRET_ACCESS_KEY=secretkey aws kinesis register-stream-consumer --endpoint-url https://127.0.0.1:4566 --no-verify-ssl --stream-arn "arn:aws:kinesis:us-east-1:000000000000:stream/test" --consumer-name consumer
/Users/elainearbaugh/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host '127.0.0.1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
{
    "Consumer": {
        "ConsumerName": "consumer",
        "ConsumerARN": "arn:aws:kinesis:us-east-1:000000000000:stream/test/consumer/consumer",
        "ConsumerStatus": "ACTIVE",
        "ConsumerCreationTimestamp": 1597244794.0
    }
}

…WS docs

elainearbaugh · Aug 12, 2020

@whummer fixed, thank you!

Once this is merged would it be possible to release a new bugfix version? I'm using localstack with TestContainers in Java, and the latest stable version of TestContainers basically just allows you to specify a docker image version for localstack (code). So I don't think I can get my tests to work without a new release version/docker image?

whummer · Aug 12, 2020

Great, thanks for the quick turnaround. We'll release a new version shortly.. 👍

…estamp (#2835)

elainearbaugh force-pushed the kinesis-enhanced-fanout branch 2 times, most recently from a87c073 to 6b8de25 Compare August 11, 2020 06:26

elainearbaugh mentioned this pull request Aug 11, 2020

Kinesis register_stream_consumer returns wrong format for ConsumerCreationTimestamp #2834

Closed

elainearbaugh force-pushed the kinesis-enhanced-fanout branch 2 times, most recently from 53fc9db to 3975071 Compare August 11, 2020 16:07

elainearbaugh force-pushed the kinesis-enhanced-fanout branch 3 times, most recently from d431862 to d3e0e66 Compare August 11, 2020 19:19

elainearbaugh force-pushed the kinesis-enhanced-fanout branch from d3e0e66 to 870470e Compare August 11, 2020 20:04

whummer reviewed Aug 12, 2020

View reviewed changes

elainearbaugh force-pushed the kinesis-enhanced-fanout branch from 870470e to 68b528b Compare August 12, 2020 15:08

elainearbaugh force-pushed the kinesis-enhanced-fanout branch from 68b528b to 159490c Compare August 12, 2020 15:40

Change register stream consumer timestamp output format to UNIX per A…

3a8bd13

…WS docs

elainearbaugh force-pushed the kinesis-enhanced-fanout branch from 159490c to 3a8bd13 Compare August 12, 2020 15:58

whummer merged commit 4aeae40 into localstack:master Aug 12, 2020

elainearbaugh deleted the kinesis-enhanced-fanout branch August 12, 2020 20:29

whummer added a commit that referenced this pull request Aug 15, 2020

release v0.11.4; change Kinesis consumer timestamp format to UNIX tim…

1bad49a

…estamp (#2835)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Change register stream consumer timestamp output format to UNIX per AWS docs#2835

Change register stream consumer timestamp output format to UNIX per AWS docs#2835
whummer merged 1 commit into
localstack:masterlocalstack/localstack:masterfrom
elainearbaugh:kinesis-enhanced-fanoutelainearbaugh/localstack:kinesis-enhanced-fanoutCopy head branch name to clipboard

elainearbaugh commented Aug 11, 2020 •

edited

Loading

Uh oh!

elainearbaugh commented Aug 11, 2020 •

edited

Loading

Uh oh!

coveralls commented Aug 11, 2020 •

edited

Loading

Uh oh!

elainearbaugh commented Aug 11, 2020

Uh oh!

elainearbaugh commented Aug 11, 2020

Uh oh!

whummer left a comment

Uh oh!

whummer Aug 12, 2020

Uh oh!

elainearbaugh Aug 12, 2020

Uh oh!

whummer Aug 12, 2020

Uh oh!

elainearbaugh Aug 12, 2020

Uh oh!

elainearbaugh commented Aug 12, 2020

Uh oh!

elainearbaugh commented Aug 12, 2020

Uh oh!

whummer commented Aug 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return microsecond_time[:-4] + microsecond_time[-1]


		def unix_timestamp(time=None):

Search code, repositories, users, issues, pull requests...

Uh oh!

Conversation

elainearbaugh commented Aug 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elainearbaugh commented Aug 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Aug 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elainearbaugh commented Aug 11, 2020

Uh oh!

elainearbaugh commented Aug 11, 2020

Uh oh!

whummer left a comment

Choose a reason for hiding this comment

Uh oh!

whummer Aug 12, 2020

Choose a reason for hiding this comment

Uh oh!

elainearbaugh Aug 12, 2020

Choose a reason for hiding this comment

Uh oh!

whummer Aug 12, 2020

Choose a reason for hiding this comment

Uh oh!

elainearbaugh Aug 12, 2020

Choose a reason for hiding this comment

Uh oh!

elainearbaugh commented Aug 12, 2020

Uh oh!

elainearbaugh commented Aug 12, 2020

Uh oh!

whummer commented Aug 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

elainearbaugh commented Aug 11, 2020 •

edited

Loading

elainearbaugh commented Aug 11, 2020 •

edited

Loading

coveralls commented Aug 11, 2020 •

edited

Loading