Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
This repository was archived by the owner on Jan 4, 2025. It is now read-only.

Fix double encoding of string parameters#263

Merged
GriceTurrble merged 1 commit intopython-amazon-mws:masterpython-amazon-mws/python-amazon-mws:masterfrom
adamantike:fix-double-encodingCopy head branch name to clipboard
Jun 4, 2021
Merged

Fix double encoding of string parameters#263
GriceTurrble merged 1 commit intopython-amazon-mws:masterpython-amazon-mws/python-amazon-mws:masterfrom
adamantike:fix-double-encodingCopy head branch name to clipboard

Conversation

@adamantike
Copy link

@adamantike adamantike commented Apr 23, 2021

Currently, the 0.8.x branch breaks string parameters by encoding them twice, first in the clean_params_dict function, and then in calc_request_description. Because of this, parameters prone to have characters that must be escaped, like NextToken and URLs, do not work correctly.

#258 was an attempt to fix this issue, but only for timestamp fields. Strings are still broken.

This solution is a backport of a commit (#65) that is already present in the develop branch, where the calc_request_description function is no longer responsible for escaping the received parameters.

It also rollbacks #258, as clean_date must encode the received values now.

Currently, the `0.8.x` branch breaks string parameters by encoding them
twice, first in the `clean_params_dict` function, and then in
`calc_request_description`. Because of this, parameters prone to have
characters that must be escaped, like `NextToken` and URLs, do not work
correctly.

#258 was an
attempt to fix this issue, but only for timestamp fields. Strings are
still broken.

This solution is a backport of a commit
(#65) that is already present in the
`develop` branch, where the `calc_request_description` function is no
longer responsible for escaping the received parameters.

It also rollbacks
#258,
as `clean_date` must encode the received values now.
@adamantike
Copy link
Author

Gentle heads up to @GriceTurrble, who reviewed the mentioned PRs.

@Bobspadger
Copy link
Member

@adamantike do you have an example of a call that failed before this fix - so I can do a live test to check ?

@adamantike
Copy link
Author

adamantike commented Apr 26, 2021

@Bobspadger, sure! A simple example is trying to get the second page of a ListOrders call (using ListOrdersByNextToken). The NextToken tends to have characters that are escaped by quote(), so you can test it like this:

import datetime
import mws

next_token = None
last_updated_after = datetime.datetime(2020, 1, 1)

orders_api = mws.Orders(
    access_key="...",
    secret_key="...",
    account_id="...",
    auth_token="...",
    region="US",
)

for _ in range(2):
    response = orders_api.list_orders(
        marketplaceids=("ATVPDKIKX0DER",),
        lastupdatedafter=last_updated_after,
        max_results=1,
        next_token=next_token,
    )
    next_token = response.parsed["NextToken"]
    assert next_token

For the second call, with version 0.8.13, I'm getting the following error response:

<ErrorResponse xmlns="https://mws.amazonservices.com/Orders/2013-09-01">
  <Error>
    <Type>Sender</Type>
    <Code>NextTokenCorrupted</Code>
    <Message>We could not decode your NextToken. Possible reasons include: a transmission error, improper quoting or a truncation problem.</Message>
  </Error>
  <RequestId>...</RequestId>
</ErrorResponse>

@GriceTurrble
Copy link
Member

My apologies for the delay on my end. Home life and such. Not ignoring this, I promise!

@adamantike
Copy link
Author

@GriceTurrble, no worries! Just to confirm, we have forked 0.8.13 and applied this patch without any issues in our systems (besides changing date strings to actual datetime objects to avoid #259).

@adamantike
Copy link
Author

@Bobspadger @GriceTurrble, gentle ping on this one!

@ihucos
Copy link

ihucos commented May 18, 2021

I also just hit this corner of the internet while working for my employer. @adamantike is the mentioned fork actually publicly available?

@adamantike
Copy link
Author

@ihucos, I haven't made it publicly available, as I would prefer for this fix to quickly be merged, and avoid more people from start depending on a fork. It's literally just this PR applied on top of 0.8.13, in case you want to temporarily fork it.

@ihucos
Copy link

ihucos commented May 20, 2021

@adamantike thanks for the message. We decided to wait for a fix upstream. I could to something to work towards it if it makes sense.

@Bobspadger
Copy link
Member

Hi Sorry, I've not forgotten about this but work has got quite hectic so not had the time to really look into this.

I am hoping to in the next week or so and try to look at what is truly happening, as this is only a new issue since there were some items back-ported from the develop branch.

I'd like to not be in a situation where we keep releasing point updates for essentially the same bug,

@GriceTurrble GriceTurrble merged commit 688e354 into python-amazon-mws:master Jun 4, 2021
@adamantike adamantike deleted the fix-double-encoding branch June 4, 2021 23:05
@adamantike
Copy link
Author

@GriceTurrble, if there are no planned changes in the short term, do you mind tagging and releasing a new patch version, to have this fix available for current 0.8.x users? :)

@adamantike
Copy link
Author

Thanks for the new release, @GriceTurrble!

cc @ihucos, this change is now available in version 0.8.14.

@adamantike
Copy link
Author

0.8.14 is still not published to PyPI, though.

@GriceTurrble
Copy link
Member

First time for me to actually perform the PyPI release, so that was a learning process.

However, 0.8.14 is now available on PyPI: https://pypi.org/project/mws/0.8.14/

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.