reverse() escapes unreserved characters

[Ola Tarkowska]

Hello,

Django==1.11.14
djangorestframework==3.8.2

I have been trying to resolve issue related to incorectly escaping reverse
url with params containing forward slash.

Constructing URL for the following pattern throws an exception as myparam
is not correctly encoded `foo/bar` rather then `foo%2Fbar`

My patterns:

1. ^v1/ ^endpoints/(?P<lineage>(.*))$ [name='endpoints-list']
2. ^v1/ ^endpoints/(?P<lineage>(.*))\.(?P<format>[a-z0-9]+)/?$
[name='endpoints-list']
3. ^v1/ ^endpoints/(?P<lineage>(.*))/something$
[name='endpoints-something-list']
4. ^v1/ ^endpoints/(?P<lineage>(.*))/something\.(?P<format>[a-z0-9]+)/?$
[name='endpoints-something-list']


That works in Django admin panel as it was fixed in https://code.djangoproject.com/ticket/22223

Am I doing somehting wrong?

Thanks in advance
Ola

[Carlton Gibson]

Hi Ola.

There’s not quite enough here to see what’s going on.

Can you post the reverse() calls, with the output and what you’re expecting?

Thanks.

Sent from my iPhone

Post by Ola Tarkowska
Hello,
Django==1.11.14
djangorestframework==3.8.2
I have been trying to resolve issue related to incorectly escaping reverse url with params containing forward slash.
Constructing URL for the following pattern throws an exception as myparam is not correctly encoded `foo/bar` rather then `foo%2Fbar`
^v1/ ^endpoints/(?P<lineage>(.*))$ [name='endpoints-list']
^v1/ ^endpoints/(?P<lineage>(.*))\.(?P<format>[a-z0-9]+)/?$ [name='endpoints-list']
^v1/ ^endpoints/(?P<lineage>(.*))/something$ [name='endpoints-something-list']
^v1/ ^endpoints/(?P<lineage>(.*))/something\.(?P<format>[a-z0-9]+)/?$ [name='endpoints-something-list']
That works in Django admin panel as it was fixed in https://code.djangoproject.com/ticket/22223
Am I doing somehting wrong?
Thanks in advance
Ola
--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
For more options, visit https://groups.google.com/d/optout.

[Ola Tarkowska]

Sorry for a slow response. I am trying to make sure that serialized object
has correctly encoded value encode correctly urls with `%2F` rather then `/`

urls.py
myrouter.register(
r'endpoints',
views.EndpointViewSet,
base_name='endpoints'
)

views.py
lookup_field = 'lineage'
lookup_value_regex = '.*'

serializers.py
class MySerializer(serializers.HyperlinkedModelSerializer):

url = serializers.HyperlinkedIdentityField(
view_name='api_v1:endpoints-detail',
lookup_field='lineage',
)


I am getting response:

"link": "http://localhost:8000/v1/endpoints/foo/bar
<http://localhost:8000/v1/annotations/organisms/Eukaryota::Bacillariophyta:Coscinodiscophyceae:Chaetocerotales:Chaetocerotaceae:Chaetoceros:Chaetoceros_sp._CCAP_1010/16>
"


that should be foo%2Fbar, what exactly am I missing?

[Ola Tarkowska]

to be clear foo/bar is an ideantifier of the resource


Ola

Post by Ola Tarkowska
Sorry for a slow response. I am trying to make sure that serialized object
has correctly encoded value encode correctly urls with `%2F` rather then `/`
urls.py
myrouter.register(
r'endpoints',
views.EndpointViewSet,
base_name='endpoints'
)
views.py
lookup_field = 'lineage'
lookup_value_regex = '.*'
serializers.py
url = serializers.HyperlinkedIdentityField(
view_name='api_v1:endpoints-detail',
lookup_field='lineage',
)
"link": "http://localhost:8000/v1/endpoints/foo/bar
<http://localhost:8000/v1/annotations/organisms/Eukaryota::Bacillariophyta:Coscinodiscophyceae:Chaetocerotales:Chaetocerotaceae:Chaetoceros:Chaetoceros_sp._CCAP_1010/16>
"
that should be foo%2Fbar, what exactly am I missing?

[Carlton Gibson]

Right, OK.

And sorry to be a pain but can you post the actual pattern that gets generated for `endpoints-detail`?
(It’s not in the list of four URLs in your original post.)

If I do something like this, I don’t get a match:

# urls.py — A sample URL from the docs.
path('bio/<username>/', view, name='bio'),

Post by Ola Tarkowska

from django.urls import reverse
reverse('bio', kwargs={"username":"foo/bar"})

Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/carlton/Documents/Django-Stack/django/django/urls/base.py", line 90, in reverse
return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
File "/Users/carlton/Documents/Django-Stack/django/django/urls/resolvers.py", line 624, in _reverse_with_prefix
raise NoReverseMatch(msg)
django.urls.exceptions.NoReverseMatch: Reverse for 'bio' with keyword arguments '{'username': 'foo/bar'}' not found. 1 pattern(s) tried: ['bio\\/(?P<username>[^/]+)\\/$']

Note the actual URL registered uses the not-slash group `[^/]`.

Also, what happens if you manually create a URL for a RetrieveAPIView with the same serialiser etc?

Thanks. Carlton.

Post by Ola Tarkowska
to be clear foo/bar is an ideantifier of the resource
Ola
Sorry for a slow response. I am trying to make sure that serialized object has correctly encoded value encode correctly urls with `%2F` rather then `/`
urls.py
myrouter.register(
r'endpoints',
views.EndpointViewSet,
base_name='endpoints'
)
views.py
lookup_field = 'lineage'
lookup_value_regex = '.*'
serializers.py
url = serializers.HyperlinkedIdentityField(
view_name='api_v1:endpoints-detail',
lookup_field='lineage',
)
"link": "http://localhost:8000/v1/endpoints/foo/bar <http://localhost:8000/v1/annotations/organisms/Eukaryota::Bacillariophyta:Coscinodiscophyceae:Chaetocerotales:Chaetocerotaceae:Chaetoceros:Chaetoceros_sp._CCAP_1010/16>"
that should be foo%2Fbar, what exactly am I missing?
--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

[Carlton Gibson]

But if I do this the old-fashioned way with a more liberal regex:

re_path(r'^bio2/(?P<username>.*)/', view, name='bio2'),

Post by Ola Tarkowska

reverse('bio2', kwargs={"username":"foo/bar"})

'/bio2/foo/bar/

i.e. your `/` is NOT escaped by `reverse()`.

(Note this is just `reverse()` — it’s nothing to do with DRF.)

If you see the discussion from this comment: comment:8 <https://code.djangoproject.com/ticket/22223#comment:8> on the ticket you linked you’ll

Something like that is certainly not going to happen.

I think this may be something you need to work around (by subclassing hyperlinked field to quote the identifier yourself).

I hope that helps
 😬

Kind Regards,

Carlton

Right, OK.
And sorry to be a pain but can you post the actual pattern that gets generated for `endpoints-detail`?
(It’s not in the list of four URLs in your original post.)
# urls.py — A sample URL from the docs.
path('bio/<username>/', view, name='bio'),

Post by Ola Tarkowska

from django.urls import reverse
reverse('bio', kwargs={"username":"foo/bar"})

File "<console>", line 1, in <module>
File "/Users/carlton/Documents/Django-Stack/django/django/urls/base.py", line 90, in reverse
return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
File "/Users/carlton/Documents/Django-Stack/django/django/urls/resolvers.py", line 624, in _reverse_with_prefix
raise NoReverseMatch(msg)
django.urls.exceptions.NoReverseMatch: Reverse for 'bio' with keyword arguments '{'username': 'foo/bar'}' not found. 1 pattern(s) tried: ['bio\\/(?P<username>[^/]+)\\/$']
Note the actual URL registered uses the not-slash group `[^/]`.
Also, what happens if you manually create a URL for a RetrieveAPIView with the same serialiser etc?
Thanks. Carlton.

Post by Ola Tarkowska
to be clear foo/bar is an ideantifier of the resource
Ola
Sorry for a slow response. I am trying to make sure that serialized object has correctly encoded value encode correctly urls with `%2F` rather then `/`
urls.py
myrouter.register(
r'endpoints',
views.EndpointViewSet,
base_name='endpoints'
)
views.py
lookup_field = 'lineage'
lookup_value_regex = '.*'
serializers.py
url = serializers.HyperlinkedIdentityField(
view_name='api_v1:endpoints-detail',
lookup_field='lineage',
)
"link": "http://localhost:8000/v1/endpoints/foo/bar <http://localhost:8000/v1/annotations/organisms/Eukaryota::Bacillariophyta:Coscinodiscophyceae:Chaetocerotales:Chaetocerotaceae:Chaetoceros:Chaetoceros_sp._CCAP_1010/16>"
that should be foo%2Fbar, what exactly am I missing?
--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

[Ola Tarkowska]

Thank you for detailed answer, I tried `[^/]`. and got the same error as
that is not going to work.
One thing is that django admin does it correctly, URL parameter is encoded.

Ola

[Carlton Gibson]

I suspect then that the parameters are escaped individually. You’ll need to have a dig around and see if you can make the hyperlinked fields do the same.

C.

Sent from my iPhone

Thank you for detailed answer, I tried `[^/]`. and got the same error as that is not going to work.
One thing is that django admin does it correctly, URL parameter is encoded.
Ola
--
You received this message because you are subscribed to the Google Groups "Django REST framework" group.
For more options, visit https://groups.google.com/d/optout.