What is the Django Way (tm) to paginate complex queries with
user-controlled parameters? That is, if I want to paginate the results
of a foos.get_list({'lots': 'of_complex', }) where
the query terms are submitted via form, how should I go about that
while maintaining the user-posted values from page to page?

Thanks in advance for your thoughts.

- John

Use the (undocumented) django.core.paginator class:http://code.djangoproject.com/browser/django/trunk/django/core/paginator.pyYou can see an example on how it works in:http://code.djangoproject.com/browser/django/trunk/django/contrib/admin/views/main.pyin the change_list function, the one that generates the lists in the
admin interface.

--
Gustavo Picon -http://tabo.aurealsys.com/

I've already dug into the paginator code. I was hoping there was a way
to do it without having to munge query parameters by hand every time I
get a new page. I thought my solution was bad but that code in the
admin interface is pretty much the same sort of edge-case-ridden sin
against nature.

Maybe django.core.paginator needs to account for more complex
scenarios, or is there yet some feature I'm missing?

- John

On 11/22/05, <> wrote:
>
> What is the Django Way (tm) to paginate complex queries with
> user-controlled parameters? That is, if I want to paginate the results
> of a foos.get_list({'lots': 'of_complex', }) where
> the query terms are submitted via form, how should I go about that
> while maintaining the user-posted values from page to page?

This looks like a case where generic views, combined with the code
from ticket #667 (specifically, the second patch), might come in
handy:http://code.djangoproject.com/ticket/667I wrote the current patch implementation, so I'd love to hear any
suggestions or criticism. :-)

On Tue, 22 Nov 2005 07:09:06 -0800 wrote:


> What is the Django Way (tm) to paginate complex queries with
> user-controlled parameters? That is, if I want to paginate the
> results of a foos.get_list({'lots': 'of_complex', }) where the query terms are submitted via form, how should I
> go about that while maintaining the user-posted values from page to
> page?

For propagating values in forms, I have a custom template tag to do
it. However, it requires that you put the request object in the
context object for your template. In my project I'm using a
'pagevars' object to hold this kind of slightly random thing (i.e.
stuff that isn't particularly related to the view), so all my
custom template tags can expect to find pagevars.request in the context.

The code looks something like this:

-----
from django.core import template
from django.utils import html

class ForwardQueryParamNode(template.Node):
def __init__(self, param_name):
self.param_name = param_name

def render(self, context):
request = context.request
return '<div><input type="hidden" name="' + self.param_name + \
'" value="' + html.escape(request.GET.get(self.param_name, '')) + \
'" /></div>'

# forward_query_param - turn a param in query string into a hidden
# input field. Needs to be able to get the request object from the context
def do_forward_query_param(parser, token):
"""
Turns a parameter in a query string into a hidden input field,
allowing it to be 'forwarded' as part of the next request in
a form submission. It requires one argument (the name of the parameter),
and also requires that the request object be in the context as
by putting the standard 'pagevars' object in the context.
"""
try:
tag_name, param_name = token.contents.split(None, 1)
except ValueError:
raise template.TemplateSyntaxError, \
"forward_query_param tag requires an argument"
param_name = param_name.strip('"')
return ForwardQueryParamNode(param_name)

------
And in the template - inside a <form> put this:

{% forward_query_param "order" %}

I also have a 'paging_control' and a 'sorting_control' which have similar
requirements, and automatically 'forward' or adjust all query string
parameters. I've attached them to this e-mail (along with an example
template and view) in case you find them useful. They will need
modification to work outside my app, but I have re-used them several
times already inside my app and found they work quite well.

This is not an official Django method at all, but it works for me. There
must be something similar to this in the admin, but last time I looked
I was unable to extract and re-use it easily. The new admin branch might
be different.

It does make me think that if we had a standard for putting the request
object into the context it would make template tags like this more
re-usable.

Finally, looking at my code, I think the <input> controls in the template
could be refactored much more nicely into their own custom template tags.

Luke


--
"Outside of a dog, a book is a man's best friend... inside of a dog,
it's too dark to read."

Luke Plant || L.Plant.98 (at) cantab.net ||http://lukeplant.me.uk/from django.views.generic import list_detail
from django.utils.httpwrappers import HttpResponseRedirect
from django.core.exceptions import Http404
from django.core.extensions import render_to_response

from django.models.members import members
from django.models.members.members import MemberDoesNotExist
from cciw.apps.cciw.common import *
from django.core.extensions import DjangoContext
from datetime import datetime, timedelta

def index(request):
lookup_args = {'dummy_member__exact' : 'False', 'hidden__exact': 'False'}
if (request.GET.has_key('online')):
lookup_args = datetime.now() - timedelta(minutes=3)

extra_context = standard_extra_context(request, title='Members')
order_option_to_lookup_arg(
{'adj': ('date_joined',),
'ddj': ('-date_joined',),
'aun': ('user_name',),
'dun': ('-user_name',),
'arn': ('real_name',),
'drn': ('-real_name',),
'als': ('last_seen',),
'dls': ('-last_seen',)},
lookup_args, request, ('user_name',))
extra_context = 'aun'

try:
search = '%' + request + '%'
lookup_args =
lookup_args =
except KeyError:
pass

return list_detail.object_list(request, 'members', 'members',
extra_context = extra_context,
template_name = 'cciw/members/index',
paginate_by=50, extra_lookup_kwargs = lookup_args,
allow_empty = True){% extends "cciw/standard" %}
{% load view_extras %}
{% load forums %}
{% block content %}Search:Online users only:{% forward_query_param "order" %}

{% if object_list %}{% paging_control %}IconMember name {% sorting_control "aun" "dun" "Sort by user name ascending" "Sort by user name descending" %}'Real' name {% sorting_control "arn" "drn" "Sort by real name ascending" "Sort by real name descending" %}CommentsJoined {% sorting_control "adj" "ddj" "Sort by 'date joined' ascending" "Sort by 'date joined' descending" %}Last seen {% sorting_control "als" "dls" "Sort by 'last seen' ascending" "Sort by 'last seen' descending" %}{% for member in object_list %}{% if member.icon %}
{{ member.icon_image }}
{% endif %}{{ member.get_link }}{{ member.real_name|escape }}{{ member.comments|bb2html }}{% if member.date_joined %}
{{ member.date_joined|date:"d M Y" }}
{% endif %}{% if member.last_seen %}
{{ member.last_seen|date:"d M Y h:i" }}
{% endif %}{% endfor %}{% paging_control %}{% else %}No members found.{% endif %}

{% endblock %}from django.core import template
from django.utils import html
from cciw.apps.cciw.settings import *
from cciw.apps.cciw.utils import *

def page_link(request, page_number, fragment = ''):
"""Constructs a link to a specific page using the request. Returns HTML escaped value"""
return html.escape(modified_query_string(request, {'page': str(page_number)}, fragment))

class PagingControlNode(template.Node):
def __init__(self, fragment = ''):
self.fragment = fragment

def render(self, context):
# context has been populated by
# generic view paging mechanism
cur_page = int(context)-1
total_pages = int(context)
request = context.request
output =
if (total_pages > 1):
output.append("&mdash; Page %d of %d &mdash;&nbsp;&nbsp; " %
(cur_page + 1, total_pages))

for i in range(0, total_pages):
if (i > 0):
output.append(" | ")

if i == cur_page:
output.append('<span class="pagingLinkCurrent">'
+ str(i+1) + '</span>')
else:
output.append(
'<a title="%(title)s" class="pagingLink" href="%(href)s">%(pagenumber)d</a>' % \
{ 'title': 'Page ' + str(i+1),
'href': page_link(request, i, self.fragment),
'pagenumber': i+1 })
output.append(" | ")
if cur_page > 0:
output.append(
'<a class="pagingLink" title="Previous page" href="%s">&laquo;</a>' % \
page_link(request, cur_page - 1, self.fragment))
else:
output.append('<span class="pagingLinkCurrent">&laquo;</span>')
output.append("&nbsp;")
if cur_page < total_pages - 1:
output.append(
'<a class="pagingLink" title="Next page" href="%s">&raquo;</a>' % \
page_link(request, cur_page + 1, self.fragment))
else:
output.append('<span class="pagingLinkCurrent">&raquo;</span>')
return ''.join(output)


def do_paging_control(parser, token):
"""
Creates a list of links to the pages of a generic view.
The paging control requires that the request object be in the context as
by putting the standard 'pagevars' object in the context.

An optional parameter can be used which contains the fragment to use for
paging links, which allows paging to skip any initial common content on the page.

Usage::

{% paging_control %}

"""
parts = token.contents.split(None, 1)
if len(parts) > 1:
return PagingControlNode(fragment = parts)
else:
return PagingControlNode()

class SortingControlNode(template.Node):
def __init__(self, ascending_param, descending_param,
ascending_title, descending_title):
self.ascending_param = ascending_param
self.descending_param = descending_param
self.ascending_title = ascending_title
self.descending_title = descending_title

def render(self, context):
request = context.request
output = '<span class="sortingControl">'
current_order = request.GET.get('order','')
if current_order == '':
try:
current_order = context
except KeyError:
current_order = ''

if current_order == self.ascending_param:
output += '<a title="' + self.descending_title +'" href="' + \
html.escape(modified_query_string(request, {'order': self.descending_param})) \
+ '"><img class="sortAscActive" src="' + CCIW_MEDIA_ROOT + \
'images/arrow-up.gif" alt="Sorted ascending" /></a>'
elif current_order == self.descending_param:
output += '<a title="' + self.ascending_title +'" href="' + \
html.escape(modified_query_string(request, {'order': self.ascending_param})) \
+ '"><img class="sortDescActive" src="' + CCIW_MEDIA_ROOT + \
'images/arrow-down.gif" alt="Sorted descending" /></a>'
else:
# query string resets page to zero if we use a new type of sorting
output += '<a title="' + self.ascending_title +'" href="' + \
html.escape(modified_query_string(request,
{'order': self.ascending_param, 'page': 0})) \
+ '"><img class="sortAsc" src="' + CCIW_MEDIA_ROOT + \
'images/arrow-up.gif" alt="Sort ascending" /></a>'

output += '</span>'
return output


def do_sorting_control(parser, token):
"""
Creates a pair of links that are used for sorting a list on a field.
Four parameters are accepted, which must be the query string parameter values that
should be used for ascending and descending sorts on a field, and the corresponding
title text for those links (in that order). All arguments must be quoted with '"'

The query string parameter name is always 'order', and the view function
will need to check that parameter and adjust accordingly. The view function
should also add 'default_order' to the context, which allows the
sorting control to determine what the current sort order is if no
'order' parameter exists in the query string.

The sorting control requires that the request object be in the context as
by putting the standard 'pagevars' object in the context. It is
used for various things, including passing on other query string parameters
in the generated URLs.

Usage::

{% sorting_control "ad" "dd" "Sort date ascending" "Sort date descending" %}

"""
bits = token.contents.split('"')
if len(bits) != 9:
raise template.TemplateSyntaxError, "sorting_control tag requires 4 quoted arguments"
return SortingControlNode(bits.strip('"'), bits.strip('"'),
bits.strip('"'), bits.strip('"'),)

class ForwardQueryParamNode(template.Node):
def __init__(self, param_name):
self.param_name = param_name

def render(self, context):
# requires the standard extra context
request = context.request
return '<div><input type="hidden" name="' + self.param_name + \
'" value="' + html.escape(request.GET.get(self.param_name, '')) + \
'" /></div>'

# forward_query_param - turn a param in query string into a hidden
# input field. Needs to be able to get the request object from the context
def do_forward_query_param(parser, token):
"""
Turns a parameter in a query string into a hidden input field,
allowing it to be 'forwarded' as part of the next request in
a form submission. It requires one argument (the name of the parameter),
and also requires that the request object be in the context as
by putting the standard 'pagevars' object in the context.
"""
try:
tag_name, param_name = token.contents.split(None, 1)
except ValueError:
raise template.TemplateSyntaxError, \
"forward_query_param tag requires an argument"
param_name = param_name.strip('"')
return ForwardQueryParamNode(param_name)

template.register_tag('paging_control', do_paging_control)
template.register_tag('sorting_control', do_sorting_control)
template.register_tag('forward_query_param', do_forward_query_param)

+1 to putting the request object into the context as standard, or at
least into DjangoContext. I've only been playing with Django for a few
days, and it's already obvious that this single change would make
writing useful custom tags a whole lot easier.

Colin

On 23 Nov 2005, at 14:42, Colin Howlett wrote:+1 to putting the request object into the context as standard, or at
least into DjangoContext. I've only been playing with Django for a few
days, and it's already obvious that this single change would make
writing useful custom tags a whole lot easier.This also means that there's more of a temptation to use the requestobject directly in template logic.Is there a way that the request object could be exposed to custom tagcode, but not the template itself?________________________________
Afternoon, man about the Internet --http://aftnn.org/

On 23 Nov 2005, at 14:59, Afternoon wrote:Is there a way that the request object could be exposed to customtag code, but not the template itself?The aim with the template system has always been to keep it de-coupled from the request/response stuff, so it can be used as astandalone component. It's hard to see how the request object couldbe exposed to custom tags without breaking that separation. Thatsaid, it would be enable some very neat custom template tag tricks.

On 23 Nov 2005, at 15:07, Simon Willison wrote:The aim with the template system has always been to keep it de-coupled from the request/response stuff, so it can be used as astandalone component. It's hard to see how the request object couldbe exposed to custom tags without breaking that separation. Thatsaid, it would be enable some very neat custom template tag tricks.Right. I think that decoupling is a Good Thing. I use templatesystems for a variety of purposes outside of web app development andit's nice to have a standard option for all requirements.I personally haven't bashed my head against Django that much yet so Ifeel I'm getting a bit out of my depth here. Perhaps the best thingis just to keep explicitly passing in the pagination arguments.________________________________
Afternoon, man about the Internet --http://aftnn.org/