Bug#775892: unblock (pre-approval): python-django/1.7.3-1

[Raphael Hertzog]

Package: release.debian.org
Severity: normal
User: ***@packages.debian.org
Usertags: unblock

I would like to upload python-django 1.7.3-1 to sid and jessie.
It's a new upstream version but it contains only bugfixes
(a few of which are security related, see #775375).

The diffstat is massive (see below) but it's partly because upstream is
very serious about its stable release process, all changes come with tests
and documentation. I would like Django to have the same kind of trust
that you have into PostgreSQL. You can have a look at
https://docs.djangoproject.com/en/1.7/internals/release-process/ and
https://docs.djangoproject.com/en/1.7/misc/api-stability/ for instance.

There a lot of bugfixes in 1.7.2 that are not in Jessie and that
are really important. See https://docs.djangoproject.com/en/1.7/releases/1.7.2/
for details. My comaintainer Brian May was hit by
https://code.djangoproject.com/ticket/23581 for example.

And like I argued previously, Django is a package with regular security
updates. By refusing to integrate new stable upstream versions, it gets harder
and harder to apply security updates over time as we diverge from the upstream
codebase. Since we want to support Jessie for 5 years, I would like
us to follow the stable releases of the 1.7.x branch throughout the jessie
lifetime.

Please let me know your thoughts.

unblock python-django/1.7.3-1

Some notes:
- the final upload will include the bug closure of #775375
- there's a small tweak of a Suggests dependency, it was not intended for
jessie but I don't see how it can hurt and did not bother to revert it

Here's the diffstat of the debdiff. I attach the filtered debdiff without the
tests & doc change.

Django.egg-info/PKG-INFO | 4
Django.egg-info/SOURCES.txt | 45
PKG-INFO | 4
debian/changelog | 20
debian/control | 2
debian/gbp.conf | 2
django/__init__.py | 2
django/apps/registry.py | 14
django/conf/locale/el/formats.py | 36
django/conf/locale/ko/LC_MESSAGES/django.po | 19
django/conf/locale/uk/formats.py | 2
django/contrib/admin/actions.py | 21
django/contrib/admin/checks.py | 7
django/contrib/admin/options.py | 23
django/contrib/admin/sites.py | 4
django/contrib/admin/static/admin/js/SelectFilter2.js | 2
django/contrib/admindocs/views.py | 17
django/contrib/auth/__init__.py | 15
django/contrib/auth/decorators.py | 5
django/contrib/auth/hashers.py | 4
django/contrib/auth/middleware.py | 20
django/contrib/auth/tests/test_forms.py | 2
django/contrib/auth/tests/test_hashers.py | 6
django/contrib/auth/tests/test_middleware.py | 53
django/contrib/auth/tests/test_views.py | 26
django/contrib/gis/db/backends/mysql/schema.py | 7
django/contrib/gis/db/backends/spatialite/creation.py | 3
django/contrib/gis/db/models/fields.py | 2
django/contrib/gis/db/models/proxy.py | 2
django/contrib/gis/db/models/sql/query.py | 2
django/contrib/gis/gdal/srs.py | 2
django/contrib/gis/tests/geoapp/tests.py | 19
django/contrib/gis/tests/gis_migrations/test_operations.py | 29
django/contrib/sites/management.py | 10
django/contrib/sites/tests.py | 61
django/contrib/webdesign/tests.py | 4
django/core/exceptions.py | 5
django/core/files/base.py | 7
django/core/management/commands/flush.py | 10
django/core/management/commands/makemessages.py | 2
django/core/management/commands/migrate.py | 3
django/core/management/commands/runserver.py | 4
django/core/management/commands/sqlsequencereset.py | 2
django/core/management/commands/squashmigrations.py | 32
django/core/serializers/xml_serializer.py | 2
django/core/servers/basehttp.py | 11
django/db/backends/mysql/base.py | 3
django/db/backends/mysql/validation.py | 4
django/db/backends/oracle/creation.py | 4
django/db/backends/postgresql_psycopg2/schema.py | 22
django/db/backends/schema.py | 162 -
django/db/backends/sqlite3/schema.py | 28
django/db/migrations/autodetector.py | 74
django/db/migrations/executor.py | 17
django/db/migrations/graph.py | 65
django/db/migrations/loader.py | 8
django/db/migrations/migration.py | 42
django/db/migrations/operations/base.py | 4
django/db/migrations/operations/fields.py | 26
django/db/migrations/operations/models.py | 31
django/db/migrations/optimizer.py | 9
django/db/migrations/questioner.py | 8
django/db/migrations/state.py | 19
django/db/migrations/writer.py | 59
django/db/models/base.py | 12
django/db/models/fields/__init__.py | 8
django/db/models/fields/related.py | 6
django/db/models/sql/compiler.py | 2
django/db/models/sql/query.py | 55
django/db/utils.py | 2
django/forms/fields.py | 17
django/forms/models.py | 47
django/middleware/csrf.py | 6
django/shortcuts.py | 7
django/template/base.py | 4
django/template/debug.py | 4
django/template/defaultfilters.py | 6
django/utils/datetime_safe.py | 12
django/utils/decorators.py | 2
django/utils/html.py | 10
django/utils/http.py | 1
django/utils/safestring.py | 10
django/utils/six.py | 354 ++-
django/utils/text.py | 6
django/views/static.py | 5
docs/Makefile | 7
docs/conf.py | 1
docs/faq/install.txt | 6
docs/howto/auth-remote-user.txt | 16
docs/howto/custom-file-storage.txt | 17
docs/howto/custom-lookups.txt | 23
docs/howto/custom-management-commands.txt | 6
docs/howto/deployment/wsgi/modwsgi.txt | 18
docs/howto/deployment/wsgi/uwsgi.txt | 2
docs/howto/error-reporting.txt | 12
docs/howto/initial-data.txt | 4
docs/howto/windows.txt | 4
docs/index.txt | 12
docs/internals/contributing/writing-code/unit-tests.txt | 10
docs/internals/deprecation.txt | 4
docs/internals/release-process.txt | 10
docs/intro/reusable-apps.txt | 21
docs/intro/tutorial02.txt | 9
docs/intro/tutorial03.txt | 2
docs/ref/applications.txt | 2
docs/ref/checks.txt | 7
docs/ref/contrib/admin/admindocs.txt | 11
docs/ref/contrib/admin/index.txt | 69
docs/ref/contrib/gis/admin.txt | 2
docs/ref/contrib/gis/geos.txt | 12
docs/ref/contrib/gis/install/geolibs.txt | 5
docs/ref/contrib/gis/install/index.txt | 9
docs/ref/contrib/gis/install/spatialite.txt | 39
docs/ref/contrib/gis/testing.txt | 12
docs/ref/contrib/sites.txt | 4
docs/ref/databases.txt | 31
docs/ref/django-admin.txt | 23
docs/ref/files/storage.txt | 19
docs/ref/forms/api.txt | 2
docs/ref/forms/formsets.txt | 2
docs/ref/forms/validation.txt | 9
docs/ref/models/fields.txt | 38
docs/ref/models/instances.txt | 28
docs/ref/models/options.txt | 4
docs/ref/models/querysets.txt | 58
docs/ref/request-response.txt | 14
docs/ref/schema-editor.txt | 17
docs/ref/settings.txt | 23
docs/ref/signals.txt | 9
docs/ref/templates/api.txt | 27
docs/ref/templates/builtins.txt | 46
docs/ref/utils.txt | 6
docs/releases/1.0-alpha-1.txt | 161 -
docs/releases/1.0-alpha-2.txt | 135 -
docs/releases/1.0-beta-2.txt | 119 -
docs/releases/1.0-beta.txt | 153 -
docs/releases/1.1-alpha-1.txt | 165 -
docs/releases/1.1-beta-1.txt | 208 --
docs/releases/1.1-rc-1.txt | 109 -
docs/releases/1.2-alpha-1.txt | 588 -----
docs/releases/1.2-beta-1.txt | 173 -
docs/releases/1.2-rc-1.txt | 101
docs/releases/1.3-alpha-1.txt | 397 ---
docs/releases/1.3-beta-1.txt | 231 --
docs/releases/1.4-alpha-1.txt | 1125 ----------
docs/releases/1.4-beta-1.txt | 1197 -----------
docs/releases/1.4.17.txt | 17
docs/releases/1.4.18.txt | 68
docs/releases/1.5-alpha-1.txt | 634 ------
docs/releases/1.5-beta-1.txt | 706 ------
docs/releases/1.5.12.txt | 14
docs/releases/1.5.txt | 28
docs/releases/1.6.10.txt | 69
docs/releases/1.6.9.txt | 16
docs/releases/1.6.txt | 49
docs/releases/1.7.2.txt | 194 +
docs/releases/1.7.3.txt | 91
docs/releases/1.7.txt | 32
docs/releases/index.txt | 32
docs/releases/security.txt | 4
docs/spelling_wordlist | 2
docs/topics/auth/customizing.txt | 6
docs/topics/auth/default.txt | 9
docs/topics/auth/passwords.txt | 2
docs/topics/cache.txt | 19
docs/topics/checks.txt | 5
docs/topics/db/aggregation.txt | 11
docs/topics/db/models.txt | 17
docs/topics/db/sql.txt | 2
docs/topics/db/transactions.txt | 2
docs/topics/email.txt | 15
docs/topics/forms/formsets.txt | 2
docs/topics/forms/index.txt | 30
docs/topics/forms/modelforms.txt | 7
docs/topics/http/sessions.txt | 4
docs/topics/i18n/translation.txt | 2
docs/topics/migrations.txt | 106 -
docs/topics/python3.txt | 23
docs/topics/serialization.txt | 12
docs/topics/signals.txt | 7
docs/topics/templates.txt | 19
docs/topics/testing/advanced.txt | 88
docs/topics/testing/tools.txt | 30
setup.cfg | 2
setup.py | 2
tests/admin_checks/tests.py | 34
tests/admin_util/models.py | 64
tests/admin_util/tests.py | 353 ---
tests/admin_utils/models.py | 64
tests/admin_utils/tests.py | 353 +++
tests/admin_validation/tests.py | 4
tests/admin_views/admin.py | 33
tests/admin_views/models.py | 36
tests/admin_views/tests.py | 90
tests/admin_widgets/tests.py | 2
tests/apps/tests.py | 40
tests/backends/tests.py | 4
tests/commands_sql/tests.py | 2
tests/csrf_tests/tests.py | 5
tests/custom_columns_regress/tests.py | 2
tests/custom_lookups/models.py | 8
tests/custom_lookups/tests.py | 34
tests/expressions/tests.py | 2
tests/field_deconstruction/tests.py | 12
tests/files/tests.py | 11
tests/fixtures_migration/fixtures/initial_data.json | 9
tests/fixtures_migration/migrations/0001_initial.py | 16
tests/fixtures_migration/models.py | 5
tests/fixtures_migration/tests.py | 34
tests/forms_tests/tests/test_forms.py | 16
tests/forms_tests/tests/test_util.py | 91
tests/forms_tests/tests/test_utils.py | 113 +
tests/i18n/test_extraction.py | 8
tests/indexes/tests.py | 47
tests/invalid_models_tests/test_custom_fields.py | 20
tests/logging_tests/tests.py | 16
tests/migrate_signals/tests.py | 22
tests/migrations/test_autodetector.py | 1351 +++++++------
tests/migrations/test_base.py | 6
tests/migrations/test_commands.py | 68
tests/migrations/test_executor.py | 96
tests/migrations/test_graph.py | 41
tests/migrations/test_operations.py | 91
tests/migrations/test_writer.py | 20
tests/model_fields/models.py | 2
tests/model_fields/tests.py | 4
tests/model_forms/models.py | 2
tests/model_forms/tests.py | 50
tests/model_inheritance/tests.py | 2
tests/model_options/test_tablespaces.py | 44
tests/model_regress/tests.py | 44
tests/project_template/test_settings.py | 29
tests/project_template/urls.py | 7
tests/project_template/views.py | 5
tests/queries/models.py | 15
tests/queries/tests.py | 61
tests/requirements/mysql.txt | 2
tests/resolve_url/tests.py | 12
tests/resolve_url/urls.py | 5
tests/runtests.py | 5
tests/schema/models.py | 12
tests/schema/tests.py | 89
tests/serializers_regress/tests.py | 4
tests/servers/test_basehttp.py | 67
tests/signed_cookies_tests/tests.py | 2
tests/staticfiles_tests/tests.py | 4
tests/template_tests/test_nodelist.py | 17
tests/test_runner/tests.py | 2
tests/test_utils/views.py | 4
tests/update_only_fields/tests.py | 2
tests/utils_tests/test_datetime_safe.py | 9
tests/utils_tests/test_http.py | 3
tests/utils_tests/test_safestring.py | 49
tests/validation/tests.py | 2
tests/view_tests/media/long-line.txt | 1
tests/view_tests/tests/test_i18n.py | 7
tests/view_tests/tests/test_static.py | 10
257 files changed, 5106 insertions(+), 8377 deletions(-)

-- System Information:
Debian Release: 8.0
APT prefers squeeze-lts
APT policy: (500, 'squeeze-lts'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

[Raphael Hertzog]

Post by Raphael Hertzog
- the final upload will include the bug closure of #775375
- there's a small tweak of a Suggests dependency, it was not intended for
jessie but I don't see how it can hurt and did not bother to revert it

I have uploaded 1.7.3-1~exp1 to experimental which is basically what I'd
like to upload to unstable. It contains one more patch compared to the
debdiff I sent to fix a build failure with Python 3.4
(https://github.com/django/django/commit/b1bf8d64fbadcab860eb98662c49b8db33db0c3c).

Cheers,

PS: I know that Neil Williams uploaded an NMU to fix the security issues but I
still want to include 1.7.3.
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[Moritz Mühlenhoff]

Post by Raphael Hertzog

Post by Raphael Hertzog
- the final upload will include the bug closure of #775375
- there's a small tweak of a Suggests dependency, it was not intended for
jessie but I don't see how it can hurt and did not bother to revert it

I have uploaded 1.7.3-1~exp1 to experimental which is basically what I'd
like to upload to unstable. It contains one more patch compared to the
debdiff I sent to fix a build failure with Python 3.4
(https://github.com/django/django/commit/b1bf8d64fbadcab860eb98662c49b8db33db0c3c).
Cheers,
PS: I know that Neil Williams uploaded an NMU to fix the security issues but I
still want to include 1.7.3.

It would still be good to unblock the NMU first to get the security
fixes into jessie.

Cheers,
Moritz

[Raphael Hertzog]

Control: retitle -1 unblock (pre-approval): python-django/1.7.4-1

Post by Raphael Hertzog
I would like to upload python-django 1.7.3-1 to sid and jessie.
It's a new upstream version but it contains only bugfixes
(a few of which are security related, see #775375).

[...]

Post by Raphael Hertzog
Please let me know your thoughts.

I want to add that my request is now to upload 1.7.4-1 since upstream
released a new bugfix version:
https://docs.djangoproject.com/en/1.7/releases/1.7.4/

BTW, that version contains a fix for an unreported FTBFS that we have in
Debian and that has been introduced by the recent security patches and thus
by Neil's NMU:
https://code.djangoproject.com/ticket/24193

(Just verified my assertion by running sbuild on version 1.7.1-1.1)

Please don't let this request languish for ever.

/me just suffered the pain of backporting Django security patches
to version 1.2 for squeeze-lts. I really want to make it easier for
us in the future by following upstream closely on their stable release
when there's really few risks in doing so.

Thanks.
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[Raphael Hertzog]

Hi,

it's been two weeks that I have opened this pre-approval request
and I got almost no feedback from the release team (except Neil saying
that he has no answer for me on IRC).

If I don't hear back from you in the next two days, I will proceed

Post by Raphael Hertzog
I want to add that my request is now to upload 1.7.4-1 since upstream
https://docs.djangoproject.com/en/1.7/releases/1.7.4/
BTW, that version contains a fix for an unreported FTBFS that we have in
Debian and that has been introduced by the recent security patches and thus
https://code.djangoproject.com/ticket/24193
(Just verified my assertion by running sbuild on version 1.7.1-1.1)

File the above bug report with severity serious and upload 1.7.4-1 to
sid closing the bug.

After all if this request must be waiting longer, we might as well benefit
from the time the package languish in sid as a proof that it's working
well for us.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[Raphael Hertzog]

Control: retitle -1 unblock: python-django/1.7.4-1

Post by Raphael Hertzog
File the above bug report with severity serious and upload 1.7.4-1 to
sid closing the bug.

Now done.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[Mehdi Dogguy]

Note that this not mean that we will accept (let's say) 1.7.5 next
time.

I assume you mean that when 1.7.5 comes out, we should make another
unblock request?

Yes. The idea is to not assume that a new minor release will be
accepted
at this stage. So, as usual, if you feel an update to the package is
needed, please file a bug report and talk to us explaining your plan
before taking actions.

Regards,

[Raphael Hertzog]

Hi Mehdi,

Post by Raphael Hertzog
it's been two weeks that I have opened this pre-approval request
and I got almost no feedback from the release team (except Neil saying
that he has no answer for me on IRC).

Neil or Niels?

Sorry, I meant Niels.

I can understand why the former doesn't have any
answer for you on this subject. The latter might not have make his
mind yet on this because it is not an easy subject.

Sure.

Post by Raphael Hertzog
If I don't hear back from you in the next two days, I will proceed

Do you think such a statement helps you in any way?

Well, it was not meant to help me, but to help Debian. If the arguments I
have put forth were not enough to convince anyone of the release team in a
reasonable timeframe, then I hoped that some real-life testing in unstable
would be a supplementary proof that it was the safe thing to do.

The non-trivial part is to try to draw a line to know what should be
allowed to be updated using new upstream releases, and what doesn't.
An effort has been made into this direction (See packages like linux,
iceweasel, postgresql, etc...) but I think that there is still room
for improvement there.

Certainly, that's why I was arguing that we could/should handle
python-django like the cases that you mention.

And if you don't agree, then I would be interested to know what could
bring you the required confidence so that we can gain that status
(maybe adding autopkgtests to reverse dependencies and running
those tests, or stuff like that).

Anyway. Based on my blabla about security stuff, I've decided to
unblock this package so that it migrates to Jessie. Note that this
doesn't mean that we will accept (let's say) 1.7.5 next time.

Thanks!
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[Mehdi Dogguy]

Post by Raphael Hertzog
And if you don't agree, then I would be interested to know what could
bring you the required confidence so that we can gain that status
(maybe adding autopkgtests to reverse dependencies and running
those tests, or stuff like that).

The list of packages that we started with are the ones that give more
trouble to the security team. Maybe the list should be expanded, I
don't know. Most probably. What we would like to have is a general
discussion (not now please... but after jessie release) to put some
criteria to help to distinguish packages that need a special treatement
and others that don't. For now, the criteria has been "security team
has trouble with X" but I don't find this criteria good enough.

The autopkgtests might be part of the answer, but not the only one
I hope.

Regards,