Upgrade to Django 2.2 LTS version

[berinhard]

Is your feature request related to a problem? Please describe.
Currently python.org is running on top of Django 2.0.13 to which the support was dropped on April, 2019. Besides of don't getting security or other updates from other LTS versions, it also makes harder for the project to use most recent versions third-party apps.

Describe the solution you'd like
To upgrade Django to 2.2 since it's a LTS one and will have support until April 2022.

Describe alternatives you've considered
Even though we already have Django 3.2 LT version, I think upgrading to Django 2.2 is a good initial step on upgrading directions. Also, we can only think about upgrading to Django 3.2 LTS after going through the 2.* upgrade pipeline.

Also, I think this process will help us to understand the consequences of such upgrade with a smaller step and, thus, better prepare ourselves for the desired Django 3.2 upgrade.

Additional context
During this process, we must consider eventual backward incompatibilities introduced by Django 2.1 and 2.2 and address them. Here's the link to both lists:

• 2.1 backward incompatibilities
• 2.2 backward incompatibilities

From what I've already seen, upgrading Django itself won't be a problem. The biggest chunk of work will be to update the Django third-party libraries used by Python.org. They can be found on base-requirements.txt and dev-requirements.txt.

If you're new to the project, please read our install instructions to run it locally on your computer. You can tag me (@berinhard) or @ewdurbin for any doubts or questions on this process.

[luzfcb]

@berinhard Migrating to python3.9 could be part of this or would find it better if handled individually in a new issue/possible pull-request?

I recently migrated two mid-size projects from Python2.7 to 3.9 / Django 1.11 to 3.2, so some things are still fresh in my memory.

I hope to get some free time to work on this.

My plan is:

1 - Adding pre-commit with some plugins to help with Python and Django migration. (can be sent in an independent pull-request)
2 - Add tox configuration to locally run tests in both Django (and Python) versions independently of CI vendor (Including update Travis-ci to run the tests via tox). (can be sent in an independent pull-request)
3 - Replace the use of os.environ with django-environ or python-decouple to then optionally support reading the .env file with environment variables (which makes local development without vagrant/docker more flexible) (can be sent in an independent pull-request)
4 - Prepare for Django version migration by removing/fixing the use of features marked for deprecation. (Can be sent in one or more independent pull requests)
5 - Update the dependencies to the latest version that supports Django 2.0 and fix compatibility preparing to upgrade the Django version. (At this step, In the past, I needed to backport some features in a temporary fork in order to facilitate the migration.)
6 - Migrate Django version to 2.2.
7 - Update the dependencies to the latest version that supports Django 2.2 and fix compatibility

Note: The vagrant machine didn't work for me...Ansible crashed...downgraded to 2.X but still not successful.

[berinhard]

Hey @luzfcb, nice to see you here =)

@berinhard Migrating to python3.9 could be part of this or would find it better if handled individually in a new issue/possible pull-request?

About the Python 3.9 update, I'm not sure if it's needed right now. Besides Python 3.6 is still being supported by the most up to date Django version, since such update may also require deployment changes. I'm not sure how easy this would be for @ewdurbin to do. From what I know, there's a lot of stuff on his plate these upcoming weeks, but let us wait on what he think about that.

1 - Adding pre-commit with some plugins to help with Python and Django migration. (can be sent in an independent pull-request)

I'm personally not a huge fan of pre-commit hooks when it comes to code lint or stuff like that. It can make the day-to-day development process slower and boring. What plugins are you thinking about to set up here?

2 - Add tox configuration to locally run tests in both Django (and Python) versions independently of CI vendor (Including update Travis-ci to run the tests via tox). (can be sent in an independent pull-request)

I don't think we need to run the tests on both Django versions. Once we upgrade the project to 2.2, we won't need to maintain it working under 2.0 anymore.

3 - Replace the use of os.environ with django-environ or python-decouple to then optionally support reading the .env file with environment variables (which makes local development without vagrant/docker more flexible) (can be sent in an independent pull-request)

I vote in favor of python-decouple because it's less Django specific.

4 - Prepare for Django version migration by removing/fixing the use of features marked for deprecation. (Can be sent in one or more independent pull requests)
5 - Update the dependencies to the latest version that supports Django 2.0 and fix compatibility preparing to upgrade the Django version. (At this step, In the past, I needed to backport some features in a temporary fork in order to facilitate the migration.)
6 - Migrate Django version to 2.2.
7 - Update the dependencies to the latest version that supports Django 2.2 and fix compatibility

This seems to be a good roadmap for me.

Note: The vagrant machine didn't work for me...Ansible crashed...downgraded to 2.X but still not successful.

Hm, that's awful =/ Actually I've never tried to run the project using Vagrant, so I ain't the best person to help you with this now. @ewdurbin do you have any suggestions on this?

Thanks again for getting involved in this discussion =)

[ewdurbin]

@berinhard Migrating to python3.9 could be part of this or would find it better if handled individually in a new issue/possible pull-request?

About the Python 3.9 update, I'm not sure if it's needed right now. Besides Python 3.6 is still being supported by the most up to date Django version, since such update may also require deployment changes. I'm not sure how easy this would be for @ewdurbin to do. From what I know, there's a lot of stuff on his plate these upcoming weeks, but let us wait on what he think about that.

Python 3.9 PR opened at #1827, I think this can be done independently and ahead of Django upgrade.

1 - Adding pre-commit with some plugins to help with Python and Django migration. (can be sent in an independent pull-request)

I'm personally not a huge fan of pre-commit hooks when it comes to code lint or stuff like that. It can make the day-to-day development process slower and boring. What plugins are you thinking about to set up here?

Agree, I prefer CI running checks which can also be run locally over pre-commit hooks. Also, configuration and maintenance of these hooks tends to need more consistent engagement than this repository has historically seen.

2 - Add tox configuration to locally run tests in both Django (and Python) versions independently of CI vendor (Including update Travis-ci to run the tests via tox). (can be sent in an independent pull-request)

I don't think we need to run the tests on both Django versions. Once we upgrade the project to 2.2, we won't need to maintain it working under 2.0 anymore.

Agree! python.org is not a library, so we only need to support ourselves :)

3 - Replace the use of os.environ with django-environ or python-decouple to then optionally support reading the .env file with environment variables (which makes local development without vagrant/docker more flexible) (can be sent in an independent pull-request)

I vote in favor of python-decouple because it's less Django specific.

I'm +1 to whatever option, but don't necessarily think it's required as part of the Django upgrade. See comment regarding Vagrant setup removal below... we probably need to readdress our dev environment setup holistically at some point.

4 - Prepare for Django version migration by removing/fixing the use of features marked for deprecation. (Can be sent in one or more independent pull requests)
5 - Update the dependencies to the latest version that supports Django 2.0 and fix compatibility preparing to upgrade the Django version. (At this step, In the past, I needed to backport some features in a temporary fork in order to facilitate the migration.)
6 - Migrate Django version to 2.2.
7 - Update the dependencies to the latest version that supports Django 2.2 and fix compatibility

This seems to be a good roadmap for me.

Agree! This seems ideal <3

Note: The vagrant machine didn't work for me...Ansible crashed...downgraded to 2.X but still not successful.

Hm, that's awful =/ Actually I've never tried to run the project using Vagrant, so I ain't the best person to help you with this now. @ewdurbin do you have any suggestions on this?

The Vagrant configuration has been unmaintained for some time, I have never seen it working... I'm going to remove it in the Python 3.9 upgrade PR... perhaps a docker compose setup would be better suited to modern dev practices.