How to translate code from Python 2 to Python 3

Almost 20 years ago, in October 2000, Python 2.0 saw the light. Many changes have since been made in the language, several versions of Python2 were released, and in December 2008, after a long testing period, Python 3.0 was introduced, which was incompatible with the second version of the language.

Due to this the developers had to face a hard choice, which was either to continue supporting current projects written in Python 2 without using a new version of the language, or to start using the new technology, despite the entire amount of work that will have to be done to ensure that everything works correctly.

The majority chose the right path and switched to Python 3, including CheckiO, since June 2017 Python 2.7 is not supported. Which, in general, is consistent with the development trends of this language, because the creators themselves officially declared that the support of Python 2.7 will eventually be discontinued in 2020.

However, some projects still use Python 2, so for them the question of how to upgrade to a new language version is very relevant. This article will help you understand this issue.

The differences

First of all, let's look at some differences between the second and third versions of the language to see those areas that can become problematic.

In Python 3 print is a function, and in Python 2 it’s an operator, so they have a different writing style and working principle. For example:

Python2: print "The answer is", 2*2
Python3: print("The answer is", 2*2)
Python2: print x,           # A comma at the end suppresses the string break
Python3: print(x, end=" ")  # Adds a space instead of a string break
Python2: print              # Prints a string break
Python3: print()            # The arguments need to be passed to the function 
Python2: print >>sys.stderr, "fatal error"
Python3: print("fatal error", file=sys.stderr)

- Some well-known methods don’t return lists in Python3 - the dictionary methods of dict.keys(), dict.items() and dict.values() return "views" instead of lists that only give a way to look at the corresponding objects, but don’t create their copy. For example, k = d.keys(); k.sort() don’t work anymore. If we try to do this, then we’ll get the error:

AttributeError: 'dict_keys' object has no attribute 'sort'

Instead, you need to use: k = sorted(d).

- The frequently used dict.iterkeys(), dict.iteritems(), and dict.itervalues() methods aren’t supported. If we look at the number of methods for working with dictionaries in Python 2 & 3, the difference will be noticeable to the naked eye.

Dictionary methods in Python 2.7:

In [19]: d = {1:100, 2:200, 3:300}

In [20]: d.
d.clear      d.get        d.iteritems  d.keys       d.setdefault
d.viewitems  d.copy       d.has_key    d.iterkeys   d.pop
d.update     d.viewkeys   d.fromkeys   d.items      d.itervalues
d.popitem    d.values     d.viewvalues

And in Python 3

 
In [21]: d = {1:100, 2:200, 3:300}

In [22]: d.
           clear()      get()        pop()        update()
           copy()       items()      popitem()    values()
           fromkeys()   keys()       setdefault()

- map() and filter() return iterators. If you really need a list, you can use list(map(...)), but often the best solution is to use list generators (especially when the original code uses lambda expressions), or you can rewrite the code so that it doesn’t need a list as such.

- range() behaves like xrange(), but works with values ​​of any size.

- zip() returns an iterator.

Python 3 simplified the rules for comparison operators.

The comparison operators (<, <=,> =,>) cause a TypeError exception when the operands can’t be streamlined. Thus, the expressions of 1 < '', 0 > None or len <= len type are no longer allowed, and, for example, None < None calls TypeError, and doesn’t return False. The consequence is that the list sorting of different types of data no longer makes sense - all the elements should be comparable to each other. Note that this doesn’t apply to operators == and !=, the objects of different incomparable types are always unequal to each other.

builtin.sorted() and list.sort() no longer accept the cmp argument, which provides a comparison function. Use the key argument instead. The arguments key and reverse are now "keyword-only".

Also, the changes affected some built-in functions (some of them were removed), the transfer of multiple function arguments, class creation, working with regular expressions, etc.

What to do with all of this? There are several ways that can help to translate code written in Python 2 to the Python 3 compatible type.

1. __future__ imports

from __future__ import division

In Python 2 the division works as follows: if you divide one integer by the other, the result will also be an integer. For example, 3/2 = 1, instead of the expected 1.5. This is because in Python 2, to get a fractional result, you must explicitly specify the type of numbers as float: 3/2.0 = 1.5.

This caused certain inconvenience and in Python 3 the division began to work as expected - regardless of whether the divisor or dividend is float or int, the result won’t be rounded to an integer, that is 3/2 = 1.5. On the other hand, now there is no automatic cast to int either - for any division, the result will be of a float type, even if it’s not necessary. For example: 4/2 = 2.0

You can read more about this imported module in PEP 238.

from __future__ import print_function

We’ve already written about the differences in the print function/operator. This import, written in the beginning of the Python 2 script, imports print as a function, and allows you to work with it in the same way as in a regular program written in Python 3.

The details are described in PEP 3105.

from __future__ import absolute_import

In order for the program written in Python 2 to import modules/packages/libraries in the same way as you are used to in Python 3, you can use the line of code presented above.

You can get familiar with its capabilities in PEP 328.

2. Unicode literals

from __future__ import unicode_literals

Unlike Python 3, where a string is always a string, in Python 2 there are 2 types of strings - str and unicode:

  
In [1]: line = 'test'

In [2]: line2 = u'тест'

The first one uses only the ASCII characters, the second one uses the full set of all possible symbols (for example, Cyrillic letters, hieroglyphs from Asian languages, etc.)

In order to be able to work with a single string type, as in Python 3, you can import unicode_literals, which functional description you can read in PEP 3112.

In case you need to ensure the backward compatibility of the program written in Python 3, you can still use the format u’some text’.

A complete list of the possibilities of the __future__ library you can read in the official documentation.

3. Min/max

As mentioned earlier, in Python 3, in order for a list to be sorted or a minimum/maximum element to be found, all elements must be comparable. If your old code written in Python 2 has lists that contain None elements, they may cause you some problems in the third version of the language. This function is designed to resolve this conflict.

 
def listmin(L):
    '''
       Returns min of an iterable L,
       Ignoring null (None) values.
       If all values are null, returns None
    '''
    values = [v for v in L if v is not None]
    return min(values) if values else None

A similar function can be written to determine the maximum element.

4. Dummy variables

There is another interesting point to consider when working with code written in different versions of Python.

from __future__ import print_function
a = [i for i in range(10)]
print(i)

If we run this code using the Python 2 interpreter, we’ll get 9 as the result, since the i variable used for the list comprehension was left in memory. If in the next part of code you forget about this and use i variable again, this can lead to the unforeseen errors.

In Python 3 everything is much simpler - the variable in this example is used only during the creation of the list and is not being saved after. Thus, when running the code, we'll see the following:

NameError: name 'i' is not defined

Conclusion

As you can see, despite the fact that the code written in Python 2 and Python 3 is considered incompatible, within some limits, you can smooth the differences between them and greatly simplify the transition of projects from the old language version to the new one.

We’ll be glad to know about your experience of working with programs written in Python 2 and useful techniques that you’ve used to adapt them to the third version.

The ideological inspirer of this article was Nick Radcliffe, who presented a speech about transferring their own product from Python 2 to Python 3 on PyData 2018.