Merge branch master into master-slp.

Author: Anselm Kruis

Doc/c-api/typeobj.rst

@@ -199,8 +199,9 @@ type objects) *must* have the :attr:`ob_size` field.
 
    This field is deprecated.  When it is defined, it should point to a function
    that acts the same as the :c:member:`~PyTypeObject.tp_getattro` function, but taking a C string
-   instead of a Python string object to give the attribute name.  The signature is
-   the same as for :c:func:`PyObject_GetAttrString`.
+   instead of a Python string object to give the attribute name.  The signature is ::
+
+      PyObject * tp_getattr(PyObject *o, char *attr_name);
 
    This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype
    inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when
@@ -213,10 +214,11 @@ type objects) *must* have the :attr:`ob_size` field.
 
    This field is deprecated.  When it is defined, it should point to a function
    that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string
-   instead of a Python string object to give the attribute name.  The signature is
-   the same as for :c:func:`PyObject_SetAttrString`, but setting
-   *v* to *NULL* to delete an attribute must be supported.
+   instead of a Python string object to give the attribute name.  The signature is ::
+
+      PyObject * tp_setattr(PyObject *o, char *attr_name, PyObject *v);
 
+   The *v* argument is set to *NULL* to delete the attribute.
    This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype
    inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when
    the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*.

Doc/howto/clinic.rst

@@ -1,3 +1,5 @@
+.. highlightlang:: c
+
 **********************
 Argument Clinic How-To
 **********************
@@ -78,17 +80,23 @@ Basic Concepts And Usage
 ========================
 
 Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``.
-If you run that script, specifying a C file as an argument::
+If you run that script, specifying a C file as an argument:
+
+.. code-block:: shell-session
 
-    % python3 Tools/clinic/clinic.py foo.c
+    $ python3 Tools/clinic/clinic.py foo.c
 
 Argument Clinic will scan over the file looking for lines that
-look exactly like this::
+look exactly like this:
+
+.. code-block:: none
 
     /*[clinic input]
 
 When it finds one, it reads everything up to a line that looks
-exactly like this::
+exactly like this:
+
+.. code-block:: none
 
     [clinic start generated code]*/
 
@@ -99,7 +107,9 @@ lines, are collectively called an Argument Clinic "block".
 When Argument Clinic parses one of these blocks, it
 generates output.  This output is rewritten into the C file
 immediately after the block, followed by a comment containing a checksum.
-The Argument Clinic block now looks like this::
+The Argument Clinic block now looks like this:
+
+.. code-block:: none
 
     /*[clinic input]
     ... clinic input goes here ...
@@ -378,9 +388,7 @@ Let's dive in!
 12. Save and close the file, then run ``Tools/clinic/clinic.py`` on
     it.  With luck everything worked---your block now has output, and
     a ``.c.h`` file has been generated! Reopen the file in your
-    text editor to see:
-
-    .. code-block:: c
+    text editor to see::
 
        /*[clinic input]
        _pickle.Pickler.dump
@@ -402,9 +410,7 @@ Let's dive in!
 
     For readability, most of the glue code has been generated to a ``.c.h``
     file.  You'll need to include that in your original ``.c`` file,
-    typically right after the clinic module block:
-
-     .. code-block:: c
+    typically right after the clinic module block::
 
        #include "clinic/_pickle.c.h"
 
@@ -1028,7 +1034,9 @@ that value, and an error has been set (``PyErr_Occurred()`` returns a true
 value), then the generated code will propagate the error.  Otherwise it will
 encode the value you return like normal.
 
-Currently Argument Clinic supports only a few return converters::
+Currently Argument Clinic supports only a few return converters:
+
+.. code-block:: none
 
     bool
     int
@@ -1607,7 +1615,9 @@ code probably looks like this::
     #endif /* HAVE_FUNCTIONNAME */
 
 And then in the ``PyMethodDef`` structure at the bottom the existing code
-will have::
+will have:
+
+.. code-block:: none
 
     #ifdef HAVE_FUNCTIONNAME
     {'functionname', ... },
@@ -1657,7 +1667,9 @@ extra code when using the "block" output preset?  It can't go in the output bloc
 because that could be deactivated by the ``#ifdef``.  (That's the whole point!)
 
 In this situation, Argument Clinic writes the extra code to the "buffer" destination.
-This may mean that you get a complaint from Argument Clinic::
+This may mean that you get a complaint from Argument Clinic:
+
+.. code-block:: none
 
     Warning in file "Modules/posixmodule.c" on line 12357:
     Destination buffer 'buffer' not empty at end of file, emptying.
@@ -1677,7 +1689,9 @@ wouldn't make any sense to the Python interpreter.  But using Argument Clinic
 to run Python blocks lets you use Python as a Python preprocessor!
 
 Since Python comments are different from C comments, Argument Clinic
-blocks embedded in Python files look slightly different.  They look like this::
+blocks embedded in Python files look slightly different.  They look like this:
+
+.. code-block:: python3
 
     #/*[python input]
     #print("def foo(): pass")

Doc/howto/urllib2.rst

@@ -578,7 +578,7 @@ Footnotes
 This document was reviewed and revised by John Lee.
 
 .. [#] Google for example.
-.. [#] Browser sniffing is a very bad practise for website design - building
+.. [#] Browser sniffing is a very bad practice for website design - building
        sites using web standards is much more sensible. Unfortunately a lot of
        sites still send different versions to different browsers.
 .. [#] The user agent for MSIE 6 is

Doc/library/asyncio-protocol.rst

@@ -25,6 +25,9 @@ the transport's kind.
 
 The transport classes are :ref:`not thread safe <asyncio-multithreading>`.
 
+.. versionchanged:: 3.6
+   The socket option ``TCP_NODELAY`` is now set by default.
+
 
 BaseTransport
 -------------

Doc/library/pkgutil.rst

@@ -224,4 +224,6 @@ support.
 
    If the package cannot be located or loaded, or it uses a :term:`loader`
    which does not support :meth:`get_data <importlib.abc.ResourceLoader.get_data>`,
-   then ``None`` is returned.
+   then ``None`` is returned.  In particular, the :term:`loader` for
+   :term:`namespace packages <namespace package>` does not support
+   :meth:`get_data <importlib.abc.ResourceLoader.get_data>`.

Doc/library/stdtypes.rst

@@ -1644,18 +1644,20 @@ expression support in the :mod:`re` module).
 
    Return true if all characters in the string are decimal
    characters and there is at least one character, false
-   otherwise. Decimal characters are those from general category "Nd". This category
-   includes digit characters, and all characters
-   that can be used to form decimal-radix numbers, e.g. U+0660,
-   ARABIC-INDIC DIGIT ZERO.
+   otherwise. Decimal characters are those that can be used to form
+   numbers in base 10, e.g. U+0660, ARABIC-INDIC DIGIT
+   ZERO.  Formally a decimal character is a character in the Unicode
+   General Category "Nd".
 
 
 .. method:: str.isdigit()
 
    Return true if all characters in the string are digits and there is at least one
    character, false otherwise.  Digits include decimal characters and digits that need
-   special handling, such as the compatibility superscript digits.  Formally, a digit
-   is a character that has the property value Numeric_Type=Digit or Numeric_Type=Decimal.
+   special handling, such as the compatibility superscript digits.
+   This covers digits which cannot be used to form numbers in base 10,
+   like the Kharosthi numbers.  Formally, a digit is a character that has the
+   property value Numeric_Type=Digit or Numeric_Type=Decimal.
 
 
 .. method:: str.isidentifier()
@@ -2199,15 +2201,12 @@ The conversion types are:
 Notes:
 
 (1)
-   The alternate form causes a leading zero (``'0'``) to be inserted between
-   left-hand padding and the formatting of the number if the leading character
-   of the result is not already a zero.
+   The alternate form causes a leading octal specifier (``'0o'``) to be
+   inserted before the first digit.
 
 (2)
    The alternate form causes a leading ``'0x'`` or ``'0X'`` (depending on whether
-   the ``'x'`` or ``'X'`` format was used) to be inserted between left-hand padding
-   and the formatting of the number if the leading character of the result is not
-   already a zero.
+   the ``'x'`` or ``'X'`` format was used) to be inserted before the first digit.
 
 (3)
    The alternate form causes the result to always contain a decimal point, even if
@@ -3303,15 +3302,12 @@ The conversion types are:
 Notes:
 
 (1)
-   The alternate form causes a leading zero (``'0'``) to be inserted between
-   left-hand padding and the formatting of the number if the leading character
-   of the result is not already a zero.
+   The alternate form causes a leading octal specifier (``'0o'``) to be
+   inserted before the first digit.
 
 (2)
    The alternate form causes a leading ``'0x'`` or ``'0X'`` (depending on whether
-   the ``'x'`` or ``'X'`` format was used) to be inserted between left-hand padding
-   and the formatting of the number if the leading character of the result is not
-   already a zero.
+   the ``'x'`` or ``'X'`` format was used) to be inserted before the first digit.
 
 (3)
    The alternate form causes the result to always contain a decimal point, even if

Doc/tutorial/modules.rst

@@ -501,7 +501,7 @@ when the ``from...import`` statement is executed.  (This also works when
 ``__all__`` is defined.)
 
 Although certain modules are designed to export only names that follow certain
-patterns when you use ``import *``, it is still considered bad practise in
+patterns when you use ``import *``, it is still considered bad practice in
 production code.
 
 Remember, there is nothing wrong with using ``from Package import

Doc/whatsnew/3.6.rst

@@ -875,6 +875,9 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
   but that use asyncio to handle them.
   (Contributed by Jim Fulton in :issue:`27392`.)
 
+* ``TCP_NODELAY`` flag is now set for all TCP transports by default.
+  (Contributed by Yury Selivanov in :issue:`27456`.)
+
 
 binascii
 --------

Lib/distutils/command/wininst-14.0-amd64.exe

@@ -1440,6 +1440,4 @@ def _install(_bootstrap_module):
     _setup(_bootstrap_module)
     supported_loaders = _get_supported_file_loaders()
     sys.path_hooks.extend([FileFinder.path_hook(*supported_loaders)])
-    if _os.__name__ == 'nt':
-        sys.meta_path.append(WindowsRegistryFinder)
     sys.meta_path.append(PathFinder)

Lib/distutils/command/wininst-14.0.exe

@@ -313,11 +313,12 @@ def add(self, message):
         # final position in order to prevent race conditions with changes
         # from other programs
         try:
-            if hasattr(os, 'link'):
+            try:
                 os.link(tmp_file.name, dest)
-                os.remove(tmp_file.name)
-            else:
+            except (AttributeError, PermissionError):
                 os.rename(tmp_file.name, dest)
+            else:
+                os.remove(tmp_file.name)
         except OSError as e:
             os.remove(tmp_file.name)
             if e.errno == errno.EEXIST:
@@ -1200,13 +1201,14 @@ def pack(self):
         for key in self.iterkeys():
             if key - 1 != prev:
                 changes.append((key, prev + 1))
-                if hasattr(os, 'link'):
+                try:
                     os.link(os.path.join(self._path, str(key)),
                             os.path.join(self._path, str(prev + 1)))
-                    os.unlink(os.path.join(self._path, str(key)))
-                else:
+                except (AttributeError, PermissionError):
                     os.rename(os.path.join(self._path, str(key)),
                               os.path.join(self._path, str(prev + 1)))
+                else:
+                    os.unlink(os.path.join(self._path, str(key)))
             prev += 1
         self._next_key = prev + 1
         if len(changes) == 0:
@@ -2076,13 +2078,14 @@ def _lock_file(f, dotlock=True):
                 else:
                     raise
             try:
-                if hasattr(os, 'link'):
+                try:
                     os.link(pre_lock.name, f.name + '.lock')
                     dotlock_done = True
-                    os.unlink(pre_lock.name)
-                else:
+                except (AttributeError, PermissionError):
                     os.rename(pre_lock.name, f.name + '.lock')
                     dotlock_done = True
+                else:
+                    os.unlink(pre_lock.name)
             except FileExistsError:
                 os.remove(pre_lock.name)
                 raise ExternalClashError('dot lock unavailable: %s' %

Lib/importlib/_bootstrap_external.py

@@ -196,7 +196,7 @@ def get_context(self, method=None):
     def get_start_method(self, allow_none=False):
         return self._name
 
-    def set_start_method(self, method=None):
+    def set_start_method(self, method, force=False):
         raise ValueError('cannot set start method of concrete context')
 
     @property

Lib/mailbox.py

@@ -217,7 +217,7 @@ def prepare(data):
         process.ORIGINAL_DIR = data['orig_dir']
 
     if 'start_method' in data:
-        set_start_method(data['start_method'])
+        set_start_method(data['start_method'], force=True)
 
     if 'init_main_from_name' in data:
         _fixup_main_from_name(data['init_main_from_name'])

Lib/multiprocessing/context.py

@@ -1204,7 +1204,10 @@ def _execute_child(self, args, executable, preexec_fn, close_fds,
                 args = list(args)
 
             if shell:
-                args = ["/bin/sh", "-c"] + args
+                # On Android the default shell is at '/system/bin/sh'.
+                unix_shell = ('/system/bin/sh' if
+                          hasattr(sys, 'getandroidapilevel') else '/bin/sh')
+                args = [unix_shell, "-c"] + args
                 if executable:
                     args[0] = executable
 

Lib/multiprocessing/spawn.py

@@ -3824,6 +3824,19 @@ def test_get_all(self):
             self.assertTrue(methods == ['fork', 'spawn'] or
                             methods == ['fork', 'spawn', 'forkserver'])
 
+    def test_preload_resources(self):
+        if multiprocessing.get_start_method() != 'forkserver':
+            self.skipTest("test only relevant for 'forkserver' method")
+        name = os.path.join(os.path.dirname(__file__), 'mp_preload.py')
+        rc, out, err = test.support.script_helper.assert_python_ok(name)
+        out = out.decode()
+        err = err.decode()
+        if out.rstrip() != 'ok' or err != '':
+            print(out)
+            print(err)
+            self.fail("failed spawning forkserver or grandchild")
+
+
 #
 # Check that killing process does not leak named semaphores
 #

Lib/subprocess.py

@@ -20,6 +20,7 @@
 import unittest
 
 from test import support
+android_not_root = support.android_not_root
 
 @contextlib.contextmanager
 def kill_on_error(proc):
@@ -311,6 +312,7 @@ def test_accept(self):
     # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203162
     @support.requires_freebsd_version(10, 3)
     @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()')
+    @unittest.skipIf(android_not_root, "mkfifo not allowed, non root user")
     def _test_open(self, do_open_close_reader, do_open_close_writer):
         filename = support.TESTFN
 

Lib/test/_test_multiprocessing.py

@@ -0,0 +1,18 @@
+import multiprocessing
+
+multiprocessing.Lock()
+
+
+def f():
+    print("ok")
+
+
+if __name__ == "__main__":
+    ctx = multiprocessing.get_context("forkserver")
+    modname = "test.mp_preload"
+    # Make sure it's importable
+    __import__(modname)
+    ctx.set_forkserver_preload([modname])
+    proc = ctx.Process(target=f)
+    proc.start()
+    proc.join()