Merge branch 'main' into issue-50977

Author: kostyafarber

.github/actions/setup-conda/action.yml

@@ -18,7 +18,7 @@ runs:
     - name: Set Arrow version in ${{ inputs.environment-file }} to ${{ inputs.pyarrow-version }}
       run: |
         grep -q '  - pyarrow' ${{ inputs.environment-file }}
-        sed -i"" -e "s/  - pyarrow/  - pyarrow=${{ inputs.pyarrow-version }}/" ${{ inputs.environment-file }}
+        sed -i"" -e "s/  - pyarrow<11/  - pyarrow=${{ inputs.pyarrow-version }}/" ${{ inputs.environment-file }}
         cat ${{ inputs.environment-file }}
       shell: bash
       if: ${{ inputs.pyarrow-version }}

.pre-commit-config.yaml

@@ -135,7 +135,7 @@ repos:
         types: [python]
         stages: [manual]
         additional_dependencies: &pyright_dependencies
-        - [email protected].276
+        - [email protected].284
     -   id: pyright_reportGeneralTypeIssues
         # note: assumes python env is setup and activated
         name: pyright reportGeneralTypeIssues

ci/code_checks.sh

@@ -187,7 +187,6 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
         pandas.show_versions \
         pandas.test \
         pandas.NaT \
-        pandas.Timestamp.unit \
         pandas.Timestamp.as_unit \
         pandas.Timestamp.ctime \
         pandas.Timestamp.date \
@@ -579,13 +578,11 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
 
     MSG='Partially validate docstrings (EX02)' ;  echo $MSG
     $BASE_DIR/scripts/validate_docstrings.py --format=actions --errors=EX02 --ignore_functions \
-        pandas.DataFrame.copy \
         pandas.DataFrame.plot.line \
         pandas.DataFrame.std \
         pandas.DataFrame.var \
         pandas.Index.factorize \
         pandas.Period.strftime \
-        pandas.Series.copy \
         pandas.Series.factorize \
         pandas.Series.floordiv \
         pandas.Series.plot.line \

ci/deps/actions-310.yaml

@@ -42,7 +42,7 @@ dependencies:
   - psycopg2
   - pymysql
   - pytables
-  - pyarrow
+  - pyarrow<11
   - pyreadstat
   - python-snappy
   - pyxlsb

ci/deps/actions-311.yaml

@@ -42,7 +42,7 @@ dependencies:
   - psycopg2
   - pymysql
   # - pytables>=3.8.0  # first version that supports 3.11
-  - pyarrow
+  - pyarrow<11
   - pyreadstat
   - python-snappy
   - pyxlsb

ci/deps/actions-38-downstream_compat.yaml

@@ -40,7 +40,7 @@ dependencies:
   - openpyxl
   - odfpy
   - psycopg2
-  - pyarrow
+  - pyarrow<11
   - pymysql
   - pyreadstat
   - pytables

ci/deps/actions-38.yaml

@@ -40,7 +40,7 @@ dependencies:
   - odfpy
   - pandas-gbq
   - psycopg2
-  - pyarrow
+  - pyarrow<11
   - pymysql
   - pyreadstat
   - pytables

ci/deps/actions-39.yaml

@@ -41,7 +41,7 @@ dependencies:
   - pandas-gbq
   - psycopg2
   - pymysql
-  - pyarrow
+  - pyarrow<11
   - pyreadstat
   - pytables
   - python-snappy

ci/deps/circle-38-arm64.yaml

@@ -40,7 +40,7 @@ dependencies:
   - odfpy
   - pandas-gbq
   - psycopg2
-  - pyarrow
+  - pyarrow<11
   - pymysql
   # Not provided on ARM
   #- pyreadstat

doc/source/development/internals.rst

@@ -15,24 +15,21 @@ Indexing
 In pandas there are a few objects implemented which can serve as valid
 containers for the axis labels:
 
-* ``Index``: the generic "ordered set" object, an ndarray of object dtype
+* :class:`Index`: the generic "ordered set" object, an ndarray of object dtype
   assuming nothing about its contents. The labels must be hashable (and
   likely immutable) and unique. Populates a dict of label to location in
   Cython to do ``O(1)`` lookups.
-* ``Int64Index``: a version of ``Index`` highly optimized for 64-bit integer
-  data, such as time stamps
-* ``Float64Index``: a version of ``Index`` highly optimized for 64-bit float data
-* ``MultiIndex``: the standard hierarchical index object
-* ``DatetimeIndex``: An Index object with ``Timestamp`` boxed elements (impl are the int64 values)
-* ``TimedeltaIndex``: An Index object with ``Timedelta`` boxed elements (impl are the in64 values)
-* ``PeriodIndex``: An Index object with Period elements
+* :class:`MultiIndex`: the standard hierarchical index object
+* :class:`DatetimeIndex`: An Index object with :class:`Timestamp` boxed elements (impl are the int64 values)
+* :class:`TimedeltaIndex`: An Index object with :class:`Timedelta` boxed elements (impl are the in64 values)
+* :class:`PeriodIndex`: An Index object with Period elements
 
 There are functions that make the creation of a regular index easy:
 
-* ``date_range``: fixed frequency date range generated from a time rule or
+* :func:`date_range`: fixed frequency date range generated from a time rule or
   DateOffset. An ndarray of Python datetime objects
-* ``period_range``: fixed frequency date range generated from a time rule or
-  DateOffset. An ndarray of ``Period`` objects, representing timespans
+* :func:`period_range`: fixed frequency date range generated from a time rule or
+  DateOffset. An ndarray of :class:`Period` objects, representing timespans
 
 The motivation for having an ``Index`` class in the first place was to enable
 different implementations of indexing. This means that it's possible for you,
@@ -43,28 +40,28 @@ From an internal implementation point of view, the relevant methods that an
 ``Index`` must define are one or more of the following (depending on how
 incompatible the new object internals are with the ``Index`` functions):
 
-* ``get_loc``: returns an "indexer" (an integer, or in some cases a
+* :meth:`~Index.get_loc`: returns an "indexer" (an integer, or in some cases a
   slice object) for a label
-* ``slice_locs``: returns the "range" to slice between two labels
-* ``get_indexer``: Computes the indexing vector for reindexing / data
+* :meth:`~Index.slice_locs`: returns the "range" to slice between two labels
+* :meth:`~Index.get_indexer`: Computes the indexing vector for reindexing / data
   alignment purposes. See the source / docstrings for more on this
-* ``get_indexer_non_unique``: Computes the indexing vector for reindexing / data
+* :meth:`~Index.get_indexer_non_unique`: Computes the indexing vector for reindexing / data
   alignment purposes when the index is non-unique. See the source / docstrings
   for more on this
-* ``reindex``: Does any pre-conversion of the input index then calls
+* :meth:`~Index.reindex`: Does any pre-conversion of the input index then calls
   ``get_indexer``
-* ``union``, ``intersection``: computes the union or intersection of two
+* :meth:`~Index.union`, :meth:`~Index.intersection`: computes the union or intersection of two
   Index objects
-* ``insert``: Inserts a new label into an Index, yielding a new object
-* ``delete``: Delete a label, yielding a new object
-* ``drop``: Deletes a set of labels
-* ``take``: Analogous to ndarray.take
+* :meth:`~Index.insert`: Inserts a new label into an Index, yielding a new object
+* :meth:`~Index.delete`: Delete a label, yielding a new object
+* :meth:`~Index.drop`: Deletes a set of labels
+* :meth:`~Index.take`: Analogous to ndarray.take
 
 MultiIndex
 ~~~~~~~~~~
 
-Internally, the ``MultiIndex`` consists of a few things: the **levels**, the
-integer **codes** (until version 0.24 named *labels*), and the level **names**:
+Internally, the :class:`MultiIndex` consists of a few things: the **levels**, the
+integer **codes**, and the level **names**:
 
 .. ipython:: python
 
@@ -80,13 +77,13 @@ You can probably guess that the codes determine which unique element is
 identified with that location at each layer of the index. It's important to
 note that sortedness is determined **solely** from the integer codes and does
 not check (or care) whether the levels themselves are sorted. Fortunately, the
-constructors ``from_tuples`` and ``from_arrays`` ensure that this is true, but
-if you compute the levels and codes yourself, please be careful.
+constructors :meth:`~MultiIndex.from_tuples` and :meth:`~MultiIndex.from_arrays` ensure
+that this is true, but if you compute the levels and codes yourself, please be careful.
 
 Values
 ~~~~~~
 
-pandas extends NumPy's type system with custom types, like ``Categorical`` or
+pandas extends NumPy's type system with custom types, like :class:`Categorical` or
 datetimes with a timezone, so we have multiple notions of "values". For 1-D
 containers (``Index`` classes and ``Series``) we have the following convention:
 

doc/source/getting_started/comparison/includes/copies.rst

@@ -11,13 +11,3 @@ or overwrite the original one:
    .. code-block:: python
 
       df = df.sort_values("col1")
-
-.. note::
-
-   You will see an ``inplace=True`` keyword argument available for some methods:
-
-   .. code-block:: python
-
-      df.sort_values("col1", inplace=True)
-
-   Its use is discouraged. :ref:`More information. <indexing.view_versus_copy>`

doc/source/user_guide/advanced.rst

@@ -609,7 +609,7 @@ are named.
 
 .. ipython:: python
 
-   s.index.set_names(["L1", "L2"], inplace=True)
+   s.index = s.index.set_names(["L1", "L2"])
    s.sort_index(level="L1")
    s.sort_index(level="L2")
 
@@ -848,125 +848,35 @@ values **not** in the categories, similarly to how you can reindex **any** panda
 
 .. _advanced.rangeindex:
 
-Int64Index and RangeIndex
-~~~~~~~~~~~~~~~~~~~~~~~~~
+RangeIndex
+~~~~~~~~~~
 
-.. deprecated:: 1.4.0
-    In pandas 2.0, :class:`Index` will become the default index type for numeric types
-    instead of ``Int64Index``, ``Float64Index`` and ``UInt64Index`` and those index types
-    are therefore deprecated and will be removed in a futire version.
-    ``RangeIndex`` will not be removed, as it represents an optimized version of an integer index.
-
-:class:`Int64Index` is a fundamental basic index in pandas. This is an immutable array
-implementing an ordered, sliceable set.
-
-:class:`RangeIndex` is a sub-class of ``Int64Index``  that provides the default index for all ``NDFrame`` objects.
-``RangeIndex`` is an optimized version of ``Int64Index`` that can represent a monotonic ordered set. These are analogous to Python `range types <https://docs.python.org/3/library/stdtypes.html#typesseq-range>`__.
-
-.. _advanced.float64index:
-
-Float64Index
-~~~~~~~~~~~~
-
-.. deprecated:: 1.4.0
-    :class:`Index` will become the default index type for numeric types in the future
-    instead of ``Int64Index``, ``Float64Index`` and ``UInt64Index`` and those index types
-    are therefore deprecated and will be removed in a future version of Pandas.
-    ``RangeIndex`` will not be removed as it represents an optimized version of an integer index.
-
-By default a :class:`Float64Index` will be automatically created when passing floating, or mixed-integer-floating values in index creation.
-This enables a pure label-based slicing paradigm that makes ``[],ix,loc`` for scalar indexing and slicing work exactly the
-same.
-
-.. ipython:: python
-
-   indexf = pd.Index([1.5, 2, 3, 4.5, 5])
-   indexf
-   sf = pd.Series(range(5), index=indexf)
-   sf
-
-Scalar selection for ``[],.loc`` will always be label based. An integer will match an equal float index (e.g. ``3`` is equivalent to ``3.0``).
+:class:`RangeIndex` is a sub-class of :class:`Index`  that provides the default index for all :class:`DataFrame` and :class:`Series` objects.
+``RangeIndex`` is an optimized version of ``Index`` that can represent a monotonic ordered set. These are analogous to Python `range types <https://docs.python.org/3/library/stdtypes.html#typesseq-range>`__.
+A ``RangeIndex`` will always have an ``int64`` dtype.
 
 .. ipython:: python
 
-   sf[3]
-   sf[3.0]
-   sf.loc[3]
-   sf.loc[3.0]
+   idx = pd.RangeIndex(5)
+   idx
 
-The only positional indexing is via ``iloc``.
+``RangeIndex`` is the default index for all :class:`DataFrame` and :class:`Series` objects:
 
 .. ipython:: python
 
-   sf.iloc[3]
+   ser = pd.Series([1, 2, 3])
+   ser.index
+   df = pd.DataFrame([[1, 2], [3, 4]])
+   df.index
+   df.columns
 
-A scalar index that is not found will raise a ``KeyError``.
-Slicing is primarily on the values of the index when using ``[],ix,loc``, and
-**always** positional when using ``iloc``. The exception is when the slice is
-boolean, in which case it will always be positional.
-
-.. ipython:: python
-
-   sf[2:4]
-   sf.loc[2:4]
-   sf.iloc[2:4]
-
-In float indexes, slicing using floats is allowed.
-
-.. ipython:: python
-
-   sf[2.1:4.6]
-   sf.loc[2.1:4.6]
-
-In non-float indexes, slicing using floats will raise a ``TypeError``.
-
-.. code-block:: ipython
-
-   In [1]: pd.Series(range(5))[3.5]
-   TypeError: the label [3.5] is not a proper indexer for this index type (Int64Index)
-
-   In [1]: pd.Series(range(5))[3.5:4.5]
-   TypeError: the slice start [3.5] is not a proper indexer for this index type (Int64Index)
-
-Here is a typical use-case for using this type of indexing. Imagine that you have a somewhat
-irregular timedelta-like indexing scheme, but the data is recorded as floats. This could, for
-example, be millisecond offsets.
-
-.. ipython:: python
-
-   dfir = pd.concat(
-       [
-           pd.DataFrame(
-               np.random.randn(5, 2), index=np.arange(5) * 250.0, columns=list("AB")
-           ),
-           pd.DataFrame(
-               np.random.randn(6, 2),
-               index=np.arange(4, 10) * 250.1,
-               columns=list("AB"),
-           ),
-       ]
-   )
-   dfir
-
-Selection operations then will always work on a value basis, for all selection operators.
-
-.. ipython:: python
-
-   dfir[0:1000.4]
-   dfir.loc[0:1001, "A"]
-   dfir.loc[1000.4]
-
-You could retrieve the first 1 second (1000 ms) of data as such:
-
-.. ipython:: python
-
-   dfir[0:1000]
-
-If you need integer based selection, you should use ``iloc``:
+A ``RangeIndex`` will behave similarly to a :class:`Index` with an ``int64`` dtype and operations on a ``RangeIndex``,
+whose result cannot be represented by a ``RangeIndex``, but should have an integer dtype, will be converted to an ``Index`` with ``int64``.
+For example:
 
 .. ipython:: python
 
-   dfir.iloc[0:5]
+   idx[[0, 2]]
 
 
 .. _advanced.intervalindex:

doc/source/user_guide/basics.rst

@@ -1479,11 +1479,6 @@ you specify a single ``mapper`` and the ``axis`` to apply that mapping to.
    df.rename({"one": "foo", "two": "bar"}, axis="columns")
    df.rename({"a": "apple", "b": "banana", "d": "durian"}, axis="index")
 
-
-The :meth:`~DataFrame.rename` method also provides an ``inplace`` named
-parameter that is by default ``False`` and copies the underlying data. Pass
-``inplace=True`` to rename the data in place.
-
 Finally, :meth:`~Series.rename` also accepts a scalar or list-like
 for altering the ``Series.name`` attribute.
 

doc/source/user_guide/categorical.rst

@@ -437,9 +437,9 @@ meaning and certain operations are possible. If the categorical is unordered, ``
 .. ipython:: python
 
     s = pd.Series(pd.Categorical(["a", "b", "c", "a"], ordered=False))
-    s.sort_values(inplace=True)
+    s = s.sort_values()
     s = pd.Series(["a", "b", "c", "a"]).astype(CategoricalDtype(ordered=True))
-    s.sort_values(inplace=True)
+    s = s.sort_values()
     s
     s.min(), s.max()
 
@@ -459,7 +459,7 @@ This is even true for strings and numeric data:
     s = pd.Series([1, 2, 3, 1], dtype="category")
     s = s.cat.set_categories([2, 3, 1], ordered=True)
     s
-    s.sort_values(inplace=True)
+    s = s.sort_values()
     s
     s.min(), s.max()
 
@@ -477,7 +477,7 @@ necessarily make the sort order the same as the categories order.
     s = pd.Series([1, 2, 3, 1], dtype="category")
     s = s.cat.reorder_categories([2, 3, 1], ordered=True)
     s
-    s.sort_values(inplace=True)
+    s = s.sort_values()
     s
     s.min(), s.max()