Skip to main content
`Ctrl`+`K`

# Version 0.13.0 (January 3, 2014)

This is a major release from 0.12.0 and includes a number of API changes, several new features and enhancements along with a large number of bug fixes.

Highlights include:

- support for a new index type `Float64Index`, and other Indexing enhancements

- `HDFStore` has a new string based syntax for query specification

- support for new methods of interpolation

- updated `timedelta` operations

- a new string manipulation method `extract`

- Nanosecond support for Offsets

- `isin` for DataFrames

Several experimental features are added, including:

- new `eval/query` methods for expression evaluation

- support for `msgpack` serialization

- an i/o interface to Google’s `BigQuery`

Their are several new or updated docs sections including:

- Comparison with SQL, which should be useful for those familiar with SQL but still learning pandas.

- Comparison with R, idiom translations from R to pandas.

- Enhancing Performance, ways to enhance pandas performance with `eval/query`.

Warning

In 0.13.0 `Series` has internally been refactored to no longer sub-class `ndarray` but instead subclass `NDFrame`, similar to the rest of the pandas containers. This should be a transparent change with only very limited API implications. See Internal Refactoring

## API changes

- `read_excel` now supports an integer in its `sheetname` argument giving the index of the sheet to read in (GH 4301).

- Text parser now treats anything that reads like inf (“inf”, “Inf”, “-Inf”, “iNf”, etc.) as infinity. (GH 4220, GH 4219), affecting `read_table`, `read_csv`, etc.

- `pandas` now is Python 2/3 compatible without the need for 2to3 thanks to @jtratner. As a result, pandas now uses iterators more extensively. This also led to the introduction of substantive parts of the Benjamin Peterson’s `six` library into compat. (GH 4384, GH 4375, GH 4372)

- `pandas.util.compat` and `pandas.util.py3compat` have been merged into `pandas.compat`. `pandas.compat` now includes many functions allowing 2/3 compatibility. It contains both list and iterator versions of range, filter, map and zip, plus other necessary elements for Python 3 compatibility. `lmap`, `lzip`, `lrange` and `lfilter` all produce lists instead of iterators, for compatibility with `numpy`, subscripting and `pandas` constructors.(GH 4384, GH 4375, GH 4372)

- `Series.get` with negative indexers now returns the same as `[]` (GH 4390)

- Changes to how `Index` and `MultiIndex` handle metadata (`levels`, `labels`, and `names`) (GH 4039):

      # previously, you would have set levels or labels directly
      >>> pd.index.levels = [[1, 2, 3, 4], [1, 2, 4, 4]]

      # now, you use the set_levels or set_labels methods
      >>> index = pd.index.set_levels([[1, 2, 3, 4], [1, 2, 4, 4]])

      # similarly, for names, you can rename the object
      # but setting names is not deprecated
      >>> index = pd.index.set_names(["bob", "cranberry"])

      # and all methods take an inplace kwarg - but return None
      >>> pd.index.set_names(["bob", "cranberry"], inplace=True)

- All division with `NDFrame` objects is now truedivision, regardless of the future import. This means that operating on pandas objects will by default use floating point division, and return a floating point dtype. You can use `//` and `floordiv` to do integer division.

  Integer division

      In [3]: arr = np.array([1, 2, 3, 4])

      In [4]: arr2 = np.array([5, 3, 2, 1])

      In [5]: arr / arr2
      Out[5]: array([0, 0, 1, 4])

      In [6]: pd.Series(arr) // pd.Series(arr2)
      Out[6]:
      0    0
      1    0
      2    1
      3    4
      dtype: int64

  True Division

      In [7]: pd.Series(arr) / pd.Series(arr2)  # no future import required
      Out[7]:
      0    0.200000
      1    0.666667
      2    1.500000
      3    4.000000
      dtype: float64

- Infer and downcast dtype if `downcast='infer'` is passed to `fillna/ffill/bfill` (GH 4604)

- `__nonzero__` for all NDFrame objects, will now raise a `ValueError`, this reverts back to (GH 1073, GH 4633) behavior. See gotchas for a more detailed discussion.

  This prevents doing boolean comparison on entire pandas objects, which is inherently ambiguous. These all will raise a `ValueError`.

      >>> df = pd.DataFrame({'A': np.random.randn(10),
      ...                    'B': np.random.randn(10),
      ...                    'C': pd.date_range('20130101', periods=10)
      ...                    })
      ...
      >>> if df:
      ...     pass
      ...
      Traceback (most recent call last):
          ...
      ValueError: The truth value of a DataFrame is ambiguous.  Use a.empty,
      a.bool(), a.item(), a.any() or a.all().

      >>> df1 = df
      >>> df2 = df
      >>> df1 and df2
      Traceback (most recent call last):
          ...
      ValueError: The truth value of a DataFrame is ambiguous.  Use a.empty,
      a.bool(), a.item(), a.any() or a.all().

      >>> d = [1, 2, 3]
      >>> s1 = pd.Series(d)
      >>> s2 = pd.Series(d)
      >>> s1 and s2
      Traceback (most recent call last):
          ...
      ValueError: The truth value of a DataFrame is ambiguous.  Use a.empty,
      a.bool(), a.item(), a.any() or a.all().

  Added the `.bool()` method to `NDFrame` objects to facilitate evaluating of single-element boolean Series:

      >>> pd.Series([True]).bool()
       True
      >>> pd.Series([False]).bool()
       False
      >>> pd.DataFrame([[True]]).bool()
       True
      >>> pd.DataFrame([[False]]).bool()
       False

- All non-Index NDFrames (`Series`, `DataFrame`, `Panel`, `Panel4D`, `SparsePanel`, etc.), now support the entire set of arithmetic operators and arithmetic flex methods (add, sub, mul, etc.). `SparsePanel` does not support `pow` or `mod` with non-scalars. (GH 3765)

- `Series` and `DataFrame` now have a `mode()` method to calculate the statistical mode(s) by axis/Series. (GH 5367)

- Chained assignment will now by default warn if the user is assigning to a copy. This can be changed with the option `mode.chained_assignment`, allowed options are `raise/warn/None`.

      In [1]: dfc = pd.DataFrame({'A': ['aaa', 'bbb', 'ccc'], 'B': [1, 2, 3]})

      In [2]: pd.set_option('chained_assignment', 'warn')

  The following warning / exception will show if this is attempted.

      In [3]: dfc.loc[0]['B'] = 1111

      Traceback (most recent call last)
         ...
      SettingWithCopyWarning:
         A value is trying to be set on a copy of a slice from a DataFrame.
         Try using .loc[row_index,col_indexer] = value instead

  Here is the correct method of assignment.

      In [4]: dfc.loc[0, 'B'] = 1111

      In [5]: dfc
      Out[5]:
           A     B
      0  aaa  1111
      1  bbb     2
      2  ccc     3

- `Panel.reindex` has the following call signature `Panel.reindex(items=None, major_axis=None, minor_axis=None, **kwargs)`

    to conform with other `NDFrame` objects. See Internal Refactoring for more information.

- `Series.argmin` and `Series.argmax` are now aliased to `Series.idxmin` and `Series.idxmax`. These return the index of the

    min or max element respectively. Prior to 0.13.0 these would return the position of the min / max element. (GH 6214)

## Prior version deprecations/changes

These were announced changes in 0.12 or prior that are taking effect as of 0.13.0

- Remove deprecated `Factor` (GH 3650)

- Remove deprecated `set_printoptions/reset_printoptions` (GH 3046)

- Remove deprecated `_verbose_info` (GH 3215)

- Remove deprecated `read_clipboard/to_clipboard/ExcelFile/ExcelWriter` from `pandas.io.parsers` (GH 3717) These are available as functions in the main pandas namespace (e.g. `pd.read_clipboard`)

- default for `tupleize_cols` is now `False` for both `to_csv` and `read_csv`. Fair warning in 0.12 (GH 3604)

- default for `display.max_seq_len` is now 100 rather than `None`. This activates truncated display (”…”) of long sequences in various places. (GH 3391)

## Deprecations

Deprecated in 0.13.0

- deprecated `iterkv`, which will be removed in a future release (this was an alias of iteritems used to bypass `2to3`’s changes). (GH 4384, GH 4375, GH 4372)

- deprecated the string method `match`, whose role is now performed more idiomatically by `extract`. In a future release, the default behavior of `match` will change to become analogous to `contains`, which returns a boolean indexer. (Their distinction is strictness: `match` relies on `re.match` while `contains` relies on `re.search`.) In this release, the deprecated behavior is the default, but the new behavior is available through the keyword argument `as_indexer=True`.

## Indexing API changes

Prior to 0.13, it was impossible to use a label indexer (`.loc/.ix`) to set a value that was not contained in the index of a particular axis. (GH 2578). See the docs

In the `Series` case this is effectively an appending operation

    In [6]: s = pd.Series([1, 2, 3])

    In [7]: s
    Out[7]:
    0    1
    1    2
    2    3
    dtype: int64

    In [8]: s[5] = 5.

    In [9]: s
    Out[9]:
    0    1
    1    2
    2    3
    5    5
    dtype: int64

    In [10]: dfi = pd.DataFrame(np.arange(6).reshape(3, 2),
       ....:                    columns=['A', 'B'])
       ....:

    In [11]: dfi
    Out[11]:
       A  B
    0  0  1
    1  2  3
    2  4  5

This would previously `KeyError`

    In [12]: dfi.loc[:, 'C'] = dfi.loc[:, 'A']

    In [13]: dfi
    Out[13]:
       A  B  C
    0  0  1  0
    1  2  3  2
    2  4  5  4

This is like an `append` operation.

    In [14]: dfi.loc[3] = 5

    In [15]: dfi
    Out[15]:
       A  B  C
    0  0  1  0
    1  2  3  2
    2  4  5  4
    3  5  5  5

A Panel setting operation on an arbitrary axis aligns the input to the Panel

    In [20]: p = pd.Panel(np.arange(16).reshape(2, 4, 2),
       ....:              items=['Item1', 'Item2'],
       ....:              major_axis=pd.date_range('2001/1/12', periods=4),
       ....:              minor_axis=['A', 'B'], dtype='float64')
       ....:

    In [21]: p
    Out[21]:
    <class 'pandas.core.panel.Panel'>
    Dimensions: 2 (items) x 4 (major_axis) x 2 (minor_axis)
    Items axis: Item1 to Item2
    Major_axis axis: 2001-01-12 00:00:00 to 2001-01-15 00:00:00
    Minor_axis axis: A to B

    In [22]: p.loc[:, :, 'C'] = pd.Series([30, 32], index=p.items)

    In [23]: p
    Out[23]:
    <class 'pandas.core.panel.Panel'>
    Dimensions: 2 (items) x 4 (major_axis) x 3 (minor_axis)
    Items axis: Item1 to Item2
    Major_axis axis: 2001-01-12 00:00:00 to 2001-01-15 00:00:00
    Minor_axis axis: A to C

    In [24]: p.loc[:, :, 'C']
    Out[24]:
                Item1  Item2
    2001-01-12   30.0   32.0
    2001-01-13   30.0   32.0
    2001-01-14   30.0   32.0
    2001-01-15   30.0   32.0

## Float64Index API change

- Added a new index type, `Float64Index`. This will be automatically created when passing floating values in index creation. This enables a pure label-based slicing paradigm that makes `[],ix,loc` for scalar indexing and slicing work exactly the same. (GH 263)

  Construction is by default for floating type values.

      In [16]: index = pd.Index([1.5, 2, 3, 4.5, 5])

      In [17]: index
      Out[17]: Index([1.5, 2.0, 3.0, 4.5, 5.0], dtype='float64')

      In [18]: s = pd.Series(range(5), index=index)

      In [19]: s
      Out[19]:
      1.5    0
      2.0    1
      3.0    2
      4.5    3
      5.0    4
      dtype: int64

  Scalar selection for `[],.ix,.loc` will always be label based. An integer will match an equal float index (e.g. `3` is equivalent to `3.0`)

      In [20]: s[3]
      Out[20]: np.int64(2)

      In [21]: s.loc[3]
      Out[21]: np.int64(2)

  The only positional indexing is via `iloc`

      In [22]: s.iloc[3]
      Out[22]: np.int64(3)

  A scalar index that is not found will raise `KeyError`

  Slicing is ALWAYS on the values of the index, for `[],ix,loc` and ALWAYS positional with `iloc`

      In [23]: s.loc[2:4]
      Out[23]:
      2.0    1
      3.0    2
      dtype: int64

      In [24]: s.iloc[2:4]
      Out[24]:
      3.0    2
      4.5    3
      dtype: int64

  In float indexes, slicing using floats are allowed

      In [25]: s[2.1:4.6]
      Out[25]:
      3.0    2
      4.5    3
      dtype: int64

      In [26]: s.loc[2.1:4.6]
      Out[26]:
      3.0    2
      4.5    3
      dtype: int64

- Indexing on other index types are preserved (and positional fallback for `[],ix`), with the exception, that floating point slicing on indexes on non `Float64Index` will now raise a `TypeError`.

      In [1]: pd.Series(range(5))[3.5]
      TypeError: the label [3.5] is not a proper indexer for this index type (Int64Index)

      In [1]: pd.Series(range(5))[3.5:4.5]
      TypeError: the slice start [3.5] is not a proper indexer for this index type (Int64Index)

  Using a scalar float indexer will be deprecated in a future version, but is allowed for now.

      In [3]: pd.Series(range(5))[3.0]
      Out[3]: 3

## HDFStore API changes

- Query Format Changes. A much more string-like query format is now supported. See the docs.

      In [27]: path = 'test.h5'

      In [28]: dfq = pd.DataFrame(np.random.randn(10, 4),
         ....:                    columns=list('ABCD'),
         ....:                    index=pd.date_range('20130101', periods=10))
         ....:

      In [29]: dfq.to_hdf(path, key='dfq', format='table', data_columns=True)

  Use boolean expressions, with in-line function evaluation.

      In [30]: pd.read_hdf(path, 'dfq',
         ....:             where="index>Timestamp('20130104') & columns=['A', 'B']")
         ....:
      Out[30]:
                         A         B
      2013-01-05 -0.424972  0.567020
      2013-01-06 -0.673690  0.113648
      2013-01-07  0.404705  0.577046
      2013-01-08 -0.370647 -1.157892
      2013-01-09  1.075770 -0.109050
      2013-01-10  0.357021 -0.674600

  Use an inline column reference

      In [31]: pd.read_hdf(path, 'dfq',
         ....:             where="A>0 or C>0")
         ....:
      Out[31]:
                         A         B         C         D
      2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
      2013-01-02  1.212112 -0.173215  0.119209 -1.044236
      2013-01-04  0.721555 -0.706771 -1.039575  0.271860
      2013-01-05 -0.424972  0.567020  0.276232 -1.087401
      2013-01-07  0.404705  0.577046 -1.715002 -1.039268
      2013-01-09  1.075770 -0.109050  1.643563 -1.469388
      2013-01-10  0.357021 -0.674600 -1.776904 -0.968914

- the `format` keyword now replaces the `table` keyword; allowed values are `fixed(f)` or `table(t)` the same defaults as prior < 0.13.0 remain, e.g. `put` implies `fixed` format and `append` implies `table` format. This default format can be set as an option by setting `io.hdf.default_format`.

      In [32]: path = 'test.h5'

      In [33]: df = pd.DataFrame(np.random.randn(10, 2))

      In [34]: df.to_hdf(path, key='df_table', format='table')

      In [35]: df.to_hdf(path, key='df_table2', append=True)

      In [36]: df.to_hdf(path, key='df_fixed')

      In [37]: with pd.HDFStore(path) as store:
         ....:     print(store)
         ....:
      <class 'pandas.HDFStore'>
      File path: test.h5

- Significant table writing performance improvements

- handle a passed `Series` in table format (GH 4330)

- can now serialize a `timedelta64[ns]` dtype in a table (GH 3577), See the docs.

- added an `is_open` property to indicate if the underlying file handle is_open; a closed store will now report ‘CLOSED’ when viewing the store (rather than raising an error) (GH 4409)

- a close of a `HDFStore` now will close that instance of the `HDFStore` but will only close the actual file if the ref count (by `PyTables`) w.r.t. all of the open handles are 0. Essentially you have a local instance of `HDFStore` referenced by a variable. Once you close it, it will report closed. Other references (to the same file) will continue to operate until they themselves are closed. Performing an action on a closed file will raise `ClosedFileError`

      In [38]: path = 'test.h5'

      In [39]: df = pd.DataFrame(np.random.randn(10, 2))

      In [40]: store1 = pd.HDFStore(path)

      In [41]: store2 = pd.HDFStore(path)

      In [42]: store1.append('df', df)

      In [43]: store2.append('df2', df)

      In [44]: store1
      Out[44]:
      <class 'pandas.HDFStore'>
      File path: test.h5

      In [45]: store2
      Out[45]:
      <class 'pandas.HDFStore'>
      File path: test.h5

      In [46]: store1.close()

      In [47]: store2
      Out[47]:
      <class 'pandas.HDFStore'>
      File path: test.h5

      In [48]: store2.close()

      In [49]: store2
      Out[49]:
      <class 'pandas.HDFStore'>
      File path: test.h5

- removed the `_quiet` attribute, replace by a `DuplicateWarning` if retrieving duplicate rows from a table (GH 4367)

- removed the `warn` argument from `open`. Instead a `PossibleDataLossError` exception will be raised if you try to use `mode='w'` with an OPEN file handle (GH 4367)

- allow a passed locations array or mask as a `where` condition (GH 4467). See the docs for an example.

- add the keyword `dropna=True` to `append` to change whether ALL nan rows are not written to the store (default is `True`, ALL nan rows are NOT written), also settable via the option `io.hdf.dropna_table` (GH 4625)

- pass through store creation arguments; can be used to support in-memory stores

## DataFrame repr changes

The HTML and plain text representations of `DataFrame` now show a truncated view of the table once it exceeds a certain size, rather than switching to the short info view (GH 4886, GH 5550). This makes the representation more consistent as small DataFrames get larger.

To get the info view, call `DataFrame.info()`. If you prefer the info view as the repr for large DataFrames, you can set this by running `set_option('display.large_repr', 'info')`.

## Enhancements

- `df.to_clipboard()` learned a new `excel` keyword that let’s you paste df data directly into excel (enabled by default). (GH 5070).

- `read_html` now raises a `URLError` instead of catching and raising a `ValueError` (GH 4303, GH 4305)

- Added a test for `read_clipboard()` and `to_clipboard()` (GH 4282)

- Clipboard functionality now works with PySide (GH 4282)

- Added a more informative error message when plot arguments contain overlapping color and style arguments (GH 4402)

- `to_dict` now takes `records` as a possible out type. Returns an array of column-keyed dictionaries. (GH 4936)

- `NaN` handing in get_dummies (GH 4446) with `dummy_na`

      # previously, nan was erroneously counted as 2 here
      # now it is not counted at all
      In [50]: pd.get_dummies([1, 2, np.nan])
      Out[50]:
           1.0    2.0
      0   True  False
      1  False   True
      2  False  False

      # unless requested
      In [51]: pd.get_dummies([1, 2, np.nan], dummy_na=True)
      Out[51]:
           1.0    2.0    NaN
      0   True  False  False
      1  False   True  False
      2  False  False   True

- `timedelta64[ns]` operations. See the docs.

  Warning

  Most of these operations require `numpy >= 1.7`

  Using the new top-level `to_timedelta`, you can convert a scalar or array from the standard timedelta format (produced by `to_csv`) into a timedelta type (`np.timedelta64` in `nanoseconds`).

      In [53]: pd.to_timedelta('1 days 06:05:01.00003')
      Out[53]: Timedelta('1 days 06:05:01.000030')

      In [54]: pd.to_timedelta('15.5us')
      Out[54]: Timedelta('0 days 00:00:00.000015500')

      In [55]: pd.to_timedelta(['1 days 06:05:01.00003', '15.5us', 'nan'])
      Out[55]: TimedeltaIndex(['1 days 06:05:01.000030', '0 days 00:00:00.000015500', NaT], dtype='timedelta64[ns]', freq=None)

      In [56]: pd.to_timedelta(np.arange(5), unit='s')
      Out[56]:
      TimedeltaIndex(['0 days 00:00:00', '0 days 00:00:01', '0 days 00:00:02',
                      '0 days 00:00:03', '0 days 00:00:04'],
                     dtype='timedelta64[ns]', freq=None)

      In [57]: pd.to_timedelta(np.arange(5), unit='d')
      Out[57]: TimedeltaIndex(['0 days', '1 days', '2 days', '3 days', '4 days'], dtype='timedelta64[ns]', freq=None)

  A Series of dtype `timedelta64[ns]` can now be divided by another `timedelta64[ns]` object, or astyped to yield a `float64` dtyped Series. This is frequency conversion. See the docs for the docs.

      In [52]: import datetime

      In [53]: td = pd.Series(pd.date_range('20130101', periods=4)) - pd.Series(
         ....:     pd.date_range('20121201', periods=4))
         ....:

      In [54]: td[2] += np.timedelta64(datetime.timedelta(minutes=5, seconds=3))

      In [55]: td[3] = np.nan

      In [56]: td
      Out[56]:
      0   31 days 00:00:00
      1   31 days 00:00:00
      2   31 days 00:05:03
      3                NaT
      dtype: timedelta64[us]

      # to days
      In [63]: td / np.timedelta64(1, 'D')
      Out[63]:
      0    31.000000
      1    31.000000
      2    31.003507
      3          NaN
      dtype: float64

      In [64]: td.astype('timedelta64[D]')
      Out[64]:
      0    31.0
      1    31.0
      2    31.0
      3     NaN
      dtype: float64

      # to seconds
      In [65]: td / np.timedelta64(1, 's')
      Out[65]:
      0    2678400.0
      1    2678400.0
      2    2678703.0
      3          NaN
      dtype: float64

      In [66]: td.astype('timedelta64[s]')
      Out[66]:
      0    2678400.0
      1    2678400.0
      2    2678703.0
      3          NaN
      dtype: float64

  Dividing or multiplying a `timedelta64[ns]` Series by an integer or integer Series

      In [57]: td * -1
      Out[57]:
      0   -31 days +00:00:00
      1   -31 days +00:00:00
      2   -32 days +23:54:57
      3                  NaT
      dtype: timedelta64[us]

      In [58]: td * pd.Series([1, 2, 3, 4])
      Out[58]:
      0   31 days 00:00:00
      1   62 days 00:00:00
      2   93 days 00:15:09
      3                NaT
      dtype: timedelta64[us]

  Absolute `DateOffset` objects can act equivalently to `timedeltas`

      In [59]: from pandas import offsets

      In [60]: td + offsets.Minute(5) + offsets.Milli(5)
      Out[60]:
      0   31 days 00:05:00.005000
      1   31 days 00:05:00.005000
      2   31 days 00:10:03.005000
      3                       NaT
      dtype: timedelta64[us]

  Fillna is now supported for timedeltas

      In [61]: td.fillna(pd.Timedelta(0))
      Out[61]:
      0   31 days 00:00:00
      1   31 days 00:00:00
      2   31 days 00:05:03
      3    0 days 00:00:00
      dtype: timedelta64[us]

      In [62]: td.fillna(datetime.timedelta(days=1, seconds=5))
      Out[62]:
      0   31 days 00:00:00
      1   31 days 00:00:00
      2   31 days 00:05:03
      3    1 days 00:00:05
      dtype: timedelta64[us]

  You can do numeric reduction operations on timedeltas.

      In [63]: td.mean()
      Out[63]: Timedelta('31 days 00:01:41')

      In [64]: td.quantile(.1)
      Out[64]: Timedelta('31 days 00:00:00')

- `plot(kind='kde')` now accepts the optional parameters `bw_method` and `ind`, passed to scipy.stats.gaussian_kde() (for scipy >= 0.11.0) to set the bandwidth, and to gkde.evaluate() to specify the indices at which it is evaluated, respectively. See scipy docs. (GH 4298)

- DataFrame constructor now accepts a numpy masked record array (GH 3478)

- The new vectorized string method `extract` return regular expression matches more conveniently.

      In [65]: pd.Series(['a1', 'b2', 'c3']).str.extract('[ab](\\d)')
      Out[65]:
           0
      0    1
      1    2
      2  NaN

  Elements that do not match return `NaN`. Extracting a regular expression with more than one group returns a DataFrame with one column per group.

      In [66]: pd.Series(['a1', 'b2', 'c3']).str.extract('([ab])(\\d)')
      Out[66]:
           0    1
      0    a    1
      1    b    2
      2  NaN  NaN

  Elements that do not match return a row of `NaN`. Thus, a Series of messy strings can be converted into a like-indexed Series or DataFrame of cleaned-up or more useful strings, without necessitating `get()` to access tuples or `re.match` objects.

  Named groups like

      In [67]: pd.Series(['a1', 'b2', 'c3']).str.extract(
         ....:     '(?P<letter>[ab])(?P<digit>\\d)')
         ....:
      Out[67]:
        letter digit
      0      a     1
      1      b     2
      2    NaN   NaN

  and optional groups can also be used.

      In [68]: pd.Series(['a1', 'b2', '3']).str.extract(
         ....:      '(?P<letter>[ab])?(?P<digit>\\d)')
         ....:
      Out[68]:
        letter digit
      0      a     1
      1      b     2
      2    NaN     3

- `read_stata` now accepts Stata 13 format (GH 4291)

- `read_fwf` now infers the column specifications from the first 100 rows of the file if the data has correctly separated and properly aligned columns using the delimiter provided to the function (GH 4488).

- support for nanosecond times as an offset

  Warning

  These operations require `numpy >= 1.7`

  Period conversions in the range of seconds and below were reworked and extended up to nanoseconds. Periods in the nanosecond range are now available.

      In [79]: pd.date_range('2013-01-01', periods=5, freq='5N')
      Out[79]:
      DatetimeIndex([          '2013-01-01 00:00:00',
                     '2013-01-01 00:00:00.000000005',
                     '2013-01-01 00:00:00.000000010',
                     '2013-01-01 00:00:00.000000015',
                     '2013-01-01 00:00:00.000000020'],
                    dtype='datetime64[ns]', freq='5N')

  or with frequency as offset

      In [69]: pd.date_range('2013-01-01', periods=5, freq=pd.offsets.Nano(5))
      Out[69]:
      DatetimeIndex([          '2013-01-01 00:00:00',
                     '2013-01-01 00:00:00.000000005',
                     '2013-01-01 00:00:00.000000010',
                     '2013-01-01 00:00:00.000000015',
                     '2013-01-01 00:00:00.000000020'],
                    dtype='datetime64[ns]', freq='5ns')

  Timestamps can be modified in the nanosecond range

      In [70]: t = pd.Timestamp('20130101 09:01:02')

      In [71]: t + pd.tseries.offsets.Nano(123)
      Out[71]: Timestamp('2013-01-01 09:01:02.000000123')

- A new method, `isin` for DataFrames, which plays nicely with boolean indexing. The argument to `isin`, what we’re comparing the DataFrame to, can be a DataFrame, Series, dict, or array of values. See the docs for more.

  To get the rows where any of the conditions are met:

      In [72]: dfi = pd.DataFrame({'A': [1, 2, 3, 4], 'B': ['a', 'b', 'f', 'n']})

      In [73]: dfi
      Out[73]:
         A  B
      0  1  a
      1  2  b
      2  3  f
      3  4  n

      In [74]: other = pd.DataFrame({'A': [1, 3, 3, 7], 'B': ['e', 'f', 'f', 'e']})

      In [75]: mask = dfi.isin(other)

      In [76]: mask
      Out[76]:
             A      B
      0   True  False
      1  False  False
      2   True   True
      3  False  False

      In [77]: dfi[mask.any(axis=1)]
      Out[77]:
         A  B
      0  1  a
      2  3  f

- `Series` now supports a `to_frame` method to convert it to a single-column DataFrame (GH 5164)

- All R datasets listed here http://stat.ethz.ch/R-manual/R-devel/library/datasets/html/00Index.html can now be loaded into pandas objects

      # note that pandas.rpy was deprecated in v0.16.0
      import pandas.rpy.common as com
      com.load_data('Titanic')

- `tz_localize` can infer a fall daylight savings transition based on the structure of the unlocalized data (GH 4230), see the docs

- `DatetimeIndex` is now in the API documentation, see the docs

- `json_normalize()` is a new method to allow you to create a flat table from semi-structured JSON data. See the docs (GH 1067)

- Added PySide support for the qtpandas DataFrameModel and DataFrameWidget.

- Python csv parser now supports usecols (GH 4335)

- Frequencies gained several new offsets:

  - `LastWeekOfMonth` (GH 4637)

  - `FY5253`, and `FY5253Quarter` (GH 4511)

- DataFrame has a new `interpolate` method, similar to Series (GH 4434, GH 1892)

      In [78]: df = pd.DataFrame({'A': [1, 2.1, np.nan, 4.7, 5.6, 6.8],
         ....:                   'B': [.25, np.nan, np.nan, 4, 12.2, 14.4]})
         ....:

      In [79]: df.interpolate()
      Out[79]:
           A      B
      0  1.0   0.25
      1  2.1   1.50
      2  3.4   2.75
      3  4.7   4.00
      4  5.6  12.20
      5  6.8  14.40

  Additionally, the `method` argument to `interpolate` has been expanded to include `'nearest', 'zero', 'slinear', 'quadratic', 'cubic', 'barycentric', 'krogh', 'piecewise_polynomial', 'pchip', 'polynomial', 'spline'` The new methods require scipy. Consult the Scipy reference guide and documentation for more information about when the various methods are appropriate. See the docs.

  Interpolate now also accepts a `limit` keyword argument. This works similar to `fillna`’s limit:

      In [80]: ser = pd.Series([1, 3, np.nan, np.nan, np.nan, 11])

      In [81]: ser.interpolate(limit=2)
      Out[81]:
      0     1.0
      1     3.0
      2     5.0
      3     7.0
      4     NaN
      5    11.0
      dtype: float64

- Added `wide_to_long` panel data convenience function. See the docs.

      In [82]: np.random.seed(123)

      In [83]: df = pd.DataFrame({"A1970" : {0 : "a", 1 : "b", 2 : "c"},
         ....:                    "A1980" : {0 : "d", 1 : "e", 2 : "f"},
         ....:                    "B1970" : {0 : 2.5, 1 : 1.2, 2 : .7},
         ....:                    "B1980" : {0 : 3.2, 1 : 1.3, 2 : .1},
         ....:                    "X"     : dict(zip(range(3), np.random.randn(3)))
         ....:                   })
         ....:

      In [84]: df["id"] = df.index

      In [85]: df
      Out[85]:
        A1970 A1980  B1970  B1980         X  id
      0     a     d    2.5    3.2 -1.085631   0
      1     b     e    1.2    1.3  0.997345   1
      2     c     f    0.7    0.1  0.282978   2

      In [86]: pd.wide_to_long(df, ["A", "B"], i="id", j="year")
      Out[86]:
                      X  A    B
      id year
      0  1970 -1.085631  a  2.5
      1  1970  0.997345  b  1.2
      2  1970  0.282978  c  0.7
      0  1980 -1.085631  d  3.2
      1  1980  0.997345  e  1.3
      2  1980  0.282978  f  0.1

- `to_csv` now takes a `date_format` keyword argument that specifies how output datetime objects should be formatted. Datetimes encountered in the index, columns, and values will all have this formatting applied. (GH 4313)

- `DataFrame.plot` will scatter plot x versus y by passing `kind='scatter'` (GH 2215)

- Added support for Google Analytics v3 API segment IDs that also supports v2 IDs. (GH 5271)

## Experimental

- The new `eval()` function implements expression evaluation using `numexpr` behind the scenes. This results in large speedups for complicated expressions involving large DataFrames/Series. For example,

      In [87]: nrows, ncols = 20000, 100

      In [88]: df1, df2, df3, df4 = [pd.DataFrame(np.random.randn(nrows, ncols))
         ....:                       for _ in range(4)]
         ....:

      # eval with NumExpr backend
      In [89]: %timeit pd.eval('df1 + df2 + df3 + df4')
      4.18 ms +- 20 us per loop (mean +- std. dev. of 7 runs, 100 loops each)

      # pure Python evaluation
      In [90]: %timeit df1 + df2 + df3 + df4
      2.91 ms +- 21 us per loop (mean +- std. dev. of 7 runs, 100 loops each)

  For more details, see the the docs

- Similar to `pandas.eval`, `DataFrame` has a new `DataFrame.eval` method that evaluates an expression in the context of the `DataFrame`. For example,

      In [91]: df = pd.DataFrame(np.random.randn(10, 2), columns=['a', 'b'])

      In [92]: df.eval('a + b')
      Out[92]:
      0   -0.685204
      1    1.589745
      2    0.325441
      3   -1.784153
      4   -0.432893
      5    0.171850
      6    1.895919
      7    3.065587
      8   -0.092759
      9    1.391365
      dtype: float64

- `query()` method has been added that allows you to select elements of a `DataFrame` using a natural query syntax nearly identical to Python syntax. For example,

      In [93]: n = 20

      In [94]: df = pd.DataFrame(np.random.randint(n, size=(n, 3)), columns=['a', 'b', 'c'])

      In [95]: df.query('a < b < c')
      Out[95]:
          a   b   c
      11  1   5   8
      15  8  16  19

  selects all the rows of `df` where `a < b < c` evaluates to `True`. For more details see the the docs.

- `pd.read_msgpack()` and `pd.to_msgpack()` are now a supported method of serialization of arbitrary pandas (and python objects) in a lightweight portable binary format. See the docs

  Warning

  Since this is an EXPERIMENTAL LIBRARY, the storage format may not be stable until a future release.

      df = pd.DataFrame(np.random.rand(5, 2), columns=list('AB'))
      df.to_msgpack('foo.msg')
      pd.read_msgpack('foo.msg')

      s = pd.Series(np.random.rand(5), index=pd.date_range('20130101', periods=5))
      pd.to_msgpack('foo.msg', df, s)
      pd.read_msgpack('foo.msg')

  You can pass `iterator=True` to iterator over the unpacked results

      for o in pd.read_msgpack('foo.msg', iterator=True):
          print(o)

- `pandas.io.gbq` provides a simple way to extract from, and load data into, Google’s BigQuery Data Sets by way of pandas DataFrames. BigQuery is a high performance SQL-like database service, useful for performing ad-hoc queries against extremely large datasets. See the docs

      from pandas.io import gbq

      # A query to select the average monthly temperatures in the
      # in the year 2000 across the USA. The dataset,
      # publicata:samples.gsod, is available on all BigQuery accounts,
      # and is based on NOAA gsod data.

      query = """SELECT station_number as STATION,
      month as MONTH, AVG(mean_temp) as MEAN_TEMP
      FROM publicdata:samples.gsod
      WHERE YEAR = 2000
      GROUP BY STATION, MONTH
      ORDER BY STATION, MONTH ASC"""

      # Fetch the result set for this query

      # Your Google BigQuery Project ID
      # To find this, see your dashboard:
      # https://console.developers.google.com/iam-admin/projects?authuser=0
      projectid = 'xxxxxxxxx'
      df = gbq.read_gbq(query, project_id=projectid)

      # Use pandas to process and reshape the dataset

      df2 = df.pivot(index='STATION', columns='MONTH', values='MEAN_TEMP')
      df3 = pd.concat([df2.min(), df2.mean(), df2.max()],
                      axis=1, keys=["Min Tem", "Mean Temp", "Max Temp"])

  The resulting DataFrame is:

      > df3
                  Min Tem  Mean Temp    Max Temp
       MONTH
       1     -53.336667  39.827892   89.770968
       2     -49.837500  43.685219   93.437932
       3     -77.926087  48.708355   96.099998
       4     -82.892858  55.070087   97.317240
       5     -92.378261  61.428117  102.042856
       6     -77.703334  65.858888  102.900000
       7     -87.821428  68.169663  106.510714
       8     -89.431999  68.614215  105.500000
       9     -86.611112  63.436935  107.142856
       10    -78.209677  56.880838   92.103333
       11    -50.125000  48.861228   94.996428
       12    -50.332258  42.286879   94.396774

  Warning

  To use this module, you will need a BigQuery account. See <https://cloud.google.com/products/big-query> for details.

  As of 10/10/13, there is a bug in Google’s API preventing result sets from being larger than 100,000 rows. A patch is scheduled for the week of 10/14/13.

## Internal refactoring

In 0.13.0 there is a major refactor primarily to subclass `Series` from `NDFrame`, which is the base class currently for `DataFrame` and `Panel`, to unify methods and behaviors. Series formerly subclassed directly from `ndarray`. (GH 4080, GH 3862, GH 816)

Warning

There are two potential incompatibilities from < 0.13.0

- Using certain numpy functions would previously return a `Series` if passed a `Series` as an argument. This seems only to affect `np.ones_like`, `np.empty_like`, `np.diff` and `np.where`. These now return `ndarrays`.

      In [96]: s = pd.Series([1, 2, 3, 4])

  Numpy Usage

      In [97]: np.ones_like(s)
      Out[97]: array([1, 1, 1, 1])

      In [98]: np.diff(s)
      Out[98]: array([1, 1, 1])

      In [99]: np.where(s > 1, s, np.nan)
      Out[99]: array([nan,  2.,  3.,  4.])

  Pandonic Usage

      In [100]: pd.Series(1, index=s.index)
      Out[100]:
      0    1
      1    1
      2    1
      3    1
      dtype: int64

      In [101]: s.diff()
      Out[101]:
      0    NaN
      1    1.0
      2    1.0
      3    1.0
      dtype: float64

      In [102]: s.where(s > 1)
      Out[102]:
      0    NaN
      1    2.0
      2    3.0
      3    4.0
      dtype: float64

- Passing a `Series` directly to a cython function expecting an `ndarray` type will no long work directly, you must pass `Series.values`, See Enhancing Performance

- `Series(0.5)` would previously return the scalar `0.5`, instead this will return a 1-element `Series`

- This change breaks `rpy2<=2.3.8`. an Issue has been opened against rpy2 and a workaround is detailed in GH 5698. Thanks @JanSchulz.

- Pickle compatibility is preserved for pickles created prior to 0.13. These must be unpickled with `pd.read_pickle`, see Pickling.

- Refactor of series.py/frame.py/panel.py to move common code to generic.py

  - added `_setup_axes` to created generic NDFrame structures

  - moved methods

    - `from_axes,_wrap_array,axes,ix,loc,iloc,shape,empty,swapaxes,transpose,pop`

    - `__iter__,keys,__contains__,__len__,__neg__,__invert__`

    - `convert_objects,as_blocks,as_matrix,values`

    - `__getstate__,__setstate__` (compat remains in frame/panel)

    - `__getattr__,__setattr__`

    - `_indexed_same,reindex_like,align,where,mask`

    - `fillna,replace` (`Series` replace is now consistent with `DataFrame`)

    - `filter` (also added axis argument to selectively filter on a different axis)

    - `reindex,reindex_axis,take`

    - `truncate` (moved to become part of `NDFrame`)

- These are API changes which make `Panel` more consistent with `DataFrame`

  - `swapaxes` on a `Panel` with the same axes specified now return a copy

  - support attribute access for setting

  - filter supports the same API as the original `DataFrame` filter

- Reindex called with no arguments will now return a copy of the input object

- `TimeSeries` is now an alias for `Series`. the property `is_time_series` can be used to distinguish (if desired)

- Refactor of Sparse objects to use BlockManager

  - Created a new block type in internals, `SparseBlock`, which can hold multi-dtypes and is non-consolidatable. `SparseSeries` and `SparseDataFrame` now inherit more methods from there hierarchy (Series/DataFrame), and no longer inherit from `SparseArray` (which instead is the object of the `SparseBlock`)

  - Sparse suite now supports integration with non-sparse data. Non-float sparse data is supportable (partially implemented)

  - Operations on sparse structures within DataFrames should preserve sparseness, merging type operations will convert to dense (and back to sparse), so might be somewhat inefficient

  - enable setitem on `SparseSeries` for boolean/integer/slices

  - `SparsePanels` implementation is unchanged (e.g. not using BlockManager, needs work)

- added `ftypes` method to Series/DataFrame, similar to `dtypes`, but indicates if the underlying is sparse/dense (as well as the dtype)

- All `NDFrame` objects can now use `__finalize__()` to specify various values to propagate to new objects from an existing one (e.g. `name` in `Series` will follow more automatically now)

- Internal type checking is now done via a suite of generated classes, allowing `isinstance(value, klass)` without having to directly import the klass, courtesy of @jtratner

- Bug in Series update where the parent frame is not updating its cache based on changes (GH 4080) or types (GH 3217), fillna (GH 3386)

- Indexing with dtype conversions fixed (GH 4463, GH 4204)

- Refactor `Series.reindex` to core/generic.py (GH 4604, GH 4618), allow `method=` in reindexing on a Series to work

- `Series.copy` no longer accepts the `order` parameter and is now consistent with `NDFrame` copy

- Refactor `rename` methods to core/generic.py; fixes `Series.rename` for (GH 4605), and adds `rename` with the same signature for `Panel`

- Refactor `clip` methods to core/generic.py (GH 4798)

- Refactor of `_get_numeric_data/_get_bool_data` to core/generic.py, allowing Series/Panel functionality

- `Series` (for index) / `Panel` (for items) now allow attribute access to its elements (GH 1903)

      In [103]: s = pd.Series([1, 2, 3], index=list('abc'))

      In [104]: s.b
      Out[104]: np.int64(2)

      In [105]: s.a = 5

      In [106]: s
      Out[106]:
      a    5
      b    2
      c    3
      dtype: int64

## Bug fixes

- `HDFStore`

  - raising an invalid `TypeError` rather than `ValueError` when appending with a different block ordering (GH 4096)

  - `read_hdf` was not respecting as passed `mode` (GH 4504)

  - appending a 0-len table will work correctly (GH 4273)

  - `to_hdf` was raising when passing both arguments `append` and `table` (GH 4584)

  - reading from a store with duplicate columns across dtypes would raise (GH 4767)

  - Fixed a bug where `ValueError` wasn’t correctly raised when column names weren’t strings (GH 4956)

  - A zero length series written in Fixed format not deserializing properly. (GH 4708)

  - Fixed decoding perf issue on pyt3 (GH 5441)

  - Validate levels in a MultiIndex before storing (GH 5527)

  - Correctly handle `data_columns` with a Panel (GH 5717)

- Fixed bug in tslib.tz_convert(vals, tz1, tz2): it could raise IndexError exception while trying to access trans[pos + 1] (GH 4496)

- The `by` argument now works correctly with the `layout` argument (GH 4102, GH 4014) in `*.hist` plotting methods

- Fixed bug in `PeriodIndex.map` where using `str` would return the str representation of the index (GH 4136)

- Fixed test failure `test_time_series_plot_color_with_empty_kwargs` when using custom matplotlib default colors (GH 4345)

- Fix running of stata IO tests. Now uses temporary files to write (GH 4353)

- Fixed an issue where `DataFrame.sum` was slower than `DataFrame.mean` for integer valued frames (GH 4365)

- `read_html` tests now work with Python 2.6 (GH 4351)

- Fixed bug where `network` testing was throwing `NameError` because a local variable was undefined (GH 4381)

- In `to_json`, raise if a passed `orient` would cause loss of data because of a duplicate index (GH 4359)

- In `to_json`, fix date handling so milliseconds are the default timestamp as the docstring says (GH 4362).

- `as_index` is no longer ignored when doing groupby apply (GH 4648, GH 3417)

- JSON NaT handling fixed, NaTs are now serialized to `null` (GH 4498)

- Fixed JSON handling of escapable characters in JSON object keys (GH 4593)

- Fixed passing `keep_default_na=False` when `na_values=None` (GH 4318)

- Fixed bug with `values` raising an error on a DataFrame with duplicate columns and mixed dtypes, surfaced in (GH 4377)

- Fixed bug with duplicate columns and type conversion in `read_json` when `orient='split'` (GH 4377)

- Fixed JSON bug where locales with decimal separators other than ‘.’ threw exceptions when encoding / decoding certain values. (GH 4918)

- Fix `.iat` indexing with a `PeriodIndex` (GH 4390)

- Fixed an issue where `PeriodIndex` joining with self was returning a new instance rather than the same instance (GH 4379); also adds a test for this for the other index types

- Fixed a bug with all the dtypes being converted to object when using the CSV cparser with the usecols parameter (GH 3192)

- Fix an issue in merging blocks where the resulting DataFrame had partially set _ref_locs (GH 4403)

- Fixed an issue where hist subplots were being overwritten when they were called using the top level matplotlib API (GH 4408)

- Fixed a bug where calling `Series.astype(str)` would truncate the string (GH 4405, GH 4437)

- Fixed a py3 compat issue where bytes were being repr’d as tuples (GH 4455)

- Fixed Panel attribute naming conflict if item is named ‘a’ (GH 3440)

- Fixed an issue where duplicate indexes were raising when plotting (GH 4486)

- Fixed an issue where cumsum and cumprod didn’t work with bool dtypes (GH 4170, GH 4440)

- Fixed Panel slicing issued in `xs` that was returning an incorrect dimmed object (GH 4016)

- Fix resampling bug where custom reduce function not used if only one group (GH 3849, GH 4494)

- Fixed Panel assignment with a transposed frame (GH 3830)

- Raise on set indexing with a Panel and a Panel as a value which needs alignment (GH 3777)

- frozenset objects now raise in the `Series` constructor (GH 4482, GH 4480)

- Fixed issue with sorting a duplicate MultiIndex that has multiple dtypes (GH 4516)

- Fixed bug in `DataFrame.set_values` which was causing name attributes to be lost when expanding the index. (GH 3742, GH 4039)

- Fixed issue where individual `names`, `levels` and `labels` could be set on `MultiIndex` without validation (GH 3714, GH 4039)

- Fixed (GH 3334) in pivot_table. Margins did not compute if values is the index.

- Fix bug in having a rhs of `np.timedelta64` or `np.offsets.DateOffset` when operating with datetimes (GH 4532)

- Fix arithmetic with series/datetimeindex and `np.timedelta64` not working the same (GH 4134) and buggy timedelta in NumPy 1.6 (GH 4135)

- Fix bug in `pd.read_clipboard` on windows with PY3 (GH 4561); not decoding properly

- `tslib.get_period_field()` and `tslib.get_period_field_arr()` now raise if code argument out of range (GH 4519, GH 4520)

- Fix boolean indexing on an empty series loses index names (GH 4235), infer_dtype works with empty arrays.

- Fix reindexing with multiple axes; if an axes match was not replacing the current axes, leading to a possible lazy frequency inference issue (GH 3317)

- Fixed issue where `DataFrame.apply` was reraising exceptions incorrectly (causing the original stack trace to be truncated).

- Fix selection with `ix/loc` and non_unique selectors (GH 4619)

- Fix assignment with iloc/loc involving a dtype change in an existing column (GH 4312, GH 5702) have internal setitem_with_indexer in core/indexing to use Block.setitem

- Fixed bug where thousands operator was not handled correctly for floating point numbers in csv_import (GH 4322)

- Fix an issue with CacheableOffset not properly being used by many DateOffset; this prevented the DateOffset from being cached (GH 4609)

- Fix boolean comparison with a DataFrame on the lhs, and a list/tuple on the rhs (GH 4576)

- Fix error/dtype conversion with setitem of `None` on `Series/DataFrame` (GH 4667)

- Fix decoding based on a passed in non-default encoding in `pd.read_stata` (GH 4626)

- Fix `DataFrame.from_records` with a plain-vanilla `ndarray`. (GH 4727)

- Fix some inconsistencies with `Index.rename` and `MultiIndex.rename`, etc. (GH 4718, GH 4628)

- Bug in using `iloc/loc` with a cross-sectional and duplicate indices (GH 4726)

- Bug with using `QUOTE_NONE` with `to_csv` causing `Exception`. (GH 4328)

- Bug with Series indexing not raising an error when the right-hand-side has an incorrect length (GH 2702)

- Bug in MultiIndexing with a partial string selection as one part of a MultIndex (GH 4758)

- Bug with reindexing on the index with a non-unique index will now raise `ValueError` (GH 4746)

- Bug in setting with `loc/ix` a single indexer with a MultiIndex axis and a NumPy array, related to (GH 3777)

- Bug in concatenation with duplicate columns across dtypes not merging with axis=0 (GH 4771, GH 4975)

- Bug in `iloc` with a slice index failing (GH 4771)

- Incorrect error message with no colspecs or width in `read_fwf`. (GH 4774)

- Fix bugs in indexing in a Series with a duplicate index (GH 4548, GH 4550)

- Fixed bug with reading compressed files with `read_fwf` in Python 3. (GH 3963)

- Fixed an issue with a duplicate index and assignment with a dtype change (GH 4686)

- Fixed bug with reading compressed files in as `bytes` rather than `str` in Python 3. Simplifies bytes-producing file-handling in Python 3 (GH 3963, GH 4785).

- Fixed an issue related to ticklocs/ticklabels with log scale bar plots across different versions of matplotlib (GH 4789)

- Suppressed DeprecationWarning associated with internal calls issued by repr() (GH 4391)

- Fixed an issue with a duplicate index and duplicate selector with `.loc` (GH 4825)

- Fixed an issue with `DataFrame.sort_index` where, when sorting by a single column and passing a list for `ascending`, the argument for `ascending` was being interpreted as `True` (GH 4839, GH 4846)

- Fixed `Panel.tshift` not working. Added `freq` support to `Panel.shift` (GH 4853)

- Fix an issue in TextFileReader w/ Python engine (i.e. PythonParser) with thousands != “,” (GH 4596)

- Bug in getitem with a duplicate index when using where (GH 4879)

- Fix Type inference code coerces float column into datetime (GH 4601)

- Fixed `_ensure_numeric` does not check for complex numbers (GH 4902)

- Fixed a bug in `Series.hist` where two figures were being created when the `by` argument was passed (GH 4112, GH 4113).

- Fixed a bug in `convert_objects` for > 2 ndims (GH 4937)

- Fixed a bug in DataFrame/Panel cache insertion and subsequent indexing (GH 4939, GH 5424)

- Fixed string methods for `FrozenNDArray` and `FrozenList` (GH 4929)

- Fixed a bug with setting invalid or out-of-range values in indexing enlargement scenarios (GH 4940)

- Tests for fillna on empty Series (GH 4346), thanks @immerrr

- Fixed `copy()` to shallow copy axes/indices as well and thereby keep separate metadata. (GH 4202, GH 4830)

- Fixed skiprows option in Python parser for read_csv (GH 4382)

- Fixed bug preventing `cut` from working with `np.inf` levels without explicitly passing labels (GH 3415)

- Fixed wrong check for overlapping in `DatetimeIndex.union` (GH 4564)

- Fixed conflict between thousands separator and date parser in csv_parser (GH 4678)

- Fix appending when dtypes are not the same (error showing mixing float/np.datetime64) (GH 4993)

- Fix repr for DateOffset. No longer show duplicate entries in kwds. Removed unused offset fields. (GH 4638)

- Fixed wrong index name during read_csv if using usecols. Applies to c parser only. (GH 4201)

- `Timestamp` objects can now appear in the left hand side of a comparison operation with a `Series` or `DataFrame` object (GH 4982).

- Fix a bug when indexing with `np.nan` via `iloc/loc` (GH 5016)

- Fixed a bug where low memory c parser could create different types in different chunks of the same file. Now coerces to numerical type or raises warning. (GH 3866)

- Fix a bug where reshaping a `Series` to its own shape raised `TypeError` (GH 4554) and other reshaping issues.

- Bug in setting with `ix/loc` and a mixed int/string index (GH 4544)

- Make sure series-series boolean comparisons are label based (GH 4947)

- Bug in multi-level indexing with a Timestamp partial indexer (GH 4294)

- Tests/fix for MultiIndex construction of an all-nan frame (GH 4078)

- Fixed a bug where `read_html()` wasn’t correctly inferring values of tables with commas (GH 5029)

- Fixed a bug where `read_html()` wasn’t providing a stable ordering of returned tables (GH 4770, GH 5029).

- Fixed a bug where `read_html()` was incorrectly parsing when passed `index_col=0` (GH 5066).

- Fixed a bug where `read_html()` was incorrectly inferring the type of headers (GH 5048).

- Fixed a bug where `DatetimeIndex` joins with `PeriodIndex` caused a stack overflow (GH 3899).

- Fixed a bug where `groupby` objects didn’t allow plots (GH 5102).

- Fixed a bug where `groupby` objects weren’t tab-completing column names (GH 5102).

- Fixed a bug where `groupby.plot()` and friends were duplicating figures multiple times (GH 5102).

- Provide automatic conversion of `object` dtypes on fillna, related (GH 5103)

- Fixed a bug where default options were being overwritten in the option parser cleaning (GH 5121).

- Treat a list/ndarray identically for `iloc` indexing with list-like (GH 5006)

- Fix `MultiIndex.get_level_values()` with missing values (GH 5074)

- Fix bound checking for Timestamp() with datetime64 input (GH 4065)

- Fix a bug where `TestReadHtml` wasn’t calling the correct `read_html()` function (GH 5150).

- Fix a bug with `NDFrame.replace()` which made replacement appear as though it was (incorrectly) using regular expressions (GH 5143).

- Fix better error message for to_datetime (GH 4928)

- Made sure different locales are tested on travis-ci (GH 4918). Also adds a couple of utilities for getting locales and setting locales with a context manager.

- Fixed segfault on `isnull(MultiIndex)` (now raises an error instead) (GH 5123, GH 5125)

- Allow duplicate indices when performing operations that align (GH 5185, GH 5639)

- Compound dtypes in a constructor raise `NotImplementedError` (GH 5191)

- Bug in comparing duplicate frames (GH 4421) related

- Bug in describe on duplicate frames

- Bug in `to_datetime` with a format and `coerce=True` not raising (GH 5195)

- Bug in `loc` setting with multiple indexers and a rhs of a Series that needs broadcasting (GH 5206)

- Fixed bug where inplace setting of levels or labels on `MultiIndex` would not clear cached `values` property and therefore return wrong `values`. (GH 5215)

- Fixed bug where filtering a grouped DataFrame or Series did not maintain the original ordering (GH 4621).

- Fixed `Period` with a business date freq to always roll-forward if on a non-business date. (GH 5203)

- Fixed bug in Excel writers where frames with duplicate column names weren’t written correctly. (GH 5235)

- Fixed issue with `drop` and a non-unique index on Series (GH 5248)

- Fixed segfault in C parser caused by passing more names than columns in the file. (GH 5156)

- Fix `Series.isin` with date/time-like dtypes (GH 5021)

- C and Python Parser can now handle the more common MultiIndex column format which doesn’t have a row for index names (GH 4702)

- Bug when trying to use an out-of-bounds date as an object dtype (GH 5312)

- Bug when trying to display an embedded PandasObject (GH 5324)

- Allows operating of Timestamps to return a datetime if the result is out-of-bounds related (GH 5312)

- Fix return value/type signature of `initObjToJSON()` to be compatible with numpy’s `import_array()` (GH 5334, GH 5326)

- Bug when renaming then set_index on a DataFrame (GH 5344)

- Test suite no longer leaves around temporary files when testing graphics. (GH 5347) (thanks for catching this @yarikoptic!)

- Fixed html tests on win32. (GH 4580)

- Make sure that `head/tail` are `iloc` based, (GH 5370)

- Fixed bug for `PeriodIndex` string representation if there are 1 or 2 elements. (GH 5372)

- The GroupBy methods `transform` and `filter` can be used on Series and DataFrames that have repeated (non-unique) indices. (GH 4620)

- Fix empty series not printing name in repr (GH 4651)

- Make tests create temp files in temp directory by default. (GH 5419)

- `pd.to_timedelta` of a scalar returns a scalar (GH 5410)

- `pd.to_timedelta` accepts `NaN` and `NaT`, returning `NaT` instead of raising (GH 5437)

- performance improvements in `isnull` on larger size pandas objects

- Fixed various setitem with 1d ndarray that does not have a matching length to the indexer (GH 5508)

- Bug in getitem with a MultiIndex and `iloc` (GH 5528)

- Bug in delitem on a Series (GH 5542)

- Bug fix in apply when using custom function and objects are not mutated (GH 5545)

- Bug in selecting from a non-unique index with `loc` (GH 5553)

- Bug in groupby returning non-consistent types when user function returns a `None`, (GH 5592)

- Work around regression in numpy 1.7.0 which erroneously raises IndexError from `ndarray.item` (GH 5666)

- Bug in repeated indexing of object with resultant non-unique index (GH 5678)

- Bug in fillna with Series and a passed series/dict (GH 5703)

- Bug in groupby transform with a datetime-like grouper (GH 5712)

- Bug in MultiIndex selection in PY3 when using certain keys (GH 5725)

- Row-wise concat of differing dtypes failing in certain cases (GH 5754)

## Contributors

A total of 77 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time.

- Agustín Herranz +

- Alex Gaudio +

- Alex Rothberg +

- Andreas Klostermann +

- Andreas Würl +

- Andy Hayden

- Ben Alex +

- Benedikt Sauer +

- Brad Buran

- Caleb Epstein +

- Chang She

- Christopher Whelan

- DSM +

- Dale Jung +

- Dan Birken

- David Rasch +

- Dieter Vandenbussche

- Gabi Davar +

- Garrett Drapala

- Goyo +

- Greg Reda +

- Ivan Smirnov +

- Jack Kelly +

- Jacob Schaer +

- Jan Schulz +

- Jeff Tratner

- Jeffrey Tratner

- John McNamara +

- John W. O’Brien +

- Joris Van den Bossche

- Justin Bozonier +

- Kelsey Jordahl

- Kevin Stone

- Kieran O’Mahony

- Kyle Hausmann +

- Kyle Kelley +

- Kyle Meyer

- Mike Kelly

- Mortada Mehyar +

- Nick Foti +

- Olivier Harris +

- Ondřej Čertík +

- PKEuS

- Phillip Cloud

- Pierre Haessig +

- Richard T. Guy +

- Roman Pekar +

- Roy Hyunjin Han

- Skipper Seabold

- Sten +

- Thomas A Caswell +

- Thomas Kluyver

- Tiago Requeijo +

- TomAugspurger

- Trent Hauck

- Valentin Haenel +

- Viktor Kerkez +

- Vincent Arel-Bundock

- Wes McKinney

- Wes Turner +

- Weston Renoud +

- Yaroslav Halchenko

- Zach Dwiel +

- chapman siu +

- chappers +

- d10genes +

- danielballan

- daydreamt +

- engstrom +

- jreback

- monicaBee +

- prossahl +

- rockg +

- unutbu +

- westurner +

- y-p

- zach powers