Skip to content

reshaping module

Functions for reshaping arrays.

Reshape functions transform a Pandas object/NumPy array in some way.

IndexFromLike _UnionGenericAlias

Any object that can be coerced into a index_from argument.

to_1d_array partial

to_1d() with raw enabled.

to_2d_array partial

to_2d() with raw enabled.

to_2d_pc_array partial

to_2d_array with expand_axis=0.

to_2d_pr_array partial

to_2d_array with expand_axis=1.

align_pd_arrays function

align_pd_arrays(
    *args,
    align_index=True,
    align_columns=True,
    to_index=None,
    to_columns=None,
    axis=None,
    reindex_kwargs=None
)

Align Pandas arrays against common index and/or column levels using reindexing and align_indexes() respectively.

broadcast function

broadcast(
    *args,
    to_shape=None,
    align_index=None,
    align_columns=None,
    index_from=None,
    columns_from=None,
    to_frame=None,
    axis=None,
    to_pd=None,
    keep_flex=None,
    min_ndim=None,
    expand_axis=None,
    post_func=None,
    require_kwargs=None,
    reindex_kwargs=None,
    merge_kwargs=None,
    tile=None,
    random_subset=None,
    seed=None,
    keep_wrap_default=None,
    return_wrapper=False,
    wrapper_kwargs=None,
    ignore_sr_names=None,
    ignore_ranges=None,
    check_index_names=None,
    index_stack_kwargs=None,
    template_context=None
)

Bring any array-like object in args to the same shape by using NumPy-like broadcasting.

See Broadcasting.

Important

The major difference to NumPy is that one-dimensional arrays will always broadcast against the row axis!

Can broadcast Pandas objects by broadcasting their index/columns with broadcast_index().

Args

*args

Objects to broadcast.

If the first and only argument is a mapping, will return a dict.

Allows using BCO, Ref, Default, Param, index_dict, IdxSetter, IdxSetterFactory, and templates. If an index dictionary, fills using ArrayWrapper.fill_and_set().

to_shape : tuple of int
Target shape. If set, will broadcast every object in args to to_shape.
align_index : bool

Whether to align index of Pandas objects using union.

Pass None to use the default.

align_columns : bool

Whether to align columns of Pandas objects using multi-index.

Pass None to use the default.

index_from : any

Broadcasting rule for index.

Pass None to use the default.

columns_from : any

Broadcasting rule for columns.

Pass None to use the default.

to_frame : bool
Whether to convert all Series to DataFrames.
axis : int, sequence or mapping
See BCO.axis.
to_pd : bool, sequence or mapping

See BCO.to_pd.

If None, converts only if there is at least one Pandas object among them.

keep_flex : bool, sequence or mapping
See BCO.keep_flex.
min_ndim : int, sequence or mapping

See BCO.min_ndim.

If None, becomes 2 if keep_flex is True, otherwise 1.

expand_axis : int, sequence or mapping
See BCO.expand_axis.
post_func : callable, sequence or mapping

See BCO.post_func.

Applied only when keep_flex is False.

require_kwargs : dict, sequence or mapping

See BCO.require_kwargs.

This key will be merged with any argument-specific dict. If the mapping contains all keys in np.require, it will be applied to all objects.

reindex_kwargs : dict, sequence or mapping

See BCO.reindex_kwargs.

This key will be merged with any argument-specific dict. If the mapping contains all keys in pd.DataFrame.reindex, it will be applied to all objects.

merge_kwargs : dict, sequence or mapping

See BCO.merge_kwargs.

This key will be merged with any argument-specific dict. If the mapping contains all keys in pd.DataFrame.merge, it will be applied to all objects.

tile : int or index_like
Tile the final object by the number of times or index.
random_subset : int

Select a random subset of parameter values.

Seed can be set using NumPy before calling this function.

seed : int
Seed to make output deterministic.
keep_wrap_default : bool
Whether to keep wrapping with Default.
return_wrapper : bool
Whether to also return the wrapper associated with the operation.
wrapper_kwargs : dict
Keyword arguments passed to ArrayWrapper.
ignore_sr_names : bool
See broadcast_index().
ignore_ranges : bool
See broadcast_index().
check_index_names : bool
See broadcast_index().
index_stack_kwargs : dict
Keyword arguments passed to stack_indexes().
template_context : dict
Template context.

For defaults, see broadcasting.

Any keyword argument that can be associated with an object can be passed as

  • a const that is applied to all objects,
  • a sequence with value per object, and
  • a mapping with value per object name and the special key _def denoting the default value.

Additionally, any object can be passed wrapped with BCO, which attributes will override any of the above arguments if not None.

Usage

  • Without broadcasting index and columns:
>>> from vectorbtpro import *

>>> v = 0
>>> a = np.array([1, 2, 3])
>>> sr = pd.Series([1, 2, 3], index=pd.Index(['x', 'y', 'z']), name='a')
>>> df = pd.DataFrame(
...     [[1, 2, 3], [4, 5, 6], [7, 8, 9]],
...     index=pd.Index(['x2', 'y2', 'z2']),
...     columns=pd.Index(['a2', 'b2', 'c2']),
... )

>>> for i in vbt.broadcast(
...     v, a, sr, df,
...     index_from='keep',
...     columns_from='keep',
...     align_index=False
... ): print(i)
   0  1  2
0  0  0  0
1  0  0  0
2  0  0  0
   0  1  2
0  1  2  3
1  1  2  3
2  1  2  3
   a  a  a
x  1  1  1
y  2  2  2
z  3  3  3
    a2  b2  c2
x2   1   2   3
y2   4   5   6
z2   7   8   9
  • Take index and columns from the argument at specific position:
>>> for i in vbt.broadcast(
...     v, a, sr, df,
...     index_from=2,
...     columns_from=3,
...     align_index=False
... ): print(i)
   a2  b2  c2
x   0   0   0
y   0   0   0
z   0   0   0
   a2  b2  c2
x   1   2   3
y   1   2   3
z   1   2   3
   a2  b2  c2
x   1   1   1
y   2   2   2
z   3   3   3
   a2  b2  c2
x   1   2   3
y   4   5   6
z   7   8   9
  • Broadcast index and columns through stacking:
>>> for i in vbt.broadcast(
...     v, a, sr, df,
...     index_from='stack',
...     columns_from='stack',
...     align_index=False
... ): print(i)
      a2  b2  c2
x x2   0   0   0
y y2   0   0   0
z z2   0   0   0
      a2  b2  c2
x x2   1   2   3
y y2   1   2   3
z z2   1   2   3
      a2  b2  c2
x x2   1   1   1
y y2   2   2   2
z z2   3   3   3
      a2  b2  c2
x x2   1   2   3
y y2   4   5   6
z z2   7   8   9
  • Set index and columns manually:
>>> for i in vbt.broadcast(
...     v, a, sr, df,
...     index_from=['a', 'b', 'c'],
...     columns_from=['d', 'e', 'f'],
...     align_index=False
... ): print(i)
   d  e  f
a  0  0  0
b  0  0  0
c  0  0  0
   d  e  f
a  1  2  3
b  1  2  3
c  1  2  3
   d  e  f
a  1  1  1
b  2  2  2
c  3  3  3
   d  e  f
a  1  2  3
b  4  5  6
c  7  8  9
  • Pass arguments as a mapping returns a mapping:
>>> vbt.broadcast(
...     dict(v=v, a=a, sr=sr, df=df),
...     index_from='stack',
...     align_index=False
... )
{'v':       a2  b2  c2
      x x2   0   0   0
      y y2   0   0   0
      z z2   0   0   0,
 'a':       a2  b2  c2
      x x2   1   2   3
      y y2   1   2   3
      z z2   1   2   3,
 'sr':       a2  b2  c2
       x x2   1   1   1
       y y2   2   2   2
       z z2   3   3   3,
 'df':       a2  b2  c2
       x x2   1   2   3
       y y2   4   5   6
       z z2   7   8   9}
  • Keep all results in a format suitable for flexible indexing apart from one:
>>> vbt.broadcast(
...     dict(v=v, a=a, sr=sr, df=df),
...     index_from='stack',
...     keep_flex=dict(_def=True, df=False),
...     require_kwargs=dict(df=dict(dtype=float)),
...     align_index=False
... )
{'v': array([0]),
 'a': array([1, 2, 3]),
 'sr': array([[1],
              [2],
              [3]]),
 'df':        a2   b2   c2
       x x2  1.0  2.0  3.0
       y y2  4.0  5.0  6.0
       z z2  7.0  8.0  9.0}
  • Specify arguments per object using BCO:
>>> df_bco = vbt.BCO(df, keep_flex=False, require_kwargs=dict(dtype=float))
>>> vbt.broadcast(
...     dict(v=v, a=a, sr=sr, df=df_bco),
...     index_from='stack',
...     keep_flex=True,
...     align_index=False
... )
{'v': array([0]),
 'a': array([1, 2, 3]),
 'sr': array([[1],
              [2],
              [3]]),
 'df':        a2   b2   c2
       x x2  1.0  2.0  3.0
       y y2  4.0  5.0  6.0
       z z2  7.0  8.0  9.0}
  • Introduce a parameter that should build a Cartesian product of its values and other objects:
>>> df_bco = vbt.BCO(df, keep_flex=False, require_kwargs=dict(dtype=float))
>>> p_bco = vbt.BCO(pd.Param([1, 2, 3], name='my_p'))
>>> vbt.broadcast(
...     dict(v=v, a=a, sr=sr, df=df_bco, p=p_bco),
...     index_from='stack',
...     keep_flex=True,
...     align_index=False
... )
{'v': array([0]),
 'a': array([1, 2, 3, 1, 2, 3, 1, 2, 3]),
 'sr': array([[1],
        [2],
        [3]]),
 'df': my_p        1              2              3
        a2   b2   c2   a2   b2   c2   a2   b2   c2
 x x2  1.0  2.0  3.0  1.0  2.0  3.0  1.0  2.0  3.0
 y y2  4.0  5.0  6.0  4.0  5.0  6.0  4.0  5.0  6.0
 z z2  7.0  8.0  9.0  7.0  8.0  9.0  7.0  8.0  9.0,
 'p': array([[1, 1, 1, 2, 2, 2, 3, 3, 3],
        [1, 1, 1, 2, 2, 2, 3, 3, 3],
        [1, 1, 1, 2, 2, 2, 3, 3, 3]])}
  • Build a Cartesian product of all parameters:
>>> vbt.broadcast(
...     dict(
...         a=vbt.Param([1, 2, 3]),
...         b=vbt.Param(['x', 'y']),
...         c=vbt.Param([False, True])
...     )
... )
{'a': array([[1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]]),
 'b': array([['x', 'x', 'y', 'y', 'x', 'x', 'y', 'y', 'x', 'x', 'y', 'y']], dtype='<U1'),
 'c': array([[False, True, False, True, False, True, False, True, False, True, False, True]])}
  • Build a Cartesian product of two groups of parameters - (a, d) and (b, c):
>>> vbt.broadcast(
...     dict(
...         a=vbt.Param([1, 2, 3], level=0),
...         b=vbt.Param(['x', 'y'], level=1),
...         d=vbt.Param([100., 200., 300.], level=0),
...         c=vbt.Param([False, True], level=1)
...     )
... )
{'a': array([[1, 1, 2, 2, 3, 3]]),
 'b': array([['x', 'y', 'x', 'y', 'x', 'y']], dtype='<U1'),
 'd': array([[100., 100., 200., 200., 300., 300.]]),
 'c': array([[False,  True, False,  True, False,  True]])}
  • Select a random subset of parameter combinations:
>>> vbt.broadcast(
...     dict(
...         a=vbt.Param([1, 2, 3]),
...         b=vbt.Param(['x', 'y']),
...         c=vbt.Param([False, True])
...     ),
...     random_subset=5,
...     seed=42
... )
{'a': array([[1, 2, 3, 3, 3]]),
 'b': array([['x', 'x', 'x', 'x', 'y']], dtype='<U1'),
 'c': array([[False,  True, False,  True, False]])}

broadcast_array_to function

broadcast_array_to(
    arr,
    target_shape,
    axis=None,
    expand_axis=None
)

Broadcast an array-like object to a target shape using vectorbt's broadcasting rules.

broadcast_arrays function

broadcast_arrays(
    *arrs,
    target_shape=None,
    axis=None,
    expand_axis=None
)

Broadcast array-like objects using vectorbt's broadcasting rules.

Optionally to a target shape.

broadcast_combs function

broadcast_combs(
    *args,
    axis=1,
    comb_func=itertools.product,
    **broadcast_kwargs
)

Align an axis of each array using a combinatoric function and broadcast their indexes.

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_combs

>>> df = pd.DataFrame([[1, 2, 3], [3, 4, 5]], columns=pd.Index(['a', 'b', 'c'], name='df_param'))
>>> df2 = pd.DataFrame([[6, 7], [8, 9]], columns=pd.Index(['d', 'e'], name='df2_param'))
>>> sr = pd.Series([10, 11], name='f')

>>> new_df, new_df2, new_sr = broadcast_combs((df, df2, sr))

>>> new_df
df_param   a     b     c
df2_param  d  e  d  e  d  e
0          1  1  2  2  3  3
1          3  3  4  4  5  5

>>> new_df2
df_param   a     b     c
df2_param  d  e  d  e  d  e
0          6  7  6  7  6  7
1          8  9  8  9  8  9

>>> new_sr
df_param    a       b       c
df2_param   d   e   d   e   d   e
0          10  10  10  10  10  10
1          11  11  11  11  11  11

broadcast_index function

broadcast_index(
    args,
    to_shape,
    index_from=None,
    axis=0,
    ignore_sr_names=None,
    ignore_ranges=None,
    check_index_names=None,
    **index_stack_kwargs
)

Produce a broadcast index/columns.

Args

args : iterable of array_like
Array-like objects.
to_shape : tuple of int
Target shape.
index_from : any

Broadcasting rule for this index/these columns.

Accepts the following values:

  • 'keep' or None - keep the original index/columns of the objects in args
  • 'stack' - stack different indexes/columns using stack_indexes()
  • 'strict' - ensure that all Pandas objects have the same index/columns
  • 'reset' - reset any index/columns (they become a simple range)
  • integer - use the index/columns of the i-th object in args
  • everything else will be converted to pd.Index
axis : int
Set to 0 for index and 1 for columns.
ignore_sr_names : bool

Whether to ignore Series names if they are in conflict.

Conflicting Series names are those that are different but not None.

ignore_ranges : bool
Whether to ignore indexes of type pd.RangeIndex.
check_index_names : bool
See is_index_equal().
**index_stack_kwargs
Keyword arguments passed to stack_indexes().

For defaults, see broadcasting.

Note

Series names are treated as columns with a single element but without a name. If a column level without a name loses its meaning, better to convert Series to DataFrames with one column prior to broadcasting. If the name of a Series is not that important, better to drop it altogether by setting it to None.

broadcast_shapes function

broadcast_shapes(
    *shapes,
    axis=None,
    expand_axis=None
)

Broadcast shape-like objects using vectorbt's broadcasting rules.

broadcast_to function

broadcast_to(
    arg1,
    arg2,
    to_pd=None,
    index_from=None,
    columns_from=None,
    **kwargs
)

Broadcast arg1 to arg2.

Argument arg2 can be a shape, an instance of ArrayWrapper, or any array-like object.

Pass None to index_from/columns_from to use index/columns of the second argument.

Keyword arguments **kwargs are passed to broadcast().

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_to

>>> a = np.array([1, 2, 3])
>>> sr = pd.Series([4, 5, 6], index=pd.Index(['x', 'y', 'z']), name='a')

>>> broadcast_to(a, sr)
x    1
y    2
z    3
Name: a, dtype: int64

>>> broadcast_to(sr, a)
array([4, 5, 6])

broadcast_to_array_of function

broadcast_to_array_of(
    arg1,
    arg2
)

Broadcast arg1 to the shape (1, *arg2.shape).

arg1 must be either a scalar, a 1-dim array, or have 1 dimension more than arg2.

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import broadcast_to_array_of

>>> broadcast_to_array_of([0.1, 0.2], np.empty((2, 2)))
[[[0.1 0.1]
  [0.1 0.1]]

 [[0.2 0.2]
  [0.2 0.2]]]

broadcast_to_axis_of function

broadcast_to_axis_of(
    arg1,
    arg2,
    axis,
    require_kwargs=None
)

Broadcast arg1 to an axis of arg2.

If arg2 has less dimensions than requested, will broadcast arg1 to a single number.

For other keyword arguments, see broadcast().

get_multiindex_series function

get_multiindex_series(
    arg
)

Get Series with a multi-index.

If DataFrame has been passed, must at maximum have one row or column.

index_to_frame function

index_to_frame(
    arg,
    reset_index=False
)

Convert Index to DataFrame.

index_to_series function

index_to_series(
    arg,
    reset_index=False
)

Convert Index to Series.

make_symmetric function

make_symmetric(
    arg,
    sort=True
)

Make arg symmetric.

The index and columns of the resulting DataFrame will be identical.

Requires the index and columns to have the same number of levels.

Pass sort=False if index and columns should not be sorted, but concatenated and get duplicates removed.

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import make_symmetric

>>> df = pd.DataFrame([[1, 2], [3, 4]], index=['a', 'b'], columns=['c', 'd'])

>>> make_symmetric(df)
     a    b    c    d
a  NaN  NaN  1.0  2.0
b  NaN  NaN  3.0  4.0
c  1.0  3.0  NaN  NaN
d  2.0  4.0  NaN  NaN

mapping_to_series function

mapping_to_series(
    arg
)

Convert a mapping-like object to Series.

repeat function

repeat(
    arg,
    n,
    axis=1,
    raw=False,
    ignore_ranges=None
)

Repeat arg n times along the specified axis.

repeat_shape function

repeat_shape(
    shape,
    n,
    axis=1
)

Repeat shape n times along the specified axis.

resolve_ref function

resolve_ref(
    dct,
    k,
    inside_bco=False,
    keep_wrap_default=False
)

Resolve a potential reference.

soft_to_ndim function

soft_to_ndim(
    arg,
    ndim,
    raw=False
)

Try to softly bring arg to the specified number of dimensions ndim (max 2).

tile function

tile(
    arg,
    n,
    axis=1,
    raw=False,
    ignore_ranges=None
)

Tile arg n times along the specified axis.

tile_shape function

tile_shape(
    shape,
    n,
    axis=1
)

Tile shape n times along the specified axis.

Identical to repeat_shape(). Exists purely for naming consistency.

to_1d function

to_1d(
    arg,
    raw=False
)

Reshape argument to one dimension.

If raw is True, returns NumPy array. If 2-dim, will collapse along axis 1 (i.e., DataFrame with one column to Series).

to_1d_array_nb function

to_1d_array_nb(
    arg
)

Resize array to one dimension.

to_1d_shape function

to_1d_shape(
    shape
)

Convert a shape-like object to a 1-dim shape.

to_2d function

to_2d(
    arg,
    raw=False,
    expand_axis=1
)

Reshape argument to two dimensions.

If raw is True, returns NumPy array. If 1-dim, will expand along axis 1 (i.e., Series to DataFrame with one column).

to_2d_array_nb function

to_2d_array_nb(
    arg,
    expand_axis=1
)

Resize array to two dimensions.

to_2d_pc_array_nb function

to_2d_pc_array_nb(
    arg
)

to_2d_array_nb() with expand_axis=0.

to_2d_pr_array_nb function

to_2d_pr_array_nb(
    arg
)

to_2d_array_nb() with expand_axis=1.

to_2d_shape function

to_2d_shape(
    shape,
    expand_axis=1
)

Convert a shape-like object to a 2-dim shape.

to_any_array function

to_any_array(
    arg,
    raw=False,
    convert_index=True
)

Convert any array-like object to an array.

Pandas objects are kept as-is unless raw is True.

to_dict function

to_dict(
    arg,
    orient='dict'
)

Convert object to dict.

to_pd_array function

to_pd_array(
    arg,
    convert_index=True
)

Convert any array-like object to a Pandas object.

to_tuple_shape function

to_tuple_shape(
    shape
)

Convert a shape-like object to a tuple.

unstack_to_array function

unstack_to_array(
    arg,
    levels=None,
    sort=True,
    return_indexes=False
)

Reshape arg based on its multi-index into a multi-dimensional array.

Use levels to specify what index levels to unstack and in which order.

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import unstack_to_array

>>> index = pd.MultiIndex.from_arrays(
...     [[1, 1, 2, 2], [3, 4, 3, 4], ['a', 'b', 'c', 'd']])
>>> sr = pd.Series([1, 2, 3, 4], index=index)

>>> unstack_to_array(sr).shape
(2, 2, 4)

>>> unstack_to_array(sr)
[[[ 1. nan nan nan]
 [nan  2. nan nan]]

 [[nan nan  3. nan]
[nan nan nan  4.]]]

>>> unstack_to_array(sr, levels=(2, 0))
[[ 1. nan]
 [ 2. nan]
 [nan  3.]
 [nan  4.]]

unstack_to_df function

unstack_to_df(
    arg,
    index_levels=None,
    column_levels=None,
    symmetric=False,
    sort=True
)

Reshape arg based on its multi-index into a DataFrame.

Use index_levels to specify what index levels will form new index, and column_levels for new columns. Set symmetric to True to make DataFrame symmetric.

Usage

>>> from vectorbtpro import *
>>> from vectorbtpro.base.reshaping import unstack_to_df

>>> index = pd.MultiIndex.from_arrays(
...     [[1, 1, 2, 2], [3, 4, 3, 4], ['a', 'b', 'c', 'd']],
...     names=['x', 'y', 'z'])
>>> sr = pd.Series([1, 2, 3, 4], index=index)

>>> unstack_to_df(sr, index_levels=(0, 1), column_levels=2)
z      a    b    c    d
x y
1 3  1.0  NaN  NaN  NaN
1 4  NaN  2.0  NaN  NaN
2 3  NaN  NaN  3.0  NaN
2 4  NaN  NaN  NaN  4.0

wrap_broadcasted function

wrap_broadcasted(
    new_obj,
    old_obj=None,
    axis=None,
    is_pd=False,
    new_index=None,
    new_columns=None,
    ignore_ranges=None
)

If the newly brodcasted array was originally a Pandas object, make it Pandas object again and assign it the newly broadcast index/columns.

BCO class

BCO(
    value,
    axis=None,
    to_pd=None,
    keep_flex=None,
    min_ndim=None,
    expand_axis=None,
    post_func=None,
    require_kwargs=None,
    reindex_kwargs=None,
    merge_kwargs=None,
    context=None
)

Class that represents an object passed to broadcast().

If any value is None, mostly defaults to the global value passed to broadcast().

Method generated by attrs for class BCO.

axis class variable

Axis to broadcast.

Set to None to broadcast all axes.

context class variable

Context used in evaluation of templates.

Will be merged over template_context.

expand_axis class variable

Axis to expand if the array is 1-dim but the target shape is 2-dim.

keep_flex class variable

Whether to keep the raw version of the output for flexible indexing.

Only makes sure that the array can broadcast to the target shape.

merge_kwargs class variable

Keyword arguments passed to column_stack_merge().

min_ndim class variable

Minimum number of dimensions.

post_func class variable

Function to post-process the output array.

reindex_kwargs class variable

Keyword arguments passed to pd.DataFrame.reindex.

require_kwargs class variable

Keyword arguments passed to np.require.

to_pd class variable

Whether to convert the output array to a Pandas object.

value class variable

Value of the object.

Default class

Default(
    value
)

Class for wrapping default values.

Method generated by attrs for class Default.

value class variable

Default value.

Ref class

Ref(
    key
)

Class for wrapping references to other values.

Method generated by attrs for class Ref.

key class variable

Reference to another key.