Vpf0 UddlmZddlmZmZddlZddlZddlmZddl Z ddl m Z ddl Z ddl ZddlZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddl m!Z!m"Z"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)m*Z*m+Z+ddl,m-Z-m.Z.e-e/cZ/Z0e.e1cZ1Z2eej3dZ4 ddd"Z5dd'Z6dd*Z7Gd+d,e Z8Gd-d.ej9Z:d/d/ddd0dd7Z;Gd8d9e ZZ>d/d/dd:dd?Z?d/d/dd:dd@Z@e jAZBdAeCdB<dCd/d/ddDddHZDdIZEd/d/dd:ddJZFddNZG dddVZH dddYZI ddd[ZJ ddd]ZKdd^ZLdd_ZMd`ZNdaZOdbZPe#eNe"dcZQejReQeOePejSeQ<eQjTddZUdeZVejWeQeVdfZXdgZYdhZZdiZ[djZ\dkZ]dlZ^dmZ_dnZ`e#eXeYdoe!dpZaeZejbea<e[ejcea<e]ejSea<e^ejdea<e_ejeea<e`ejfea<dqZgejWeaegdrZhdsZidtZjduZkdvZle#eheidwZmejejbem<ekejcem<elejSem<dxZnejWemendyZodzZpd{Zqd|Zrd}Zsd~ZtdZudZvdZwdZxdZydZzdZ{e#eteode!dpZ|ej}e|ewdexejce|<eyejSe|<e{ejfe|<ddZ~dZejWe|edZdZdZdZdZdZdZe#eede!dpZeejbe<eejce<eeeejSe<e#eede!dpZdZej}eddeeejce<eeeejSe<dZe#eede!dpZeeeejSe<eeeejbe<e#eede!dpZeeeejSe<eeeejbe<dZdZe#eede!dpZeejbe<eejce<eeeejSe<dZdZejWeeejWeeejWeeejWeeejWeedZdZejWeedddZdS)) annotations)CallableSequenceN)partial) NamedTuple)ad_util)config)core)dispatch)dtypes)source_info_util)util)ad)batching)mlir) partial_eval)lax)_argnum_weak_type _input_dtypestandard_primitive)ir)hlo)Array ArrayLikeShape)safe_mapsafe_zipT) canonicalizeoperandr start_indices Sequence[int] limit_indicesstridesSequence[int] | Nonereturnrct|t|t||dnt|S)aWraps XLA's `Slice `_ operator. Args: operand: an array to slice start_indices: a sequence of ``operand.ndim`` start indices. limit_indices: a sequence of ``operand.ndim`` limit indices. strides: an optional sequence of ``operand.ndim`` strides. Returns: The sliced array Examples: Here are some examples of simple two-dimensional slices: >>> x = jnp.arange(12).reshape(3, 4) >>> x Array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) >>> lax.slice(x, (1, 0), (3, 2)) Array([[4, 5], [8, 9]], dtype=int32) >>> lax.slice(x, (0, 0), (3, 4), (1, 2)) Array([[ 0, 2], [ 4, 6], [ 8, 10]], dtype=int32) These two examples are equivalent to the following Python slicing syntax: >>> x[1:3, 0:2] Array([[4, 5], [8, 9]], dtype=int32) >>> x[0:3, 0:4:2] Array([[ 0, 2], [ 4, 6], [ 8, 10]], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.slice_in_dim` - :func:`jax.lax.index_in_dim` - :func:`jax.lax.dynamic_slice` Nr r"r#)slice_pbindtuple)rr r"r#s T/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/_src/lax/slicing.pyslicer,9sKf gU=-A-A$)-$8$8&-odd5>>  K KKArray | np.ndarray(Array | np.ndarray | Sequence[ArrayLike] slice_sizesrct||}tjjrt j|\}}ng}t j|}tj |g||Rdt|iS)a\Wraps XLA's `DynamicSlice `_ operator. Args: operand: an array to slice. start_indices: a list of scalar indices, one per dimension. These values may be dynamic. slice_sizes: the size of the slice. Must be a sequence of non-negative integers with length equal to `ndim(operand)`. Inside a JIT compiled function, only static values are supported (all JAX arrays inside JIT must have statically known size). Returns: An array containing the slice. Examples: Here is a simple two-dimensional dynamic slice: >>> x = jnp.arange(12).reshape(3, 4) >>> x Array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) >>> dynamic_slice(x, (1, 1), (2, 3)) Array([[ 5, 6, 7], [ 9, 10, 11]], dtype=int32) Note the potentially surprising behavior for the case where the requested slice overruns the bounds of the array; in this case the start index is adjusted to return a slice of the requested size: >>> dynamic_slice(x, (1, 1), (2, 4)) Array([[ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.slice` - :func:`jax.lax.dynamic_slice_in_dim` - :func:`jax.lax.dynamic_index_in_dim` r0) _dynamic_slice_indicesr dynamic_shapesvaluer_extract_tracers_dyn_shaper canonicalize_shapedynamic_slice_pr)r*)rr r0 dynamic_sizes static_sizess r+ dynamic_slicer:qs`)-@@-  8"%"@"M"MM<<M*;77L  g ?  ?  ? ? ?*/ *=*= ? ??r-updateArray | Sequence[ArrayLike]cHt||}tj||g|RS)aWraps XLA's `DynamicUpdateSlice `_ operator. Args: operand: an array to slice. update: an array containing the new values to write onto `operand`. start_indices: a list of scalar indices, one per dimension. Returns: An array containing the slice. Examples: Here is an example of updating a one-dimensional slice update: >>> x = jnp.zeros(6) >>> y = jnp.ones(3) >>> dynamic_update_slice(x, y, (2,)) Array([0., 0., 1., 1., 1., 0.], dtype=float32) If the update slice is too large to fit in the array, the start index will be adjusted to make it fit >>> dynamic_update_slice(x, y, (3,)) Array([0., 0., 0., 1., 1., 1.], dtype=float32) >>> dynamic_update_slice(x, y, (5,)) Array([0., 0., 0., 1., 1., 1.], dtype=float32) Here is an example of a two-dimensional slice update: >>> x = jnp.zeros((4, 4)) >>> y = jnp.ones((2, 2)) >>> dynamic_update_slice(x, y, (1, 2)) Array([[0., 0., 0., 0.], [0., 0., 1., 1.], [0., 0., 1., 1.], [0., 0., 0., 0.]], dtype=float32) See Also: - :attr:`jax.numpy.ndarray.at` - :attr:`lax.dynamic_update_index_in_dim` - :attr:`lax.dynamic_update_slice_in_dim` )r2dynamic_update_slice_pr))rr;r s r+dynamic_update_slicer?s/Z)-@@-  $Wf E} E E EEr-c2eZdZUdZded<ded<ded<dS)GatherDimensionNumbersa} Describes the dimension number arguments to an `XLA's Gather operator `_. See the XLA documentation for more details of what the dimension numbers mean. Args: offset_dims: the set of dimensions in the `gather` output that offset into an array sliced from `operand`. Must be a tuple of integers in ascending order, each representing a dimension number of the output. collapsed_slice_dims: the set of dimensions `i` in `operand` that have `slice_sizes[i] == 1` and that should not have a corresponding dimension in the output of the gather. Must be a tuple of integers in ascending order. start_index_map: for each dimension in `start_indices`, gives the corresponding dimension in the `operand` that is to be sliced. Must be a tuple of integers with size equal to `start_indices.shape[-1]`. Unlike XLA's `GatherDimensionNumbers` structure, `index_vector_dim` is implicit; there is always an index vector dimension and it must always be the last dimension. To gather scalar indices, add a trailing dimension of size 1. ztuple[int, ...] offset_dimscollapsed_slice_dimsstart_index_mapN__name__ __module__ __qualname____doc____annotations__r-r+rArAsB*''''""""""r-rAceZdZdZejZejZejZe ddZ dS)GatherScatterModea^ Describes how to handle out-of-bounds indices in a gather or scatter. Possible values are: CLIP: Indices will be clamped to the nearest in-range value, i.e., such that the entire window to be gathered is in-range. FILL_OR_DROP: If any part of a gathered window is out of bounds, the entire window that is returned, even those elements that were otherwise in-bounds, will be filled with a constant. If any part of a scattered window is out of bounds, the entire window will be discarded. PROMISE_IN_BOUNDS: The user promises that indices are in bounds. No additional checking will be performed. In practice, with the current XLA implementation this means that out-of-bounds gathers will be clamped but out-of-bounds scatters will be discarded. Gradients will not be correct if indices are out-of-bounds. sstr | GatherScatterMode | Nonect|tr|S|dkr tjS| |dks|dkr tjS|dkr tjSt d|d)Nclipfilldroppromise_in_boundszUnknown gather mode "") isinstancerMCLIP FILL_OR_DROPPROMISE_IN_BOUNDS ValueError)rNs r+from_anyzGatherScatterMode.from_anysy!&'' hF{{  ##yAKK1;;  ++   00 3q333 4 44r-N)rNrO) rFrGrHrIenumautorWrXrY staticmethodr[rKr-r+rMrMsf( $,dikk 5 5 5< 5 5 5r-rMFunique_indicesindices_are_sortedmode fill_valuedimension_numbersr`boolrarbrOc | tj}t|}|tjkr|t |} t j| tjr tj }nt j| tj rt j | j }nat j| tj rt j | j}n(| t jkrd}nt!d| nd}t"|||t'j|t+|t+|||S)a~Gather operator. Wraps `XLA's Gather operator `_. The semantics of gather are complicated, and its API might change in the future. For most use cases, you should prefer `Numpy-style indexing `_ (e.g., `x[:, (1,4,7), ...]`), rather than using `gather` directly. Args: operand: an array from which slices should be taken start_indices: the indices at which slices should be taken dimension_numbers: a `lax.GatherDimensionNumbers` object that describes how dimensions of `operand`, `start_indices` and the output relate. slice_sizes: the size of each slice. Must be a sequence of non-negative integers with length equal to `ndim(operand)`. indices_are_sorted: whether `indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements gathered from ``operand`` are guaranteed not to overlap with each other. If ``True``, this may improve performance on some backends. JAX does not check this promise: if the elements overlap the behavior is undefined. mode: how to handle indices that are out of bounds: when set to ``'clip'``, indices are clamped so that the slice is within bounds, and when set to ``'fill'`` or ``'drop'`` gather returns a slice full of ``fill_value`` for the affected slice. The behavior for out-of-bounds indices when set to ``'promise_in_bounds'`` is implementation-defined. fill_value: the fill value to return for out-of-bounds slices when `mode` is ``'fill'``. Ignored otherwise. Defaults to ``NaN`` for inexact types, the largest negative value for signed types, the largest positive value for unsigned types, and ``True`` for booleans. Returns: An array containing the gather output. NTz(Unsupported dtype for gather fill_value rdr0r`rarbrc)rMrYr[rX_dtyper issubdtypenpinexactnan signedintegeriinfominunsignedintegermaxbool_rZgather_pr)r r6re) rr rdr0r`rarbrc parsed_modedtypes r+gatherrvsEX \  .D!**400+%222Wooe  5"* - - MV  UB$4 5 5M\%((,  UB$6 7 7M\%((, FL  KEKKLLLJ  }0A)+66.))011    r-c2eZdZUdZded<ded<ded<dS)ScatterDimensionNumbersa Describes the dimension number arguments to an `XLA's Scatter operator `_. See the XLA documentation for more details of what the dimension numbers mean. Args: update_window_dims: the set of dimensions in the `updates` that are window dimensions. Must be a tuple of integers in ascending order, each representing a dimension number. inserted_window_dims: the set of size 1 window dimensions that must be inserted into the shape of `updates`. Must be a tuple of integers in ascending order, each representing a dimension number of the output. These are the mirror image of `collapsed_slice_dims` in the case of `gather`. scatter_dims_to_operand_dims: for each dimension in `scatter_indices`, gives the corresponding dimension in `operand`. Must be a sequence of integers with size equal to `scatter_indices.shape[-1]`. Unlike XLA's `ScatterDimensionNumbers` structure, `index_vector_dim` is implicit; there is always an index vector dimension and it must always be the last dimension. To scatter scalar indices, add a trailing dimension of size 1. r!update_window_dimsinserted_window_dimsscatter_dims_to_operand_dimsNrErKr-r+rxrxesB*$###%%%%------r-rxrar`rbscatter_indicesupdatesc tjtjtjtj|d\}}t ||||||||t| S)aScatter-add operator. Wraps `XLA's Scatter operator `_, where addition is used to combine updates and values from `operand`. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. updates: the updates that should be scattered onto `operand`. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `scatter_indices`, `updates` and the output relate. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the sum of `operand` and the scattered updates. r update_jaxpr update_constsrdrar`rb) r_reduction_jaxpradd _abstractify_const scatter_add_pr)rMr[ rr}r~rdrar`rbjaxprconstss r+ scatter_addr}N&sw'*'7 7A8N8N'O'OQQ-%    e.?+N  % %d + +  - --r-c tjtjtjtj|d\}}t ||||||||t| S)aScatter-multiply operator. Wraps `XLA's Scatter operator `_, where multiplication is used to combine updates and values from `operand`. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. updates: the updates that should be scattered onto `operand`. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `start_indices`, `updates` and the output relate. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the sum of `operand` and the scattered updates. r) rrmulrr scatter_mul_pr)rMr[rs r+ scatter_mulrrr-c tjtjtjtj|d\}}t ||||||||t| S)aScatter-min operator. Wraps `XLA's Scatter operator `_, where the `min` function is used to combine updates and values from `operand`. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. updates: the updates that should be scattered onto `operand`. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `start_indices`, `updates` and the output relate. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the sum of `operand` and the scattered updates. rr) rrrorr scatter_min_pr)rMr[rs r+ scatter_minrrr-c tjtjtjtj|d\}}t ||||||||t| S)aScatter-max operator. Wraps `XLA's Scatter operator `_, where the `max` function is used to combine updates and values from `operand`. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. updates: the updates that should be scattered onto `operand`. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `start_indices`, `updates` and the output relate. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the sum of `operand` and the scattered updates. rr) rrrqrr scatter_max_pr)rMr[rs r+ scatter_maxr rr-zweakref.WeakKeyDictionary_scatter_apply_cacherK) update_shaperar`rbfuncCallable[[Array], Array]rc tj|d|j}fd} t| } n#t $rYnwxYwtj| tjtj|\} } t |||| | |||t | S)aScatter-apply operator. Wraps `XLA's Scatter operator `_, where values from ``operand`` are replaced with ``func(operand)``, with duplicate indices resulting in multiple applications of ``func``. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Note that in the current implementation, ``scatter_apply`` is not compatible with automatic differentiation. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. func: unary function that will be applied at each index. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `start_indices`, `updates` and the output relate. update_shape: the shape of the updates at the given indices. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the result of applying `func` to `operand` at the given indices. rc|SNrK)x_rs r+zscatter_apply..nsQr-r) rfullrur setdefault TypeErrorrr_zero scatter_pr)rMr[) rr}rrdrrar`rbunused_applyrrs ` r+ scatter_applyr>s^ 8L!W] 3 3&    & ! , ,T6 : :FF    D &vs/? '@R@R/S/STT-%  U.?+N  % %d + +  - --s? A  A c|SrrK)rys r+rr|sar-c xt|||dd|||t| S)aScatter-update operator. Wraps `XLA's Scatter operator `_, where updates replace values from `operand`. If multiple updates are performed to the same index of operand, they may be applied in any order. The semantics of scatter are complicated, and its API might change in the future. For most use cases, you should prefer the :attr:`jax.numpy.ndarray.at` property on JAX arrays which uses the familiar NumPy indexing syntax. Args: operand: an array to which the scatter should be applied scatter_indices: an array that gives the indices in `operand` to which each update in `updates` should be applied. updates: the updates that should be scattered onto `operand`. dimension_numbers: a `lax.ScatterDimensionNumbers` object that describes how dimensions of `operand`, `start_indices`, `updates` and the output relate. indices_are_sorted: whether `scatter_indices` is known to be sorted. If true, may improve performance on some backends. unique_indices: whether the elements to be updated in ``operand`` are guaranteed to not overlap with each other. If true, may improve performance on some backends. JAX does not check this promise: if the updated elements overlap when ``unique_indices`` is ``True`` the behavior is undefined. mode: how to handle indices that are out of bounds: when set to 'clip', indices are clamped so that the slice is within bounds, and when set to 'fill' or 'drop' out-of-bounds updates are dropped. The behavior for out-of-bounds indices when set to 'promise_in_bounds' is implementation-defined. Returns: An array containing the sum of `operand` and the scattered updates. NrKr)rr)rMr[)rr}r~rdrar`rbs r+scatterr~sET  d*;+N  % %d + +  - --r-srcidxsaxesc 0tjd|Dd}tjtjfd|Dt t |jdz }||z}tj }|D]}d||<t t dj|j dz dz}t|t |t |}t||t |S)Nc8g|]}tj|dSr)r expand_dims.0is r+ zindex_take..s$DDD!S_Q55DDDr-rc*g|]}j|SrKshape)raxrs r+rzindex_take..s%C%C%Ccim%C%C%Cr-rBrCrDrdr0) r concatenaterrjarrayr*rangendimlistrrArv) rrrindicesmax_idxr0rrBdnumss ` r+ index_takers ODDtDDDa H H' OBH%C%C%C%Cd%C%C%CDD!% q(8"9"9:: < <' g 'SY+ bKOOeAsx'-*::Q>??@@+  ;;Dkk # # #% W!+.. 0 0 00r-r start_index int | None limit_indexstrideintaxiscdg|jz}t|j}dg|jz}|j|}|tj|nd} |tj|n|} | dkr| |z} | dkr| |z} t |}| ||<| ||<tj|||<t ||||S)a Convenience wrapper around :func:`lax.slice` applying to only one dimension. This is effectively equivalent to ``operand[..., start_index:limit_index:stride]`` with the indexing applied on the specified axis. Args: operand: an array to slice. start_index: an optional start index (defaults to zero) limit_index: an optional end index (defaults to operand.shape[axis]) stride: an optional stride (defaults to 1) axis: the axis along which to apply the slice (defaults to 0) Returns: An array containing the slice. Examples: Here is a one-dimensional example: >>> x = jnp.arange(4) >>> lax.slice_in_dim(x, 1, 3) Array([1, 2], dtype=int32) Here are some two-dimensional examples: >>> x = jnp.arange(12).reshape(4, 3) >>> x Array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]], dtype=int32) >>> lax.slice_in_dim(x, 1, 3) Array([[3, 4, 5], [6, 7, 8]], dtype=int32) >>> lax.slice_in_dim(x, 1, 3, axis=1) Array([[ 1, 2], [ 4, 5], [ 7, 8], [10, 11]], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.slice` - :func:`jax.lax.index_in_dim` - :func:`jax.lax.dynamic_slice_in_dim` rr)rrrr _canonicalize_dimensionrr,) rrrrrr r"r#len_axisstart_index_intlimit_index_ints r+ slice_in_dimrsd# $-w}%%- C', ']4 (#/1+>>>56$/1+>>>5=q%0Oq%0O T$'-'-.v66'$- w }g > >>r-indexkeepdimscFtj|t|}}|j|}|dkr||zn|}d|cxkr|ks(nd}t ||||t |||dzd|}|r|Stj||fS)aConvenience wrapper around :func:`lax.slice` to perform int indexing. This is effectively equivalent to ``operand[..., start_index:limit_index:stride]`` with the indexing applied on the specified axis. Args: operand: an array to index. index: integer index axis: the axis along which to apply the index (defaults to 0) keepdims: boolean specifying whether the output array should preserve the rank of the input (default=True) Returns: The subarray at the specified index. Examples: Here is a one-dimensional example: >>> x = jnp.arange(4) >>> lax.index_in_dim(x, 2) Array([2], dtype=int32) >>> lax.index_in_dim(x, 2, keepdims=False) Array(2, dtype=int32) Here are some two-dimensional examples: >>> x = jnp.arange(12).reshape(3, 4) >>> x Array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) >>> lax.index_in_dim(x, 1) Array([[4, 5, 6, 7]], dtype=int32) >>> lax.index_in_dim(x, 1, axis=1, keepdims=False) Array([1, 5, 9], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.slice` - :func:`jax.lax.slice_in_dim` - :func:`jax.lax.dynamic_index_in_dim` rz2index {} is out of bounds for axis {} with size {}r) r rrr IndexErrorformatrrsqueeze)rrrr axis_size wrapped_indexmsgresults r+ index_in_dimr s^,U33SYY%mD!)',qyy%)##e- m ' ' ' 'i ' ' ' ' >C SZZtY77 8 88  0A1d K K& ( M ;vw ' ''r- slice_sizectj|dg|jz}t|j}t |}|||<t j|||<t|||S)aConvenience wrapper around :func:`lax.dynamic_slice` applied to one dimension. This is roughly equivalent to the following Python indexing syntax applied along the specified axis: ``operand[..., start_index:start_index + slice_size]``. Args: operand: an array to slice. start_index: the (possibly dynamic) start index slice_size: the static slice size axis: the axis along which to apply the slice (defaults to 0) Returns: An array containing the slice. Examples: Here is a one-dimensional example: >>> x = jnp.arange(5) >>> dynamic_slice_in_dim(x, 1, 3) Array([1, 2, 3], dtype=int32) Like `jax.lax.dynamic_slice`, out-of-bound slices will be clipped to the valid range: >>> dynamic_slice_in_dim(x, 4, 3) Array([2, 3, 4], dtype=int32) Here is a two-dimensional example: >>> x = jnp.arange(12).reshape(3, 4) >>> x Array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) >>> dynamic_slice_in_dim(x, 1, 2, axis=1) Array([[ 1, 2], [ 5, 6], [ 9, 10]], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.slice_in_dim` - :func:`jax.lax.dynamic_slice` - :func:`jax.lax.dynamic_index_in_dim` r) rrrrrrr rr:)rrrrr r0s r+dynamic_slice_in_dimrHskb%(J{A$>$>#?',#N-W]##+ T$#-2:>>+d w { ; ;;r- int | ArraycZt||d|}|r|Stj||fS)aConvenience wrapper around dynamic_slice to perform int indexing. This is roughly equivalent to the following Python indexing syntax applied along the specified axis: ``operand[..., index]``. Args: operand: an array to slice. index: the (possibly dynamic) start index axis: the axis along which to apply the slice (defaults to 0) keepdims: boolean specifying whether the output should have the same rank as the input (default = True) Returns: An array containing the slice. Examples: Here is a one-dimensional example: >>> x = jnp.arange(5) >>> dynamic_index_in_dim(x, 1) Array([1], dtype=int32) >>> dynamic_index_in_dim(x, 1, keepdims=False) Array(1, dtype=int32) Here is a two-dimensional example: >>> x = jnp.arange(12).reshape(3, 4) >>> x Array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], dtype=int32) >>> dynamic_index_in_dim(x, 1, axis=1, keepdims=False) Array([1, 5, 9], dtype=int32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.index_in_dim` - :func:`jax.lax.dynamic_slice` - :func:`jax.lax.dynamic_slice_in_dim` r)rrr)rrrrrs r+dynamic_index_in_dimrs9Z 4 8 8& ( M ;vw ' ''r-ct|}tj|dgtj|z}|||<t |||S)afConvenience wrapper around :func:`dynamic_update_slice` to update a slice in a single ``axis``. Args: operand: an array to slice. update: an array containing the new values to write onto `operand`. start_index: a single scalar index axis: the axis of the update. Returns: The updated array Examples: >>> x = jnp.zeros(6) >>> y = jnp.ones(3) >>> dynamic_update_slice_in_dim(x, y, 2, axis=0) Array([0., 0., 1., 1., 1., 0.], dtype=float32) If the update slice is too large to fit in the array, the start index will be adjusted to make it fit: >>> dynamic_update_slice_in_dim(x, y, 3, axis=0) Array([0., 0., 0., 1., 1., 1.], dtype=float32) >>> dynamic_update_slice_in_dim(x, y, 5, axis=0) Array([0., 0., 0., 1., 1., 1.], dtype=float32) Here is an example of a two-dimensional slice update: >>> x = jnp.zeros((4, 4)) >>> y = jnp.ones((2, 4)) >>> dynamic_update_slice_in_dim(x, y, 1, axis=0) Array([[0., 0., 0., 0.], [1., 1., 1., 1.], [1., 1., 1., 1.], [0., 0., 0., 0.]], dtype=float32) Note that the shape of the additional axes in ``update`` need not match the associated dimensions of the ``operand``: >>> y = jnp.ones((2, 3)) >>> dynamic_update_slice_in_dim(x, y, 1, axis=0) Array([[0., 0., 0., 0.], [1., 1., 1., 0.], [1., 1., 1., 0.], [0., 0., 0., 0.]], dtype=float32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.dynamic_update_slice` - :func:`jax.lax.dynamic_update_index_in_dim` - :func:`jax.lax.dynamic_slice_in_dim` r)rrr_ndimr?)rr;rrr s r+dynamic_update_slice_in_dimrsOp T$$'J{A$>$>#?#)GBTBT#T-#- gv} = ==r-c"t|}tj|tj|krEtj|dztj|ksJtj||f}t ||||S)alConvenience wrapper around :func:`dynamic_update_slice` to update a slice of size 1 in a single ``axis``. Args: operand: an array to slice. update: an array containing the new values to write onto `operand`. index: a single scalar index axis: the axis of the update. Returns: The updated array Examples: >>> x = jnp.zeros(6) >>> y = 1.0 >>> dynamic_update_index_in_dim(x, y, 2, axis=0) Array([0., 0., 1., 0., 0., 0.], dtype=float32) >>> y = jnp.array([1.0]) >>> dynamic_update_index_in_dim(x, y, 2, axis=0) Array([0., 0., 1., 0., 0., 0.], dtype=float32) If the specified index is out of bounds, the index will be clipped to the valid range: >>> dynamic_update_index_in_dim(x, y, 10, axis=0) Array([0., 0., 0., 0., 0., 1.], dtype=float32) Here is an example of a two-dimensional dynamic index update: >>> x = jnp.zeros((4, 4)) >>> y = jnp.ones(4) >>> dynamic_update_index_in_dim(x, y, 1, axis=0) Array([[0., 0., 0., 0.], [1., 1., 1., 1.], [0., 0., 0., 0.], [0., 0., 0., 0.]], dtype=float32) Note that the shape of the additional axes in ``update`` need not match the associated dimensions of the ``operand``: >>> y = jnp.ones((1, 3)) >>> dynamic_update_index_in_dim(x, y, 1, 0) Array([[0., 0., 0., 0.], [1., 1., 1., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]], dtype=float32) See Also: - :attr:`jax.numpy.ndarray.at` - :func:`jax.lax.dynamic_update_slice` - :func:`jax.lax.dynamic_update_index_in_dim` - :func:`jax.lax.dynamic_index_in_dim` r)rrrrr)rr;rrs r+dynamic_update_index_in_dimrst T$Yv#)G,,,, 9V  q CIg$6$6 6 6 6 6 _VdW - -F $WfeT B BBr-cltjdd|tjdd||jt|kr*d}t |||jt|t|kr%d}t |||tttj |j|s*d}t |||jtd|Ds$d}t ||tj j sMtttj ||s%d }t |||tttj||}|(t|d t|jzkr|Stjdd |t||jkr*d }t |||jtd |Ds$d}t ||tdt!||DS)Nr,r r"z{slice start_indices must have length equal to the number of dimensions of the operand, got indices {} for operand shape {}.zjslice limit_indices must have the same length as start_indices, got start_indices {} and limit_indices {}.zkslice limit_indices must be less than or equal to operand shape, got limit_indices {} for operand shape {}.c3"K|] }|dkV dSrNrK)rsis r+ z$_slice_shape_rule..Ds& - -R1W - - - - - -r-zSslice start_indices must be greater than or equal to zero, got start_indices of {}.znslice limit_indices must be greater than or equal to start_indices, got start_indices {} and limit_indices {}.rr#zuslice strides must have length equal to the number of dimensions of the operand, got strides {} for operand shape {}.c3"K|] }|dkV dSrrKrrNs r+rz$_slice_shape_rule..Vs& % %Q!V % % % % % %r-z&slice strides must be positive, got {}c3JK|]\}}tj|d|VdS)r) window_size window_strideN)r stride_dim)rdrNs r+rz$_slice_shape_rule..YsK//1aqaqAAA//////r-)r_check_shapelikerlenrrrallmapoperatorger r3r4r*subzip)rr r"r#rdiffs r+_slice_shape_ruler5sw???w??? \S'''' MC CJJ}gm<< = ==3}---- 8C CJJ}m<< = == Sgm] ; ; < <> 8C CJJ}gm<< = == - -} - - - - -/ &C CJJ}-- . ..   $@ s8; }== > >@:c cjj >> ? ?? s8< >> ? ?$ _g$W]1C1C*CCC Kw 7333\\W\!! CC CJJw 66 7 77 % %W % % % % %) 2C CJJw'' ( (( //tW--/// / //r-c tj|sJ|jj}|'t jt j|dr5t|t j||dt|z}nt j |t j t j |jdkdt j dt j t j|jd|}t|t j||t j|d}tj|tj|d|}|j|ksJd|jd||gS)Nrrrz result.shape=z operand_shape=)ris_undefined_primalavalrrjrequalrsubtractrrwherermultiplyrpadr) trr r"r# operand_shapepads real_limitsrs r+_slice_transpose_ruler\s^  ( ((((,$- _rx3344_ }bk-GGc-((( * *DD&hrx  A%qvaR[!%<%.s&99c999999r-zOslice slice_sizes must be greater than or equal to zero, got slice_sizes of {}.c3,K|]}|jdkVdSrrridxs r+rz,_dynamic_slice_shape_rule..(003Q000000r-zGstart_indices arguments to dynamic_slice must be scalars, got indices )r split_listrrrrrrrrranyr*r_merge_dyn_shape)rr0starts_and_dyn_sizesr dynrs r+_dynamic_slice_shape_ruler/s';gl^LL- \S'''' PC CJJ}gm<< = ==3{++++ LC CJJs=11;?? @ @@ .j L L89 M!$* * 4  qw 3 3 3 L L L L L Lr-zKindex arguments to dynamic_slice must be integers of the same type, got: {}, c3.K|]}|jjVdSrrunamers r+rz,_dynamic_slice_dtype_rule..&(M(M!(M(M(M(M(M(Mr-)rr*rr+rrjoinru)rr0r-r.rr s @r+_dynamic_slice_dtype_ruler<s';gl^LL- L L L L=J L L LLLP C CJJtyy(M(M}(M(M(MMMNN O OO r-c|d}t|tjurtj|g|ddRd|i}tj|dg|ddRd|i|fS)Nrrr0)typerZeror7r))primalstangentsr0 tangent_outs r+_dynamic_slice_jvprCs{ + +gl**!&{ZWQRR[ZZZkZZK  gaj P7122; P P PK P PR] ]]r-ctj|sJtd|DsJ|jj|jj}}t |tjur.tj|jgdgt|zzStj |d|}tj ||g|Rgdgt|zzS)Nc3@K|]}tj| VdSrrrrs r+rz0_dynamic_slice_transpose_rule..0 B Bq'** * B B B B B Br-r)rrrrrrur>rr?rrrr>r))r rr0r r  operand_dtypezeross r+_dynamic_slice_transpose_rulerJs  ( (((( B BM B B B B BBBB!(!3W\5G- !WW  L & & '4&3}3E3E*E EE H]A} 5 5E # ( BM B B B C FS'' ' ()r-cvt|dkrtjgddfSt}t dt ||D||ur!t jd|DddfSt jfdt ||Dd}|dfS)Nrint32c3:K|]\}}||j|VdSrr)rrrs r+rz/_batch_dynamic_slice_indices..s,KKdaQ]qwqz]]]]KKr-c8g|]}tj|dSr)r broadcastrs r+rz0_batch_dynamic_slice_indices..s$DDDqCM!T22DDDr-cPg|]"\}}tj|df|dnd#S)rNrrK)broadcast_dimensions)rbroadcast_in_dim)rrrsizes r+rz0_batch_dynamic_slice_indices..sU&&& A !dAY89 2PPP&&&r-r) dimension)rrjrobjectnextrrr)rbdims empty_markerrSs @r+_batch_dynamic_slice_indicesrYs\\Q 8B $ &&, KKc'5&9&9KKK  $ \ ?DDGDDDa H H$ NN O&&&&We$$&&&    ' !r-c ~|^}}|^}}|j|tjurdndz }tt |}t j||g\} } t j||g\} } t|d|} t| | \}}t||g| ||g| | |ddtj dS)NrrrKrTrg) rr not_mappedr*rrr*rArY_gather_batching_rulerMrY)rrr0rstart_indices_and_dyn operand_bdstart_idx_and_dyn_bdsndimsdimsr dyn_slice_sizes start_idx_bdsdyn_slice_size_bdsrr index_bdims r+_dynamic_slice_batching_rulerfs%1!' !'1$*$ ,zX-@@@!!a H% uU||  $#'?3H5'#R#R -&*o6KeW&U&U#-# T15 7 7 7%2=-PP%  e&o&101UDT  ,  ? ? ??r-c8tj||jg\}}|s.|t|g|Rt |St j||}tj ||j d}t j |t||g|Rd|iS)N)r0Fr0) rr*rdefault_process_primitiver7dictrr,r DShapedArrayru_dyn_shape_staging_rule)tracerr0r-r r.rrs r+_dynamic_slice_staging_rulerms';afXFF- J  * *?Q&: > > >1< > >>r-crtj||jjg\}}|s,t j|jgd|DRd|i\}}|g|fSt j||}d|D}tj t||jj |jj }|gtj fS)Nc3$K|] }|jV dSr)rrrs r+rz0_dynamic_slice_typecheck_rule..s$00Q!&000000r-r0cVg|]&}t|tjur|jn|'SrK)r>r Literalvalrps r+rz1_dynamic_slice_typecheck_rule.. s0LLLQ$q''T\11qLLLr-)rr*rrr7 abstract_evalrr,r rjr*ru weak_type no_effects) rrr0r-r r.out_avaleffects out_shapes r+_dynamic_slice_typecheck_rulerzs';afk]KK- ''5 K00-000KKK>IKKHg :w $[#66ILL)LLLI y!1!116rur bintbound)rars r+rz/_dynamic_slice_padding_rule..sH + + +a ==DI55!'--1 + + +r-cVg|]&}t|tjur|jn|'SrK)r>r DArrayrsrps r+rz/_dynamic_slice_padding_rule..s0MMM!Q4;..quuAMMMr-)rr*rrrr,r:) in_avals out_avalsrr0starts_and_dynx_avalstart_indices_avals dyn_avalsr r.dyn_ slice_sizes_ start_idxs r+_dynamic_slice_padding_rulers+/?8a[+Q+Q(& y~x@@- + +)S)) + + +$%k488,MM}MMM) 9l 3 3 44r-r:)weak_type_rulec|j^}}tj||jg\}}|j\}|r)|t j||}tj ||||gS)Nrr ) avals_inrr*rrr;rr,rr:) rrr0r-rrr r.r s r+_dynamic_slice_lowerr"sw|*&1';fk]KK-m)(MS%9+s%K%KLLH  S(A] K K K LLr-c*|j|jkr/d}t||j|j|jt |kr*d}t|||jt t tj|j|js/d}t||j|jtd|Drtd||jS)Nzidynamic_update_slice update must have the same rank as operand, got update shape {} for operand shape {}.zvdynamic_update_slice start_indices must have length equal to the rank of operand, got indices {} for operand shape {}.zodynamic_update_slice update shape must be smaller than operand shape, got update shape {} for operand shape {}.c3,K|]}|jdkVdSrr&r's r+rz3_dynamic_update_slice_shape_rule..Br)r-zMstart_indices arguments to dynamic_update_slice must be scalars, got indices ) rrrrrrrrrr+rr;r rs r+ _dynamic_update_slice_shape_ruler5s \V[   7C CJJv|W];; < << \S'''' CC CJJ}gm<< = == SgmV\ : : ; ;= >C CJJv|W];; < <<00-00000= <,9<< = == r-ctjd||tfdDrAd}t|ddD|jS)Nr?c3K|]?}|jdjkp$tj|jtj V@dSrr2r4s r+rz3_dynamic_update_slice_dtype_rule..Ir5r-zQindex arguments to dynamic_update_slice must be integers of the same type, got {}r6c3.K|]}|jjVdSrr8rs r+rz3_dynamic_update_slice_dtype_rule..Mr:r-)rcheck_same_dtypesr+rrr;rurs ` r+ _dynamic_update_slice_dtype_rulerGs.@@@ L L L L=J L L LLLP C CJJtyy(M(M}(M(M(MMMNN O OO r-c|dd\}}|dd}|dd\}}tj||g|R}t|tjur;t|tjur tj|}n;t j|}t j|}tj||g|R}||fS)N)r>r)r>rr? from_valuerinstantiate_zeros) r@rArr;r g_operandg_updateval_outrBs r+_dynamic_update_slice_jvprPsBQBK/'6!""+- ! )X " ' H- H H H' )__ $$h7<)G)G,))'22KK$Y//I#H--H(-iRMRRRK + r-ctd|DsJtj|r |jj}n|j}t |t jur_tj|rt j|jnd}tj|rt j|jnd}nptj }tj }tj ||} tj|r ||| g|Rnd}tj|r ||g|Rd|ind}||gdgt|zzS)Nc3@K|]}tj| VdSrrF)rrs r+rz7_dynamic_update_slice_transpose_rule..^rGr-rr0)rrrrrr>rr?r>r)r7r_zerosr) r rr;r r operand_tupdate_tdusdsrIs r+$_dynamic_update_slice_transpose_ruler]sV B BM B B B B BBBBF## ;$LLm>>>>>>fjH X $#m*<*`_ operator and following the outline of the implementation of ShapeInference::InferGatherShape in TensorFlow. rrzTGather index leaf dimension must be within [0, rank(indices) + 1). rank(indices) is z$ and gather index leaf dimension is rrvrBzOffset dimension z$ in gather op is out of bounds; got z, but should have been in [0, )zGather op has zI elements in start_index_map and the bound of dimension index_vector_dim= of indices is z". These two numbers must be equal.z'Invalid start_index_map; domain is [0, ), got: ->rDrCzTGather op must have one slice size for every input dimension; got: len(slice_sizes)=z, input_shape.rank=zAll components of the offset index in a gather op must either be a offset dimension or explicitly collapsed; got len(slice_sizes)=z, output_slice_sizes=z, collapsed_slice_dims=zSlice size at index z2 in gather op is out of range, must be within [0, z + 1), got zBGather op can only collapse slice dims with bound 1, but bound is z for index z at position ) rBrCrD_rankrrrrrrr_gather_shape_computation)rrrdr0r`rarbrcrBrCrDindex_vector_dimoutput_offset_dim_countoutput_shape_rankr offset_dimoperand_dim_for_start_index_ircorresponding_input_sizer~s r+_gather_shape_rulers"-+*?%5/7^^a' 7^^&&&*:Q*>*> J7.(s>7741a!555555577r-c3\K|]&}|vrtntV'dSr)rV)rrindices_shape_genrBslice_sizes_gens r+rz,_gather_shape_computation..*s`NN/0()K'7'7d?###)**NNNNNNr-) rBrCrrrrappendpopiter enumerater*r) rrdr0rrexpanded_indices_shapeansrCrrBrs @@@@r+rrs,!-+*?+&&w7!;7^^a' ..   $444!!!$$$-...12277779[#9#9777/ NNNNNN49:K4L4LNNN N N# *r-c|}ttjtj} t j|j} t j|tj}t|jdz } | | |j t j|| |j z } t j t j |tjdt j |t j | tt| } t j| | g} | t|jz}tjtj||j}t)|||||t*j}t jt j| |||t j||S)zHLowers a FILL_OR_DROP gather as a PROMISE_IN_BOUNDS gather with masking.rurr)rarb)rc)rrjrint64rshape_as_valuerconvert_element_typerrD bitwise_andrlerr*r _reduce_andrBrarangervrMrYselectrR full_like)rrrdr0r`rarc output_shaperintarray operand_dimsnum_batch_dims upper_boundmask output_ndimsbatch_dims_in_output gather_outs r+ _gather_fillr/s % RXRX . . .(#GM22,  $Wbh 7 7'w}%%).88E1223 %%hhu/D&E&EFG  fWbhqkk"" fWcok5~9N9N3O3OPPQQ S S$ / 0 0$ #e&7"8"88,29\#:#:#(#466 gw{);,>@@@* |-ABB jZ@@@ B BBr-c 0t|||||||dS)Nrr_)rv) grrrdr0r`rarbrcs r+_gather_jvp_rulerQs/ 7-{-#5D   r-c ttj|sJ|jj} t |t jurt j|j} n^tj| tj |} t|j |j |j } t| ||| |||} | dgS)Nr)r`rarb)rrrrr>rr?rrrrxrBrCrDr) r rrrdr0r`rarbrcr rrI scatter_dnumss r+_gather_transpose_rulerYs  ( ((((,$- !WW  ,w| $ $CC H]CIaLL 1 1E+*6,A#4#DFFFM eWa%3); ! ! !C tr-c |^}} } |^} } } d| D}| | tj|| d\}} |jdf|z}dtt jd|jz}tt jd|j}tt jd|j}t|||}t| tj rtj | |}ttj|| |D]@\}}t|tjr!t#j||jst(Atj| jt/| ||}n| }t1|| |tj|||||||fS| k| itj| | d} td|jD}t||j|j}t1|| ||dd||dfStj|| d}tj| | d} t#j|jddrt7t#j|jdd|jt#j| jddt=j| j||||||}tj d|ztj!|dfStE| j| jd}tG| j}d|d <tj$|t|d}tj%|| &|gtO|dz } d |z}dtt jd|jz}tt jd|j}dtt jd|jz}t|||}t1|| ||||||dfS) Nc&g|]}|jjSrK)rur~)rbs r+rz)_gather_batching_rule..qsBBBQ17=BBBr-rrrrrgc3 K|] }d|zV dS)rNrKrps r+rz(_gather_batching_rule..s&EE!AEEEEEEr-Frr)(rmove_stacked_axisrr*rjrrBrCrDrArV RaggedAxis bdim_as_shaperrr,IndexedAxisSizer same_referentlengthsNotImplementedError shape_as_bdim stacked_axisrrvmoveaxisdefinitely_equalr ShapedArrayrur rrr_promote_dtype_for_sizerbroadcasted_iotarastyper)rrrdr0r`rarbrcrrrb operand_bdim indices_bdimrddyn_slice_size_boundsrBrCrDrragged_slice_sizesorig fabricatedbdim_outr index_dtype count_shapecountss r+r\r\ls(4$'7_4>1, 1BB/BBB,"6$6w aPPG\=#% 3Krva):)FGGHHHK +<+Q!R!RSSBF1&7&GHHIIO "1' ) ) )E, 344#1, LL! {O < <  & & $  j(": ; ; &#D**<== && %'  # #GU4F G GIIhhh E(6KLL%-D     !)  )) 8q99GEE'8'DEEEEEK ".C)9 ; ; ;E '7e)%%** N N NOP QQ q99Gq99G W]1-q11B'  7=,gm < <  7=,!4W]CC E E-;' > AA *'-q9IJJKw}%%KKO  !+u[/A/A1 E EFovw~~k'B'BC!+..244G$K%q2C2X(Y(Y"Z"ZZq"3"?@@AAKU26!->-N#O#OPPPO "1' ) ) )E '7e).%7d' ) ) )+, ,,r-ctj|tjs|S t |}n:#tj|jdkr|ntjdcYSxYw|tj|jkr|S|tjtj jkrtjdStj tj S)N rL) r rirjr3rrnbitsrurqrLrr)rurSs r+rrs  5"* - - LF t99DDFHUOO(B..55BHW4E4EEEE RXe__   L rx!!%%% 8G    $RX . ..s 35A*c|\} } td| jDrt|tjkrtt |||||| gS)Nc3JK|]}t|tjVdSr)rVpeBoundedAxisSizerps r+rz#_gather_pad_rule..s/GGqAr) * *GGGGGGr-rdr0rbrc)r+rr rMrYrv) rrrrrdr0r`rarbrc operand_aval indices_avals r+_gather_pad_ruler(sz (, GGL4FGGGGG   000  '5F(t  L L L MMr-rvir.Valuec |j\} } |j\| jj| jj} fdt t| D} |g|j | R}g|| R}tt||||||} tj || ||tj| | gtjg\}|S)Nc$g|] }j|z SrKr&)rraval_ys r+rz(_gather_lower_opaque..sIIIa&+/IIIr-)rBrgrr)rrru_rulesphysical_element_avalrrr_replacerBr _gather_lowerrdelegate_loweringr physical_aval)rrrrdr0r`rarbrcaval_x aval_indices elt_shapetrailing_offset_dims gather_lowerresr,s @r+_gather_lower_opaquer:s&, M'&l!77 EEK)IIII5Y3H3HIII'00I%1I4HII1KK*+* **+'8n+$:OOO,   <'"6**L9#F++, . . .$# *r-c |j\} tj| jtjrt ||||||||| gS|t jkr4tj td} | ||||||||| j  S|t j t j fvs J|tjt#|jggt'|jdj dz t#|jt#|j} t/j|sxtj||}tj| g} |||g} | t6j|d}tj| | |jStj ||| tj!|t6j| gS) NrgFmultiple_results)rdr0r`rarcrr)rCoperand_batching_dimsstart_indices_batching_dimsrrBrD)rdra)resultsoperands attributes)ra)"rr riruextendedr:rMrXr lower_funrrrYrWrrAgetrrCrrrBrDr is_constant_shapeeval_dynamic_shape_as_tensoraval_to_ir_typerBoolAttrDynamicGatherOp build_genericr@rvdense_int_array)rrrrdr0r`rarbrcr gather_fill_fnrr@rArBs r+r1r1 s7m)( x~v77  Wg1B-D        +++^L5IIIN > Wg+%:LHN  < < << #5#(* * * *+/ * * * $ ( ( 1 FGG"$3<?011A5(455,<== )  %   , ,A3CEEK #H--.G+.H" koo.@AAJ   , ,(z - C CCJK J  [));??+=>>  @ @ @ AAr-c tj|jtjst dt jd||tj|jdS)NrrTr) r rirurjr3rZrrr)rrr~rs r+_scatter_dtype_rulerO@sW  7="* 5 5: 8 9 99 7G444  "7=t L L LLr-c |j|j|j} t|dz } t|| ks| dkr#t dt|d| dt |j} t| | kr| dt| dz tz} t|| kr#t d| dt|dtdd tdd tt|dd tdd tdd ttdd ttz} t| kr#t d | d tdt| |j| kr1t d t| d| d|j| dtt| D]I}| |}|dks|tkr&t dtd|d|dJt| ddfdttjD}ttD]K}|}|||j|kr*t d|d|j|d||dLfdtt|D}d}|D]b}|| kr|dz }tj|j|| |s*t d|d|j|d| |d|dz }cjS)adValidates the well-formedness of the ``dimension_numbers`` argument to Scatter. The code implements the checks based on the detailed operation semantics of XLA's `Scatter `_ operator and following the outline of the implementation of ShapeInference::InferScatterShape in TensorFlow. rrzUScatter index leaf dimension must be within [0, rank(indices) + 1). rank(indices) is z% and scatter index leaf dimension is rzUpdates tensor must be of rank z; got rryrzzScatter op has window of size z ; doesn't match operand of rank zScatter op has zV elements in scatter_dims_to_operand_dims and the bound of dimension index_vector_dim=rz!. These two numbers must be equalz.s@CCC#$,@(A(A#A#A$M!,#A#A#Ar-zBounds of the window dimensions of updates must not exceed the bounds of the corresponding dimensions of operand. For dimension z, updates bound is z, operand bound is c6g|]}|tv|SrK)r)rdimrys r+rz'_scatter_shape_rule..sB222/00B1B1B1B1B1r-zBounds of the scatter dimensions of updates must be the same as the bounds of the corresponding dimensions of scatter indices. For scatter dimension z, indices bound is )ryrzr{rrrrrrrrrrr r)rrr~rrrdrar`rbr{rrexpected_updates_rankrrrSmax_update_slice_sizesupdate_window_dimupdate_scatter_dimsscatter_dims_seenrzrys` @@r+_scatter_shape_rulerYFsn);*?!2!O7^^a' 7^^&&&*:Q*>*> O>--- . .. ,@AAA'4HIII*E'NNI,... !9.DEEE)96LMMM,eGnni.000&''#.B*C*CC+ 7^^{"" ?[??-27^^??? @ @@ &'' m$%&& -c*F&G&G--$4--#*-0@"A--- . .. 122 3 3EEa &q )C Qww#w'' D %gDD89DD=@DDD E EE(193555CCCCCeC .sO#P#P'($+=#3k!n#D#P#P#P#P#P#Pr-)rrrrzrryr*r{rrrorrjuint64rnrurqrrRclamp) rrr~rposr upper_boundsrr0s ` @r+_clamp_scatter_indicesr`s+ # W]## $ $aE &&&u'?'DEFFF Qhcc"#P#P#P#P#P,1,N#P#P#PPP,"<00+ 028GM;R;R;V1W1W24(<<==+$['-&)'-&8&81&<%>@@+ 28A;; 8"( K K   r-c |\}} } |\} } } ~ t|| | |||||| }t| tjur;t| tjur tj|}nKt j| } t j| } t| | | |||||| }||fS)Nr)rr)r>rr?rrr)r@rArrrdrar`rbrrr~r g_indices g_updatesrrBs r+_scatter_add_jvprds&'7G$,!)Y     wl!5F+N    '  )__ $$iGL)H)H,))'22KK$Y//I$Y//I$$7IL#7H-n  %K + r-c(tj|rJtj|r |jj} n|j} t |t jur_tj|rt j|jnd} tj|rt j|jnd} ndx} } tj|r|} tj|rt|j|j |j } g}d}tt|jD]L}||j vr| d!| | |j||dz }Mt||| || d} | d| gS)Nrrrr%)rrrrr>rr?rAryrzr{rrrrvr rrr~rrrdrar`rb updates_shaperr gather_dnumsr0r^rs r+_scatter_add_transpose_ruleris #G , ,,,,G$$"L&MMMM !WW .0.DW.M.MW W\***SWI-/-CG-L-LVw|GL)))RVHHI g&&i g&&J+%8.C)FHHHlk cS\\""! !6 6 6   Q       ]+<+OPS+TU V V V (##7l$/dqJJJh T8 $$r-c tj|rJtj|r |jj} n|j} t |t jur`tj|rt j|jnd} tj|rt j|jnd} ndx} } tj|rt||||||| } tj|r|stdt|j |j |j } g}d}tt|jD]L}||j vr|d!|| |j ||dz }Mt!t#j|||| || d} | d| gS)NrCscatter_mul gradients are only implemented if `unique_indices=True`rrrr%)rrrrr>rr?rr rAryrzr{rrrrvrrrfs r+_scatter_mul_transpose_rulerls #G , ,,,,G$$"L&MMMM !WW .0.DW.M.MW W\***SWI-/-CG-L-LVw|GL)))RVHHI g&& Wg1B/i g&&1 Q! OQQ Q+%8.C)FHHHlk cS\\""! !6 6 6   Q       ]+<+OPS+TU V V V (##7++W*6K!a111h T8 $$r-c r|\} } } |\} } }tdt||D}tj| | |} d} tj| ||} | t t jd|j}dt t jd|jz}t t jd|j }t|||}| | | | |||||| dfStj| | |} t| j }d|d<tj| jt |d}tj|| gt%|dz } t t jd|j}dt t jd|jz}dt t jd|j z}t|||}| | | | |||||| dfS)Nc3:K|]\}}||j|VdSrr)rrrs r+rz)_scatter_batching_rule..0s9!!eagbk!!r-rrrr)rdrar`rbrrr)rVrr bdim_at_frontr*rjrrzryr{rxr)rrrrrurr) scatter_oprrrrrdrar`rbrrr~rr updates_bdimrSrzryr{rrrs r+_scatter_batching_rulerr(s{+'7G-7*, l !!s<'D'D!!! ! !$  "7L$ ? ?',  "7L$ ? ?' +<+Q!R!RSSbfQ0A0T&U&U V VV#(3D3a)b)b#c#c #-1%A C C CE ?? w5+N l-  I IKL LL  "7L$ ? ?'W]##++b/   u[/A/A1 E E& OVW-s;/?/?!/C D D'RVA'8'KLLMMbfQ0A0V&W&W X XX!%bfQ8I8f.g.g(h(h!h !+/#? A A A%  w5+N l-  I IKL LLr-z scatter-addz scatter-mulc |stdtj|tt j|||||||S)Nrkr)r rrrrzeros_like_jaxval) rrrrrdrar`rbkws r+_scatter_mul_jvp_rhsrvfsg M K M MM K ""Aqrr?rrrrAryrzr{rrrrvrjrrrrr_ones)#rpr@rArrrdrar`rbrrr~rrbrcrrgrrBrhr0r^r initial_vals target_valssuccessful_updatesretained_values num_updatesnum_refsupdates_normalizer updates_coefoperand_normalizer operand_coeftarget_tangentstangent_updatess# r+_scatter_extremal_jvprxs&'7G$,!)Y #--- OO wl!]+#$  0 0'  )__ $$iGL)H)H,))'22KK$Y//I$Y//I*!4*?%BDDDL K C 3w}%% & & m0 0 01=)I#)NOPPP q,(=(=??L,(=(=??K"[0#{2O Jw   J)39W+=+=z'** , ,      KCJw''Ig&&! # #  HO$';?$;$'+$577:00!j1133LO$';?$;$'J{$;$;==--9L7L"(;*?*?AAO'5 </0Oi%-+1C-;#' )))K + r-z scatter-minz scatter-maxc "|td|\}} } |\} } } |}t| tjur_t| tjurDt|| | |||||| }|tj|fStj| } tj| } |rJt|| | ||||d| }t| | | ||||d| }||fSt| j }|j D]}d||<tj |}tj|rC|dzt!jt jjkr t jn t j}n t j}t+jt+jt+j|||t+j| |}t5t+j|j d|| |||||}t9|j |j|j}g}d}t?tA|j D]Q}||jvr|!d!|!| j |j ||dz }RtE|| || }t+j#t+j$|t+j%||t+j%|}t+j#t+j$||| t+j%| }t+j#t+j$|t+j%|| t+j%| }t+j#t+j$||| t+j%| }tM|| ||||| }tM|| ||||| }||fS) Nz!scatter_apply JVP not implementedrTrrrr|rrr)'r r>rr?rr)rrrrrrymathprodr is_constant_dimrjrnuint32rqr\rrreshapeiotaryrrrArzr{rrrrvreqrr) r@rArrrdrar`rbrrr~rrbrcrrrB ids_shape update_dimnum_idsid_dtype update_ids scattered_idsrhr0r^rgathered_update_idsmasked_operandmasked_updatesmasked_g_operandmasked_g_updatess r+ _scatter_jvprs: A B BB%'7G$,!)Y % )__ $$iGL)H)Hnn wl!U+N  G GL++G44 44"9--)"9--)  nn wl!U+DtMMG..),!U+Dt!MMK K  7=!!),jIj Ii ' '""$q[BHRY,?,?,CCCryyHHyHws{38Hg#>#> JJy999;;*#(7=!X>>!:u-?)7dDDD- ((36888,+ # ]()) * *aE &&&u'?'DEFFF Qhcc}g1=+6888 :cf]CJ}4M4MNN%sz'':':<<.:cfZ2EFF%sz'':':<<.Z}cj6O6O P P )3:i+@+@BBZz3F G G )3:i+@+@BB */+='5D B B B',g7G.3/A+9FFF+ + r-c |stdtj|rJtj|r |jj} n|j} t |t jur`tj|rt j|jnd} tj|rt j|jnd} ndx} } tj|r0t||tj | d|j ||d| } tj|rt|j |j|j} g}d}t!t#|jD]L}||jvr|d!|| |j ||dz }Mt'||| || d} | d| gS) Nz>scatter transpose is only implemented whereunique_indices=TruerrTrrrr%)r rrrrr>rr?rrrrurAryrzr{rrrrvrfs r+_scatter_transpose_rulerP s 5 4 5 55  #G , ,,,,G$$"L&MMMM !WW .0.DW.M.MW W\***SWI-/-CG-L-LVw|GL)))RVHHI g&&:!Wch}aqw&O&O&O,=-?)-D:::i  g&&&+%8.C)FHHHlk cS\\""! !6 6 6   Q       ]+<+OPS+TU V V V (##7l$/d#$&&&h T8 $$r-rc |j\} } |j\} | jj| jj} fdt t| D}|g|j |R}tt|||||| }tj |||||tj| | tjgtj| g\}|S)Nc$g|] }j|z SrKr&)rr aval_updatess r+rz)_scatter_lower_opaque.. s!OOOA,+a/OOOr-)ryrrrdr`rarbr-)rrrur.r/rrrr0ryr_scatter_lowerrr2r r3)rrrr~rrrdr`rarbr4r5r,r6trailing_window_dims scatter_lowerr9rs @r+_scatter_lower_opaquer s!(+|$&, M'&l!77 EEK)OOOOs9~~9N9NOOO'001,?1/11122<}).+$888-   ='7G"6**L"<002#F++,  . . .$# *r-c |F|rJ|jdj} tjtt jd| \}}|j\} tj | jtj rt||||||||||  gS| tj krAtjt d} | |d||||\}|} t$jt+| jt+| jggt+| jt3|jdjdz }tj| }|g}|g}t%j|f||||t:j|t:j| }tjt jd| j}|jj !||}t;j"|5tGj$}|j%rtMd tj'|j(||tj)||j*d|j*d|j+ \}}t%j,tj-|dddn #1swxYwY|j.S) NrrKrFr<rrrryrzinput_batching_dimsscatter_indices_batching_dimsscattered_dims_to_operand_dimsrrar`z!Cannot lower effectful `scatter`.)dim_var_values)/rrurr_scatter_reduction_computationr rrr rirCrrMrWrrDr`replacerrxrErryrzr{rrrH ScatterOprrIupdate_computationblocksrInsertionPointr new_name_stackrxr  jaxpr_subcompmodule_contextTokenSet argumentsrreturn_flatten_ir_valuesr@)rrrr~rrrdrar`rbrHr clip_fnrrrop scalar_typer; name_stack out_nodesrs r+rr sLO)M"%"6&(8](K(K#M#ML-m)( x~v77; ! Wgw!+N-D  : : : ;;   ###n3eLLLGt44gw!):<<._scatter s. >L >Lm ;??+=>>{~66888G&t'7J'G'GHHKoa '..{KHHG 7 # # Iw( ) 0c k3% >s?.C99C=C=)rrurj complex128rrMrWrrDr`rrrrxrErryrzr{rrrrHr rrealimagcomplex)rrrr~rrrdrar`rboperand_aval_inrupdates_aval_inrr rrrrrrrs ` `` @@@r+_scatter_add_lower_gpur s), %/1obm++ #w'3(5,=-?)7d  D D DD  ###n3eLLLGws{{T{22GWg.000HGm)( %-11e677 :;;$&%)%*L%M%M3<?011A5 2-8>**** x~z2244& #(7##SXg%6%6 7 7$ #(7##SXg%6%6 7 7$ +dD ! ! ""r-gpu)platformlist[ArrayLike]c t||jkr7d}t|t||jt |t tfsA|jdkr'td|jt|}g}t||jD]\}}tj t|tj r||Ht |ttjfrUt#j|rA|t'j|dkr||zn|t|t#j|}t |ttjfrB||dkr%|t'j|t|zn|5t'j|t|}|t'j|dk||z||S)NzJLength of slice indices must match number of operand dimensions ({} vs {})rz+Slice indices must be a 1D sequence, got {}r)rrrZrrrVr*rrr rirhrjrprrr3r rrrdimension_as_valuer)rr rrrrd_arrs r+r2r2 s  7<'' C SZZM 2 2GMBB C CC ME4= 1 1(Q D}233 5 55''M&-//33da B$677 mmA!c2:&''D,@,C,C mmC,a!eeQUUF1IINNOOO ""A!c2:&'' mm!a%%A0F1II>>>>QOOO  $Qq 2 2E MM#*QUAIq112222 -r-r) rrr r!r"r!r#r$r%r)rr.r r/r0rr%r)rr.r;rr r<r%r)rrr rrdrAr0rr`rerarerbrOr%r)rrr}rr~rrdrxrarer`rerbrOr%r)rrr}rrrrdrxrrrarer`rerbrOr%r)rrrrrr!r%r)rr) rr.rrrrrrrrr%r)rT) rr.rrrrrrer%rr) rr.rrrrrrr%r) rr.rrrrrrer%r) rr.r;rrrrrr%r) rr.r;rrrrrr%r)r%r))rr.r r/r%r) __future__rcollections.abcrrr\r functoolsrrtypingrweakrefnumpyrjrjax._srcrr r r r r rjax._src.interpretersrrrrr# jax._src.laxrjax._src.lax.utilsrrrjax._src.lib.mlirrjax._src.lib.mlir.dialectsrjax._src.typingrrr jax._src.utilrrr unsafe_mapr unsafe_ziprurhr,r:r?rAEnumrMrvrxrrrrWeakKeyDictionaryrrJrrrrrrrrrrrrrr( deflinear2primitive_batchersdef_implrr!register_loweringr/r<rCrJrYrfrmrzrr7primitive_jvpsprimitive_transposescustom_staging_rulescustom_typechecks padding_rulesrrrrrrr>rrrrrrrrrrrr\rr(rsdefjvpr:r1rOrYr`rdrirlrrrrrvrrrrrrrrrrr2rKr-r+rs #""""""........   %%%%%%$$$$$$******&&&&&&444444 ! ******3333333333,,,,,,,,CZCZ D 1 1 1 +/5K5K5K5K5Kp7?7?7?7?t.F.F.F.Fb#####Z###6$5$5$5$5$5 $5$5$5V#(&+26DDDDDDN.....j...:#5)- ------------d#5)- ------------d#5)- ------------d#5)- ------------`3L'2K2M2MMMMM "5)- ;-;-;-;-;-;-|"0 #5)- .-.-.-.-.-.-`0000*/0H?H?H?H?H?VGH"&9(9(9(9(9(|787<7<7<7<71(1(1(1(1(h;>;>;>;>|>C>C>C>CB%/%/%/N$(  . g F F  g,---';G$  ;;;ddd w ---7770^^^ ) ) )   ???, > > > ' ' '555%$8/$$Q'')))&8/"+H(/KO,+F(*G'$?!MMM(<===$   ==="+++,,+$&F-F()(./( 23BBB -/JKKKMMM #"LLL 9 9 9MMMcLcLcLL   0BBBD&d,d,d,L / / / M M M  *H$$Q'' ) ) )  ($d+++$:!(=H%-    (2A2A2Ahx///MMM fffR   2.%%%B%%%%%%P0L0L0Ld#",m$$Q''))) $4- )D & ' -00 M*#",m$$Q'')))   - D D     *E & ' -00 M*cccJ#",m$$Q'')))  ' -00 M*#*7+@-#P#P- "",m$$Q'')))  ' -00 M*#*7+@-#P#P- cccJ(%(%(%T  ,i$$Q'' ) ) )  ,)%< " ' ),, I&   ,555ny.111}n555}n555}n555}n5555444#4#4#l}&