Vpfw"ddlmZddlmZmZddlZddlZddlmZm Z m Z ddl m Z ddlmZddlmZddlmZmZmZmZeed fZejZeed fZeeZe d d ddZe d d ddZ e d d ddZ!ej"GddZ#ej"GddZ$e ej%GddZ%dS)) annotations)MappingSequenceN)safe_zip use_cpp_classcache) xla_bridge) xla_client)sdy)are_op_shardings_equalget_num_ways_dim_shardedis_op_sharding_replicatedop_sharding_to_indices.iF)max_sizetrace_context_in_keyshardingSharding global_shapeShapereturnMapping[Device, Index | None]c|||jrSt|drfd|jjDSdDS)N_internal_device_listc"i|] }|| Sr).0d global_maps Q/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/_src/sharding.py z4_addressable_devices_indices_map..*s5 M M M z!} M M Mc^i|]*\}}|j|jk'||+Sr process_indexclient)rrinds rr z4_addressable_devices_indices_map..,sB : : :VQ _ 6 6 8 8 8 8 S 8 8 8r!)devices_indices_mapis_fully_addressablehasattrraddressable_device_listitems)rrrs @r _addressable_devices_indices_mapr,#s++L99* "  X.//M M M M M3K M M MM : :z//11 : : ::r!Mapping[Device, Index]c|||t|}t||t|j}t t |j|SN) shard_shape_to_xla_hlo_shardinglenr_device_assignmentdictr)sr hlo_shardingindicess rcommon_devices_indices_mapr8/sk-- ''L(9(9::, "<#&q';#<#< > >' hq+W55 6 66r!cT|t|}t|r|St|\}}t|t|ks&Jt|t|fg}t t ||D]o\}\}} t ||\} } n#t$rtwxYw| dkr"td|d|d|d|d|d|d | | pt|S) Nrz Sharding z implies that array axis z is partitioned z" times, but the dimension size is z (full shape: z , per-dimension tiling factors: z should evenly divide the shape)) r1r2rr enumeraterdivmod TypeErrorNotImplementedError ValueErrorappendtuple) selfrr6 partitions_outdimr5pquotient remainders r_common_shard_shaperI8s~**3|+<+<==,|,, *<88-*a ZC -- - - -J\ARAR/S - - - #x jAABB  kc6Aq "1aLLh     A~~  d  S     23  &  ,6      JJx ss 4CCc:eZdZUded<ded<dZded<d d ZdS) SdyDimShardingz Sequence[str]axesbool is_closedNz int | Nonepriorityrsdy.DimensionShardingAttrcxtjd|jD|j|jS)WBuilds the attribute. NOTE: An MLIR context is required as a context manager. cLg|]!}tj|"Sr)r AxisRefAttrget)raxiss r z(SdyDimSharding.build..]s(999t  T " "999r!)rNrO)r DimensionShardingAttrrUrLrNrOrAs rbuildzSdyDimSharding.buildWsA  $ ( (99ty999. )   r!)rrP)__name__ __module__ __qualname____annotations__rOrZrr!rrKrKQsP///(      r!rKc,eZdZUded<ded<d dZdS) SdyArrayShardingstr mesh_namezSequence[SdyDimSharding]dimension_shardingsrsdy.TensorShardingAttrcjtj|jd|jDS)rRc6g|]}|Sr)rZ)r dim_shardings rrWz*SdyArraySharding.build..ns$KKK,    KKKr!)r TensorShardingAttrrUrbrcrYs rrZzSdyArraySharding.buildgs;  ! % % KK$2JKKK M MMr!N)rrd)r[r\r]r^rZrr!rr`r`bsE...////MMMMMMr!r`ceZdZdZed#dZed$dZed$dZed%d Zd&d Z ed'dZ d(dZ d)dZ e jd#dZd*dZd+dZe jd'dZd,dZd-d!Zd"S).rzADescribes how a :class:`jax.Array` is laid out across devices. r set[Device]c td)zThe set of devices that this :class:`Sharding` spans. In multi-controller JAX, the set of devices is global, i.e., includes non-addressable devices from other processes. (Subclasses should implement this method.r=rYs r device_setzSharding.device_setw H I IIr!rMc td)zIs this sharding fully replicated? A sharding is fully replicated if each device has a complete copy of the entire data. rlrmrYs ris_fully_replicatedzSharding.is_fully_replicatedror!c td)zIs this sharding fully addressable? A sharding is fully addressable if the current process can address all of the devices named in the :class:`Sharding`. ``is_fully_addressable`` is equivalent to "is_local" in multi-process JAX. rlrmrYs rr(zSharding.is_fully_addressables H I IIr! str | Nonec td)z(Returns the memory kind of the sharding.rlrmrYs r memory_kindzSharding.memory_kinds H I IIr!kindrac td)z?Returns a new Sharding instance with the specified memory kind.z'Subclasses should implement this methodrm)rArvs rwith_memory_kindzSharding.with_memory_kinds G H HHr!XLADeviceAssignmentc tdNrlrmrYs rr3zSharding._device_assignments H I IIr!num_dimensionsintxc.HloShardingc tdr{rmrAr|s rr1zSharding._to_xla_hlo_sharding H I IIr!r`c tdr{rmrs r_to_sdy_shardingzSharding._to_sdy_shardingrr!c`tjdkr|jSd|jDS)zdThe set of devices in the :class:`Sharding` that are addressable by the current process. cVh|]&}|j|jk$|'Srr#rrs r z/Sharding.addressable_devices..s< < < z:Sharding._addressable_device_assignment..sLAAqOqx'='='?'???????AAr!)r(r3r)r@rr*rYs r_addressable_device_assignmentz'Sharding._addressable_device_assignmentsv  %  $$t,--G 4-E F FF AAD3AAA A AAr!c"t||S)zReturns the shape of the data on each device. The shard shape returned by this function is calculated from ``global_shape`` and the properties of the sharding. )rIrs rr0zSharding.shard_shapes t\ 2 22r!rAotherndimc t||||o|j|jko|j|jkS#t$r ||kcYSwxYw)aCReturns ``True`` if two shardings are equivalent. Two shardings are equivalent if they place the same logical array shards on the same devices. For example, a :class:`NamedSharding` may be equivalent to a :class:`PositionalSharding` if both place the same shards of the array on the same devices. )r r1rrur=)rArrs ris_equivalent_tozSharding.is_equivalent_tos $T%>%>t%D%D%*%?%?%E%EGG4,0KK4%"335  U]sAAA+*A+N)rrj)rrM)rrs)rvrarr)rry)r|r}rr~)r|r}rr`)rrrrrrrr-rrrr)rArrrrr}rrM)r[r\r]__doc__propertyrnrqr(rurxr3r1r functoolscached_propertyrrr'rr0rrr!rrrqs JJJ 8J JJJ 8J JJJ 8J JJJ 8JIIII JJJ 8JJJJJJJJJ  <<<<@@@@:::: AAAA3333r!)rrrrrrrr)& __future__rcollections.abcrr dataclassesr jax._src.utilrrrjax._srcr r jax._src.libr xcjax._src.lib.mlir.dialectsr jax._src.op_shardingsr r rrr@r}rDevicesliceIndexryr,r8rI dataclassrKr`rrr!rrse#"""""--------8888888888%%%%%%))))))****** c3h  eSjv&5111 : : :21 :51117772175111210             M M M M M M M Mr{wwwwwwwwwwr!