Vpf%kUddlmZddlmZmZmZddlZddlZddlm Z m Z ddl m Z ddl ZddlmZddlmZddlmZdd lmZdd lmZmZmZmZmZmZmZmZmZdd lm Z dd lm!Z"dd l#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+ddlm,Z,e,j-e.e&Z/ddZ0ddZ1ddZ2ddZ3e"j4dZ5dZ6e"j4dZ7dZ8dd#Z9e"j4d$Z:Gd%d&Z;ej<j=ej<j>fZ?dd,Z@ej<j=ej<jAfZBej<j>ej<jCfZDdd/ZEdd1ZFdd7ZGdd<ZHe"jId=ZJddAZKe"jIddDZLe dEF dddIZMddJZNdKZOdLdLdMdNZPddQZQe"j4dRZRdSZSej<j>ZTddWZUdd`ZVdaZWdbZXdcZYddZZiZ[dee\df<dgZ]e]e[e^<ddjZ_e_e[ej`<ddlZae[bdmeDddoZce[bdpejdDddrZeddxZfddyZgddzZhdd|Zie"j4d}Zj dddZkddZldZmGddZnddZoepZqde\d<dS)) annotations)CallableIterableSequenceN)partial lru_cache)Any)core)dtypes)numpy_scalar_types) ShapedArray) PyTreeDef tree_flattentree_unflattentree_maptreedef_childrengenerate_key_pathskeystrbroadcast_prefix prefix_errors)_replace_nones) linear_util)TracingDebugInfo)safe_map WrapKwArgsHashableHashableFunction Unhashablesafe_zip)traceback_utilxr returnint | tuple[int, ...]ctjd|d} tj|S#t$r*t t tj|cYSwxYw)z2Ensure x is either an index or a tuple of indices.N/expected a static index or sequence of indices.r concrete_or_erroroperatorindex TypeErrortuplemapr!s Q/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/_src/api_util.py _ensure_indexr/+sg T1&WXX!) >!   ))) X^Q'' ( (((()s,1A A tuple[int, ...]ctjd|d} tj|fS#t$r*t t tj|cYSwxYw)z Convert x to a tuple of indices.Nr%r&r-s r._ensure_index_tupler23sj T1&WXX!) N1    ))) X^Q'' ( (((()s-1A! A!strcTt|tstd||S)Nzargument is not a string: ) isinstancer3r*r-s r. _ensure_strr6;s0 As  6 444 5 55 (str | Iterable[str]tuple[str, ...]cvt|tr|fSttt|S)z Convert x to a tuple of strings.)r5r3r+r,r6r-s r._ensure_str_tupler;@s33& 4K [!$$ % %%r7c'^Kt||\}}||fV}t|VdSNrr)in_tree args_flatpy_args py_kwargsanss r. flatten_funrDGsD%gy99'9y   #Sr7c|\}}t|if\}}||krtd|d|||}t||SNz Expected z, got rr*rfunio_treerAin_tree_expectedout_treeargsr?rCs r.apply_flat_funrNMsi&H}---$    A 0AAAA B BB T # # & &&r7c'XKt||}|ifV}t|VdSr=r>)r?r@rArCs r.flatten_fun_nokwargsrPUs> 7I . .'rk#Sr7c|\}}t|\}}||krtd|d|||}t||SrFrGrHs r.apply_flat_fun_nokwargsrR[se&Hw''-$    A 0AAAA B BB T # # & &&r7fn lu.WrappedFun6tuple[PyTreeDef, Callable[[], PyTreeDef], bool] | NonectttrttjdksJtt trtt jdksJtjdt jdh fdt |j|jD\\\}}|fd|fS#t$rYdSwxYw)Nrc3ZK|]%\\}}}|v |||tjdufV&dS)rN)rDrM).0frMstoreflattenss r. z(flattened_fun_in_tree..qsT1P1P IQuh ua;+A../AN1P1Pr7cjSr=val)out_tree_storesr.z'flattened_fun_in_tree..ws N.r7) r5rDrlenrMrPzip transformsstores ValueError)rSr? has_kwargsr\ras @@r.flattened_fun_in_treerics K ) )Hc+2B.C.Cq.H.H.H.H )7 3 3/ "' ( (A - - - -q!#7#Kt||}|ifV}t|ttfrt |dkr"t dt |d||\}}t|\}}t|\}} ||f|| ffVdS)NzNexpected function with aux output to return a two-element tuple, but got type z with value )rr5listr+rcr*typer) r?r@rApairrCauxans_flatans_treeaux_flataux_trees r.flatten_fun_nokwargs2rtys 7I . .'{  $ D4- ( (MCIINN L+/::LLCGLL M MM (#s#C(((H#C(((H 8x2222222r7c*eZdZdZdgZdZdZdZdS)_HashableWithStrictTypeEqualityz|Box object used when comparing static arguments as a jit key. Requires exact type equality using `is` and value equality.r`c||_dSr=r_selfr`s r.__init__z(_HashableWithStrictTypeEquality.__init__ DHHHr7c*t|jSr=)hashr`rys r.__hash__z(_HashableWithStrictTypeEquality.__hash__s >>r7crt|jt|juo|j|jkSr=)rmr`ryothers r.__eq__z&_HashableWithStrictTypeEquality.__eq__s+ >>T%)__ , FUY1FFr7N)__name__ __module__ __qualname____doc__ __slots__rzrrr7r.rvrvs[AAg)GGGGGr7rvsiginspect.Signatureargnums argnums_nameNonec (d}|jD]1}|jtvr|dz }|jtjjurdS2|r@t| |kst||krtd|d|d|ddSdS)z Validate that the argnums are sensible for a given function. For functions that accept a variable number of positions arguments (`f(..., *args)`) all positive argnums are considered valid. rrWNzJitted function has =z, but only accepts z positional arguments.) parametersvalueskind_POSITIONAL_ARGUMENTSinspect ParameterVAR_POSITIONALminmaxrg)rrr n_pos_argsparams r._validate_argnumsrs*~$$&&  e z***Aojj w(7 7 7 ff 8 M3w<<-*,,G 0J0J LLLL7LL)3LLL M MMMM0J0Jr7argnames argnames_namecd}t}t}|jD]g\}}|jtvr||)|jt jjurd}D|jtvr||h|t|zx}rtd|d|d|rdSt||z x}rtd|d|ddS)z Validate that the argnames are sensible for a given function. For functions that accept a variable keyword arguments (`f(..., **kwargs)`) all argnames are considered valid except those marked as position-only (`f(pos_only, /, ...)`). FTz%Jitted function has invalid argnames z in z. These are positional-onlyNz$. Function does not take these args.) setritemsr_KEYWORD_ARGUMENTSaddrr VAR_KEYWORD_INVALID_KEYWORD_ARGUMENTSrg) rrr var_kwargs valid_kwargsinvalid_kwargs param_namerinvalid_argnamess r._validate_argnamesrsp*55, UU.>//11%%j% z'''z"""" w(4 4 4jj 1 1 1$$$)3x==89G F=MFF(FFF G GG  Fh--,67P O=MOO(OOO P PPPPr7Tc ttdt|ryg}tD]f\}}|vr t |s+t d|dt |d|jd|t|gnfdtD}tfdD}t|t||fS)NFuNon-hashable static arguments are not supported, as this can lead to unexpected cache-misses. Static argument (index ) of type  for function  is non-hashable.c<g|]\}}|v t|Sr)r)rYiarg dyn_argnumss r. z#argnums_partial..s7+++fak))S//)))r7c3(K|] }|V dSr=rrYrrMs r.r]z"argnums_partial..'00q47000000r7) r2_ensure_inboundsrc enumerate is_hashablergrmrappendrvr+_argnums_partial)rZrrMrequire_static_args_hashable fixed_argsrrdyn_argss `` r.argnums_partialrsW#K00+ D ;??+! +JD//>>3 k  8   G FBC F FCyy F F()  F F FGG G7<<====>++++ $+++J 0000K000 0 0( ![% *;*; < .s-MMAQn5L5La5L5L5L5LMMr7c3(K|] }|V dSr=rrs r.r]z)argnums_partial_except..rr7rrrr) rrcr+rangerrgrmrrrvr) rZrrMrrrrr static_args `` r.argnums_partial_exceptrsb  d7N#M3t99nMM.MMMMs4yy!1!1MMMMM+ 0000K000 0 0(*  E Eac$iiaJ z " "E  K@A K K *   K K-.Z K K K L LL 7 CCDDDD ![% *;*; < .s1 F F FA1==${    a F F Fr7)objectrcrditerr) rrrkwargsrMrrrCrrs @@r.rrs XX( s:X6 7$K**faDGGZ  + F F F F F F F F$ k8 $ $ 0 0 0 0Fl# )))))r7static_argnamesrdict[str, Any]c s||fSfd|D}i}|D]e\}}||vr\ t|t|||<,#t$r,t d|dt |d|jdwxYwft|t||fS)Nc$i|] \}}|v || Srr)rYkvrs r. z+argnames_partial_except..)s)LLLA1O3K3K13K3K3Kr7ztNon-hashable static arguments are not supported, as this can lead to unexpected cache-misses. Static argument (name rrr) rr}rr*rgrmr_argnames_partialr)rZrr dyn_kwargs fixed_kwargsrrs ` r.argnames_partial_exceptr%s  f9LLLLLLL*!#,  ( (fa ( S #3-- Q GGG FAB F FCyy F F()  F F FGG GG 1j66 7 7 CCs A**6B rrc/zKtd|jDfi|}||fV}|VdS)Nc$i|] \}}||jSrr_)rYrrs r.rz%_argnames_partial..<s ???daAE???r7)dictr`r)rrMrrrCs r.rr:sU ??l&6&<&<&>&>??? N N: N N&Fl# )))))r7i)maxsizekwstuple[bool, ...]cg}|rt|\}}n|d}}t|D]4\}}t||v} || f|jz5|bt |d|D]'\} } | |v} || f| jz(t|S)afReturns a tuple with a boolean value for each leaf in args and kwargs. What if a user specifies donate_argnums but calls the function with kwargs or vice-versa? In that case, in `resolve_argnums` using the signature of the function, the counterpart (donate_argnames or donate_argnums respectively) is calculated so when this function is called both donate_argnums and donate_argnames are available. This allows JAX to donate kwargs when only donate_argnums is specified and vice-versa. When both donate_argnums and donate_argnames are specified, only the args and kwargs specified are donated. NrW) rrchildrenrextend num_leavesr node_datar+) donate_argnumsdonate_argnamesr?rres args_tree kwargs_treerrdonatekeyr`s r.donation_vectorrAs#+-g66I{{$d{I),,..//++fa !~% & &FJJy3>)****[2244Q79M9M9O9OPP--So%f jj&S^+,,,, sr7cH|s|stt|Stt|}tt|}dx}x}}g}|t|kr|t|kr(||||krt d|d|d|t|kr||||kr |dz }|dz }n#||||z |dz }|t|kt|S)aEShifts donate to account for static. >>> rebase_donate_argnums((3, 4), (0, 1)) (1, 2) Args: donate_argnums: An iterable of ints. static_argnums: An iterable of ints. Returns: A tuple of unique, sorted integer values based on donate_argnums with each element offset to account for static_argnums. rz`static_argnums` z and `donate_argnums` z cannot intersect.rW)r+sortedrrcrgr)rrrjoouts r.rebase_donate_argnumsr^sb )N) '' ( ((#n--...#n--...-!-a! # C  3~  >!#4q8I#I#I N>NN+9NNN O OO 3~  >!#4~a7H#H#H1fa1faa jj"Q&'''1fa C   sr7cH t|dS#t$rYdSwxYw)NTF)r}r*)rs r.rrs9III 4  55s  !!F)r tupled_argsc R t t|tg|jz}g fd} t|t ||n#t $r|rt |\}}|\}}d}|r[|d|dz }t|dkr- t|||f|d|dz }n#t $rYnwxYwt |d|d |d |dwxYw fd  D t |jksJ S) Ncv|gtt|dzS)Nr)rrcr)rr!axess r.rbzflatten_axes..s-DKKc,q//!2D.E.E(EFFr7z Note that za that are non-trivial pytrees should always be wrapped in a tuple representing the argument list.rWzE In particular, you're passing in a single argument which means that z/ might need to be wrapped in a singleton tuple.zS specification must be a tree prefix of the corresponding value, got specification z for value tree rc g|] }|urdn| Sr=r)rYaproxys r.rz flatten_axes..s% 2 2 2!u**$$! 2 2 2r7) rrrrrrgrrcr flatten_axes) nametreedef axis_treerrdummy add_leaves_hintrrs @@r.rrs ((% 688*w/A"A B B% $FFFF*C Zy995AAAA CCC $G,,jgqli D * ETEEEFd W     A % % * tWyl 3 3 3 )!%))) *$$    $  88?H88'.881588 9 9>BC'C, 3 2 2 2T 2 2 2$ Tg( ( ( ( ( +s0AAC<4C C< CC<CC<out_spectuple[lu.WrappedFun, Callable]ct|\}}t|t||\}}|t|t||ffS)Nclosure)r_flat_out_axesr+r)rZrleavesr out_axess r. flat_out_axesrsT!**/&'q%--99+!X Xf w/G H H H HHr7c/K||fV}t||} tt||d}nG#t$r:t ||^}}|dj\} | dz } t| dwxYw||fVdS)Nc |duSr=rr-s r.rbz _flat_out_axes..s AIr7)is_leafz pmap out_axesz The full pytree is the output of the pmapped function. Ensure that the `out_axes` argument to `pmap` is a pytree prefix of the pmapped function's output.)rr+rrgrrM) rr rMrrCspec spec_flater msgs r.rrsFl#  ( ($ $&tS:M:MNNNOOII $$$ $ $ $EA 1_   "DC )*C S//t#$ Ys ;AA?ct|trtd|t|std|t j|rtd|dS)Nz.staticmethod arguments are not supported, got zExpected a callable value, got z/Expected a function, got a generator function: )r5 staticmethodr*callablerisgeneratorfunctionrIs r.check_callabler$s\""L JSJJ K KK #= ;c;; < <<  %%M KcKK L LLMMr7int | Iterable[int] | Nonestr | Iterable[str] | None'tuple[tuple[int, ...], tuple[str, ...]]cdS$"ttfS|j}NJttfdt |DnIttfdt |DfS)z?Infer missing argnums and argnames for a function with inspect.N)rrc3PK|] \}\}}|jtk|v|V!dSr=r_POSITIONAL_OR_KEYWORD)rYrrrrs r.r]z-infer_argnums_and_argnames..sJa!U :/ / /AMM 4AMMMr7c3PK|] \}\}}|jtk|v|V!dSr=r*)rYrrrrs r.r]z-infer_argnums_and_argnames..sJa!U :/ / /ALL 4@LLLr7)r2r;rr+rr)rrrrs `` r.infer_argnums_and_argnamesr-s  _) 6 X1!'**G **H H ~* _     **H()9)9););<<GG "'**G()9)9););<<H ( r7rIr signatureinspect.Signature | Nonerint | Sequence[int] | NonerItuple[tuple[int, ...], tuple[str, ...], tuple[int, ...], tuple[str, ...]]c|U|dnt|}|dnt|}|dnt|}|td|d|Jd}nlt|||\}}t|||\}}t ||dt ||dt ||dt ||dt ||t||}||||fS) aValidates and completes the argnum/argname specification for a jit. * fills in any missing pieces (e.g., names given numbers, or vice versa), * validates the argument names/numbers against the function signature, * validates that donated and static arguments don't intersect. * rebases the donated arguments so they index into the dynamic arguments, (after static arguments have been removed), in the order that parameters are passed into the compiled function. Nrz"Getting the signature of function z8 failed. Pass donate_argnums instead of donate_argnames.rrrr)r2r;rgr-rr_assert_no_intersectionr)rIr.rrrrs r.resolve_argnumsr4s_"*1RR7J88N+3bb9J::O)1RR7J88N" ICIII J JJ  " " "OO 'A>?'4'4#NO&@>?'4'4#NOi1ABBBy/3DEEEi1ABBBy/3DEEE/?;;;(HH. ./ IIr7ct|t|}|rtd|ddS)NzEstatic_argnames and donate_argnames cannot intersect. Argument names z3 appear in both static_argnames and donate_argnames)r intersectionrg)rrrs r.r3r3'sj O))#o*>*>??#E  D  D D D E EEEEr7c tj|S#t$r%tjt|dcYSwxYw)Ndtype)r result_typerggetattrr-s r._dtyper;/sT3  a  333  ga11 2 22223s,AAc tjt|tjr|ntj|S#t $rYnwxYwt |dd}t|drtj |j d}n t dt|dtj tj|||S) N weak_typeFr8Tallow_extended_dtypezCannot interpret value of type z9 as an abstract array; it does not have a dtype attributer=)r raise_to_shapedr5 AbstractValueget_avalr*r:hasattrr canonicalize_dtyper8rmr npshape)r!r=r8s r._shaped_abstractify_slowrH5s   a+ , ,Baa$-2B2B D DD    D ae,,) Q+  %agD I I IEE  *$q'' * * * + ++  "(1++u B B BBsAA AActt|d}| ||nt|Sr=)_shaped_abstractify_handlersgetrmrH)r!handlers r.shaped_abstractifyrMFs= ( , ,T!WWd ; ;'*0H0K0KKr7z,dict[Any, Callable[[Any], core.ShapedArray]]rJcHtd|dt|d)Nz Argument 'z ' of type z is not a valid JAX type)r*rmr-s r._str_abstractifyrOMs*MqMMDGGMMMNNNr7 np.ndarrayr c|j}tj|t|jtj|dSNTr>)r8r check_valid_dtyper rGrEr!r8s r._numpy_array_abstractifyrUQsF '%5!!! QW DAAA C CCr7 np.genericctj|}tj|t tj|tj|dSrR)rFr8r rSr rGrErTs r._np_scalar_abstractifyrXXsR (1++%5!!! RXa[[ DAAA C CCr7c#(K|] }|tfVdSr=)rXrYts r.r]r]]sD$A$A()&'(>$?$A$A$A$A$A$Ar7int | float | complex | boolct|}tj||}td||tjvS)Nrr@)rmr _scalar_type_to_dtyper _weak_types)r!typr8s r._python_scalar_abstractifyra`s< Q#  &sA . .% R#1C*C D D DDr7c#(K|] }|tfVdSr=)rarZs r.r]r]dsD$J$J()&'(B$C$J$J$J$J$J$Jr7tagc|Sr=r)rIrcs r.api_hookreis *r7 traced_forsrc str | None fun_signatureTracingDebugInfo | NonecTt|||||}|dSt|||dS)zGTry to build trace-time debug info for fun when applied to args/kwargs.N) _arg_namesr)rfrgrirMrrr arg_namess r. debug_infornms= fn(**) 4 *c9d ; ;;r7c\ tj|S#ttf$rYdSwxYwr=)rr.rgr*r#s r.ririys?  S ! !! i  44s ++ct|tr|j}t|ttj|} |jj}|jj}|jd|d|S#t$rYdSwxYw)Nz at :) r5rfuncrunwrap__code__ co_filenameco_firstlinenorAttributeError)rIfilenamelinenos r.fun_sourceinforzs3   (C 3  s#|'H \ (Fl 3 3 3 36 3 33  44s&A.. A<;A<tuple[str, ...] | Nonec |dSttdt|| t| fdt |D}fd|D} |j|i|}n#ttf$rYdSwxYwtfd|j DS)NTc&g|] \}}|vrn|Srr)rYrr!staticstatic_argnums_s r.rz_arg_names..s+ M M MTQQ/))66q M M Mr7c(i|]\}}||vrn|Srr)rYrr!r~static_argnames_s r.rz_arg_names..s. P P PAA---ff1 P P Pr7c3vK|]3\}}t|D]\}}|u |t|V4dSr=)rr)rYrr!pathlr~s r.r]z_arg_names..slHH74033HH4q&t &&7FHHr7) rrrcrrrbindrgr*r+ arguments) fn_signaturerMrrrargs_bar~rrs @@@r.rlrls$$ 88&$T3t99nEE/)) M M M M MYt__ M M M% P P P P P P P P&  E ,V , ,BB i  44 HHHHbl6H6H6J6JHHH H HHs BB$#B$c/PK||fV}|dt|DfVdS)zKlinear_util transform to get output pytree paths of pre-flattened function.c2g|]\}}t|Sr)r)rYrr s r.rz result_paths..s" B B BwtQfTll B B Br7N)r)rMrrCs r. result_pathsrsFFl# B B*||||it |t |t ||fSr=)rrm_class_with_attrsr_HashableByObjectIdrr+)rZ flat_argsidxsobjs flat_args_rr!s r.hoist_obj_attrsrsr2j$ ""da Aww### kk%a(()))) kk!nnn !U4[[%++ 6 6 BBr7c&eZdZdgZdZdZdZdS)rr`c||_dSr=r_rxs r.rzz_HashableByObjectId.__init__r{r7c*t|jSr=)idr`r~s r.rz_HashableByObjectId.__hash__s dh<<r7c|j|juSr=r_rs r.rz_HashableByObjectId.__eq__s 8uy  r7N)rrrrrzrrrr7r.rrsHg)!!!!!r7rr[rmc:t|dSr=)rr)r[s r.register_class_with_attrsrsr7z set[type]r)r!r r"r#)r!r r"r0)r!r3r"r3)r!r8r"r9)rSrTr"rU)rrrr0rr3r"r)rrrr9rr3r"r)T)rrrrrrr"r0)rZrTrr0rMrrr)rZrTrr9rr)rr)rrr"r)r"r0)rZrTrr r"r)rrrr%rr&r"r')rIrr.r/rr0rr&rr0rr&r"r1)r!rPr"r )r!rVr"r )r!r\r"r )rcr3)rfr3rgrhrir/rMrrrrr0rr9r"rj)rIrr"r/)rIrr"rh)r"r{r=)rrrrjrr{r"r)rZrTrrjrrr"rT)r[rmr"r)r __future__rcollections.abcrrrrr( functoolsrrtypingr numpyrFjax._srcr r jax._src.abstract_arraysr jax._src.corer jax._src.tree_utilrrrrrrrrrrrrjax._src.linear_utilr jax._src.utilrrrrrrr register_exclusion__file__r,r/r2r6r;transformation_with_auxrDrNrPrRrirtrvrPOSITIONAL_ONLYPOSITIONAL_OR_KEYWORDrrrr KEYWORD_ONLYrrrrrtransformationrrrrrrrrrr$r+r-r4r3r;rHrMrJ__annotations__rOr3rUndarrayrXupdaterapython_scalar_dtypesrernrirzrlrrrrrrrrrr7r.rs#""""""8888888888((((((((777777%%%%%%.-----&&&&&&1111111111111111111111######!!(+++))))))))    &&&& ''' ''';;;;, 3 3 3 G G G G G G G G  # ) MMMM* # " )  #P#P#P#PLGGGG&    GGGG2     DDDD*     4 $8    F38U$$$$$LIIII   MMM!*@@2J2J2J2JjEEE333 CCC"LLLNPOOOOOOO$4S!CCCC ,DRZ(CCCC ##$A$A-?$A$A$AAAAEEEE##$J$J-3-H$J$J$JJJJ      < < < <     H H H HCCC =A . . . . .EEEECCC!!!!!!!!"suu$$$$$$r7