Vpf{  ddlmZddlmZmZmZmZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZmZmZmZmZmZmZddlmZddlmZddlmZddlmZdd lmZe jeZed Z d5dZ!d6dZ"Gddee Z#GddZ$dZ%e$Z&e&j'Z'e&j(Z(e&j)Z)GddZ*e*Z+GddZ,e,Z-e j.Z/Gddee Z0dZ1dZ2ddd d!d"d7d,Z3ddd-d8d3Z4ddd-d9d6Z5ddd-d:d:Z6ddd-d;d?Z7ddd-ddGZ:GdHdIee Z;d?dKZ<d@dMZ=dAdOZ>dBdQZ?dBdRZ@d aAGdSdTeZBdUZCGdVdWeZDGdXdYe j.ZEeEZFdZZGe3d[d d\]ZHe3d^e!d_d`da]ZIe6dbe"dcddd]ZJe6dee"dfdgdh]ZKe3die!djd dk]ZLe9dlddm]ZMe9dnddo]ZNe3dpd dq]ZOe3drd ds]ZPe3dtd du]ZQejReQd`ZSe3dvd dw]ZTe3dxd dy]ZUe3dzd d{]ZVe3d|d d}]ZWe3d~d d]ZXe3dd d`dZYdZZdZ[e3dd d`eZe[dZ\e4dddgddZ]e3dd d]Z^e6dddddZ_e4dgdddZ`e3dd d`dZae4dgdddZbe3dd d`dddZce3dd dddZde3dd d`dddZee3dd d`dZfe3dd d]Zge7ddd]Zhe6ddd]Zie3dd d]Zje9ddd]Zke3dd`d]Zle6ddd]Zme3dd d]Zne3dd d]Zoe6dddì]Zpe3dd ddƄdDŽZqe6dddd˄d̄Zre6dddϬ]Zse3dd`dѬ]Zte9dddӬ]Zue6ddd֬]Zve4dddgddڬZwe4dddgddd߄dZxdZydZze3dd deyezZ{e&j|}de~e$deddZdZdZe:dddeeeZdZdZe3dd deeZe4dgdddddZe5dgdddddZe4dgdddZe3dd d]Ze3dee jdd!dddZe3dd`d]Ze3d d`d`d Ze3d d d`d Ze3d d d]Ze6dddddZejdCdZejdCdZdZe5dgdddddZe5dgdddddZe5d gddd!d"d#Zd$Ze5d%gddd&e'ZejdDd)ZdEd+Ze8d,d!d-e.e3d/d d0]Ze3d1d d`d2d3d4ZdS(F) annotations)CallableHashableIteratorSequenceN)AnyGeneric NamedTupleNoReturnProtocolTypeVarcast)lib)jax_jit)transfer_guard_lib) xla_client)logging_config_Tvarnamestrdefaultboolreturnctj|t|}|}|dvrdS|dvrdSt d|d|)abRead an environment variable and interpret it as a boolean. True values are (case insensitive): 'y', 'yes', 't', 'true', 'on', and '1'; false values are 'n', 'no', 'f', 'false', 'off', and '0'. Args: varname: the name of the variable default: the default boolean value Raises: ValueError if the environment variable is anything else. )yyesttrueon1T)nnoffalseoff0Fzinvalid truth value z for environment )osgetenvrlower ValueError)rrvals O/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/_src/config.pybool_envr-&si '3w<<((# # 000 4 333 5 OCOOGOO P PPintc`ttj|t|S)zz!Config.values..is O O O<4D&, O O Or.)rEitemsrJs r,valuesz Config.valuesgs% O O43F3L3L3N3N O O OOr.cl||jvrtd|d||j|<|||f|j|<dS)NzConfig option z already defined)rE ExceptionrF)r:rOr[opt_type meta_args meta_kwargss r, add_optionzConfig.add_optionksM t""" =t=== > >> &DK8DIdOOOr.c v ddlm}ddlm}m d_ _t jt j t j t j d ji}jD].\}\}}}j|}||||jg|Ri|/| fddS)aRegisters absl flags for the JAX configs. E.g., for each JAX config defined using bool_state(), this method registers an absl boolean flag, with the same name. This is the recommended method to call if you use `app.run(main)` and you need JAX flags. Examples: ```python from absl import app import jax ... if __name__ == '__main__': jax.config.config_with_absl() app.run(main) ``` rN)appflagsTenumc.Sr7)complete_absl_config) absl_flagsr:sr,z)Config.config_with_absl..s 9 9* E Er.) absl.flagsrgabslrfrGrkr DEFINE_boolr/DEFINE_integerfloat DEFINE_floatr DEFINE_string DEFINE_enumrFr]rEr4call_after_init) r: absl_FLAGSrf absl_defsrO flag_typerbrcr[rks ` @r,config_with_abslzConfig.config_with_abslqs,$#####--------DM DO . 10 0*0 3I 6:Y__5F5FJJ11y)["4(fi 4I III[IIIIEEEEEFFFFFr.c|jD]E\}} |j|}n#t$rY wxYw|jr||jFdSr7)rEr]FLAGSrVpresentr;r4)r:rkrOr[flags r,rjzConfig.complete_absl_configs+1133   f%      DJ   s . ;;ctstjdtj}dgd|D}ddl}||j|d| |jdadSdS)aParses command-line args that start with --jax. This method should be used only by advanced users. Most users should use :meth:`config_with_absl` instead. This method has serious limitations: e.g., although it parses only the --jax* command-line args, it runs the validators of all registered absl flags, even non-JAX ones that have not been set yet; as such, for the non-JAX flags, the validators run on the default flag values, not on the values indicated by the command-line args. c|dkS)Nz--r8)as r,rlz.Config.parse_flags_with_absl..s qDyr.c3DK|]}|d|VdS)z--jaxN) startswith)rZrs r, z/Config.parse_flags_with_absl..s3EEaq||G/D/DEEEEEEEr.rNT) known_only) already_configured_with_absl itertools takewhilesysargvrmryrgr{rj)r:jax_argvrns r,parse_flags_with_abslzConfig.parse_flags_with_absls ( *$%8%8#(CChFEE8EEEFh  jxD111  +++%)""" * *r.N)r=r>r?_HAS_DYNAMIC_ATTRIBUTESrKrPrTrRpropertyr^rdryrjrr8r.r,rCrCLs ''' ((( BBB  PP 8P999 $G$G$GL   *****r.rCctj}d}d}|j}|r|j|j}|r|jr|j}||t jtjtjtjtjtjtjtjtjtjt jt"jt$jt&jt(jt*jt,jt.jfS)zReturns a tuple of configuration values that affect tracing. These values are included in the cache key for linear_util.cache. Values included in this set should also most likely be included in the C++ JIT state, which is handled separately. r8)rthread_local_stateextra_jit_contextaxis_env_statemesh_context_manager enable_x64r4numpy_rank_promotiondefault_matmul_precisiondynamic_shapesnumpy_dtype_promotiondefault_devicerandom_seed_offsetthreefry_partitionablethreefry_gpu_kernel_loweringsoftmax_custom_jvpenable_memories disable_jitdebug_key_reusejax_xla_profile_version&hlo_source_file_canonicalization_regexpgle_profiling_runs enable_pgleuse_shardy_partitioner)tlsrrcontexts r, trace_contextrs "$$#.&' ,'3+N 8-8"7 . 0@  $&>&D   5 ;   2 8 & & ,  "       ! ' 0 6  #   & ((r.ceZdZdS) NoDefaultNr=r>r?r8r.r,rrr.rceZdZdS)_UnsetNrr8r.r,rrrr.rczeZdZdZddddefddZddZddZeddZ e j efd dZ dZ dS)!State)_name_value_update_thread_local_hook_update_global_hook _validator_default_context_manager_valuer@r=NrrOrrrupdate_global_hookCallable[[_T], None] | Noneupdate_thread_local_hook"Callable[[_T | None], None] | None validatorCallable[[Any], None] | Noneextra_descriptiondefault_context_manager_valuerc ||_|dr |ddn||_d|d|d||_||_||_||_||_||dS)Njax_zContext manager for `z` config optionz. ) rrr=r@rrrrr;) r:rOrhelprrrrrs r,rKzState.__init__sDJ $ 7 7ADHHTDM6D66(66/366DL1D%=D"DO*GD'IIgr.rr cjtdt|jNz[bool() not supported for instances of type '{0}' (did you mean to use '{0}.value' instead?) TypeErrorformattyper=rJs r,__bool__zState.__bool__4  55;V JJ 6!6! " ""r.r4r5c|jr||||_|jr||dSdSr7)rrrr9s r,r;z State._setsT  ooeDK & u%%%%%&&r.ctj|jt}|turt t |n|jSr7)_thread_local_state__dict__getrunsetrrr)r:r+s r,r4z State.values:  & * *4:u = =Cu,,4C===$+=r.new_valc#K|tur.|jtur|j}ntd|jd|jr||t t |jt}tt |j||j r| | dV|tur:tt |j|j r| ddSdStt |j||j r*| tt|dSdS#|tur8tt |j|j r| dwwtt |j||j r)| tt|wwxYw)NzContext manager for zU config option requires an argument representing the new value for the config option.) no_defaultrrr=rgetattrrrrsetattrrdelattrrr)r:rprev_vals r,__call__zState.__call__!s* ,J > >5-t}---.. .  oog*DJ>>H W555 %. $$W--- = eee U  #TZ000  ) /  ( ( . . . . . / / #TZ:::  ) =  ( (b();); < < < < < = = U  #TZ000  ) /  ( ( . . . . / #TZ:::  ) =  ( (b();); < < < < =s .EBGcB||_||_||jdS)zmPrivate method that adds hooks to an existing context-manager. Used to avoid cyclic import dependencies.N)rrr)r:rrs r, _add_hookszState._add_hooks>s-&>D"1Dt{#####r.)rOrrrrrrrrrrrrrrr r<)rr)rr)r=r>r? __slots__rrKrr;rr4 contextlibcontextmanagerrrr8r.r,rrs)9=EI04!+5*"""" &&&& >>> 8>$.=====8$$$$$r.rz This will be enabled by default in future versions of JAX, at which point all uses of the flag will be considered deprecated (following the `API compatibility policy `_).z (transient)Fr)rrupgraderrOrrCallable[[bool], None] | Noner$Callable[[bool | None], None] | Nonerr State[bool]c dt|ts>td|dtt |dt |t ||}|}|r|dtzz }|tz }tj |tt||||||dt|tgd|itt |t#fd S) a=Set up thread-local state and return a contextmanager for managing it. This function is a convenience wrapper. It defines a flag, environment variable, and corresponding thread-local state, which can be managed via the contextmanager it returns. The thread-local state value can be read via the ``config.`` attribute, where ``config`` is the singleton ``Config`` instance. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: boolean, a default value for the option. help: string, used to populate the flag help information as well as the docstring of the returned context manager. update_global_hook: a optional callback that is called with the updated value of the global state when it is altered or set initially. update_thread_local_hook: a optional callback that is called with the updated value of the thread-local state when it is altered or set initially. upgrade: optional indicator that this flag controls a canonical feature upgrade, so that it is `True` for the incoming functionality, `False` for the outgoing functionality to be deprecated. extra_description: string, optional: extra information to add to the summary description. Returns: A contextmanager to control the thread-local state value. Examples: ENABLE_FOO = config.bool_state( name='jax_enable_foo', default=False, help='Enable foo.') # Now the JAX_ENABLE_FOO shell environment variable and --jax_enable_foo # command-line flag can be used to control the process-level value of # the configuration option, in addition to using e.g. # ``config.update("jax_enable_foo", True)`` directly. We can also use a # context manager: with enable_foo(True): ... The value of the thread-local state or flag can be accessed via ``config.jax_enable_foo``. Reading it via ``config.FLAGS.jax_enable_foo`` is an error. z(Default value must be of type bool, got of type r= T)rrrrrrbrccjSr7rY_ss r,rlzbool_state.. 17r.) isinstancerrrrr-upperr)UPGRADE_BOOL_HELPUPGRADE_BOOL_EXTRA_DESCconfigrIaddrrdrrCr)rOrrrrrrrs @r, bool_staterPsCx GT " "T SwSS&tG}}j$w--PPSS T TT TZZ\\7 + +' $ 1C# ##D00""4((( Dk GT.@7)OOO! D!TRfd^LLL &$!2!2!2!233444 (r.)rr enum_values Sequence[str]Callable[[str], None] | None#Callable[[str | None], None] | None State[str]c pt|ts>td|dtt |dt ||}t j||}|vrtd|d|tj |fd}tt||||||t|dg|d  tt |t#fd S) a}Set up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. enum_values: list of strings representing the possible values for the option. default: string, default value. help: string, used to populate the flag help information as well as the docstring of the returned context manager. Returns: A contextmanager to control the thread-local state value. 'Default value must be of type str, got rr=Invalid value "" for JAX flag c t|tus|vr&tdd|dt|ddS)Nznew enum value must be in , got r.rrr*rrs r,rzenum_state..validatorsy G}}C7+#=#= AKAA%AA04W AAA B BB$>#=r.rrrrhrrrcjSr7rYrs r,rlzenum_state..rr.rrrrrr)r'r(rr*rrIrrrdrrCr)rOrrrrrrrs ` @r, enum_statersy4 GS ! !T SgSS&tG}}j$w--PPSS T TT $ IdjjllG , ,' K GGGGG H HH""4(((BBBBB  Cj   +7 !  Av"-t<<  &$!2!2!2!233444 (r. str | NoneState[str | None]c l|St|ts>td|dtt |dt ||}t j||}||vrtd|d|tj |fd}td||||||t|d g|d  tt |t#fd S) aSet up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. enum_values: list of strings representing the possible values for the option. default: optional string, default value. help: string, used to populate the flag help information as well as the docstring of the returned context manager. Returns: A contextmanager to control the thread-local state value. N/Default value must be of type str or None, got rr=rrc |@t|tus|vr(tdd|dt|ddSdS)Nz"new enum value must be None or in rrrrrs r,validatez%optional_enum_state..validates G}}C7+#=#= AKAA%AA04W AAA B BB #=#=r.rrhrrcjSr7rYrs r,rlz%optional_enum_state.. rr.r)rOrrrrrr rs ` @r,optional_enum_stater sz4 GS!9!9 SgSS&tG}}j$w--PPSS T TT $ IdjjllG , ,' WK77 GGGGG H HH""4(((BBBBB  L GT-/G!  Av"-t<<  &$!2!2!2!233444 (r.Callable[[int], None] | None#Callable[[int | None], None] | None State[int]c t|ts>td|dtt |dt ||}t j|}|4 t|}n##t$rtd|d|wxYwtj |d}tt||||||t|tgd|i tt |t#fd S) a+Set up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: optional int, default value. help: string, used to populate the flag help information as well as the docstring of the returned context manager. Returns: A contextmanager to control the thread-local state value. z'Default value must be of type int, got rr=Nrrc||7t|ts$td|dt|dSdS)Nz6new int config value must be None or of type int, got r)rr/r*rrs r,r zint_state..validate1sd:gs#;#; @%@@04W @@ A AAr.rrcjSr7rYrs r,rlzint_state..9rr.)rr/rrrr)r'r(rr*rrIrrrdrrCrrOrrrr default_envr rs @r, int_staters}. GS ! !T SgSS&tG}}j$w--PPSS T TT $ $**,,''+OK  gg OOO M+MMtMM N NNO""4(((AAA  Cjw&8)855!D!SBVTNKKK &$!2!2!2!233444 ( B"" CrqCallable[[float], None] | None%Callable[[float | None], None] | None State[float]c t|ts>td|dtt |dt ||}t j|}|4 t|}n##t$rtd|d|wxYwtj |d}tt||||||t|tgd|i tt |t#fd S) aSet up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: default value. help: string, used to populate the flag help information as well as the docstring of the returned context manager. Returns: A contextmanager to control the thread-local state value. z)Default value must be of type float, got rr=Nrrc|>t|ttfs$td|dt |dSdS)Nz:new float config value must be None or of type float, got r)rrqr/r*rrs r,r zfloat_state..validate`s`:gs|#D#D  1 1 1!%g 1 1 2 22r.rrcjSr7rYrs r,rlzfloat_state..irr.)rrqrrrr)r'r(rr*rrIrrrdrrCrrs @r, float_stater=sz. GU # #T SSS&tG}}j$w--PPSS T TT $ $**,,''+Ok""gg OOO M+MMtMM N NNO""4(((222  El4$(:+X77!D!UbvtnMMM &$!2!2!2!233444 (rc t|ts>td|dtt |dt |d}t ||||||S)alSet up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: string, a default value for the option. help: string, used to populate the flag help information as well as the docstring of the returned context manager. update_global_hook: an optional callback that is called with the updated value of the global state when it is altered or set initially. update_thread_local_hook: an optional callback that is called with the updated value of the thread-local state when it is altered or set initially. Returns: A contextmanager to control the thread-local state value. rrr=cvt|ts#td|dt|ddS)Nz1new string config value must be of type str, got rr)rrrrrs r,rzstring_state..validatorsf gs # #C B&BB15gBBB C CCCCr.rrrrrrstring_or_object_staterOrrrrrs r, string_stater%ms8 GS ! !T SgSS&tG}}j$w--PPSS T TTCCC  GT+7    r.c |St|ts>td|dtt |dt |d}t ||||||S)auSet up thread-local state and return a contextmanager for managing it. See docstring for ``bool_state``. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: optional string, a default value for the option. help: string, used to populate the flag help information as well as the docstring of the returned context manager. update_global_hook: an optional callback that is called with the updated value of the global state when it is altered or set initially. update_thread_local_hook: an optional callback that is called with the updated value of the thread-local state when it is altered or set initially. Returns: A contextmanager to control the thread-local state value. Nr rr=c~|8t|ts%td|dt|ddSdS)Nz9new string config value must be None or of type str, got rr)rrr*rrs r,rz(optional_string_state..validatorsj:gs#;#; B&BB15gBBB C CCr.rr"r$s r,optional_string_stater(s8 GS!9!9 SgSS&tG}}j$w--PPSS T TTCCC  GT+7    r.rrrr State[Any]c|}tj||}tj|tt||||||tt|tfdt |tgd|iS)aSet up thread-local state and return a contextmanager for managing it. Similar to ``string_state``, except the context manager will accept any object, not just a string. Any value passed via command line flag or environment variable will be treated as a string. Args: name: string, converted to lowercase to define the name of the config option (and absl flag). It is converted to uppercase to define the corresponding shell environment variable. default: string, a default value for the option. help: string, used to populate the flag help information as well as the docstring of the returned context manager. update_global_hook: an optional callback that is called with the updated value of the global state when it is altered or set initially. update_thread_local_hook: an optional callback that is called with the updated value of the thread-local state when it is altered or set initially. validator: an optional callback that is called with the new value on any update, and should raise an error if the new value is invalid. Returns: A contextmanager to control the thread-local state value. cjSr7rYrs r,rlz(string_or_object_state..rr.rr)r)r'r(rrrIrrrrrCrrdr)rOrrrrrrs @r,r#r#sD $ IdjjllG , ,'""4((( Cj GT-/G! &$!2!2!2!233444D!SBVTNKKK (r.cNeZdZUdZded<ded<ded< ddd ZddZddZdS)Flag)rr4 _update_hookrrrr4rr.NrOr update_hookcL||_||_||dSr7)rr.r;)r:rOrr/s r,rKz Flag.__init__s)DJ#DIIgr.rr cjtdt|jrrrJs r,rz Flag.__bool__rr.r5cP||_|j||dSdSr7)r4r.r9s r,r;z Flag._sets4DJ $ %$r.r7)rOrrrr/rrr<)r=r>r?rrArKrr;r8r.r,r-r-s0) *** ))),,,,<@ """" r.r- Flag[bool]c|dd}t|||}t||t|||SNr/)popr-rrdrrOrargskwargsr/r[s r, bool_flagr: sG =$//+ g{ + +&D&$f555 -r. Flag[int]c|dd}t|||}t||t|||Sr5)r6r-rrdr/r7s r,int_flagr=G =$//+ g{ + +&D&#tV444 -r. Flag[float]c|dd}t|||}t||t|||Sr5)r6r-rrdrqr7s r, float_flagrAsG =$//+ g{ + +&D&%v666 -r. Flag[str]c|dd}t|||}t||t|||Sr5)r6r-rrdrr7s r, string_flagrD!r>r.c|dd}t|||}t||d|||S)Nr/rh)r6r-rrdr7s r, enum_flagrF(sG =$//+ g{ + +&D&&$777 -r.ceZdZUdZded<dZded<dZded<dZded <d Zd ed <dZ ded <dZ ded<dZ ded<d Z d ed<d Z d ed<dZded<dZded<dS)_GlobalExtraJitContextNrrr Any | NonerFrrrr/rrrrxla_profile_versionrrr)r=r>r?rrArrrrrrrrJrrrr8r.r,rHrH5s%)))))&*****)-----.!&&&&&',,,,,"""""+!&&&&&&&r.rHc ztj}|jp t}|jdi||_dSNr8)r global_staterrH_replace)kwgsrs r,_update_global_jit_staterQDsE"  <$:$<$<'))//B//"r.ceZdZUdZdZded<dZded<dZded<dZd ed <dZ d ed <dZ ded <dZ d ed<dZ ded<dZ d ed<dZd ed<dZd ed<dZded<dZded<dZd ed<dS)_ThreadLocalExtraJitContexta\A namedtuple containing states to add to the cache key. Just in time compilation (for jit, pmap, etc) behavior is configurable through global and thread-local options, used in the cache key. The initialization, which uses both config.py and core.py is done using `_update_thread_local_jit_state` in core.py to prevent circular imports. NrIdynamic_trace_stater8rrrrrrrz bool | Nonerz int | NonerrrrrJrr)r=r>r?r@rTrArrrrrrrrrrrJrrr8r.r,rSrSJs%)((((.#%%%%%&*))))&*****)----- $.$$$$#'''''(,,,,,.22222$((((($((((($(((((!+!!!!!!r.rSceZdZdZdZdS)_ThreadLocalStateCachea"A thread local cache for _ThreadLocalExtraJitContext The extra_jit_context in jax_jit.thread_local_state() may get updated and thus incurring dispatch overhead for comparing this python object during jit calls. We want to deduplicate the objects that have the same hash/equality to also have the same object ID, since the equality check is much faster if the object IDs match. cLtjdd|_dS)Nc|Sr7r8)xs r,rlz1_ThreadLocalStateCache.__init__..ss1r.) functools lru_cache canonicalizerJs r,rKz_ThreadLocalStateCache.__init__rs'0 +C00==Dr.N)r=r>r?r@rKr8r.r,rVrVis->>>>>r.rVc tj}|jp t}|jdi|}t ||_dSrL)rrrrSrN_thread_local_state_cacher])rOrrtmps r,update_thread_local_jit_stateraysZ"$$#  ! B%@%B%B'2#3@@EE#r."jax2tf_associative_scan_reductionsaJAX has two separate lowering rules for the cumulative reduction primitives (cumsum, cumprod, cummax, cummin). On CPUs and GPUs it uses a lax.associative_scan, while for TPUs it uses the HLO ReduceWindow. The latter has a slow implementation on CPUs and GPUs. By default, jax2tf uses the TPU lowering. Set this flag to True to use the associative scan lowering usage, and only if it makes a difference for your application. See the jax2tf README.md for more details.)rOrr#jax2tf_default_native_serialization#JAX2TF_DEFAULT_NATIVE_SERIALIZATIONTzSets the default value of the native_serialization parameter to jax2tf.convert. Prefer using the parameter instead of the flag, the flag may be removed in the future. Starting with JAX 0.4.31 non-native serialization is deprecated.jax_serialization_versionJAX_SERIALIZATION_VERSIONz6DEPRECATED: use jax_export_calling_convention_version.%jax_export_calling_convention_version%JAX_EXPORT_CALLING_CONVENTION_VERSION a The calling convention version number to use for exporting. This must be within the range of versions supported by the tf.XlaCallModule used in your deployment environment. See https://jax.readthedocs.io/en/latest/export/shape_poly.html#calling-convention-versions.'jax_export_ignore_forward_compatibility&JAX_EXPORT_IGNORE_FORWARD_COMPATIBILIYzWhether to ignore the forward compatibility lowering rules. See https://jax.readthedocs.io/en/latest/export/export.html#compatibility-guarantees-for-custom-calls. jax_platformsayComma-separated list of platform names specifying which platforms jax should initialize. If any of the platforms in this list are not successfully initialized, an exception will be raised and the program will be aborted. The first platform in the list will be the default platform. For example, config.jax_platforms=cpu,tpu means that CPU and TPU backends will be initialized, and the CPU backend will be used unless otherwise specified. If TPU initialization fails, it will raise an exception. By default, jax will try to initialize all available platforms and will default to GPU or TPU if available, and fallback to CPU otherwise.jax_pjrt_client_create_optionsz}A set of key-value pairs in the format of "k1:v1;k2:v2" strings provided to a device platform pjrt client as extra arguments.jax_enable_checkszBTurn on invariant checking for JAX internals. Makes things slower.jax_debug_key_reuseahTurn on experimental key reuse checking. With this configuration enabled, typed PRNG keys (i.e. keys created with jax.random.key()) will have their usage tracked, and incorrect reuse of a previously-used key will lead to an error. Currently enabling this leads to a small Python overhead on every call to a JIT-compiled function with keys as inputs or outputs.jax_check_tracer_leaksaMTurn on checking for leaked tracers as soon as a trace completes. Enabling leak checking may have performance impacts: some caching is disabled, and other overheads may be added. Additionally, be aware that some Python debuggers can cause false positives, so it is recommended to disable any debuggers while leak checking is enabled.jax_debug_nanszAdd nan checks to every operation. When a nan is detected on the output of a jit-compiled computation, call into the un-compiled version in an attempt to more precisely identify the operation which produced the nan.jax_debug_infszAdd inf checks to every operation. When an inf is detected on the output of a jit-compiled computation, call into the un-compiled version in an attempt to more precisely identify the operation which produced the inf.jax_log_compileszLog a message each time `jit` or `pmap` compiles an XLA computation. Logging is performed with `logging`. When this option is set, the log level is WARNING; otherwise the level is DEBUG.jax_explain_cache_misseszEach time there is a miss on one of the main caches (e.g. the tracing cache), log an explanation.. Logging is performed with `logging`. When this option is set, the log level is WARNING; otherwise the level is DEBUG.jax_log_checkpoint_residualszLog a message every time jax.checkpoint (aka jax.remat) is partially evaluated (e.g. for autodiff), printing what residuals are saved.jax_pmap_shmap_mergez/If True, pmap and shard_map API will be merged.)rOrrrcL|tj_dSr7)rrrMrr+s r,_update_jax_memories_globalrys/2#+,,,r.cL|tj_dSr7)rrrrrxs r,!_update_jax_memories_thread_localr{s58#+  ""222r.jax_enable_memorieszaIf True, will allow fetching memory kinds available on executable and annotate Shardings with it.)rrrrr jax_spmd_mode allow_all allow_jitaDecides whether Math on `jax.Array`'s that are not fully addressable (i.e. spans across multiple processes) is allowed. The options are: * allow_jit: Default, `pjit` and `jax.jit` computations are allowed to execute on non-fully addressable `jax.Array`s * allow_all: `jnp`, normal math (like `a + b`, etc), `pjit`, `jax.jit` and all other operations are allowed to execute on non-fully addressable `jax.Array`s.)rOrrrjax_distributed_debugzEnable logging useful for debugging multi-process distributed computations. Logging is performed with `logging` at WARNING level.jax_random_seed_offsetz?Offset to all random seeds (e.g. argument to jax.random.key()).c"t|SN)rrQrxs r,rlrl@#;$ $ $ r.c"t|Srrarxs r,rlrlB)F* * * r.)rOrrrrjax_legacy_prng_key)allowwarnerrorrzFSpecify the behavior when raw PRNG keys are passed to jax.random APIs.jax_enable_custom_prngzlEnables an internal upgrade that allows one to define custom pseudo-random number generator implementations.jax_default_prng_impl) threefry2x32rbg unsafe_rbgrzaSelect the default PRNG implementation, used when one is not explicitly provided at seeding time.jax_threefry_partitionableaVEnables internal threefry PRNG implementation changes that render it automatically partitionable in some cases. Without this flag, using the standard jax.random pseudo-random number generation may result in extraneous communication and/or redundant distributed computation. With this flag, the communication overheads disappear in some cases.c"t|SN)rrrxs r,rlrlfs#;"$$$$$$r.c"t|Srrrxs r,rlrlhs)F"*$*$*$r.)rOrrrrr jax_threefry_gpu_kernel_loweringzOn GPU, lower threefry PRNG operations to a kernel implementation. This makes compile times faster at a potential runtime memory cost.c"t|SN)rrrxs r,rlrlqs#;%($*$*$*r.c"t|Srrrxs r,rlrlss)F%(******r.jax_softmax_custom_jvpzUse a new custom_jvp rule for jax.nn.softmax. The new rule should improve memory usage and stability. Set True to use new behavior. See https://github.com/google/jax/pull/15677c"t|SN)rrrxs r,rlrl~rr.c"t|Srrrxs r,rlrlrr.)jax_enable_custom_vjp_by_custom_transposezyEnables an internal upgrade that implements `jax.custom_vjp` by reduction to `jax.custom_jvp` and `jax.custom_transpose`.!jax_raise_persistent_cache_errorsaGIf true, exceptions raised when reading or writing to the persistent compilation cache will be allowed through, halting program execution if not manually caught. If false, exceptions are caught and raised as warnings, allowing program execution to continue. Defaults to false so cache bugs or intermittent issues are non-fatal.*jax_persistent_cache_min_compile_time_secsg?zThe minimum compile time of a computation to be written to the persistent compilation cache. This threshold can be raised to decrease the number of entries written to the cache.)jax_persistent_cache_min_entry_size_bytesatThe minimum size (in bytes) of an entry that will be cached in the persistent compilation cache: * -1: disable the size restriction and prevent overrides. * Leave at default (0) to allow for overrides. The override will typically ensure that the minimum size is optimal for the filesystem being used for the cache. * > 0: the actual minimum size desired; no overrides.-jax_compilation_cache_include_metadata_in_keyaInclude metadata, such as file names and line numbers, in the compilation cache key. If false, the cache will still get hits even if functions or files are moved, etc. However, it means that executables loaded from the cache may have stale metadata, which may show up in, e.g., profiles.*jax_hlo_source_file_canonicalization_regexaKUsed to canonicalize the source_path metadata of HLO instructions by removing the given regex. If set, re.sub() is called on each source_file with the given regex, and all matches are removed. This can be used to avoid spurious cache misses when using the persistent compilation cache, which includes HLO metadata in the cache key.(jax_include_full_tracebacks_in_locationszAInclude Python tracebacks in MLIR locations in IR emitted by JAX. jax_traceback_in_locations_limit zLimit the number of frames at the Python traceback frames included in MLIR locations. If set to the negative value, traceback will not be limited.'jax_share_autotune_config_between_hostsaIf set to True, the coordinator process will share autotune configs other participants. This will increase overall compilation time, but will lead to equal compiled modules in each process. If both jax_share_binary_between_hosts and jax_share_autotune_config_between_hosts are set, compiled HLO will be shared when it's possible and autotune config sharing will be used as a fallback.jax_share_binary_between_hostszJIf set to True, the compiled module will be shared between hosts directly.)jax_share_binary_between_hosts_timeout_msiOz&Timeout for the compiled module share.jax_enable_pglezIf set to True and the property jax_pgle_profiling_runs is set to greater than 0, the modules will be recompiled after running specified number times with collected data provided to the profile guided latency estimator.c"t|SN)rrrxs r,rlrls#;#L#L#Lr.c"t|Srrrxs r,rlrls)F***r.jax_pgle_profiling_runszQAmount of times module should be profiled before recompilation when PGLE is used.c"t|SN)rrrxs r,rlrls#;$$$r.c"t|Srrrxs r,rlrls)F***r.jax_pgle_aggregation_percentileZzPPercentile used to aggregate performance data between devices when PGLE is used.jax_enable_compilation_cachezIf set to False, the compilation cache will be disabled regardless of whether set_cache_dir() was called. If set to True, the path could be set to a default value or via a call to set_cache_dir().jax_compilation_cache_dirzPath for the cache. Precedence: 1. A call to compilation_cache.set_cache_dir(). 2. The value of this flag set in the command line or by default.jax_compilation_cache_max_sizea[The maximum size (in bytes) allowed for the persistent compilation cache. When set, the least recently accessed cache entry(s) will be deleted once the total cache directory size exceeds the specified limit. Caching will be disabled if this value is set to 0. A special value of -1 indicates no limit, allowing the cache size to grow indefinitely.jax_default_dtype_bits3264zSpecify bit width of default dtypes, either 32-bit or 64-bit. This is a temporary flag that will be used during the process of deprecating the ``jax_enable_x64`` flag.jax_numpy_dtype_promotionstandardstrictzSpecify the rules used for implicit type promotion in operations between arrays. Options are "standard" or "strict"; in strict-mode, binary operations between arrays of differing strongly-specified dtypes will result in an error.c"t|SN)rrrxs r,rlrl>sS999r.c"t|Srrrxs r,rlrl@s##>>>r.)rOrrrrrcL|tj_dSr7)rrrMrrxs r,_update_x64_globalrCs*-#+'''r.cL|tj_dSr7)rrrrrxs r,_update_x64_thread_localrFs03#+  ""---r.jax_enable_x64zEnable 64-bit types to be used x64_enabledctjSr7)rr4)rs r,rlrlSs *2Br.cL|tj_dSr7)rrrMrrxs r,_update_default_device_globalrUs.1#++++r.cL|tj_dSr7)rrrrrxs r,#_update_default_device_thread_localrYs47#+  ""111r.c|t|tjsjdtt |vr8t dt|t |dStd|dSdS)NDevicez=Allowing non-`xla_client.Device` default device: %s, type: %szWjax.default_device must be passed a Device object (e.g. `jax.devices('cpu')[0]`), got: ) rrrrrloggerinforeprr*rxs r,_validate_default_devicer]s_ZZ->??_3tCyy>>!! kk I s))T#YY    f ?7:?? @ @@___r.jax_default_deviceaConfigure the default device for JAX operations. Set to a Device object (e.g. ``jax.devices("cpu")[0]``) to use that Device as the default device for JAX operations and jit'd function calls (there is no effect on multi-device computations, e.g. pmapped function calls). Set to None to use the system default device. See :ref:`faq-data-placement` for more information on device placement.r$cL|tj_dSr7)rrrMrrxs r,_update_disable_jit_globalrzs+.#+(((r.cL|tj_dSr7)rrrrrxs r, _update_disable_jit_thread_localr}s14#+  ""...r.jax_disable_jitz6Disable JIT compilation and just call original Python.jax_numpy_rank_promotion)rrraisezXControl NumPy-style automatic rank promotion broadcasting ("allow", "warn", or "raise").c"t|SN)rrrxs r,rlrlsC888r.c"t|Srrrxs r,rlrls#===r.jax_default_matmul_precision)rhighhighestbfloat16 tensorfloat32float32aKControl the default matmul and conv precision for 32bit inputs. Some platforms, like TPU, offer configurable precision levels for matrix multiplication and convolution computations, trading off accuracy for speed. The precision can be controlled for each operation; for example, see the :func:`jax.lax.conv_general_dilated` and :func:`jax.lax.dot` docstrings. But it can be useful to control the default behavior obtained when an operation is not given a specific precision. This option can be used to control the default precision level for computations involved in matrix multiplication and convolution on 32bit inputs. The levels roughly describe the precision at which scalar products are computed. The 'bfloat16' option is the fastest and least precise; 'float32' is similar to full float32 precision; 'tensorfloat32' is intermediate. c"t|SN)rrrxs r,rlrls<<<r.c"t|Srrrxs r,rlrls#SAAAr.jax_traceback_filtering)r% tracebackhide remove_framesquiet_remove_framesautoraxControls how JAX filters internal frames out of tracebacks. Valid values are: * "off": disables traceback filtering. * "auto": use "tracebackhide" if running under a sufficiently new IPython, or "remove_frames" otherwise. * "tracebackhide": adds "__tracebackhide__" annotations to hidden stack frames, which some traceback printers support. * "remove_frames": removes hidden frames from tracebacks, and adds the unfiltered traceback as a __cause__ of the exception. * "quiet_remove_frames": removes hidden frames from tracebacks, and adds a brief message (to the __cause__ of the exception) describing that this has happened. jax_bcoo_cusparse_loweringz&Enables lowering BCOO ops to cuSparse.jax_dynamic_shapesJAX_DYNAMIC_SHAPESzOEnables experimental features for staging out computations with dynamic shapes.c"t|SN)rrrxs r,rlrlsc222r.c"t|Srrrxs r,rlrls#3777r.jax_remat_opt_barrierz9Enables using optimization-barrier op for lowering remat.jax_eager_pmapz9Enable eager-mode pmap when jax_disable_jit is activated."jax_custom_vjp_disable_shape_checkz>Disable the check from #19009 to enable some custom_vjp hacks.*jax_experimental_unsafe_xla_runtime_errorsaEnable XLA runtime errors for jax.experimental.checkify.checks on CPU and GPU. These errors are async, might get lost and are not very readable. But, they crash the computation and enable you to write jittable checks without needing to checkify. Does not work under pmap/pjit.rzOptional profile version for XLA compilation. This is meaningful only when XLA is configured to support the remote compilation profile feature.c"t|SN)rJrrxs r,rlrls#;$!$!$!r.c"t|Srrrxs r,rlrls)F*!*!*!r.Iterator[None]c#|Ktj}|j}d|_ dV||_dS#||_wxYw)zEIndicates that the current context is an explicit device_put*() call.TN)rrexplicit_device_putstateprevs r,explicit_device_put_scoperV  / 1 1%  "$"%% EEE $EE$$$$2 ;c#|Ktj}|j}d|_ dV||_dS#||_wxYw)zDIndicates that the current context is an explicit device_get() call.TN)rrexplicit_device_getrs r,explicit_device_get_scoper rrc|t||ddS|dkr"t||tjjdS|dkr"t||tjjdS|dkr"t||tjjdS|dkr"t||tjjdS|dkr"t||tjjdSJd|) z;Applies the transfer guard level within transfer_guard_lib.Nrlogdisallow log_explicitdisallow_explicitFzInvalid transfer guard level )rrTransferGuardLevelALLOWLOGDISALLOW LOG_EXPLICITDISALLOW_EXPLICIT)rkeyr+s r,_update_transfer_guardrs[ E3 g~~ E3*=CDDDDD e|| E3*=ABBBBB j E3*=FGGGGG n E3*=JKKKKK !!! E3*=OPPPPP77#77777r.!jax_transfer_guard_host_to_device)rrrrr zQSelect the transfer guard level for host-to-device transfers. Default is "allow".cFttjd|SNhost_to_devicerrrMrxs r,rlrl/$#9'))+;S$B$Br.cFttjd|Srrrrrxs r,rlrl1$)?-//1A3*H*Hr.#jax_transfer_guard_device_to_devicezSSelect the transfer guard level for device-to-device transfers. Default is "allow".cFttjd|SNdevice_to_devicerrxs r,rlrl>s$#9'))+=s$D$Dr.cFttjd|Srrrxs r,rlrl@s$)?-//1CS*J*Jr.!jax_transfer_guard_device_to_hostzQSelect the transfer guard level for device-to-host transfers. Default is "allow".cFttjd|SNdevice_to_hostrrxs r,rlrlMrr.cFttjd|Sr"rrxs r,rlrlOrr.cFdD]}t||dS)N)rrr )rrP)r+rOs r,!_update_all_transfer_guard_globalr&Rs64d MM$r.jax_transfer_guardzSelect the transfer guard level for all transfers. This option is set-only; the transfer guard level for a specific direction should be read using the per-transfer direction option. Default is "allow".)rOrrrrrc#zKtj5}|t||t ||t ||t |dVddddS#1swxYwYdS)a A contextmanager to control the transfer guard level for all transfers. For more information, see https://jax.readthedocs.io/en/latest/transfer_guard.html Args: new_val: The new thread-local transfer guard level for all transfers. Yields: None. N)r ExitStack enter_contexttransfer_guard_host_to_devicetransfer_guard_device_to_devicetransfer_guard_device_to_host_transfer_guard)rstacks r,transfer_guardr0fs  5g>>??? 7@@AAA 5g>>??? 00111 EEE                   sB B00B47B4module_names_strctj|sdS|d}|D]}tj|dS)N,)rdisable_all_debug_loggingsplitenable_debug_logging)r1 module_names module_names r,_update_debug_log_modulesr9{s^*,,,   F!'',,,!55k' 444455r.jax_debug_log_moduleszyComma-separated list of module names (e.g. "jax" or "jax._src.xla_bridge,jax._src.dispatch") to enable debug logging for.)rOrrrjax_pmap_no_rank_reductionzCIf True, pmap shards have a the same rank as their enclosing array.jax_use_shardy_partitionerzWhether to lower to Shardy. Shardy is a new open sourced propagation framework for MLIR. Currently Shardy is experimental in JAX. See www.github.com/openxla/shardyc"t|SN)rrrxs r,rlrls#;"$$$r.c"t|Sr>rrxs r,rlrls)F"***r.)rrrrrr)rrrr/rr/)rOrrrrrrrrrrrrrrr)rOrrrrrrrrrrrrr)rOrrrrrrrrrrrrr) rOrrr/rrrrrrrr) rOrrrqrrrrrrrr) rOrrrrrrrrrrr) rOrrrrrrrrrrr)rOrrrrrrrrrrrrr))rr3)rr;)rr?)rrB)rr)rrrr)r1r) __future__rcollections.abcrrrrrr[rloggingr'r threadingtypingrr r r r r rjax._srcr jax._src.librrrr getLoggerr=rrr-r1r3rCrrrRrPrrrrrlocalrrrrrrr rrr%r(r#r-r:r=rArDrFrrHrQrSrVr_rarbrcrerg#export_ignore_forward_compatibilityrlrm enable_checksrcheck_tracer_leakspartialchecking_leaks debug_nans debug_infs log_compilesexplain_cache_misseslog_checkpoint_residualspmap_shmap_mergeryr{r spmd_modedistributed_debugrlegacy_prng_keyenable_custom_prngdefault_prng_implrrr%enable_custom_vjp_by_custom_transposeraise_persistent_cache_errors&persistent_cache_min_compile_time_secs%persistent_cache_min_entry_size_bytes)compilation_cache_include_metadata_in_keyr$include_full_tracebacks_in_locationstraceback_in_locations_limit#share_autotune_config_between_hostsshare_binary_between_hosts%share_binary_between_hosts_timeout_msrrpgle_aggregation_percentileenable_compilation_cachecompilation_cache_dircompilation_cache_max_sizedefault_dtype_bitsrrrrrIremoverrrrrrrrrrrtraceback_filteringbcoo_cusparse_loweringrr(rremat_opt_barrier eager_pmapcustom_vjp_disable_shape_checkxla_runtime_errorsrrrrrr+r,r-r&r.r0r9pmap_no_rank_reductionrr8r.r,rpsH#"""""BBBBBBBBBBBB NNNNNNNNNNNNNNNNNN ++++++############  8 $ $ WT]]QQQQ(//// ) ) ) ) )(2, ) ) )s*s*s*s*s*s*s*s*l(((B   4 Y[[ %io''Q$Q$Q$Q$Q$GBKQ$Q$Q$jH )9=EIL L L L L L j8FJ - - - - - - j8 > > > > >Y_ > > >3244FFF&0Z -  5 & & &"'1j . H:D A A K ' ' '#&I $ G/ 3 3@ )2 0 G;Q ? ? g ) ) )%'1j 2 H=u E E q '''#&%       "7!6 )  J"L"L"L    MOOO *   SUUU Z !  EGGG#"#5t<< Z   $&&& Z   $&&& z    "z #  *,,,&: '  :    : <<< 333999*  2> , ... J k*  ? A A A J  Y ! K    * ***    Z !   < >>>J 555  1 333$ %    $$$$ % % % *z +  **** + + + Z !   C     ! ! !)3 4   F )H)H)H%!+ ,  !!!*5 5  A*C*C*C&)2 4  B )D )D )D%-7J 8  + - - -)*?)> 5  ***&(2z 3 K ((($ )y +      '1j 2   ' ' '#(Z )   )2 4  1)))% j   ML     i "     (i *  &: '  .- $  M'Y )  '    Z !t  8 :::# $X&  ,::?? @ @ @...444Z   ))5 777 ##$4555 xx(B(BCCDDD222888 @ @ @('   N 5@& ( ( (///555j   B1= ??? "z #***  +99>> ? ? ?/. 'VVV  I ==BB-CCC2!j $   *$ %  2555  D/44 5 5 3388999J  EHHH Z    D FFF ", -   I "K"K"K  Z 5  "$) "  !!!!    %%%% %%%%888"!4 3 ,  BBHH !I !I !I#6"5 .  DDJJ #K #K #K!4 3 ,  BBHH !I !I !I &%   9 : : :     (5555  1 2222$ % M $ %   (r.