Vpf * ddlmZddlZddlmZmZmZmZddlm Z ddl Z ddl Z ddl m Z ddl ZddlZddlmZmZmZmZmZddlmZddlmZdd lmZmZdd lmZed Zeje ed Z!ed e"eZ#edeZ$eZ%ej&Z&ej'Z'e(e'_)de'_(ej*dddddZ+e(e+_)de+_(ej*dddddZ,e(e,_)de,_(e dddZ-edd!Z.e ddd#Z/e ddd%Z0edd(Z1edd*Z2edd,Z3dd-Z4e ddd0Z5ed1eeZ6ed2eZ7edd:Z8edd<Z9edd=ddAZ:eddCZ;eddHZ<ej=dIdJdKgZ>e?e>dLdMe@e>dNdOeAe>dPdQe"de>dRdSiZBdTZCeDZEedd=ddWZFe dddYZFeeEdfddZZFedd=dd[ZGe8ejHd\d]d^ZIe8ejJeId_Gd`daZKeGdbdce j ZLe8eLddde dddiZMddlZN dddnZO dddrZPddsZQdtZRee duGdvdwZSee duGdxdyZTee duGdzd{ZUee duGd|d}ZVeeSeTeUeVfZWed~eZXe?eXdfZYeddZZGddeZ[ddZ\iZ]e\e?de\e@de\eAde\ejJde\ejHde dddZ^eddZ_eddZ`eddZae dddZbe dddZc dddZdedZe dddZfedd=ddZgddZh dddZidS)) annotationsN)CallableHashableIterableSequence) dataclass)partial)Any NamedTupleTypeVarUnionoverload)traceback_util)pytree)safe_zip set_module)unzip2z jax.tree_utilTTyp)boundHdefault_registryFT) enable_none enable_tupleenable_namedtuple enable_list enable_dictnone_leaf_registrydispatch_registrytreer is_leafCallable[[Any], bool] | Nonereturntuple[list[Leaf], PyTreeDef]c8t||S)z"Alias of :func:`jax.tree.flatten`.rflattenr r!s R/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/_src/tree_util.py tree_flattenr*Is  ! !$ 0 00treedef PyTreeDefleavesIterable[Leaf]c,||S)z$Alias of :func:`jax.tree.unflatten`.) unflatten)r,r.s r)tree_unflattenr2Qs   6 " ""r+ list[Leaf]cDt||dS)z!Alias of :func:`jax.tree.leaves`.rr&r(s r) tree_leavesr5W  ! !$ 0 0 33r+None | Callable[[Any], bool]cDt||dS)z$Alias of :func:`jax.tree.structure`.r&r(s r)tree_structurer:_r6r+treedefsIterable[PyTreeDef]cPtjtt|S)a Makes a tuple treedef from an iterable of child treedefs. Args: treedefs: iterable of PyTree structures Returns: a single treedef representing a tuple of the structures Examples: >>> import jax >>> x = [1, 2, 3] >>> y = {'a': 4, 'b': 5} >>> x_tree = jax.tree.structure(x) >>> y_tree = jax.tree.structure(y) >>> xy_tree = jax.tree_util.treedef_tuple([x_tree, y_tree]) >>> xy_tree == jax.tree.structure((x, y)) True See Also: - :func:`jax.tree_util.treedef_children` )rtuplerlist)r;s r) treedef_tupler@gs. &X 7 77r+list[PyTreeDef]c*|S)aReturn a list of treedefs for immediate children Args: treedef: a single PyTreeDef Returns: a list of PyTreeDefs representing the children of treedef. Examples: >>> import jax >>> x = [(1, 2), 3, {'a': 4}] >>> treedef = jax.tree.structure(x) >>> jax.tree_util.treedef_children(treedef) [PyTreeDef((*, *)), PyTreeDef(*), PyTreeDef({'a': *})] >>> _ == [jax.tree.structure(vals) for vals in x] True See Also: - :func:`jax.tree_util.treedef_tuple` )childrenr,s r)treedef_childrenrEs,     r+boolc|jdkS)aReturn True if the treedef represents a leaf. Args: treedef: tree to check Returns: True if treedef is a leaf (i.e. has a single node); False otherwise. Examples: >>> import jax >>> tree1 = jax.tree.structure(1) >>> jax.tree_util.treedef_is_leaf(tree1) True >>> tree2 = jax.tree.structure([1, 2]) >>> jax.tree_util.treedef_is_leaf(tree2) False r9) num_nodesrDs r)treedef_is_leafrIs&  a r+c.|jdko |jdkS)Nr9)rH num_leavesrDs r)treedef_is_strict_leafrLs  a  ;G$6!$;;r+iterable Iterable[Any]c|tjt|St|}|t ||kS)a,Tests whether all elements in the given iterable are all leaves. This function is useful in advanced cases, for example if a library allows arbitrary map operations on a flat iterable of leaves it may want to check if the result is still a flat iterable of leaves. Args: iterable: Iterable of leaves. Returns: A boolean indicating if all elements in the input are leaves. Examples: >>> import jax >>> tree = {"a": [1, 2, 3]} >>> assert all_leaves(jax.tree_util.tree_leaves(tree)) >>> assert not all_leaves([tree]) )r all_leavesrr?r5)rMr!lsts r)rPrPs=* _  -x 8 88 x..C +c7++ ++r+ _Children_AuxDatanodetypetype[T] flatten_func)Callable[[T], tuple[_Children, _AuxData]]unflatten_func"Callable[[_AuxData, _Children], T]Nonect|||t|||t|||t ||t |<dS)aC Extends the set of types that are considered internal nodes in pytrees. See :ref:`example usage `. Args: nodetype: a Python type to register as a pytree. flatten_func: a function to be used during flattening, taking a value of type ``nodetype`` and returning a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable auxiliary data to be stored in the treedef and to be passed to the ``unflatten_func``. unflatten_func: a function taking two arguments: the auxiliary data that was returned by ``flatten_func`` and stored in the treedef, and the unflattened children. The function should return an instance of ``nodetype``. See also: - :func:`~jax.tree_util.register_static`: simpler API for registering a static pytree. - :func:`~jax.tree_util.register_dataclass`: simpler API for registering a dataclass. - :func:`~jax.tree_util.register_pytree_with_keys` - :func:`~jax.tree_util.register_pytree_node_class` - :func:`~jax.tree_util.register_pytree_with_keys_class` Examples: First we'll define a custom type: >>> class MyContainer: ... def __init__(self, size): ... self.x = jnp.zeros(size) ... self.y = jnp.ones(size) ... self.size = size If we try using this in a JIT-compiled function, we'll get an error because JAX does not yet know how to handle this type: >>> m = MyContainer(size=5) >>> def f(m): ... return m.x + m.y + jnp.arange(m.size) >>> jax.jit(f)(m) # doctest: +IGNORE_EXCEPTION_DETAIL Traceback (most recent call last): ... TypeError: Cannot interpret value of type as an abstract array; it does not have a dtype attribute In order to make our object recognized by JAX, we must register it as a pytree: >>> def flatten_func(obj): ... children = (obj.x, obj.y) # children must contain arrays & pytrees ... aux_data = (obj.size,) # aux_data must contain static, hashable data. ... return (children, aux_data) ... >>> def unflatten_func(aux_data, children): ... # Here we avoid `__init__` because it has extra logic we don't require: ... obj = object.__new__(MyContainer) ... obj.x, obj.y = children ... obj.size, = aux_data ... return obj ... >>> jax.tree_util.register_pytree_node(MyContainer, flatten_func, unflatten_func) Now with this defined, we can use instances of this type in JIT-compiled functions. >>> jax.jit(f)(m) Array([1., 2., 3., 4., 5.], dtype=float32) N)r register_noderr_RegistryEntry _registry)rTrVrXs r)register_pytree_noder_sgH  <HHH""8\>JJJ!!(L.III&|^DD)Hr+clscVt|tjd|j|S)aExtends the set of types that are considered internal nodes in pytrees. This function is a thin wrapper around ``register_pytree_node``, and provides a class-oriented interface. Args: cls: a type to register as a pytree Returns: The input class ``cls`` is returned unchanged after being added to JAX's pytree registry. This return value allows ``register_pytree_node_class`` to be used as a decorator. See also: - :func:`~jax.tree_util.register_static`: simpler API for registering a static pytree. - :func:`~jax.tree_util.register_dataclass`: simpler API for registering a dataclass. - :func:`~jax.tree_util.register_pytree_node` - :func:`~jax.tree_util.register_pytree_with_keys` - :func:`~jax.tree_util.register_pytree_with_keys_class` Examples: Here we'll define a custom container that will be compatible with :func:`jax.jit` and other JAX transformations: >>> import jax >>> @jax.tree_util.register_pytree_node_class ... class MyContainer: ... def __init__(self, x, y): ... self.x = x ... self.y = y ... def tree_flatten(self): ... return ((self.x, self.y), None) ... @classmethod ... def tree_unflatten(cls, aux_data, children): ... return cls(*children) ... >>> m = MyContainer(jnp.zeros(4), jnp.arange(4)) >>> def f(m): ... return m.x + 2 * m.y >>> jax.jit(f)(m) Array([0., 2., 4., 6.], dtype=float32) r*)r_op methodcallerr2)r`s r)register_pytree_node_classrds)XsBON;;S=OPPP *r+r!fCallable[..., Any]restct||\}|gfd|Dz}fdt|DS)zAlias of :func:`jax.tree.map`.c:g|]}|S flatten_up_to.0rr,s r) ztree_map..Vs'BBB70033BBBr+c3"K|] }|V dSNrkroxsrfs r) ztree_map..Ws)==b11b6======r+)r*r1zip)rfr r!rhr.rPr,s` @r)tree_maprxOsh !w///&'xBBBBTBBBB*   ====C,<=== = ==r+ruc,||S)aBuild a treedef from a nested iterable structure Args: treedef: the PyTreeDef structure to build. xs: nested iterables matching the arity as the treedef Returns: object with structure defined by treedef See Also: - :func:`jax.tree.unflatten` Examples: >>> import jax >>> tree = [(1, 2), {'a': 3, 'b': 4}] >>> treedef = jax.tree.structure(tree) Both ``build_tree`` and :func:`jax.tree_util.tree_unflatten` can reconstruct the tree from new values, but ``build_tree`` takes these values in terms of a nested rather than flat structure: >>> jax.tree_util.build_tree(treedef, [[10, 11], [12, 13]]) [(10, 11), {'a': 12, 'b': 13}] >>> jax.tree_util.tree_unflatten(treedef, [10, 11, 12, 13]) [(10, 11), {'a': 12, 'b': 13}] )from_iterable_tree)r,rus r) build_treer{Zs8  # #B ' ''r+ outer_treedef inner_treedefPyTreeDef | Nonepytree_to_transposec t|\}}|(t||d}|j |j}|j |zkr*||}t d|d|t | fdt|D}t|}ttt||} t|| S)z$Alias of :func:`jax.tree.transpose`.Nrz Mismatch z != cFg|]}fdtDS)c.g|]}tSrk)next)ro_ iter_flats r)rqz-tree_transpose...s2221tI222r+)range)ro__ inner_sizers r)rqz"tree_transpose..sC   792222j 1 1222   r+) r*r:rmrKcompose TypeErroriterrrwmapr r2) r|r}rflatr, outer_sizeexpected_treedefloltransposed_lolsubtreesrrs @@r)tree_transposerys 233-$"=#>#>?R#S#STU#VWWM'*'* J344$,,];; DDD2BDD E EE4jj)     =B:=N=N   #9. 77 H H(  x 0 00r+r]to_iter from_iterc |dfSrsrkrus r)rs b$Zr+c t|Srsr>rrus r)rrs uRyyr+c |dfSrsrkrs r)rrs RJr+c t|Srs)r?rs r)rrs d2hhr+cptt|dddS)N)rsorteditemsrs r)rrs)F6"((**+=+=$>$>ttt$Dr+c<tt||Srs)dictrw)keysrus r)rrs$s4}}*=*=r+cdS)N)rkNrk)zs r)rrsr+cdSrsrkrs r)rrs4r+ct|\}}tfd|}||S)z0Replaces ``None`` in ``tree`` with ``sentinel``.c|n|Srsrk)xsentinels r)rz _replace_nones..sQYAr+)rr'rr1)rr r.r,s` r)_replace_nonesrsG&..t44/&' 5555v > >&   6 " ""r+functionCallable[[T, Any], T]cdSrsrk)rr r!s r) tree_reducer Cr+ initializercdSrsrkrr rr!s r)rrrr+c|tur$tj|t||Stj|t|||S)z!Alias of :func:`jax.tree.reduce`.re)no_initializer functoolsreducer5rs r)rrsQ N""  Hk$&H&H&H I II  Hk$&H&H&H+ V VVr+c>tt||S)zAlias of :func:`jax.tree.all`.re)allr5r(s r)tree_allrs [w / / / 0 00r+ct|t|fSrs)r>valuesrrs r)rrs)U188::  affhh 0r+cFtjt||Srs) collections OrderedDictr)rrs r)rrs{.xf/E/EFFr+ctt}tfd|Dj|ffS)Nc3(K|] }|V dSrsrk)rokds r)rvz'_flatten_defaultdict..s'""qt""""""r+)r>rdefault_factory)rrs` r)_flatten_defaultdictrsF vayy  $ """"T""" " "Q%6$= ==r+c`tj|dt|d|SNrr9)r defaultdictr)srs r)rrs%K+AaD(1Q42H2HIIr+c0eZdZdZdZdZdZdZdZdS)_HashableCallableShimzGObject that delegates __call__, __hash__, and __eq__ to another object.c||_dSrsfun)selfrs r)__init__z_HashableCallableShim.__init__s DHHHr+c|j|i|Srsr)rargskws r)__call__z_HashableCallableShim.__call__s 48T R  r+c*t|jSrs)hashrrs r)__hash__z_HashableCallableShim.__hash__s >>r+cbt|tr|j|jkS|j|kSrs) isinstancerr)rothers r)__eq__z_HashableCallableShim.__eq__s1%.//# X "" 8u r+cd|jdS)Nz_HashableCallableShim()rrs r)__repr__z_HashableCallableShim.__repr__s 1DH 1 1 11r+N) __name__ __module__ __qualname____doc__rrrrrrkr+r)rrseOO!!! 22222r+rc"eZdZdZfdZxZS)PartialamA version of functools.partial that works in pytrees. Use it for partial function evaluation in a way that is compatible with JAX's transformations, e.g., ``Partial(func, *args, **kwargs)``. (You need to explicitly opt-in to this behavior because we didn't want to give functools.partial different semantics than normal function closures.) For example, here is a basic usage of ``Partial`` in a manner similar to ``functools.partial``: >>> import jax.numpy as jnp >>> add_one = Partial(jnp.add, 1) >>> add_one(2) Array(3, dtype=int32, weak_type=True) Pytree compatibility means that the resulting partial function can be passed as an argument within transformed JAX functions, which is not possible with a standard ``functools.partial`` function: >>> from jax import jit >>> @jit ... def call_func(f, *args): ... return f(*args) ... >>> call_func(add_one, 2) Array(3, dtype=int32, weak_type=True) Passing zero arguments to ``Partial`` effectively wraps the original function, making it a valid argument in JAX transformed functions: >>> call_func(Partial(jnp.add), 1, 2) Array(3, dtype=int32, weak_type=True) Had we passed ``jnp.add`` to ``call_func`` directly, it would have resulted in a ``TypeError``. Note that if the result of ``Partial`` is used in the context where the value is traced, it results in all bound arguments being traced when passed to the partially-evaluated function: >>> print_zero = Partial(print, 0) >>> print_zero() 0 >>> call_func(print_zero) # doctest:+ELLIPSIS Tracedwith ct|tjrU|}t|}t j||g|Ri|}|j|_|j|_|j|_|St j||g|Ri|Srs) rrr rsuper__new__funcrkeywords)klassrrr original_funcout __class__s r)rzPartial.__new__s $ )** 7m "= 1 1d EGGOE4 5$ 5 5 5" 5 5c$di$di#,dm j UWW_UD 64 6 6 62 6 66r+)rrrrr __classcell__)rs@r)rrsC..`777777777r+rc,|j|jf|jfSrs)rrr)partial_s r)rr0sx}h&78(-Hr+c:t|g|dRi|dSr)r)rrus r)rr1s&WT3BqE333RU33r+ prefix_tree full_tree list[Any]cFgdfd}t||||S)Nc*t|jSrs)r:rK)ts r)rz"broadcast_prefix..=s**5r+cH|g|zSrs)extend)rsubtreerKresults r)rz"broadcast_prefix..>s$&--jj6I6I0I"J"Jr+re)rx)rrr! add_leavesrKrs @@r)broadcast_prefixr6s@ &55*JJJJJ* :{Iw???? -r+rtuple[Iterable[Any], Hashable]c|t|}|tdt||S)aFlatten the given pytree node by one level. Args: pytree: A valid pytree node, either built-in or registered via :func:`register_pytree_node` or related functions. Returns: A pair of the pytrees flattened children and its hashable metadata. Raises: ValueError: If the given pytree is not a built-in or registered container via ``register_pytree_node`` or ``register_pytree_with_keys``. Examples: >>> import jax >>> from jax._src.tree_util import flatten_one_level >>> flattened, meta = flatten_one_level({'a': [1, 2], 'b': {'c': 3}}) >>> flattened ([1, 2], {'c': 3}) >>> meta ('a', 'b') Nzcan't tree-flatten type: )rflatten_one_level ValueErrortype)rrs r)rrDs>. **622#[ ?f?? @ @@ Jr+!list[Callable[[str], ValueError]]c@ttd|||SNrk)r? _prefix_error)rrr!s r) prefix_errorsr cs  mB Y@@ A AAr+tree1tree2'Iterable[tuple[KeyPath, str, str, str]]c#:Ktd|||Ed{VdS)aHelper to describe structural differences between two pytrees. Args: tree1, tree2: pytrees known to have different structure. Usage: raise Exception( "Value 1 and value 2 must have the same pytree structure, but they have " "the following structural differences: " + (" ".join( f" - {keystr(path)} is a {thing1} in value 1 and a {thing2} in " f" value 2, so {explanation}. " for path, thing1, thing2, explanation in equality_errors(val1, val2)))) rkN)_equality_errors)r r r!s r)equality_errorsrjs4&b% 8 8888888888r+c tdtftdddi}tt||g|jzt||g|jzS)z0Like `equality_errors` but invoked on PyTreeDef.LeafMetacdS)Nz pytree leafrk)rs r)rz+equality_errors_pytreedef..s=r+)rLeafrk)rrrr2rK)r r leafs r)equality_errors_pytreedefrs [ Jj4'41H1H#I#I#I J J6SUWY Z Z \ \$ v8H/HII'v8H/HII K KKr+c #PKtt||r tt||rdSt|t|kr>|tt|tt|dfVdSt |t t frt|t|ksJt|t|krR|t|jdt|t|jdt|dfVdSt|\}}t|\}}t |}t |}t|t|} } d dt| t| D} n #d} YnxYwt|t|kr|t|dt|d t|d krd ndt|dt|d t|d krd ndd | od | dzfVdS||kr.|t|d|t|d|dfVdS|| ksJd|d| t|||D]#\} } } tg|| R| | |Ed{V$dS)Nreztheir Python types differ of length zthe lengths do not match c3>K|]}t|jVdSrs)reprkeyrors r)rvz#_equality_errors..sBEEADKKEEEEEEr+z with z childr9renz$the numbers of children do not matchz., with the symmetric difference of key sets: {}z with pytree metadata z'the pytree node metadata does not matchz-equal pytree nodes gave different tree keys:  and )rLr:rstrrr?r>lenrr _child_keysjoinsetsymmetric_differencerwr)patht1t2r! t1_childrent1_meta t2_childrent2_metat1_keyst2_keysdiffrc1c2s r)rrs^B@@@AAK^B@@@AAKDJF "XXb DHH s488}}.I IIII FT5M"" 88tBxx     2ww#b'' R! 7 7c"gg 7 7R! 7 7c"gg 7 7 ' )))) f*2..+w*2..+wk""+k""+ __k"oo7' 88EELL55c'llCCEEE E EDD DDD[)))) 2hh 6 6c+.. 6 6;''!++ee 6 62hh 6 6c+.. 6 6;''!++ee 6 6 1  OOtOOO Q      F  2hh 7 7g 7 72hh 7 7g 7 7 4 6666 F G   MgMMGMM   w [99==iaR $  BG<<<<<<<<<<==s 0A G==H)frozenc$eZdZUdZded<dZdS) SequenceKeyDStruct for use with :func:`jax.tree_util.register_pytree_with_keys`.intidxcd|jdSN[])r9rs r)__str__zSequenceKey.__str__ tx   r+Nrrrr__annotations__r>rkr+r)r6r6s7ML (((r+r6c$eZdZUdZded<dZdS)DictKeyr7rrcd|jdSr;rrs r)r>zDictKey.__str__r?r+Nr@rkr+r)rCrCs7ML---r+rCc$eZdZUdZded<dZdS) GetAttrKeyr7r"namecd|jS)N.rHrs r)r>zGetAttrKey.__str__s ty??r+Nr@rkr+r)rGrGs7ML )))r+rGc$eZdZUdZded<dZdS)FlattenedIndexKeyr7r8rcd|jdS)Nz []rErs r)r>zFlattenedIndexKey.__str__s '48 ' ' ''r+Nr@rkr+r)rMrMs7ML ((((((((r+rMKeyEntry.rKeyPathc@dd|DS)a=Helper to pretty-print a tuple of keys. Args: keys: A tuple of ``KeyEntry`` or any class that can be converted to string. Returns: A string that joins all string representations of the keys. Examples: >>> import jax >>> keys = (0, 1, 'a', 'b') >>> jax.tree_util.keystr(keys) '01ab' rc,g|]}t|Srk)r"rs r)rqzkeystr..s'''Q#a&&'''r+)r%)rs r)keystrrSs% ''$''' ( ((r+c$eZdZUded<ded<dS)_RegistryWithKeypathsEntryrgflatten_with_keysrXN)rrrrArkr+r)rUrUs*''''$$$$$$r+rUtyhandler#Callable[[T], tuple[KeyEntry, ...]]c|fd}tvr*t|tjt<dSdS)Nct|\}}tt|||fSrs)r^rr?rw)rurCr,rXrWs r)rVz-_register_keypaths..flatten_with_keyssD!" --b11Hg GGBKK** + +W 44r+)r^rUr_registry_with_keypaths)rWrXrVs`` r)_register_keypathsr]s[5555559__"<9R=2##B_r+chtdtt|DS)Nc34K|]}t|VdSrsr6rois r)rvz..s(CCqKNNCCCCCCr+r>rr#rs r)rrs)eCCE#b''NNCCCCCr+chtdtt|DS)Nc34K|]}t|VdSrsr`ras r)rvz..s(BBa;q>>BBBBBBr+rcrs r)rrs)UBB5R>>BBBBBr+cNtdt|DS)Nc34K|]}t|VdSrsrCrs r)rvz..s()I)I'!**)I)I)I)I)I)Ir+)r>rrs r)rrs#E)I)IfRjj)I)I)I$I$Ir+cXtd|DS)Nc34K|]}t|VdSrsrhrs r)rvz..(,J,JAWQZZ,J,J,J,J,J,Jr+r>rrs r)rr%u,J,J,J,J,J'J'Jr+cXtd|DS)Nc34K|]}t|VdSrsrhrs r)rvz..rkr+rlrs r)rrrmr+rV>Callable[[T], tuple[Iterable[tuple[KeyEntry, Any]], _AuxData]]&Callable[[_AuxData, Iterable[Any]], T]4None | Callable[[T], tuple[Iterable[Any], _AuxData]]cl|sfd}|}t|||t|t|<dS)a Extends the set of types that are considered internal nodes in pytrees. This is a more powerful alternative to ``register_pytree_node`` that allows you to access each pytree leaf's key path when flattening and tree-mapping. Args: nodetype: a Python type to treat as an internal pytree node. flatten_with_keys: a function to be used during flattening, taking a value of type ``nodetype`` and returning a pair, with (1) an iterable for tuples of each key path and its child, and (2) some hashable auxiliary data to be stored in the treedef and to be passed to the ``unflatten_func``. unflatten_func: a function taking two arguments: the auxiliary data that was returned by ``flatten_func`` and stored in the treedef, and the unflattened children. The function should return an instance of ``nodetype``. flatten_func: an optional function similar to ``flatten_with_keys``, but returns only children and auxiliary data. It must return the children in the same order as ``flatten_with_keys``, and return the same aux data. This argument is optional and only needed for faster traversal when calling functions without keys like ``tree_map`` and ``tree_flatten``. Examples: First we'll define a custom type: >>> class MyContainer: ... def __init__(self, size): ... self.x = jnp.zeros(size) ... self.y = jnp.ones(size) ... self.size = size Now register it using a key-aware flatten function: >>> from jax.tree_util import register_pytree_with_keys_class, GetAttrKey >>> def flatten_with_keys(obj): ... children = [(GetAttrKey('x'), obj.x), ... (GetAttrKey('y'), obj.y)] # children must contain arrays & pytrees ... aux_data = (obj.size,) # aux_data must contain static, hashable data. ... return children, aux_data ... >>> def unflatten(aux_data, children): ... # Here we avoid `__init__` because it has extra logic we don't require: ... obj = object.__new__(MyContainer) ... obj.x, obj.y = children ... obj.size, = aux_data ... return obj ... >>> jax.tree_util.register_pytree_node(MyContainer, flatten_with_keys, unflatten) Now this can be used with functions like :func:`~jax.tree_util.tree_flatten_with_path`: >>> m = MyContainer(4) >>> leaves, treedef = jax.tree_util.tree_flatten_with_path(m) c<|\}}d|D|fS)Ncg|]\}}|Srkrk)rorcs r)rqzHregister_pytree_with_keys..flatten_func_impl..es ) ) )DAqa ) ) )r+rk)r key_childrenr,rVs r)flatten_func_implz4register_pytree_with_keys..flatten_func_implcs1//55lG ) )L ) ) )7 22r+N)r_rUr\)rTrVrXrVrxs ` r)register_pytree_with_keysry"s`@ %33333%Lx~>>>&@''(###r+ct|drtjdnd}t|tjd|j||S)aoExtends the set of types that are considered internal nodes in pytrees. This function is similar to ``register_pytree_node_class``, but requires a class that defines how it could be flattened with keys. It is a thin wrapper around ``register_pytree_with_keys``, and provides a class-oriented interface: Args: cls: a type to register as a pytree Returns: The input class ``cls`` is returned unchanged after being added to JAX's pytree registry. This return value allows ``register_pytree_node_class`` to be used as a decorator. See also: - :func:`~jax.tree_util.register_static`: simpler API for registering a static pytree. - :func:`~jax.tree_util.register_dataclass`: simpler API for registering a dataclass. - :func:`~jax.tree_util.register_pytree_node` - :func:`~jax.tree_util.register_pytree_with_keys` - :func:`~jax.tree_util.register_pytree_node_class` Examples: >>> from jax.tree_util import register_pytree_with_keys_class, GetAttrKey >>> @register_pytree_with_keys_class ... class Special: ... def __init__(self, x, y): ... self.x = x ... self.y = y ... def tree_flatten_with_keys(self): ... return (((GetAttrKey('x'), self.x), (GetAttrKey('y'), self.y)), None) ... @classmethod ... def tree_unflatten(cls, aux_data, children): ... return cls(*children) r*Ntree_flatten_with_keys)hasattrrbrcryr2)r`rVs r)register_pytree_with_keys_classr}ns^N*1n)E)EObon%%%4 2?3 4 4c6H *r+ data_fields Sequence[str] meta_fieldscttfd}fd}fd}ttttttt ttt ||t<t||t<S)aK Extends the set of types that are considered internal nodes in pytrees. This differs from ``register_pytree_with_keys_class`` in that the C++ registries use the optimized C++ dataclass builtin instead of the argument functions. See :ref:`extending-pytrees` for more information about registering pytrees. Args: nodetype: a Python type to treat as an internal pytree node. This is assumed to have the semantics of a :obj:`~dataclasses.dataclass`: namely, class attributes represent the whole of the object state, and can be passed as keywords to the class constructor to create a copy of the object. All defined attributes should be listed among ``meta_fields`` or ``data_fields``. meta_fields: auxiliary data field names. These fields *must* contain static, hashable, immutable objects, as these objects are used to generate JIT cache keys. In particular, ``meta_fields`` cannot contain :class:`jax.Array` or :class:`numpy.ndarray` objects. data_fields: data field names. These fields *must* be JAX-compatible objects such as arrays (:class:`jax.Array` or :class:`numpy.ndarray`), scalars, or pytrees whose leaves are arrays or scalars. Note that ``data_fields`` may be ``None``, as this is recognized by JAX as an empty pytree. Returns: The input class ``nodetype`` is returned unchanged after being added to JAX's pytree registry. This return value allows ``register_dataclass`` to be partially evaluated and used as a decorator as in the example below. Examples: >>> from dataclasses import dataclass >>> from functools import partial >>> >>> @partial(jax.tree_util.register_dataclass, ... data_fields=['x', 'y'], ... meta_fields=['op']) ... @dataclass ... class MyStruct: ... x: jax.Array ... y: jax.Array ... op: str ... >>> m = MyStruct(x=jnp.ones(3), y=jnp.arange(3), op='add') >>> m MyStruct(x=Array([1., 1., 1.], dtype=float32), y=Array([0, 1, 2], dtype=int32), op='add') Now that this class is registered, it can be used with functions in :mod:`jax.tree_util`: >>> leaves, treedef = jax.tree.flatten(m) >>> leaves [Array([1., 1., 1.], dtype=float32), Array([0, 1, 2], dtype=int32)] >>> treedef PyTreeDef(CustomNode(MyStruct[('add',)], [*, *])) >>> jax.tree.unflatten(treedef, leaves) MyStruct(x=Array([1., 1., 1.], dtype=float32), y=Array([0, 1, 2], dtype=int32), op='add') In particular, this registration allows ``m`` to be passed seamlessly through code wrapped in :func:`jax.jit` and other JAX transformations: >>> @jax.jit ... def compiled_func(m): ... if m.op == 'add': ... return m.x + m.y ... else: ... raise ValueError(f"{m.op=}") ... >>> compiled_func(m) Array([1., 2., 3.], dtype=float32) cztfdD}tfdD}||fS)Nc38K|]}t|VdSrsgetattrrorHrs r)rvz@register_dataclass..flatten_with_keys..-::dD!!::::::r+c3VK|]#}t|t|fV$dSrsrGrrs r)rvz@register_dataclass..flatten_with_keys..s:NN$*T""GAt$4$45NNNNNNr+rrmetadatar~rs` r)rVz-register_dataclass..flatten_with_keyssR ::::k::: : :D NNNN+NNN N ND :r+ctt|}tt|}t||z}di|Sr)r>rwr)rr meta_args data_argskwargsr~rrTs r)rXz*register_dataclass..unflatten_funcsYc+t,,--Ic+t,,--I )i' ( (F 8  f  r+cztfdD}tfdD}||fS)Nc38K|]}t|VdSrsrrs r)rvz;register_dataclass..flatten_func..rr+c38K|]}t|VdSrsrrs r)rvz;register_dataclass..flatten_func..rr+rrs` r)rVz(register_dataclass..flatten_funcsR ::::k::: : :D ::::k::: : :D :r+) r>rregister_dataclass_noder?rrr]r^rUr\)rTr~rrVrXrVs``` r)register_dataclassrs0Vk""+k""+   **8T+5F5F[HYHYZZZ,,XtK7H7H${J[J[\\\++Hd;6G6GkIZIZ[[[&|^DD)H&@''(# /r+type[H]c4d}d}t||||S)a Registers `cls` as a pytree with no leaves. Instances are treated as static by :func:`jax.jit`, :func:`jax.pmap`, etc. This can be an alternative to labeling inputs as static using ``jit``'s ``static_argnums`` and ``static_argnames`` kwargs, ``pmap``'s ``static_broadcasted_argnums``, etc. Args: cls: type to be registered as static. Must be hashable, as defined in https://docs.python.org/3/glossary.html#term-hashable. Returns: The input class ``cls`` is returned unchanged after being added to JAX's pytree registry. This allows ``register_static`` to be used as a decorator. Examples: >>> import jax >>> @jax.tree_util.register_static ... class StaticStr(str): ... pass This static string can now be used directly in :func:`jax.jit`-compiled functions, without marking the variable static using ``static_argnums``: >>> @jax.jit ... def f(x, y, s): ... return x + y if s == 'add' else x - y ... >>> f(1, 2, StaticStr('add')) Array(3, dtype=int32, weak_type=True) c d|fSrrk)objs r)rz!register_static..&s S r+c|Srsrk)rempty_iter_childrens r)rz!register_static..'ssr+)ry)r`r'r1s r)register_staticrs,@ " !'22)C)444 *r++tuple[list[tuple[KeyPath, Any]], PyTreeDef]cLt||\}}t|||fS)aFlattens a pytree like ``tree_flatten``, but also returns each leaf's key path. Args: tree: a pytree to flatten. If it contains a custom type, it must be registered with ``register_pytree_with_keys``. Returns: A pair which the first element is a list of key-leaf pairs, each of which contains a leaf and its key path. The second element is a treedef representing the structure of the flattened tree. )r*_generate_key_paths)r r!rtree_defs r)tree_flatten_with_pathr,s,T7+++!X T7 + +X 55r+list[tuple[KeyPath, Any]]c"t||S)aGets the leaves of a pytree like ``tree_leaves`` and returns each leaf's key path. Args: tree: a pytree. If it contains a custom type, it must be registered with ``register_pytree_with_keys``. Returns: A list of key-leaf pairs, each of which contains a leaf and its key path. See Also: - :func:`jax.tree_util.tree_leaves` - :func:`jax.tree_util.tree_flatten_with_path` )rr(s r)tree_leaves_with_pathr>s T7 + ++r+c>ttd||Sr)r?_generate_key_paths_r(s r)generate_key_pathsrRs "2tW55 6 66r+key_pathIterable[tuple[KeyPath, Any]]c#K|r|r|fVdStt}|r>|\}}|D]!\}}t g||R||Ed{V"dSt }||fVdSttrbtdrR|dtkr9fdj D}|D]!\}}t g||R||Ed{V"dSt|dD]0\} }t| }t g||R||Ed{V1dS)N_fieldsr9cNg|]!}t|t|f"Srkr)rorr s r)rqz(_generate_key_paths_..qs.LLL!Z]]GD!$4$45LLLr+r) r\getrrVrrrrr>r|r enumeraterM) rr r! key_handlerrwrrrvrrbs ` r)rrZs    D. F'++DJJ77+ !33D99OL!BB1%nn!nnaAAAAAAAAAA F  + +D 1 1$ \ D. Fu '$ ":":  1gdLLLLt|LLLLBB1%nn!nnaAAAAAAAAAA FQ  @@da!A#NhNNNAw??????????@@r+ct||\}tt|}|fd|Dz}fdt|DS)a4Maps a multi-input function over pytree key path and args to produce a new pytree. This is a more powerful alternative of ``tree_map`` that can take the key path of each leaf as input argument as well. Args: f: function that takes ``2 + len(rest)`` arguments, aka. the key path and each corresponding leaves of the pytrees. tree: a pytree to be mapped over, with each leaf's key path as the first positional argument and the leaf itself as the second argument to ``f``. *rest: a tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. Returns: A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``f(kp, x, *xs)`` where ``kp`` is the key path of the leaf at the corresponding leaf in ``tree``, ``x`` is the leaf value and ``xs`` is the tuple of values at corresponding nodes in ``rest``. See Also: - :func:`jax.tree_util.tree_map` - :func:`jax.tree_util.tree_flatten_with_path` - :func:`jax.tree_util.tree_leaves_with_path` c:g|]}|Srkrlrns r)rqz&tree_map_with_path..s'(P(P(Pa)>)>q)A)A(P(P(Pr+c3"K|] }|V dSrsrkrts r)rvz%tree_map_with_path..s)EEb11b6EEEEEEr+)rr?rwr1)rfr r!rhkeypath_leavesall_keypath_leavesr,s` @r)tree_map_with_pathr{s{:34AA.'^,--.%(P(P(P(P4(P(P(PP   EEEEC1C,DEEE E EEr+ctt|rJtt |}|r2t d||dDSt|t r.t|drt d|j DSttt|}t dt|DS)Nc3 K|] \}}|V dSrsrk)rorrs r)rvz_child_keys..s&DDtq!DDDDDDr+rrc34K|]}t|VdSrs)rG)rors r)rvz_child_keys..s(771A777777r+c34K|]}t|VdSrs)rMras r)rvz_child_keys..s+CC!"1%%CCCCCCr+) rLr:r\rrr>rVrr|rr#rEr)rrX num_childrens r)r$r$s #N6$:$: ; ;;;; # ' 'V 5 5' D DDw88@@CDDD D DD&%  DWVY%?%?D 77777 7 77'v(>(>??@@L CCu\/B/BCCC C CCr+%Iterable[Callable[[str], ValueError]]c # Ktt|rdSttkr fdVdSt\}t\ }t t  t t  t ttfrpytree structure error: different types at key path {name}F At that key path, the prefix pytree {name} has a subtree of type zN but at the same key path the full pytree has a subtree of different type rJrKrrSrformat)rHrrrs r)rz_prefix_error..smz H%%  +    )__    !'D 1 1 33r+ctdjdtdjdtdtd |S)Nz-pytree structure error: different lengths of z at key path {name}zB At that key path, the prefix pytree {name} has a subtree of type rzC, but the full pytree has a subtree of the same type but of length rJrK)rrrSr#r)rHrrrrWs r)rz_prefix_error..s L"+ L L)) L L[ L L&)%5%5 L L;>i..  L L L 6t6   r+ctdtdtdtdddDdtdddDd | d n!d dd DzS)NzSpytree structure error: different numbers of pytree children at key path {name}rz with z child keys rc3>K|]}t|jVdSrsr"rrs r)rvz2_prefix_error....s*==qAE ======r+zR but at the same key path the full pytree has a subtree of the same type but with c3>K|]}t|jVdSrsrrs r)rvz2_prefix_error....s*;;qAE ;;;;;;r+ rKrz/so the symmetric difference on key sets is c3>K|]}t|jVdSrsrrs r)rvz2_prefix_error....s*44!3qu::444444r+)rrSrr#r%r)rHr1full_tree_childrenfull_tree_keysrrprefix_tree_childrenprefix_tree_keyss r)rz_prefix_error..s& @h'' @ @K   @ @())  @ @ xx==,<=====  @ @/00 @ @xx;;N;;;;; @ @ @ T  22 7((44t44444 7 7 8 9 9r+rz )prefixc tdtdtddd |S)NzHpytree structure error: different pytree metadata at key path {name}rz with metadata z_ but at the same key path the full pytree has a subtree of the same type but with metadata z6 so the diff in the metadata at these pytree nodes is rKr)rHfull_tree_meta_strr metadata_diffrprefix_tree_meta_strs r)rz_prefix_error..sz H%%  +     "         T** , ,r+z4equal pytree nodes gave differing prefix_tree_keys: r!)rLr:rrr>r$rr?r#r&r'r"textwrapindentr%difflibndiff splitlinesrwr )rrrr!prefix_tree_metafull_tree_metarr)r*r1rrrrrrrrWs``` @@@@@@@@@r)r r saN;HHHII  F +$y//)) 3 3 3 3 3 3333 F ,=[+I+I(('8'C'C$n344/00 --y)). dE]++   ;3y>>))   b        f* ! " " 7 7N8K8K L Ldd ddd   C(:$;$;;; 9 9 9 9 9 9 9 9 9 9 9 9 9 f''/00^,,O M.99;;,7799 ; ; < < M , , , , , , , , , , , F ^ + + +000-00 , + +')=?QRR55iaR^x^^^R444444444455s /EE rs)r r r!r"r#r$)r,r-r.r/r#r )r r r!r"r#r3)r r r!r7r#r-)r;r<r#r-)r,r-r#rA)r,r-r#rF)rMrNr!r"r#rF)rTrUrVrWrXrYr#rZ)r`rr#r) rfrgr r rhr r!r"r#r )r,r-rur r#r )r|r-r}r~rr r#r )rrr r r!r"r#r) rrr r rrr!r"r#r) rrr r rr r!r"r#r)r r r!r"r#rF)rr rr r!r"r#r)rr r#r)rr rr r!r"r#r)r r r r r!r"r#r )r r-r r-r#r )rrP)rWrUrXrYr#rZ)rTrUrVrprXrqrVrr)rTrr~rrrr#r)r`rr#r)r r r!r"r#r)r r r!r"r#r)rrPr r r!r"r#r)rr r#rP) rrPrr rr r!r"r#r)j __future__rrcollections.abcrrrr dataclassesrrrr operatorrbrtypingr r r r rjax._srcr jax._src.libr jax._src.utilrrrexportregister_exclusion__file__rrrrrr-rrrPyTreeRegistryrrr*r2r5r:r@rErIrLrPrRrSr_rdrxr{r namedtupler]r>r?rr^robjectrrrrrrrrrrr rrrr6rCrGrMBuiltInKeyEntryrOrPrSrUr]r\ryr}rrrrrrrrr$r rkr+r)rsF #"""""BBBBBBBBBBBB!!!!!!<<<<<<<<<<<<<<######........  O $ $!!(+++ GCLL ge49%%% GCx       *6*,,'.+V*DD$(((!)2*F)44$((( (09=11111#### 8<444448<44444888820    ,<<<<7;,,,,,6 GKx} 5 5 5 7:X . . .FEFEFEFER, , , , ^6:>>>>>>((((<1111.('(89k:RSS >>//1H1H I I....0F0F G G..DD== ? ?DJJ335G5GHH   ### 9=  9= $28<WWWWWCG111111  00FFHHH >>> IIKKK 22222222*@7@7@7@7@7i@7@7@7F HH33>B     @;?BBBBBEI99999*KKKK6=6=6=r $ $ $ $((((((((  Wj:KKL 7:X . . . # ))))$%%%%%%%%     C C B B4IIJJJJJJJ  HHHHHV, , , , ^ddddN" " " " J7;66666"7;,,,,,(8<77777)-1@@@@@B@DFFFFFFD D D D D"-1 `5`5`5`5`5`5`5r+