§ Vp«f²+ãóž—dZddlmZmZmZmZmZddlmZm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3ddl4m5Z5ddl6m7Z7m8Z8m9Z9m:Z:ddl;mZ>ddl?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHdd lImJZJmKZKmLZLmMZMmNZNmOZOmPZPmQZQmRZRmSZSdd lTmUZUdd lVmWZWmXZXdd lYmZZZd S)a­ .. currentmodule:: jax.experimental.sparse The :mod:`jax.experimental.sparse` module includes experimental support for sparse matrix operations in JAX. It is under active development, and the API is subject to change. The primary interfaces made available are the :class:`BCOO` sparse array type, and the :func:`sparsify` transform. Batched-coordinate (BCOO) sparse matrices ----------------------------------------- The main high-level sparse object currently available in JAX is the :class:`BCOO`, or *batched coordinate* sparse array, which offers a compressed storage format compatible with JAX transformations, in particular JIT (e.g. :func:`jax.jit`), batching (e.g. :func:`jax.vmap`) and autodiff (e.g. :func:`jax.grad`). Here is an example of creating a sparse array from a dense array: >>> from jax.experimental import sparse >>> import jax.numpy as jnp >>> import numpy as np >>> M = jnp.array([[0., 1., 0., 2.], ... [3., 0., 0., 0.], ... [0., 0., 4., 0.]]) >>> M_sp = sparse.BCOO.fromdense(M) >>> M_sp BCOO(float32[3, 4], nse=4) Convert back to a dense array with the ``todense()`` method: >>> M_sp.todense() Array([[0., 1., 0., 2.], [3., 0., 0., 0.], [0., 0., 4., 0.]], dtype=float32) The BCOO format is a somewhat modified version of the standard COO format, and the dense representation can be seen in the ``data`` and ``indices`` attributes: >>> M_sp.data # Explicitly stored data Array([1., 2., 3., 4.], dtype=float32) >>> M_sp.indices # Indices of the stored data Array([[0, 1], [0, 3], [1, 0], [2, 2]], dtype=int32) BCOO objects have familiar array-like attributes, as well as sparse-specific attributes: >>> M_sp.ndim 2 >>> M_sp.shape (3, 4) >>> M_sp.dtype dtype('float32') >>> M_sp.nse # "number of specified elements" 4 BCOO objects also implement a number of array-like methods, to allow you to use them directly within jax programs. For example, here we compute the transposed matrix-vector product: >>> y = jnp.array([3., 6., 5.]) >>> M_sp.T @ y Array([18., 3., 20., 6.], dtype=float32) >>> M.T @ y # Compare to dense version Array([18., 3., 20., 6.], dtype=float32) BCOO objects are designed to be compatible with JAX transforms, including :func:`jax.jit`, :func:`jax.vmap`, :func:`jax.grad`, and others. For example: >>> from jax import grad, jit >>> def f(y): ... return (M_sp.T @ y).sum() ... >>> jit(grad(f))(y) Array([3., 3., 4.], dtype=float32) Note, however, that under normal circumstances :mod:`jax.numpy` and :mod:`jax.lax` functions do not know how to handle sparse matrices, so attempting to compute things like ``jnp.dot(M_sp.T, y)`` will result in an error (however, see the next section). Sparsify transform ------------------ An overarching goal of the JAX sparse implementation is to provide a means to switch from dense to sparse computation seamlessly, without having to modify the dense implementation. This sparse experiment accomplishes this through the :func:`sparsify` transform. Consider this function, which computes a more complicated result from a matrix and a vector input: >>> def f(M, v): ... return 2 * jnp.dot(jnp.log1p(M.T), v) + 1 ... >>> f(M, y) Array([17.635532, 5.158883, 17.09438 , 7.591674], dtype=float32) Were we to pass a sparse matrix to this directly, it would result in an error, because ``jnp`` functions do not recognize sparse inputs. However, with :func:`sparsify`, we get a version of this function that does accept sparse matrices: >>> f_sp = sparse.sparsify(f) >>> f_sp(M_sp, y) Array([17.635532, 5.158883, 17.09438 , 7.591674], dtype=float32) Support for :func:`sparsify` includes a large number of the most common primitives, including: - generalized (batched) matrix products & einstein summations (:obj:`~jax.lax.dot_general_p`) - zero-preserving elementwise binary operations (e.g. :obj:`~jax.lax.add_p`, :obj:`~jax.lax.mul_p`, etc.) - zero-preserving elementwise unary operations (e.g. :obj:`~jax.lax.abs_p`, :obj:`jax.lax.neg_p`, etc.) - summation reductions (:obj:`~jax.lax.reduce_sum_p`) - general indexing operations (:obj:`~jax.lax.slice_p`, `lax.dynamic_slice_p`, `lax.gather_p`) - concatenation and stacking (:obj:`~jax.lax.concatenate_p`) - transposition & reshaping ((:obj:`~jax.lax.transpose_p`, :obj:`~jax.lax.reshape_p`, :obj:`~jax.lax.squeeze_p`, :obj:`~jax.lax.broadcast_in_dim_p`) - some higher-order functions (:obj:`~jax.lax.cond_p`, :obj:`~jax.lax.while_p`, :obj:`~jax.lax.scan_p`) - some simple 1D convolutions (:obj:`~jax.lax.conv_general_dilated_p`) Nearly any :mod:`jax.numpy` function that lowers to these supported primitives can be used within a sparsify transform to operate on sparse arrays. This set of primitives is enough to enable relatively sophisticated sparse workflows, as the next section will show. Example: sparse logistic regression ----------------------------------- As an example of a more complicated sparse workflow, let's consider a simple logistic regression implemented in JAX. Notice that the following implementation has no reference to sparsity: >>> import functools >>> from sklearn.datasets import make_classification >>> from jax.scipy import optimize >>> def sigmoid(x): ... return 0.5 * (jnp.tanh(x / 2) + 1) ... >>> def y_model(params, X): ... return sigmoid(jnp.dot(X, params[1:]) + params[0]) ... >>> def loss(params, X, y): ... y_hat = y_model(params, X) ... return -jnp.mean(y * jnp.log(y_hat) + (1 - y) * jnp.log(1 - y_hat)) ... >>> def fit_logreg(X, y): ... params = jnp.zeros(X.shape[1] + 1) ... result = optimize.minimize(functools.partial(loss, X=X, y=y), ... x0=params, method='BFGS') ... return result.x >>> X, y = make_classification(n_classes=2, random_state=1701) >>> params_dense = fit_logreg(X, y) >>> print(params_dense) # doctest: +SKIP [-0.7298445 0.29893667 1.0248291 -0.44436368 0.8785025 -0.7724008 -0.62893456 0.2934014 0.82974285 0.16838408 -0.39774987 -0.5071844 0.2028872 0.5227761 -0.3739224 -0.7104083 2.4212713 0.6310087 -0.67060554 0.03139788 -0.05359547] This returns the best-fit parameters of a dense logistic regression problem. To fit the same model on sparse data, we can apply the :func:`sparsify` transform: >>> Xsp = sparse.BCOO.fromdense(X) # Sparse version of the input >>> fit_logreg_sp = sparse.sparsify(fit_logreg) # Sparse-transformed fit function >>> params_sparse = fit_logreg_sp(Xsp, y) >>> print(params_sparse) # doctest: +SKIP [-0.72971725 0.29878938 1.0246326 -0.44430563 0.8784217 -0.77225566 -0.6288222 0.29335397 0.8293481 0.16820715 -0.39764675 -0.5069753 0.202579 0.522672 -0.3740134 -0.7102678 2.4209507 0.6310593 -0.670236 0.03132951 -0.05356663] é)ÚjacfwdÚjacobianÚjacrevÚgradÚvalue_and_grad)Úbcoo_broadcast_in_dimÚbcoo_concatenateÚbcoo_conv_general_dilatedÚbcoo_dot_generalÚbcoo_dot_general_pÚbcoo_dot_general_sampledÚbcoo_dot_general_sampled_pÚbcoo_dynamic_sliceÚ bcoo_extractÚbcoo_extract_pÚbcoo_fromdenseÚbcoo_fromdense_pÚ bcoo_gatherÚbcoo_multiply_denseÚbcoo_multiply_sparseÚbcoo_update_layoutÚbcoo_reduce_sumÚ bcoo_reshapeÚbcoo_revÚ bcoo_sliceÚbcoo_sort_indicesÚbcoo_sort_indices_pÚbcoo_spdot_general_pÚ bcoo_squeezeÚbcoo_sum_duplicatesÚbcoo_sum_duplicates_pÚ bcoo_todenseÚbcoo_todense_pÚbcoo_transposeÚbcoo_transpose_pÚBCOO) Úbcsr_broadcast_in_dimÚbcsr_concatenateÚbcsr_dot_generalÚbcsr_dot_general_pÚ bcsr_extractÚbcsr_extract_pÚbcsr_fromdenseÚbcsr_fromdense_pÚbcsr_sum_duplicatesÚ bcsr_todenseÚbcsr_todense_pÚBCSR)Ú JAXSparse)ÚemptyÚeyeÚtodenseÚ todense_p)ÚCuSparseEfficiencyWarningÚSparseEfficiencyErrorÚSparseEfficiencyWarning) Ú coo_fromdenseÚcoo_fromdense_pÚ coo_matmatÚ coo_matmat_pÚ coo_matvecÚ coo_matvec_pÚ coo_todenseÚ coo_todense_pÚCOO) Ú csr_fromdenseÚcsr_fromdense_pÚ csr_matmatÚ csr_matmat_pÚ csr_matvecÚ csr_matvec_pÚ csr_todenseÚ csr_todense_pÚCSCÚCSR)Ú random_bcoo)ÚsparsifyÚ SparseTracer)ÚlinalgN)[Ú__doc__Újax.experimental.sparse.adrrrrrÚjax.experimental.sparse.bcoorr r r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&Újax.experimental.sparse.bcsrr'r(r)r*r+r,r-r.r/r0r1r2Újax.experimental.sparse._baser3Újax.experimental.sparse.apir4r5r6r7Újax.experimental.sparse.utilr8r9r:Újax.experimental.sparse.coor;r<r=r>r?r@rArBrCÚjax.experimental.sparse.csrrDrErFrGrHrIrJrKrLrMÚjax.experimental.sparse.randomrNÚ!jax.experimental.sparse.transformrOrPÚjax.experimental.sparserQ©óú`/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/experimental/sparse/__init__.pyúrasøððnðnðfðððððððððððððð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ðD ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ððððððððððððððððððððððððððððð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ð ðFÐEÐEÐEÐEÐEððððððððð 5Ð4Ð4Ð4Ð4Ð4Ð4Ð4r_