Vpf,:dZddlmZddlmZmZddlZddlZddlZddl Z ddl m Z ddl m Z ddlZddlmZddlZddlmZddlmZddlZddlZddlZddlZe jd d d Zd Z d Z!dZ"gdZ#dZ$dZ%d"dZ&GddZ'GddZ( d#d$d!Z)dS)%zDefinitions of two versions of MNIST (model and training code ). One definition uses pure JAX (for those who prefer an example with fewer moving parts, at the expense of code size), and another using Flax. See README.md for how these are used. ) annotations)CallableSequenceN)Any)flags)linen)pyplot mock_dataFzUse fake data, for testing.)r )ir g?gMbP?split tfds.Split batch_sizeintcXtjrtj|5 tjd|}na#t $rT}tjdt|}|r)d| dd}t|||d}~wwxYw dddn #1swxYwYntjd|}d }| |}| d |d }|S) a Loads either training or test MNIST data. Args: split: either tfds.Split.TRAIN or tfds.Split.TEST. Returns: an iterator with pairs (images, labels). The images have shape (B, 28, 28, 1) and the labels have shape (B, 10), where B is the batch_size. ) num_examplesmnist)rz,metadata files were not found in (.+/)mnist/zTFDS mock_data is missing the mnist metadata files. Run the `saved_model_main.py` binary and see where TFDS downloads the mnist data set (typically ~/tensorflow_datasets/mnist). Copy the `mnist` directory to rz and re-run the testNctj|dtjdz }tj|dd}||fS)Nimagego@labelr)tfcastfloat32one_hot)xrrs j/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/jax/experimental/jax2tf/examples/mnist_lib.py_prepare_examplez$load_mnist.._prepare_exampleWs= GAgJ + +e 3E Jqz2 & &E 5>iT)drop_remainder) _MOCK_DATAvaluetfdstestingr load Exceptionresearchstrgroup ValueErrormapcacheshufflebatch)rrdsemmsgr!s r load_mnistr7<s)   Z  8 8    Ywe , , ,     IEs1vv N N  S34''!**SSS#3Q &'                  7% ( ( (B  vv" xxzz$%%j%FF" )s5B4AB4 B$ABB$$B44B8;B8ceZdZdZdZedddZedZedd Zed Z edd Z d S) PureJaxMNISTzAn MNIST model written using pure JAX. There is an option for the model to skip the classifier layer, for demonstrating reuse of the classifier-less model into a larger model. See README.md. mnist_pure_jaxTparamsSequence[tuple[Any, Any]]c`||jddf}|ddD]1\}}tj|||z}tj|}2|s|S|d\}}tj|||z}|t jj|ddz S)a_The prediction function. Args: params: a list with pairs of weights and biases for each layer. inputs: the batch of images (B, 28, 28, 1) with_classifier: whether to include the classifier layer. Returns: either the predictions (B, 10) if with_classifier=True, or the final set of logits of shape (B, 512). rNrT)axiskeepdims) reshapeshapejnpdottanhjaxscipyspecial logsumexp) r;inputswith_classifierrwbfinal_wfinal_blogitss r predictzPureJaxMNIST.predictls  Q,--Ass 1 '!Q--! a (1++aa  hbzGW WQ 7 *F CI%// 1t0%% %%r"ct||d}tjtj||zd SNTrKrr?)r9rQrCmeansumr;rJlabels predictionss r losszPureJaxMNIST.losssC&&vvt&LLK HSW[61::: ; ; ;;r"rQrctjfdfdtj|D}t jt j|S)Nctj|d}tj|d}tj||kS)NrrU)rCargmaxrV)rJrY target_classpredicted_classr;rQs r _per_batchz)PureJaxMNIST.accuracy.._per_batchsLZQ///l 7766#:#:CCCo Xo5 6 66r"c.g|]\}}||Src).0rJrYras r z)PureJaxMNIST.accuracy..s6%3VVjj  r")rFjitr&as_numpyrCrVstack)rQr;datasetbatchedras`` @r accuracyzPureJaxMNIST.accuracys W77777 W7 7;}W7M7MG 8CIg&& ' ''r"ctjtj|||}dt ||DS)NcTg|]%\\}}\}}|t|zz |t|zz f&Src) step_size)rdrLrMdwdbs r rez'PureJaxMNIST.update..sK 8 8 8 ARR Y^!3 4 8 8 8r")rFgradr9r[zip)r;rJrYgradss r updatezPureJaxMNIST.updatesM 'CH\& ' ' ? ?E 8 8$'$6$6 8 8 88r"cX tjd fdttddtddD}t |D]"}t j}tj|D].\}}tj tj |||}/t j|z } t tj ||} t tj ||} tjdtj|| tjdtjd| ztjd tjd| z$t#jtj | |fS) agTrains a pure JAX MNIST predictor. Returns: a tuple with two elements: - a predictor function with signature "(Params, ImagesBatch) -> Predictions". If `with_classifier=False` then the output of the predictor function is the last layer of logits. - the parameters "Params" for the predictor function rcg|]X\}}ttj||fzttj|fzfYSrc) param_scalerFrandomnormal)rdr5nrngs r rez&PureJaxMNIST.train..soCCC!QSZ..sQF;;;SZ..sQD999;CCCr"Nr>r%s: Epoch %d in %0.2f sec!%s: Training set accuracy %0.2f%%Y@%s: Test set accuracy %0.2f%%rT)rFrxPRNGKeyrr layer_sizesrangetimer&rgrfr9rtrkrQlogginginfoname functoolspartial) train_dstest_ds num_epochsrKr;epoch start_timerJrY epoch_time train_acctest_accr{s @r trainzPureJaxMNIST.trains *  Q  CCCCC{3B3/QRRAACCCFz"" $ $9;;j M(33FF.&&-,--fffEE9;;+j'' (D FFr"NT)r;r<)rQr) __name__ __module__ __qualname____doc__r staticmethodrQr[rkrtrrcr"r r9r9bs $%%%%<%0<<<< ( ( (< (88<8 FFF<FFFr"r9ceZdZdZdZGddejZeZed dZ edZ edZ ed d Z d S) FlaxMNISTzAn MNIST model using Flax. mnist_flaxc4eZdZdZejddZdS)FlaxMNIST.ModulezA simple CNN model for MNIST. There is an option for the model to skip the classifier layer, for demonstrating reuse of the classifier-less model into a larger model. See README.md. TcBtjdd|}tj|}tj|dd}tjdd|}tj|}tj|dd}||jddf}tjd |}tj|}|s|Stjd |}tj|}|S) N )r)features kernel_size)r) window_shapestrides@rr>)rr)nnConvreluavg_poolrArBDense log_softmax)selfrrKs r __call__zFlaxMNIST.Module.__call__s 2"'26 2 2 21 5 5a '!**a +aff = = =a 2"'26 2 2 21 5 5a '!**a +aff = = =a ))QWQZ$ % %a "(C  # #a '!**a  "(B    " "a .  a hr"Nr)rrrrrcompactrrcr"r Modulers@ZZr"rTcJtjd|i||S)Nr;rT)rmodelapply)r;rJrKs r rQzFlaxMNIST.predicts1 ? (F!3!'1@ ! B BBr"ct||d}tjtj||zd SrS)rrQrCrVrWrXs r r[zFlaxMNIST.losssE##FFD#IIK HSW[61::: ; ; ;;r"ctjtj|||}|||\}}t j||}||fSN)rFrqrr[rtoptax apply_updates)txr; opt_staterJrYrqupdatess r rtzFlaxMNIST.updatesV #38IN # #FFF ; ;D433GY   1 1F 9 r"c tjd}d}tjdt ztj}tj ||d}tj t|}| |} t|D])} tj} tj|D]5\} } tjtjd||| | | \}} 6tj| z }t&tj||}t&tj||}t-jdtj| |t-jdtjd |zt-jd tjd |z+t3jtj| }||fS) aiTrains a pure JAX MNIST predictor. Returns: a tuple with two elements: - a predictor function with signature "(Params, ImagesBatch) -> Predictions". If `with_classifier=False` then the output of the predictor function is the last layer of logits. - the parameters "Params" for the predictor function rg?)rr;) learning_ratemomentum)static_argnumsr|r}r~rrT)rFrxrrCones input_shaperrrinitrsgdrnrrr&rgrfrtr9rkrQrrrrr)rrrrKr{ momentum_mass init_shaper;rrrrrJrYrrr predict_fns r rzFlaxMNIST.trains *  Q  CM$,ck::J _ ! !#z 2 28 rs#"""""........   $$$$$$ """" !U !+u"?AA  !!!   # # # # L\F\F\F\F\F\F\F\F~^ ^ ^ ^ ^ ^ ^ ^ J15 " " " " " " " r"