Wpf:3dZddlZddlmZmZmZmZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddl m!Z!ej"Z#e j$Z%ej&Z'ej(Z)ej*Z+ej,Z-ej.Z/dZ0dZ1dZ2dZ3dZ4dZ5dZ6dZ7dZ8dZ9ej:GddZ;Gddej<Z=dS)zMediaPipe audio embedder task.N)CallableMappingListOptional)packet_creator) packet_getter)packet) audio_embedder_graph_options_pb2)embeddings_pb2)embedder_options_pb2)audio_task_running_mode)base_audio_task_api) audio_data)embedding_result) base_options) task_info) doc_controlsaudio_inAUDIOembeddings_out EMBEDDINGSsample_rate_in SAMPLE_RATEz7mediapipe.tasks.audio.audio_embedder.AudioEmbedderGraphtimestamped_embeddings_outTIMESTAMPED_EMBEDDINGSiceZdZUdZeed<ejZeed<dZ e e ed<dZ e e ed<dZ e eeegdfed<ejdefd ZdS) AudioEmbedderOptionsaIOptions for the audio embedder task. Attributes: base_options: Base options for the audio embedder task. running_mode: The running mode of the task. Default to the audio clips mode. Audio embedder task has two running modes: 1) The audio clips mode for running embedding extraction on independent audio clips. 2) The audio stream mode for running embedding extraction on the audio stream, such as from microphone. In this mode, the "result_callback" below must be specified to receive the embedding results asynchronously. l2_normalize: Whether to normalize the returned feature vector with L2 norm. Use this option only if the model does not already contain a native L2_NORMALIZATION TF Lite Op. In most cases, this is already the case and L2 norm is thus achieved through TF Lite inference. quantize: Whether the returned embedding should be quantized to bytes via scalar quantization. Embeddings are implicitly assumed to be unit-norm and therefore any dimension is guaranteed to have a value in [-1.0, 1.0]. Use the l2_normalize option if this is not the case. result_callback: The user-defined result callback for processing audio stream data. The result callback should only be specified when the running mode is set to the audio stream mode. r running_modeN l2_normalizequantizeresult_callbackreturnc|j}|jtjkrdnd|_t |j|j}t||S)z2Generates an AudioEmbedderOptions protobuf object.FT)rr )rembedder_options) rto_pb2r _RunningMode AUDIO_CLIPSuse_stream_mode_EmbedderOptionsProtorr _AudioEmbedderGraphOptionsProto)selfbase_options_protoembedder_options_protos k/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/mediapipe/tasks/python/audio/audio_embedder.pyr%zAudioEmbedderOptions.to_pb2Ssx*1133262C|G_2_2_ei&2&@@@ +'/ 1 1 11)__name__ __module__ __qualname____doc__ _BaseOptions__annotations__r&r'rrrboolr r!rAudioEmbedderResultintrdo_not_generate_docsr*r%r/r.rr5s,+7, 777!%,%%%!(HTN!!!JN/8H&93%?%EFGNNN$ 15 1 1 1%$ 1 1 1r/rceZdZdZededdfdZededdfdZde de e fdZ d e d e dd fd Zd S) AudioEmbedderaAClass that performs embedding extraction on audio clips or audio stream. This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output embedding tensor. Input tensor: (kTfLiteFloat32) - input audio buffer of size `[batch * samples]`. - batch inference is not supported (`batch` is required to be 1). - for multi-channel models, the channels must be interleaved. At least one output tensor with: (kTfLiteUInt8/kTfLiteFloat32) - `N` components corresponding to the `N` dimensions of the returned feature vector for this output layer. - Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`. model_pathr"ct|}t|tj}||S)aZCreates an `AudioEmbedder` object from a TensorFlow Lite model and the default `AudioEmbedderOptions`. Note that the created `AudioEmbedder` instance is in audio clips mode, for embedding extraction on the independent audio clips. Args: model_path: Path to the model. Returns: `AudioEmbedder` object that's created from the model file and the default `AudioEmbedderOptions`. Raises: ValueError: If failed to create `AudioEmbedder` object from the provided file such as invalid file path. RuntimeError: If other types of error occurred. )model_asset_path)rr)r4rr&r'create_from_options)clsr=roptionss r.create_from_model_pathz$AudioEmbedder.create_from_model_pathtsG& <<.packets_callbacks# !##,U6TUl / 0 9 9 ; ; 2 . . .  > > >-=??%%  !.1H"I J JLLL   - -.D E E r/:) task_graph input_streamsoutput_streams task_optionsF)enable_flow_limitingN)rstrr Packet _TaskInfo_TASK_GRAPH_NAMEjoin _AUDIO_TAG_AUDIO_IN_STREAM_NAME_SAMPLE_RATE_TAG_SAMPLE_RATE_IN_STREAM_NAME_EMBEDDINGS_TAGrH_TIMESTAMPTED_EMBEDDINGS_TAG$_TIMESTAMPTED_EMBEDDINGS_STREAM_NAMEgenerate_graph_configrr!)rArBrSrs ` r.r@z!AudioEmbedder.create_from_optionss" fm1C)D      # HHj"78 9 9 HH&(CD E E HHo'>? @ @#((,4LCC    I 3 ''U'CC#3=  ? ??r/ audio_clipc|jjstd|tt j|jdtt j |jji}g}tj |t}|D]W}tj}|||t"|X|S)aPerforms embedding extraction on the provided audio clips. The audio clip is represented as a MediaPipe AudioData. The method accepts audio clips with various length and audio sample rate. It's required to provide the corresponding audio sample rate within the `AudioData` object. The input audio clip may be longer than what the model is able to process in a single inference. When this occurs, the input audio clip is split into multiple chunks starting at different timestamps. For this reason, this function returns a vector of EmbeddingResult objects, each associated ith a timestamp corresponding to the start (in milliseconds) of the chunk data on which embedding extraction was carried out. Args: audio_clip: MediaPipe AudioData. Returns: An `AudioEmbedderResult` object that contains a list of embedding result objects, each associated with a timestamp corresponding to the start (in milliseconds) of the chunk data on which embedding extraction was carried out. Raises: ValueError: If any of the input arguments is invalid, such as the sample rate is not provided in the `AudioData` object. RuntimeError: If audio embedding extraction failed to run. 1Must provide the audio sample rate in audio data.T transpose) audio_format sample_rate ValueError_process_audio_clipr`r create_matrixbufferrb create_doublerget_proto_listrer rMrNappendr7rP)r+rgrE output_listembeddings_proto_listprotorRs r.embedzAudioEmbedder.embeds8  " .L J K KK--  ():d K K K#  ()@)L M M /N K)8;<>>&GG-=??%%e,,,  - -.D E EGGGG r/ audio_blockrQNc|jjstd|js2|jj|_|t |jn:|jj|jkr%td|jjd|jd|ttj |j d |tzidS)aSends audio data (a block in a continuous audio stream) to perform audio embedding extraction. Only use this method when the AudioEmbedder is created with the audio stream running mode. The input timestamps should be monotonically increasing for adjacent calls of this method. This method will return immediately after the input audio data is accepted. The results will be available via the `result_callback` provided in the `AudioEmbedderOptions`. The `embed_async` method is designed to process auido stream data such as microphone input. The input audio data may be longer than what the model is able to process in a single inference. When this occurs, the input audio block is split into multiple chunks. For this reason, the callback may be called multiple times (once per chunk) for each call to this function. The `result_callback` provides: - An `AudioEmbedderResult` object that contains a list of embeddings. - The input timestamp in milliseconds. Args: audio_block: MediaPipe AudioData. timestamp_ms: The timestamp of the input audio data in milliseconds. Raises: ValueError: If any of the followings: 1) The sample rate is not provided in the `AudioData` object or the provided sample rate is inconsistent with the previously received. 2) The current input timestamp is smaller than what the audio embedder has already processed. riz.The audio sample rate provided in audio data: z/ is inconsistent with the previously received: .TrjN) rlrmrn_default_sample_rate_set_sample_raterb_send_audio_stream_datar`rrprqatrK)r+ryrQs r. embed_asynczAudioEmbedder.embed_asyncs@  # /L J K KK  $D"-":"Fd 7 57777  ! -1J J J  C  % 1 C C&*&? C C C D DD      ();t L L L O O== ? ?"r/)r0r1r2r3 classmethodrZrCrr@ _AudioDatarr7rxr8rr:r/r.r<r<`s&,c,o,,,;,.0?#70?r3 dataclassestypingrrrrmediapipe.pythonrr$mediapipe.python._framework_bindingsr -mediapipe.tasks.cc.audio.audio_embedder.protor .mediapipe.tasks.cc.components.containers.protor .mediapipe.tasks.cc.components.processors.protor !mediapipe.tasks.python.audio.corer running_mode_moduler,mediapipe.tasks.python.components.containersraudio_data_modulerembedding_result_modulemediapipe.tasks.python.corerbase_options_modulertask_info_module1mediapipe.tasks.python.core.optional_dependenciesrrMr7AudioEmbedderGraphOptionsr* AudioDatar BaseOptionsr4EmbedderOptionsr)AudioTaskRunningModer&TaskInfor\r`r_rHrcrbrar]rerdrK dataclassrBaseAudioTaskApir<r:r/r.rs%$444444444444++++++******777777ZZZZZZIIIIIIOOOOOO\\\\\\AAAAAAXXXXXXddddddKKKKKKEEEEEEJJJJJJ-="B"\  ( ". ,<"7  % "  *. L'C$7!% '1'1'1'1'1'1'1'1T}}}}}'8}}}}}r/