WpfU:dZddlZddlmZmZmZmZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddl m!Z!ej"Z#e j$Z%ej&Z'ej(Z)ej*Z+ej,Z-ej.Z/dZ0dZ1dZ2dZ3dZ4dZ5dZ6dZ7dZ8dZ9ej:GddZ;Gddej<Z=dS)z MediaPipe audio classifier task.N)CallableMappingListOptional)packet_creator) packet_getter)packet)"audio_classifier_graph_options_pb2)classifications_pb2)classifier_options_pb2)audio_task_running_mode)base_audio_task_api) audio_data)classification_result) base_options) task_info) doc_controlsaudio_inAUDIOclassifications_outCLASSIFICATIONSsample_rate_in SAMPLE_RATEz;mediapipe.tasks.audio.audio_classifier.AudioClassifierGraphtimestamped_classifications_outTIMESTAMPED_CLASSIFICATIONSic$eZdZUdZeed<ejZeed<dZ e e ed<dZ e e ed<dZe eed<dZe ee ed<dZe ee ed <dZe eee gdfed <ejd efd ZdS) AudioClassifierOptionsaGOptions for the audio classifier task. Attributes: base_options: Base options for the audio classifier task. running_mode: The running mode of the task. Default to the audio clips mode. Audio classifier task has two running modes: 1) The audio clips mode for running classification on independent audio clips. 2) The audio stream mode for running classification on the audio stream, such as from microphone. In this mode, the "result_callback" below must be specified to receive the classification results asynchronously. display_names_locale: The locale to use for display names specified through the TFLite Model Metadata. max_results: The maximum number of top-scored classification results to return. score_threshold: Overrides the ones provided in the model metadata. Results below this value are rejected. category_allowlist: Allowlist of category names. If non-empty, classification results whose category name is not in this set will be filtered out. Duplicate or unknown category names are ignored. Mutually exclusive with `category_denylist`. category_denylist: Denylist of category names. If non-empty, classification results whose category name is in this set will be filtered out. Duplicate or unknown category names are ignored. Mutually exclusive with `category_allowlist`. result_callback: The user-defined result callback for processing audio stream data. The result callback should only be specified when the running mode is set to the audio stream mode. r running_modeNdisplay_names_locale max_resultsscore_thresholdcategory_allowlistcategory_denylistresult_callbackreturnc|j}|jtjkrdnd|_t |j|j|j |j |j }t||S)z4Generates an AudioClassifierOptions protobuf object.FT)r!r"r#rr )rclassifier_options) rto_pb2r _RunningMode AUDIO_CLIPSuse_stream_mode_ClassifierOptionsProtor!r"r#rr !_AudioClassifierGraphOptionsProto)selfbase_options_protoclassifier_options_protos m/var/www/html/nettyfy-visnx/env/lib/python3.11/site-packages/mediapipe/tasks/python/audio/audio_classifier.pyr(zAudioClassifierOptions.to_pb2\s*1133262C|G_2_2_ei&6,20!6$  & & & -'3 5 5 55)__name__ __module__ __qualname____doc__ _BaseOptions__annotations__r)r*rrrstrr intr!floatr"rr#r$rAudioClassifierResultrdo_not_generate_docsr-r(r2r1rr5s8+7, 777(, ,,,#+x}###%)/8E?))),0htCy)000+/Xd3i(///LP/8H&;S%A4%GHIPPP$ 57 5 5 5%$ 5 5 5r2rceZdZdZededdfdZededdfdZde de e fdZ d e d e dd fd Zd S) AudioClassifieraClass that performs audio classification on audio data. This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor. Input tensor: (kTfLiteFloat32) - input audio buffer of size `[batch * samples]`. - batch inference is not supported (`batch` is required to be 1). - for multi-channel models, the channels must be interleaved. At least one output tensor with: (kTfLiteFloat32) - `[1 x N]` array with `N` represents the number of categories. - optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `category_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `AudioClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled. model_pathr%ct|}t|tj}||S)aYCreates an `AudioClassifier` object from a TensorFlow Lite model and the default `AudioClassifierOptions`. Note that the created `AudioClassifier` instance is in audio clips mode, for classifying on independent audio clips. Args: model_path: Path to the model. Returns: `AudioClassifier` object that's created from the model file and the default `AudioClassifierOptions`. Raises: ValueError: If failed to create `AudioClassifier` object from the provided file such as invalid file path. RuntimeError: If other types of error occurred. )model_asset_path)rr)r7rr)r*create_from_options)clsrAroptionss r1create_from_model_pathz&AudioClassifier.create_from_model_pathsG& <<.packets_callbacks# &((1%;YZl 4 5 > > @ @ !" 5 5 5| E E E$7$L$N$N!!**  !.1M"N O OQQQ   / /0K L L r2:) task_graph input_streamsoutput_streams task_optionsF)enable_flow_limitingN)rr9r Packet _TaskInfo_TASK_GRAPH_NAMEjoin _AUDIO_TAG_AUDIO_IN_STREAM_NAME_SAMPLE_RATE_TAG_SAMPLE_RATE_IN_STREAM_NAME_CLASSIFICATIONS_TAGrL _TIMESTAMPED_CLASSIFICATIONS_TAG(_TIMESTAMPED_CLASSIFICATIONS_STREAM_NAMEgenerate_graph_configrr$)rErFrWrs ` r1rDz#AudioClassifier.create_from_optionss" fm1C)D      # HHj"78 9 9 HH&(CD E E HH*,HI J J HH08      I 3 ''U'CC#3=  ? ??r2 audio_clipc|jjstd|tt j|jdtt j |jji}g}tj |t}|D]W}tj}|||t"|X|S)a Performs audio classification on the provided audio clip. The audio clip is represented as a MediaPipe AudioData. The method accepts audio clips with various length and audio sample rate. It's required to provide the corresponding audio sample rate within the `AudioData` object. The input audio clip may be longer than what the model is able to process in a single inference. When this occurs, the input audio clip is split into multiple chunks starting at different timestamps. For this reason, this function returns a vector of ClassificationResult objects, each associated ith a timestamp corresponding to the start (in milliseconds) of the chunk data that was classified, e.g: ClassificationResult #0 (first chunk of data): timestamp_ms: 0 (starts at 0ms) classifications #0 (single head model): category #0: category_name: "Speech" score: 0.6 category #1: category_name: "Music" score: 0.2 ClassificationResult #1 (second chunk of data): timestamp_ms: 800 (starts at 800ms) classifications #0 (single head model): category #0: category_name: "Speech" score: 0.5 category #1: category_name: "Silence" score: 0.1 Args: audio_clip: MediaPipe AudioData. Returns: An `AudioClassifierResult` object that contains a list of classification result objects, each associated with a timestamp corresponding to the start (in milliseconds) of the chunk data that was classified. Raises: ValueError: If any of the input arguments is invalid, such as the sample rate is not provided in the `AudioData` object. RuntimeError: If audio classification failed to run. 1Must provide the audio sample rate in audio data.T transpose) audio_format sample_rate ValueError_process_audio_cliprcr create_matrixbufferre create_doublerget_proto_listrhr rQrRappendr<rT)r.rjrI output_list classification_result_proto_listprotorVs r1classifyzAudioClassifier.classifys^  " .L J K KK--  ():d K K K#  ()@)L M M /N K'4'C?@(B(B$1NN$7$L$N$N!!**5111  / /0K L LNNNN r2 audio_blockrUNc|jjstd|js2|jj|_|t |jn:|jj|jkr%td|jjd|jd|ttj |j d |tzidS)aSends audio data (a block in a continuous audio stream) to perform audio classification. Only use this method when the AudioClassifier is created with the audio stream running mode. The input timestamps should be monotonically increasing for adjacent calls of this method. This method will return immediately after the input audio data is accepted. The results will be available via the `result_callback` provided in the `AudioClassifierOptions`. The `classify_async` method is designed to process auido stream data such as microphone input. The input audio data may be longer than what the model is able to process in a single inference. When this occurs, the input audio block is split into multiple chunks. For this reason, the callback may be called multiple times (once per chunk) for each call to this function. The `result_callback` provides: - An `AudioClassifierResult` object that contains a list of classifications. - The input timestamp in milliseconds. Args: audio_block: MediaPipe AudioData. timestamp_ms: The timestamp of the input audio data in milliseconds. Raises: ValueError: If any of the followings: 1) The sample rate is not provided in the `AudioData` object or the provided sample rate is inconsistent with the previously received. 2) The current input timestamp is smaller than what the audio classifier has already processed. rlz.The audio sample rate provided in audio data: z/ is inconsistent with the previously received: .TrmN) rorprq_default_sample_rate_set_sample_ratere_send_audio_stream_datarcrrsrtatrO)r.r|rUs r1classify_asynczAudioClassifier.classify_asyncs@  # /L J K KK  $D"-":"Fd 7 57777  ! -1J J J  C  % 1 C C&*&? C C C D DD      ();t L L L O O== ? ?"r2)r3r4r5r6 classmethodr9rGrrD _AudioDatarr<r{r:rr>r2r1r@r@ms2,c,6G,,,;,.1?#91?>O1?1?1?;1?f??5J0K????B0 0#0$000000r2r@)>r6 dataclassestypingrrrrmediapipe.pythonrr$mediapipe.python._framework_bindingsr /mediapipe.tasks.cc.audio.audio_classifier.protor .mediapipe.tasks.cc.components.containers.protor .mediapipe.tasks.cc.components.processors.protor !mediapipe.tasks.python.audio.corer running_mode_moduler,mediapipe.tasks.python.components.containersraudio_data_modulerclassification_result_modulemediapipe.tasks.python.corerbase_options_modulertask_info_module1mediapipe.tasks.python.core.optional_dependenciesrrQr<AudioClassifierGraphOptionsr- AudioDatar BaseOptionsr7ClassifierOptionsr,AudioTaskRunningModer)TaskInfor_rcrbrLrfrerdr`rhrgrO dataclassrBaseAudioTaskApir@r>r2r1rs'&444444444444++++++******777777^^^^^^NNNNNNQQQQQQ\\\\\\AAAAAAXXXXXXnnnnnnKKKKKKEEEEEEJJJJJJ4I$F$b!  ( ". 0B"7  % "  4(. P+L(#@ !% 4545454545454545nWWWWW):WWWWWr2