[Tut] Python Video to Text – Speech Recognition

[Tut] Python Video to Text – Speech Recognition - Printable Version

+- Sick Gaming (https://sickgaming.net)
+-- Forum: Programming (https://sickgaming.net/forum-76.html)
+--- Forum: Python (https://sickgaming.net/forum-83.html)
+--- Thread: [Tut] Python Video to Text – Speech Recognition (/thread-100631.html)

[Tut] Python Video to Text – Speech Recognition - xSicKxBot - 01-23-2023

Python Video to Text – Speech Recognition

5/5 – (1 vote)

A good friend and his wife recently founded an AI startup in the lifestyle niche that uses machine learning to discover specific real-world patterns from videos.

For their business system, they need a pipeline that takes a video file, converts it to audio, and transcribes the audio to standard text that is then used for further processing. I couldn’t help but work on a basic solution to help fix their business problem.

Project Overview

[Image: image-276-1024x568.png]

I finished the project in three steps:

First, install the necessary libraries.
Second, convert the video to an audio file (
Code:
.mp4
to
Code:
.wav
)
Third, convert the audio file to a speech file (
Code:
.wav
to
Code:
.txt
). We first break the large audio file into smaller chunks and convert each of them separately due to the size restrictions of the used API.

Let’s get started!

Step 1: Install Libraries

We need the following

Code:
import

statements in our code:

Code:
# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp

Consequently, you need to

Code:
pip install

the following three libraries in your shell — assuming you run Python version 3.9:

Code:
pip3.9 install pydub
pip3.9 install SpeechRecognition
pip3.9 install moviepy

The

Code:
os

module is already preinstalled as a Python Standard Library.

If you need an additional guide on how to install Python libraries, check out this tutorial:

Recommended: Python Install Library Guide

Step 2: Video to Audio

[Image: image-252.png]

Before you can do speech recognition on the video, we need to extract the audio as a

Code:
.wav

file using the

Code:
moviepy.editor.VideoFileClip().audio.write_audiofile()

method.

Here’s the code:

Code:
def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path)

Recommended: Python Video to Audio

Step 3: Audio to Text

After extracting the audio file, we can start transcribing the speech from the

Code:
.wav

file using Google’s powerful speech recognition library on chunks of the potentially large audio file.

Using chunks instead of passing the whole audio file avoids an error for large audio files — Google has some restrictions on the audio file size.

However, you can play around with the splitting thresholds of 700ms silence—it can be more or less, depending on your concrete file.

Here’s the audio to text code function that worked for me:

Code:
def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text

Need more info? Check out the following deep dive:

Recommended: Large Audio to Text? Here’s My Speech Recognition Solution in Python

Step 4: Putting It Together

[Image: image-255-1024x683.png]

Finally, we can combine our functions. First, we extract the audio from the video. Second, we chunk the audio into smaller files and recognize speech independently on each chunk using Google’s speech recognition module.

I added comments to annotate the most important parts of this code:

Code:
# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path) def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text # Create a speech recognition object
r = sr.Recognizer() # Video to audio to text
video_to_audio('sample_video.mp4', 'sample_audio.wav')
result = large_audio_to_text('sample_audio.wav') # Print to shell and file
print(result)
print(result, file=open('result.txt', 'w'))

Store this code in a folder next to your video file

Code:
'sample_video.mp4'

and run it. It will create an audio file

Code:
'sample_audio.wav'

and chunk the audio and print the result to the shell, as well as to a file called

Code:
'result.txt'

. This contains the transcription of the video file.