{"id":46812,"date":"2024-09-18T00:00:00","date_gmt":"2024-09-18T07:00:00","guid":{"rendered":"https:\/\/griddb-linux-hte8hndjf8cka8ht.westus-01.azurewebsites.net\/blog\/building-video-summarizer-using-ai\/"},"modified":"2025-11-13T12:57:05","modified_gmt":"2025-11-13T20:57:05","slug":"building-video-summarizer-using-ai","status":"publish","type":"post","link":"https:\/\/griddb-linux-hte8hndjf8cka8ht.westus-01.azurewebsites.net\/en\/blog\/building-video-summarizer-using-ai\/","title":{"rendered":"Building Video Summarizer Using AI"},"content":{"rendered":"

With the recent advancements in AI technology, such as OpenAI, it has become possible to automate tasks that were previously too tedious to perform manually. An example of this is a video summarizer. Previously, the process of summarizing video content relied mainly on human vision and hearing. However, with AI models such as GPT-4<\/a> and Whisper<\/a>, it is now possible to automate this task.<\/p>\n

We will be utilizing the following technologies: OpenAI, Node.js, React, and GridDB. This blog will teach you how to create a basic web application for uploading a video and receiving a summary of the content.<\/p>\n

Getting Started<\/h2>\n

Source code can be found here:<\/p>\n

$ git clone https:\/\/github.com\/griddbnet\/Blogs --branch voice_summarizer<\/code><\/p>\n

This project running on Ubuntu 20.04 LTS. These are the mandatory stack requirements that you need to run for this project:<\/p>\n

OpenAI Key<\/h3>\n

To access any OpenAI services, we need a valid key. Go to this link<\/a> and create a new OpenAI key.<\/p>\n

\"\"<\/a><\/p>\n

The OpenAI key is on a project basis, so we need to create a project first in the OpenAI platform and you need also to enable any models that you use on a project. For this project, we will need gpt-4o<\/code> and whisper<\/code> models.<\/p>\n

\"\"<\/a><\/p>\n

The OpenAI key will be saved on the .env<\/code> file and make sure not to include it in version control by adding it to the .gitignore<\/code>.<\/p>\n

Node.js<\/h3>\n

This project will run on the Node.js platform. You need to install it from here<\/a>. For this project, we will use the nvm<\/code> package manager and Node.js v16.20.2 LTS version.<\/p>\n

\n
# installs nvm (Node Version Manager)\ncurl -o- https:\/\/raw.githubusercontent.com\/nvm-sh\/nvm\/v0.39.7\/install.sh | bash\n\n# download and install Node.js\nnvm install 16\n\n# verifies the right Node.js version is in the environment\nnode -v # should print `v16.20.2`\n\n# verifies the right NPM version is in the environment\nnpm -v # should print `8.19.4``<\/code><\/pre>\n<\/div>\n
To connect Node.js and GridDB database, you need the gridb-node-api<\/a> npm package which is a Node.js binding developed using GridDB C Client and Node addon API.<\/p>\n
FFmpeg<\/h3>\n
This project utilizes the fluent-ffmpeg<\/code><\/a> npm package, which requires FFmpeg to be installed on the system. For Ubuntu, you can use the following command to install it:<\/p>\n
\n
sudo apt update\nsudo apt install ffmpeg<\/code><\/pre>\n<\/div>\n
For more installation information, please go to the FFmpeg official website<\/a>.<\/p>\n
GridDB<\/h3>\n
To save the video summary and video data, we will use the GridDB database. Please look at the guide<\/a> for detailed installation. We will use Ubuntu 20.04 LTS here.<\/p>\n
Run GridDB and check if the service is running. Use this command:<\/p>\n
\n
sudo systemctl status gridstore<\/code><\/pre>\n<\/div>\n
If not running try to run the database with this command:<\/p>\n
\n
sudo systemctl start gridstore<\/code><\/pre>\n<\/div>\n
Run the Project<\/h2>\n
Pull the Source Code<\/h3>\n
To run the project, you need to clone the code from this repository<\/a>.<\/p>\n
Run this command to pull the source code.<\/p>\n
\n
git clone https:\/\/github.com\/junwatu\/video-summarizer-nodejs-griddb.git<\/code><\/pre>\n<\/div>\n
Change the directory to the app<\/code> folder and install any project dependencies using this command:<\/p>\n
\n
cd video-summarizer-nodejs-griddb\nnpm install<\/code><\/pre>\n<\/div>\n
Setup .env<\/code><\/h3>\n
This project needs a few environment variables. Copy the env.example<\/code> file to the .env<\/code> file.<\/p>\n
\n
cp .env .example .env<\/code><\/pre>\n<\/div>\n
You need to fill in the environment variables in this file:<\/p>\n
\n
OPENAI_API_KEY=sk-....\nVITE_API_URL=http:\/\/localhost:3000<\/code><\/pre>\n<\/div>\n
It is important to note that the project must be restarted every time you change the VITE_API_URL<\/code> environment variable.<\/p>\n
Start the Project<\/h3>\n
Run this command to run the project:<\/p>\n
\n
npm run start:build<\/code><\/pre>\n<\/div>\n
Open Web App<\/h3>\n
Open the web app using the browser. The default URL is http:\/\/localhost:3000<\/code>. Upload any videos and it’s recommended to upload a short video for fast processing. You can also use the video in the test\/video<\/code> folder. Depending on the video duration, it will take a minute to process.<\/p>\n
\"\"<\/a><\/p>\n
How it Works?<\/h2>\n
The user flow for this project or web app involves opening the web app, uploading the video, waiting for processing, and receiving the summary result. It uses the GPT-4o and Whisper models from OpenAI to summarize the uploaded user video. This project requires two models because OpenAI models cannot process video directly, however they can process images or audio files. On the other side, in Node.js, to separate the video into images and audio files, we use the fluent-ffmpeg<\/code><\/a> npm package.<\/p>\n
These are the primary preparation steps for videos before we input them into OpenAI models for the summarization process.<\/p>\n
1. Video Processing<\/h3>\n
While it’s not possible to directly send a video to the API, GPT-4o can understand videos if you sample frames and then provide them as images. It performs better at this task than the earlier GPT-4 Turbo model.<\/p>\n
This function, extractFrames()<\/code>, will extract images from the video file and save them in the frames<\/code> folder.<\/p>\n
\n
export function extractFrames(videoPath, secondsPerFrame, outputFolder) {\n    return new Promise((resolve, reject) => {\n        const frameRate = 1 \/ secondsPerFrame\n        const framePattern = path.join(outputFolder, 'frame-%03d.png')\n\n        ffmpeg(videoPath)\n            .outputOptions([`-vf fps=${frameRate}`])\n            .output(framePattern)\n            .on('end', () => {\n                fs.readdir(outputFolder, (err, files) => {\n                    if (err) {\n                        reject(err)\n                    } else {\n                        const framePaths = files.map(file => path.join(outputFolder, file))\n                        resolve(framePaths)\n                    }\n                })\n            })\n            .on('error', reject)\n            .run()\n    })\n}<\/code><\/pre>\n<\/div>\n
In the function extractFrames<\/code>, the parameter secondsPerFrame<\/code> defines the interval between frames that you want to extract from the video. Specifically, secondsPerFrame<\/code> determines how many seconds should elapse between each frame that is extracted.<\/p>\n
Here\u00e2\u0080\u0099s how it works:<\/p>\n
Frame Rate Calculation: The frame rate is calculated as the reciprocal of secondsPerFrame, i.e., frameRate = 1 \/ secondsPerFrame. This means:<\/p>\n