Video to text

Get a textual description of a video

Overview

The Video to text feature is an advanced multimodal ML model designed to transform visual content into comprehensive textual narratives. By analyzing the temporal and spatial data within a video, the model generates descriptive summaries that capture key actions, objects, and contexts. This feature significantly streamlines content accessibility and archival workflows, allowing users to understand video substance without manual playback.

Key Capabilities

Configurable Description Length: Users can tailor the output to their specific needs, ranging from concise, one-sentence "shorthand" summaries to detailed, long-form paragraph descriptions.
Multilingual Support: The model integrates a powerful translation layer, enabling the generation of descriptions in several languages. This ensures global reach and cross-regional collaboration for localized video assets.

Typical use cases

Search & Discovery: Generate searchable summaries to help users find specific clips in large video libraries without watching them.
Accessibility: Provide automated "Alt-text" or audio description scripts for visually impaired users to meet global compliance standards.
Content Localization: Instantly translate video summaries into multiple languages for international audiences and global distribution.
Social Media Management: Create quick, catchy captions and descriptions for platforms like YouTube, TikTok, or Instagram.

API endpoints

An up-to-date reference with all API endpoints is available here:

Scaleflex API for Digital Asset Management (DAM), Visual AI and Media OptimizationScaleflex API

Example API responses

Input video

API response

{
    "status":"success",
    "version":"3.34.1",
    "video_to_text":"A woman with long brown hair, wearing a pink tank top and white shorts, sits on a sandy beach, facing the ocean. The water is calm with gentle waves, and the sky is blue with scattered white clouds. The scene evokes a sense of peace and tranquility as she gazes out at the horizon.",
    "request_uuid":"0ddc7a38-e63a-40e6-89ce-687efee663fb",
    "sha1":"1ae8b2fcce5b37d8a0d6afe21a0ed9757cf78cbc",
    "translations":{
        "en":"A woman with long brown hair, wearing a pink tank top and white shorts, sits on a sandy beach, facing the ocean. The water is calm with gentle waves, and the sky is blue with scattered white clouds. The scene evokes a sense of peace and tranquility as she gazes out at the horizon.",
        "fr":"Une femme aux longs cheveux bruns, vêtue d\'un débardeur rose et d\'un short blanc, est assise sur une plage de sable, face à l\'océan. L\'eau est calme avec des vagues douces et le ciel est bleu avec des nuages blancs épars. La scène évoque un sentiment de paix et de tranquillité alors qu\'elle contemple l\'horizon.",
        "es":"Una mujer de pelo largo y castaño, con un top rosa y pantalones cortos blancos, se sienta en una playa de arena, de cara al océano. El agua está en calma con suaves olas y el cielo es azul con nubes blancas dispersas. La escena evoca una sensación de paz y tranquilidad mientras ella mira hacia el horizonte.",
        "it":"Una donna con lunghi capelli castani, che indossa un top rosa e pantaloncini bianchi, è seduta su una spiaggia sabbiosa, di fronte all\'oceano. L\'acqua è calma con onde dolci e il cielo è blu con nuvole bianche sparse. La scena evoca un senso di pace e tranquillità mentre contempla l\'orizzonte."
    },
    "file_downloaded":"sample-mp4-files-sample_960x540.mp4",
    "file_downloaded_in":0.85,
    "duration_sec":13

}

PreviousFace detection NextSearch & find assets

Last updated 2 months ago

{
    "status":"success",
    "version":"3.34.1",
    "video_to_text":"A woman with long brown hair, wearing a pink tank top and white shorts, sits on a sandy beach, facing the ocean. The water is calm with gentle waves, and the sky is blue with scattered white clouds. The scene evokes a sense of peace and tranquility as she gazes out at the horizon.",
    "request_uuid":"0ddc7a38-e63a-40e6-89ce-687efee663fb",
    "sha1":"1ae8b2fcce5b37d8a0d6afe21a0ed9757cf78cbc",
    "translations":{
        "en":"A woman with long brown hair, wearing a pink tank top and white shorts, sits on a sandy beach, facing the ocean. The water is calm with gentle waves, and the sky is blue with scattered white clouds. The scene evokes a sense of peace and tranquility as she gazes out at the horizon.",
        "fr":"Une femme aux longs cheveux bruns, vêtue d\'un débardeur rose et d\'un short blanc, est assise sur une plage de sable, face à l\'océan. L\'eau est calme avec des vagues douces et le ciel est bleu avec des nuages blancs épars. La scène évoque un sentiment de paix et de tranquillité alors qu\'elle contemple l\'horizon.",
        "es":"Una mujer de pelo largo y castaño, con un top rosa y pantalones cortos blancos, se sienta en una playa de arena, de cara al océano. El agua está en calma con suaves olas y el cielo es azul con nubes blancas dispersas. La escena evoca una sensación de paz y tranquilidad mientras ella mira hacia el horizonte.",
        "it":"Una donna con lunghi capelli castani, che indossa un top rosa e pantaloncini bianchi, è seduta su una spiaggia sabbiosa, di fronte all\'oceano. L\'acqua è calma con onde dolci e il cielo è blu con nuvole bianche sparse. La scena evoca un senso di pace e tranquillità mentre contempla l\'orizzonte."
    },
    "file_downloaded":"sample-mp4-files-sample_960x540.mp4",
    "file_downloaded_in":0.85,
    "duration_sec":13

}