Highlevel API

A simple highlevel API is built on top of OpenVIP core library. This API does not cover all OpenVIP features, it is intended to make most common tasks very easy. Unlike the core components, this API has C interface. Python binding is available as well.

This section gives brief tutorial on using the API. It does not describe used data structures, returned values etc. in detail; consult openvip/highlevel_api.h header for that. There's an example application in src/tools/cli/cli.cpp, too.

Initialization and Shutdown

An application that wishes to use OpenVIP highlevel API has to link against openvip_hl library (libopenvip_hl.so on Unix, openvip_hl.dll on Windows). The library must be initialized prior to using it and properly deinitialized before the application exits:

#include <openvip/highlevel_api.h>

int main()
{
  openvip_library_t *lib;
  lib = openvip_create(NULL);

  ...

  openvip_destroy(lib);
}

The argument to openvip_create will most often be NULL, but can be used to specify the directory where OpenVIP library is installed. You can see that openvip_create creates an instance of openvip_library_t. This handle is passed to all OpenVIP functions, which makes it possible to use the library in multi-threaded environment.

After initializing the library, application should set UI callback. UI callback is set of functions that are called to show progress indicator during lengthy operations (such as rendering or indexing multimedia files) and also to report errors and warnings in human-readable form. The application is not required to set the callback, but if it doesn't do it, all error and progress messages are discarded. openvip_set_ui_callback takes openvip_ui_callback_t structure and arbitrary cookie as arguments. The cookie is pointer to any information the application needs to pass to the callbacks. It is not interpreted in any way, only passed to the callbacks. Following example will make it clearer:

void show_gauge(void *cookie)
{
    printf("%s: busy\n", (const char*)cookie);
}
int update_gauge(float value, void *cookie)
{
    printf("%s: %0.2f%%\n", (const char*)cookie, value*100);
    return 1; /* continue working */
}
void hide_gauge(void *cookie) 
{
    printf("%s: done\n", (const char*)cookie);
}
void info_gauge(const char *text, void *cookie)
{ 
    printf("%s: busy info: %s\n", (const char*)cookie, text);
}
void log_message(const char *text, void *cookie)
{
    printf("%s: error: %s\n", (const char*)cookie, text);
}
    
openvip_ui_callback_t my_callback =
{
    show_gauge,
    update_gauge,
    hide_gauge,
    info_gauge,
    log_message, /* log informative message */
    log_message, /* log error */
    log_message, /* log warning */
};

...

openvip_set_ui_callback(lib, my_callback, argv[1]);

Processing

OpenVIP library uses networks as the input. When you are using the highlevel API, you must first load the network into memory and then render it.

Loading Networks

The most common ways to load a network are to load it from file using openvip_load_network_file or from string contained in memory using openvip_load_network_from_string. The latter is useful if you need to generate the network dynamically, for example when previewing a filter in GUI.

Networks loaded this way, however, can only store rendered video into file. Many applications need ability to further process the output and so they need to get raw data from OpenVIP in some kind of memory buffer. Function openvip_load_network_with_memoutput_from_string is designed to meet this need (note that it only works with video output, audio is not supported). The network is given as a string (as in openvip_load_network_from_string), but there is additional requirement on the network: it must contain exactly one terminal module whose type must be openvip.highlevel_api.MemOutput. ID of the module is given as another argument. The last argument is openvip_dest_t structure that has a pointer to callback function. The callback is called for every generated frame with raw data in RGB24 format as argument.

A typical network for processing with memory output looks like this:

<?xml version="1.0" ?>
<!DOCTYPE network PUBLIC
          "-//OPENVIP//DTD Network Format V1.0//EN" 
          "http://openvip.sourceforge.net/dtds/openvip-network.dtd">

<network xmlns="http://openvip.sourceforge.net/network-format" version="1.0">

    <module id="loader0" class="Input">
        <param name="filename">test.avi</param>
    </module>

    <module id="OUTPUT" class="openvip.highlevel_api.MemOutput">
        <param name="filename">clasa.wav</param>
        <param name="format">Wav</param>
    </module>

    <connect module_in="loader0" conn_in="audio0" module_out="OUTPUT" conn_out="audio0"/>

</network>

And the call to create rendering network:

void callback_func(unsigned char *data, size_t len, void *cookie)
{
    /* process len bytes in data... */
}
...
openvip_task_t *task;
openvip_dest_t dest;
dest.cookie = NULL; /*we don't use it*/
dest.callback = callback_func;
task = openvip_load_network_with_memoutput_from_string(lib, bufferWithNetworkCode, "OUTPUT", &dest);

Rendering

There are two functions for rendering loaded networks: openvip_render and openvip_render_single_frame. The former completely render the task and returns when rendering is completed. It has two optional parameters to specify UI callback (analogically to openvip_set_ui_callback). If not NULL, then this callback is used instead of the one set using openvip_set_ui_callback.

The other one only works with video networks created using openvip_load_network_with_memoutput_from_string or openvip_create_thumbnails_generator. It renders only one (specified) frame of the output instead of rendering everything and returns rendered raw data. Returned buffer is same as the one passed to openvip_dest_t callback when using openvip_render on the same network.

After you are done with rendering, you must destroy the task using openvip_destroy_task.

openvip_task_t *task = openvip_load_network_file(lib, "foo.avi");
openvip_render(lib, task);
openvip_destroy_task(lib, task);

Thumbnails

The API contains function for effective creation of thumbnails, too. Thumbnails are very small and short video sequences that represent video or audio stream. Images from the sequence are intended to be layed alongside to form a strip, as in this example (note that there are two thumbnails in the image, for video stream and for audio below it):

Thumbnails are rendered by network obtained from openvip_create_thumbnails_generator fuction. The function takes file- and streamname as arguments, together with desired dimensions of produced images and number of thumbnail frames. The function returns OpenVIP network renderable using openvip_render or openvip_render_single_frame. Network's output is passed to callback function specified as the last argument to openvip_create_thumbnails_generator in exactly same way as when using load_network_with_memoutput_from_string. The network will produce one video stream with requested dimensions and specified number of frames long.

Thumbnails generator caches generated thumbnails in temporary directory. Cache's data structures are designed in such way that not only are subsequent uses of the generator with same file and arguments much faster, but it also speeds up creation of thumbnail strips with same dimensions but different number of frames (not even same dimensions are required for audio streams). This is accomplished by storing thumbnail images for every frame of the stream and composing final N-frames thumbnail from N such "subthumbnails" uniformly distributed along streams' length.

Additional Functions

Getting Information About Files

Information about multimedia file's content can be obtained easily using the highlevel API. openvip_get_file_info function returns pointer to openvip_file_info_t structure which contains filename and two arrays: one of them is list of all video streams contained in the file, the other lists audio streams. Stream information structure for video streams contains width, height, FPS, aspect ratio and length in frames. Audio stream information consists of sample rate, number of channels (1-mono, 2-stereo, ...) and length in samples.

unsigned i;
openvip_file_info_t *fi = openvip_get_file_info(lib, "foo.avi");
for (i = 0; i < fi->video_cnt; i++)
    printf("video stream %s: %ix%i\",
    fi->video_streams[i].name, fi->video_streams[i].width, fi->video_streams[i].height);
for (i = 0; i < fi->audio_cnt; i++)
    printf("audio stream %s: %iHz %s\",
    fi->audio_streams[i].name, fi->audio_streams[i].sample_rate,
    fi->audio_streams[i].channels == 1 ? "mono" : "stereo");
openvip_free(lib, fi);

Enumerating Plugins

Function openvip_enum_plugins can be used to enumerate all plugins of certain type (one of OPENVIP_PLUGIN_VIDEO_FILTER, OPENVIP_PLUGIN_AUDIO_FILTER, OPENVIP_PLUGIN_VIDEO_TRANSITION, OPENVIP_PLUGIN_AUDIO_TRANSITION, OPENVIP_PLUGIN_INPUT and OPENVIP_PLUGIN_OUTPUT). The function returns openvip_plugins_info_t structure which is esentially an array of name and description fields for every plugin. The former is formal name of the plugin to be used in OpenVIP network format files, the latter is descriptive name intented for humans (and as such is suitable for use in GUIs).

unsigned i;
openvip_plugins_info_t *p = openvip_enum_plugins(lib, OPENVIP_PLUGIN_VIDEO_FILTER);
for (i = 0; i < p->cnt; i++)
    printf("filter '%s' has description '%s'\n", p->plugins[i].name, p->plugins[i].description);
openvip_free(lib, p);

Python Binding

Highlevel API is available as Python extensions module as well. There are few differences from C API that are imposed by the way things are usually done in Python:

  • the API was translated into Python classes

  • exceptions derived from openvip.Error are used to report errors

Inicialization is done by creating instance of openvip.Library class:

import openvip
lib = openvip.Library()

Functions do not take library argument, but are methods of openvip.Library (or openvip.Task in case of openvip_render and openvip_render_single_frame) instead. openvip_ prefix is omitted, too. An application that renders network given as command line argument would look like this in Python:

import sys, openvip
lib = openvip.Library()
task = lib.load_network_file(sys.argv[0])
task.render()

dest argument to openvip.Library.load_network_with_memoutput_from_string and openvip.Library.create_thumbnails_generator is instance of class derived from openvip.DestCallback or openvip.DestCallbackPIL. Constructors of both classes take reference to callback function as its argument. The former calls the callback with three arguments (data string, width and height), the latter creates Image object (from Python Imaging Library, which must be installed if you use DestCallbackPIL) and passes it to the callback as single argument. Typical usage:

filename = sys.argv[2]
def callback(img):
    img.save(filename)
task = lib.create_thumbnails_generator(sys.argv[1], 'video0', w, h, frames, DestCallbackPIL(callback))

openvip.Library.enum_plugins returns Python list of openvip.PluginInfo objects. openvip.Library.get_file_info and openvip.Library.get_task_file_info return openvip.FileInfo object. See openvip.py for their definition; they are almost identical to their C counterparts, except that Python lists are used instead of linked lists from C API.

Finally, there are several example applications written using Python highlevel API binding in src/tools.