FFmpeg之SDL(四)

610 阅读6分钟

1、环境搭建:

官网:www.libsdl.org/ 文档:wiki.libsdl.org/Introductio…

下载编译好的库,目录结构如下:

image.png 把这个SDL2-2.0.14放到工程路径下:

image.png

配置头文件目录:写的相对路径,是相对于你的cpp文件而言。 image.png 配置链接库:

image.png 再指定下具有链接的库:

image.png

这时候需要设置动态库dll的路径了,有两种方法: 放在和main.cpp相同的路径下或者生成的exe相同的路径下。

image.png 或者 image.png

测试程序,打开一个窗口:

#include<iostream>
#include <SDL.h>

int main(int argc, char* argv[])
{
	SDL_Window* window = NULL;
	SDL_Init(SDL_INIT_VIDEO);
	window = SDL_CreateWindow("Basic Window",
							SDL_WINDOWPOS_UNDEFINED, // 设置为0,0窗口在屏幕的左上角,设置为 SDL_WINDOWPOS_UNDEFINED在屏幕中间显示
							SDL_WINDOWPOS_UNDEFINED, // 同上
							640,
							480,
							SDL_WINDOW_OPENGL | SDL_WINDOW_RESIZABLE);
	if (!window) 
	{
		std::cout << "error:"<< SDL_GetError() << std::endl;
		return 1;
	}

	SDL_Delay(5000); // delay 5000ms
	SDL_DestroyWindow(window);

	SDL_Quit(); // free
}

SDL子系统

SDL将功能分成以下子系统:

  • SDL_INIT_TIMER:定时器
  • SDL_INIT_AUDIO:音频
  • SDL_INIT_VIDEO:视频
  • SDL_INIT_JOYSTICK:摇杆
  • SDL_INIT_HAPTIC:触摸屏
  • DL_INIT_GAMECONTROLLER:游戏控制器
  • SDL_INIT_EVENTS:事件
  • SDL_INIT_EVERYTHING:包含上述所有选项

主要API

SDL_Init():初始化SDL系统 
SDL_CreateWindow():创建窗口SDL_Window 
SDL_CreateRenderer():创建渲染器SDL_Renderer
SDL_CreateTexture():创建纹理SDL_Texture
SDL_UpdateTexture():设置纹理的数据 
SDL_RenderCopy():将纹理的数据拷贝给渲染器 
SDL_RenderPresent():显示 
SDL_Delay():工具函数,用于延时 
SDL_Quit():退出SDL系统

SDL事件

SDL_Event:代表一个事件

SDL_WaitEvent():等待一个事件 
SDL_PushEvent():发送一个事件 
SDL_PumpEvents():将硬件设备产生的事件放入事件队列,用于读取事件,在调用该函数之前,必须调用SDL_PumpEvents搜集键盘等事件
SDL_PeepEvents():从事件队列提取一个事件

示例:

#include<iostream>
#include <SDL.h>
#include <windows.h>
#define FF_QUIT_EVENT    (SDL_USEREVENT + 2) // 用户自定义事件
#undef main
int main(int argc, char* argv[])
{
    SDL_Window* window = NULL;              // Declare a pointer
    SDL_Renderer* renderer = NULL;

    SDL_Init(SDL_INIT_VIDEO);               // Initialize SDL2

    // Create an application window with the following settings:
    window = SDL_CreateWindow(
        "An SDL2 window",                  // window title
        SDL_WINDOWPOS_UNDEFINED,           // initial x position
        SDL_WINDOWPOS_UNDEFINED,           // initial y position
        640,                               // width, in pixels
        480,                               // height, in pixels
        SDL_WINDOW_SHOWN | SDL_WINDOW_BORDERLESS// flags - see below
    );

    // Check that the window was successfully created
    if (window == NULL)
    {
        // In the case that the window could not be made...
        printf("Could not create window: %s\n", SDL_GetError());
        return 1;
    }

    /* We must call SDL_CreateRenderer in order for draw calls to affect this window. */
    renderer = SDL_CreateRenderer(window, -1, 0);

    /* Select the color for drawing. It is set to red here. */
    SDL_SetRenderDrawColor(renderer, 255, 0, 0, 255);

    /* Clear the entire screen to our selected color. */
    SDL_RenderClear(renderer);

    /* Up until now everything was drawn behind the scenes.
       This will show the new, red contents of the window. */
    SDL_RenderPresent(renderer);

    SDL_Event event;
    int b_exit = 0;
    for (;;)
    {
        SDL_WaitEvent(&event); // 监听事件
        switch (event.type)
        {
        case SDL_KEYDOWN:	/* 键盘事件 */
            switch (event.key.keysym.sym)
            {
            case SDLK_a:
                printf("key down a\n");
                break;
            case SDLK_s:
                printf("key down s\n");
                break;
            case SDLK_d:
                printf("key down d\n");
                break;
            case SDLK_q:
                printf("key down q and push quit event\n");
                SDL_Event event_q;
                event_q.type = FF_QUIT_EVENT;
                SDL_PushEvent(&event_q); // 发送自定义事件
                break;
            default:
                printf("key down 0x%x\n", event.key.keysym.sym);
                break;
            }
            break;
        case SDL_MOUSEBUTTONDOWN:			/* 鼠标按下事件 */
            if (event.button.button == SDL_BUTTON_LEFT)
            {
                printf("mouse down left\n");
            }
            else if (event.button.button == SDL_BUTTON_RIGHT)
            {
                printf("mouse down right\n");
            }
            else
            {
                printf("mouse down %d\n", event.button.button);
            }
            break;
        case SDL_MOUSEMOTION:		/* 鼠标移动事件 */
            printf("mouse move (%d,%d)\n", event.button.x, event.button.y);
            break;
        case FF_QUIT_EVENT:
            printf("receive quit event\n");
            b_exit = 1;
            break;
        }
        if (b_exit) // 等于1,直接退出循环
            break;
    }

    //destory renderer
    if (renderer)
        SDL_DestroyRenderer(renderer);

    // Close and destroy the window
    if (window)
        SDL_DestroyWindow(window);

    // Clean up
    SDL_Quit();
    return 0;
}


SDL多线程

SDL线程创建:SDL_CreateThread 
SDL线程等待:SDL_WaitThead 
SDL互斥锁:SDL_CreateMutex/SDL_DestroyMutex 
SDL锁定互斥:SDL_LockMutex/SDL_UnlockMutex 
SDL条件变量(信号量):SDL_CreateCond/SDL_DestoryCond 
SDL条件变量(信号量)等待/通知:SDL_CondWait/SDL_CondSingal

示例:

#include<iostream>
#include <SDL.h>
#include <windows.h>
SDL_mutex* s_lock = NULL;
SDL_cond* s_cond = NULL;

int thread_work(void* arg)
{
    SDL_LockMutex(s_lock);
    printf("                <============thread_work sleep\n");
    Sleep(10 * 1000);      // 用来测试获取锁,Sleep不会让出锁
    printf("                <============thread_work wait\n");
    // 释放s_lock资源,并等待signal。之所以释放s_lock是让别的线程能够获取到s_lock
    SDL_CondWait(s_cond, s_lock); //阻塞,直到主线程发送signal并且主线程释放lock后,这个函数退出

    printf("                <===========thread_work receive signal, continue to do ~_~!!!\n");
    printf("                <===========thread_work end\n");
    SDL_UnlockMutex(s_lock);
    return 0;
}

int main(int argc, char* argv[])
{
    s_lock = SDL_CreateMutex();
    s_cond = SDL_CreateCond();
    SDL_Thread* t = SDL_CreateThread(thread_work, "thread_work", NULL);
    if (!t)
    {
        printf("  %s", SDL_GetError);
        return -1;
    }

    for (int i = 0; i < 2; i++)
    {
        Sleep(2 * 1000);
        printf("main execute =====>\n");
    }
    printf("main SDL_LockMutex(s_lock) before ====================>\n");
    SDL_LockMutex(s_lock);  // 获取锁,但是子线程还拿着锁, 阻塞直到可以拿到锁
    printf("main ready send signal====================>\n");
    printf("main SDL_CondSignal(s_cond) before ====================>\n");
    SDL_CondSignal(s_cond); // 发送信号,唤醒等待的线程
    printf("main SDL_CondSignal(s_cond) after ====================>\n");
    Sleep(10 * 1000);
    SDL_UnlockMutex(s_lock);// 释放锁,让其他线程可以拿到锁
    printf("main SDL_UnlockMutex(s_lock) after ====================>\n");

    SDL_WaitThread(t, NULL);
    SDL_DestroyMutex(s_lock);
    SDL_DestroyCond(s_cond);
    return 0;
}

简单分析: 首先最重要要明确: 唤醒等待的线程+拿到锁,线程才能继续执行。

主线程创建出子线程,子线程立即获得锁,并且进入休眠sleep 10s,sleep不会释放锁。同时,主线程也会继续执行,间隔2s,打印main execute.执行完后再打印'main SDL_LockMutex(s_lock) before ====================>'.这时候主线程执行SDL_LockMutex去获取锁,不巧的是,子线程此时还抱着锁再休眠。主线程进行等待。 子线程完成了休眠,打印'<============thread_work wait',然后执行SDL_CondWait(),它会释放锁,并且阻塞等待着signal来唤醒。与此同时,主线程发现子线程已经释放锁,立即持有锁,继续执行,打印'main ready send signal====================>' 和 'main SDL_CondSignal(s_cond) before ====================>',主线程执行SDL_CondSignal(s_cond),发送signal唤醒线程。并且进入了休眠,主线程没有释放锁。同时,子线程收到signal,但是却没有持有锁,仍然需要等待主线程释放锁资源让自己持有。等到主线程休眠完成,释放锁。子线程终于可以继续后面的事情了...

结果:

image.png

SDL视频显示

流程:

graph TD
SDL_Init --> SDL_CreateWindow --> SDL_Window --> SDL_CreateRenderer --> SDL_Renderer --> SDL_CreateTexture --> SDL_Texture --> SDL_UpdateTexture -->
SDL_RenderClear --> SDL_RenderCopy --> SDL_RenderPresent

为了方便,我们直接将YUV显示出来,只需要将一帧帧数据构造成Texture,塞到流程中即可。 这里先说明,根据YUV的类型,如何正确的读取一帧数据,请参考FFmpeg之YUV基础(二)。 我们现在用I420测试。 生成YUV数据: ffmpeg -i source.mp4 -an -c:v rawvideo -pixel_format yuv420p source.yuv 注意:可以用mediainfo查看视频的宽高,然后示例中的YUV设置的宽高保持一致。

示例:

#include<iostream>
#include <SDL.h>
//自定义消息类型
#define REFRESH_EVENT   (SDL_USEREVENT + 1)     // 请求画面刷新事件
#define QUIT_EVENT      (SDL_USEREVENT + 2)     // 退出事件

//定义分辨率
// YUV像素分辨率
#define YUV_WIDTH  1280 //320
#define YUV_HEIGHT 720 //240
//定义YUV格式
#define YUV_FORMAT  SDL_PIXELFORMAT_IYUV

int s_thread_exit = 0;  // 退出标志 = 1则退出

int refresh_video_timer(void *data)
{
    while (!s_thread_exit)
    {
        SDL_Event event;
        event.type = REFRESH_EVENT;
        SDL_PushEvent(&event);
        SDL_Delay(40);
    }

    s_thread_exit = 0;

    //push quit event
    SDL_Event event;
    event.type = QUIT_EVENT;
    SDL_PushEvent(&event);

    return 0;
}
#undef main
int main(int argc, char* argv[])
{
    //初始化 SDL
    if(SDL_Init(SDL_INIT_VIDEO))
    {
        fprintf( stderr, "Could not initialize SDL - %s\n", SDL_GetError());
        return -1;
    }

    // SDL
    SDL_Event event;                            // 事件
    SDL_Rect rect;                              // 矩形
    SDL_Window *window = NULL;                  // 窗口
    SDL_Renderer *renderer = NULL;              // 渲染
    SDL_Texture *texture = NULL;                // 纹理
    SDL_Thread *timer_thread = NULL;            // 请求刷新线程
    uint32_t pixformat = YUV_FORMAT;            // YUV420P,即是SDL_PIXELFORMAT_IYUV

    // 分辨率
    // 1. YUV的分辨率
    int video_width = YUV_WIDTH;
    int video_height = YUV_HEIGHT;
    // 2.显示窗口的分辨率
    int win_width = YUV_WIDTH;
    int win_height = YUV_WIDTH;

    // YUV文件句柄
    FILE *video_fd = NULL;
    const char *yuv_path = "C:\\Users\\Lenovo\\Desktop\\source.yuv";

    size_t video_buff_len = 0;

    uint8_t *video_buf = NULL; //读取数据后先把放到buffer里面

    // 我们测试的文件是YUV420P格式
    uint32_t y_frame_len = video_width * video_height;
    uint32_t u_frame_len = video_width * video_height / 4;
    uint32_t v_frame_len = video_width * video_height / 4;
    uint32_t yuv_frame_len = y_frame_len + u_frame_len + v_frame_len;

    //创建窗口
    window = SDL_CreateWindow("Simplest YUV Player",
                           SDL_WINDOWPOS_UNDEFINED,
                           SDL_WINDOWPOS_UNDEFINED,
                           video_width, video_height,
                           SDL_WINDOW_OPENGL|SDL_WINDOW_RESIZABLE);
    if(!window)
    {
        fprintf(stderr, "SDL: could not create window, err:%s\n",SDL_GetError());
        goto _FAIL;
    }
    // 基于窗口创建渲染器
    renderer = SDL_CreateRenderer(window, -1, 0);
    // 基于渲染器创建纹理
    texture = SDL_CreateTexture(renderer,
                                pixformat,
                                SDL_TEXTUREACCESS_STREAMING,
                                video_width,
                                video_height);

    // 分配空间
    video_buf = (uint8_t*)malloc(yuv_frame_len);
    if(!video_buf)
    {
        fprintf(stderr, "Failed to alloce yuv frame space!\n");
        goto _FAIL;
    }

    // 打开YUV文件
    video_fd = fopen(yuv_path, "rb");
    if( !video_fd )
    {
        fprintf(stderr, "Failed to open yuv file\n");
        goto _FAIL;
    }
    // 创建请求刷新线程
    timer_thread = SDL_CreateThread(refresh_video_timer,
                                    NULL,
                                    NULL);

    while (1)
    {
        // 收取SDL系统里面的事件
        SDL_WaitEvent(&event);

        if(event.type == REFRESH_EVENT) // 画面刷新事件
        {
            video_buff_len = fread(video_buf, 1, yuv_frame_len, video_fd);
            if(video_buff_len <= 0)
            {
                fprintf(stderr, "Failed to read data from yuv file!\n");
                goto _FAIL;
            }
            // 设置纹理的数据 video_width = 320, plane
            SDL_UpdateTexture(texture, NULL, video_buf, video_width);

            // 显示区域,可以通过修改w和h进行缩放
            rect.x = 0;
            rect.y = 0;
            float w_ratio = win_width * 1.0 /video_width;
            float h_ratio = win_height * 1.0 /video_height;
            // 320x240 怎么保持原视频的宽高比例
            rect.w = video_width * w_ratio;
            rect.h = video_height * h_ratio;
//            rect.w = video_width * 0.5;
//            rect.h = video_height * 0.5;

            // 清除当前显示
            SDL_RenderClear(renderer);
            // 将纹理的数据拷贝给渲染器
            SDL_RenderCopy(renderer, texture, NULL, &rect);
            // 显示
            SDL_RenderPresent(renderer);
        }
        else if(event.type == SDL_WINDOWEVENT)
        {
            //If Resize
            SDL_GetWindowSize(window, &win_width, &win_height);
            printf("SDL_WINDOWEVENT win_width:%d, win_height:%d\n",win_width,
                   win_height );
        }
        else if(event.type == SDL_QUIT) //退出事件
        {
            s_thread_exit = 1;
        }
        else if(event.type == QUIT_EVENT)
        {
            break;
        }
    }

_FAIL:
    s_thread_exit = 1;      // 保证线程能够退出
    // 释放资源
    if(timer_thread)
        SDL_WaitThread(timer_thread, NULL); // 等待线程退出
    if(video_buf)
        free(video_buf);
    if(video_fd)
        fclose(video_fd);
    if(texture)
        SDL_DestroyTexture(texture);
    if(renderer)
        SDL_DestroyRenderer(renderer);
    if(window)
        SDL_DestroyWindow(window);

    SDL_Quit();

    return 0;
}

简单分析:

1、明确两个自定义事件,依次是刷新和退出事件;另外还有一个resize事件,window窗口改变大小触发。

2、首先分析REFRESH_EVENT事件,可以看到有一个子线程用来将发送刷新事件,可以看到if(event.type == REFRESH_EVENT)这个分支的处理:读取yuv数据,更新纹理数据,将数据拷贝到render中渲染并在窗口中显示。

只要这个s_thread_exit标志位置为1,那么不再发送刷新事件。if(event.type == REFRESH_EVENT)将不再进行处理。

3、rect用来真正的显示图像数据,这应该就是所谓的"视口"。rect可以方便对调整视频显示的策略。一般来说:

  • 可以将像素数据铺满全屏,如果窗口恰好被你拉伸的扁扁的,那么势必图像会变形。

image.png

  • 另外一种处理是,可以保持原视频的宽高比例,可以通过以宽或者高为基准,对高或宽进行等比列的压缩。当然,这时候不出意外,rect不能盖住window上,这就会出现黑边。

image.png

  • 第三种处理是,当窗口大小小于原始数据的宽高时,可直接进行裁剪。

image.png

4、SDL_QUIT事件相对就简单了,直接break,打破循环,释放资源结束程序。