ffmpeg 应用滤镜搭建播放倍速
FFmpeg 的滤镜可以实现，对原始音视频流进行裁剪、缩放、变速、降噪等基础处理；添加特效（如水印、字幕、转场、滤镜效果）；修复音视频缺陷（如去除杂音、校正色彩）；适配不同场景的需求（如调整分辨率以适应设备、混音以匹配多声道输出）等功能。下面本文将使用滤镜来实现播放倍速功能。
AVFilterGraph* m_filterGraph = nullptr;
AVFilterContext* m_videoSrcFilterCtx = nullptr;
AVFilterContext* m_videoSinkFilterCtx = nullptr;
AVFilterContext* m_audioSrcFilterCtx = nullptr;
AVFilterContext* m_audioSinkFilterCtx = nullptr;这个有几个成员变量，其中 m_filterGraph 是用来管理滤镜的，我们创建滤镜，配置滤镜图的时候都需要用到它。简单来说，我们创建这样一个对象，然后在接下来跟滤镜有关的操作中，参数为 AVFilterGraph 的，都把它传进去就行了。
下面几个分别是视频源滤镜上下文，视频输出滤镜上下文，音频源滤镜上下文，音频输出滤镜上下文。我们会将 AVFrame 传入源滤镜上下文，然后从输出滤镜上下文获取新的 AVFrame.
const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
const AVFilter* setpts = avfilter_get_by_name("setpts");
const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");
const AVFilter* atempo = avfilter_get_by_name("atempo");这里有几个 AVFilter 对象，其中 buffer 对应 视频源滤镜，buffersink 对应视频输出滤镜，abuffer 和 abuffersink 则是音频输入和输出滤镜。setpts 滤镜是用来改变视频帧的 pts 的，而 atempo 滤镜是用来改变音频播放速度的。
int ret = 0;
const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
AVRational time_base = m_videoCodecContext->time_base;
char args[512];
snprintf(args, sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
time_base.num, time_base.den,
m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);
ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in", args, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out", NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffersink 滤镜" << std::endl;
return false;
}
AVFilterContext* videoSpeedCtx = nullptr;
const AVFilter* setpts = avfilter_get_by_name("setpts");
if ((ret = avfilter_graph_create_filter(&videoSpeedCtx,
setpts, "speed",
"0.5*PTS", nullptr, m_filterGraph)) < 0) {
std::cout << "无法创建视频滤镜 " << std::endl;
return -1;
}avfilter_graph_create_filter 用于创建和初始化滤镜，第一个参数是滤镜上下文，第二个参数是 AVFilter，第三个参数是名称，没啥用，随便设置即可，第四个参数是一个字符串，用于描述滤镜的属性。第五个参数没用，第六个参数则是滤镜图。
重点说明下第四个参数，输入滤镜的第四个参数是视频帧/音频帧的一些属性
视频参数
必须设置
- video_size (视频尺寸)
格式：widthxheight
示例：1280x720 - pix_fmt (像素格式)
格式：整数
示例：0(AV_PIX_FMT_YUV420P) - time_base (时间基)
格式：num/den
示例：1/1000
推荐 - pixel_aspect (像素宽高比)
格式：num/den
示例：1/1
描述像素形状，影响显示比例
可选（根据滤镜链需求） - frame_rate (帧率)
格式：num/den或 小数
示例：30000/1001(29.97fps)
说明：当滤镜链需要知道帧率时设置 - sar (样本宽高比)
格式：num/den
示例：1/1
说明：与pixel_aspect类似，有时需要同时设置 - sws_param (缩放参数)
格式：字符串
示例：flags=bicubic
说明：指定缩放算法，当需要调整大小时使用
音频参数
- time_base (时间基)
格式：num/den
示例：1/44100 - sample_rate (采样率)
格式：整数
示例：44100、48000 - sample_fmt (采样格式)
格式：字符串
示例：s16、fltp - channel_layout (声道布局)
格式：十六进制或字符串
示例：0x3、stereo、mono
输出滤镜的参数放空即可，setpts 和 atempo 则跟滤镜有关。
if (avfilter_link(m_videoSrcFilterCtx, 0, videoSpeedCtx, 0) < 0 ||
avfilter_link(videoSpeedCtx, 0, m_videoSinkFilterCtx, 0) < 0) {
std::cerr << "连接视频滤镜失败" << std::endl;
return false;
}avfilter_link 用于链接滤镜，上面的代码表示数据从 m_videoSrcFilterCtx 流向 videoSpeedCtx，再从 videoSpeedCtx 流向 m_videoSinkFilterCtx，很好理解，数据从输入滤镜开始，流向中间的滤镜，有中间的滤镜进行处理，再流向下一个滤镜，最终流入到输出滤镜。
0 表示滤镜输入，输出端口的索引，有的滤镜有多个输入或输出，比如我们如果想把两段音频合并到一起，第一段音频就传入到融合滤镜的第一个端口，第二段音频传入到第二个端口。
if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
std::cerr << "滤镜图配置失败 " << std::endl;
return -1;
}这段代码用于检查滤镜图中所有链接和格式的有效性并进行配置。
至此，滤镜的相关设置就结束了。接下去只需将 AVFrame 传入输入滤镜上下文，再从输出滤镜上下文接收新的 AVFrame 即可
av_buffersrc_add_frame(m_videoSrcFilterCtx, frame);
while (av_buffersink_get_frame(m_videoSinkFilterCtx, filterFrame) >= 0) {
// do something
}需要注意的是，虽然 setpts 的功能只是用于改变视频帧的 pts，但是实际上 filterFrame 的 format 和 frame 不同，因此格式转化的参数需要从 filterFrame 获取。
下面介绍滤镜连接的另外一种方式，这种方式与上一种的区别在于不需要手动连接滤镜，只需要创建 AVFilterInOut 对象，然后将其和输入滤镜和输出滤镜绑定，通过 avfilter_graph_parse_ptr 函数，自动地去链接滤镜。
m_filterGraph = avfilter_graph_alloc();
if (!m_filterGraph) {
return false;
}
// 创建视频滤镜
int ret = 0;
const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
AVRational time_base = m_videoCodecContext->time_base;
char args[512];
snprintf(args, sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
time_base.num, time_base.den,
m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);
ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in",
args, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
// 设置buffersink参数
ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out",
NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffersink 滤镜" << std::endl;
return false;
}前面的部分也是差不多的，创建滤镜图，创建源滤镜和输出滤镜。
AVFilterInOut* outputs = avfilter_inout_alloc();
AVFilterInOut* inputs = avfilter_inout_alloc();
if (!outputs || !inputs) {
return false;
}
outputs->name = av_strdup("in");
outputs->filter_ctx = m_videoSrcFilterCtx;
outputs->pad_idx = 0;
outputs->next = NULL;
inputs->name = av_strdup("out");
inputs->filter_ctx = m_videoSinkFilterCtx;
inputs->pad_idx = 0;
inputs->next = NULL;
if ((ret = avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2",
&inputs, &outputs, NULL)) < 0) {
std::cout << "无法解析滤镜描述" << std::endl;
return false;
}
avfilter_inout_free(&inputs);
avfilter_inout_free(&outputs);这里创建了两个 AVFilterInOut 对象，并设置参数，需要注意的是 outputs 对应的是输入滤镜，inputs 对应的是输出滤镜，可以理解为数据从输入滤镜输出到 AVFilterInOut ，所以是 output。
那么为什么不把 inputs 取名成 outputs 这个就跟 avfilter_graph_parse_ptr 有关了。
avfilter_graph_parse_ptr 函数声明如下：
int avfilter_graph_parse_ptr(AVFilterGraph *graph, const char *filters,
AVFilterInOut **inputs, AVFilterInOut **outputs,
void *log_ctx);AVFilterInOut **inputs 参数就对应上面代码中的 inputs 对象，另外一个同理。
avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2", &inputs, &outputs, NULL))avfilter_graph_parse_ptr 将根据传入的 filters 自动地去创建滤镜链
if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
std::cerr << "滤镜图配置失败 " << std::endl;
return -1;
}最后同样需要调用 avfilter_graph_config 配置滤镜图
不过倍速后播放，音频和视频会有些偏差，暂时不确定是哪方面的问题
VideoDecoder.h 代码如下
#ifndef VIDEODECODER_H
#define VIDEODECODER_H
#include <atomic>
#include <functional>
#include <string>
#include <vector>
#include <mutex>
#include <condition_variable>
extern "C" {
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libavutil/avutil.h"
#include "libswscale/swscale.h"
#include "libswresample/swresample.h"
#include "libavutil/channel_layout.h"
#include "libswresample/swresample.h"
#include "libavutil/opt.h"
#include "libavutil/time.h"
#include "libavfilter/avfilter.h" // 滤镜系统
#include "libavfilter/buffersrc.h" // 缓冲源滤镜
#include "libavfilter/buffersink.h" // 缓冲接收滤镜
#include "libavutil/pixdesc.h"
}
#include "Timer.h"
#include "VideoStructs.h"
using namespace std;
class VideoDecoder
{
public:
VideoDecoder();
~VideoDecoder();
bool open(const string& filePath);
VideoParams getVideoParam() { return m_videoParam;}
AudioParams getAudioParam() { return m_audioParam;}
bool isStop() { return m_stop; }
void addVideoCallback(std::function<bool(AVFrame*)> callback) {
m_videoCallbacks.emplace_back(callback);
}
void addAudioCallback(std::function<bool(AVFrame*)> callback) {
m_audioCallbacks.emplace_back(callback);
}
void play();
void pause();
void stop();
void close();
private:
void showError(int ret);
void readPacket();
void decodeVideo();
void decodeAudio();
// 格式转换
bool initSwrContext(const AVFrame* frame);
bool initSwsContext(const AVFrame* frame);
// 滤镜
bool initFilter();
bool initFilterByDescribe();
// 硬解码
AVHWDeviceType findSupportedHWDeviceType(const AVCodec* codec);
bool initHWDevice();
bool initHWFramesContext();
bool hwFrameToSwFrame(AVFrame* hwFrame, AVFrame* swFrame);
private:
VideoParams m_videoParam;
AudioParams m_audioParam;
// 播放控制及音视频同步
bool m_play = false;
bool m_stop = true;
Timer* m_timer;
std::vector<AVPacket> m_videoPackets;
std::vector<AVPacket> m_audioPackets;
std::mutex m_videoMutex;
std::mutex m_audioMutex;
std::condition_variable m_videoAvailable;
std::condition_variable m_audioAvailable;
std::mutex m_fullMutex;
std::condition_variable m_queueFull;
std::mutex m_playMutex;
std::condition_variable m_playCondition;
std::atomic<int> m_threadCount;
// 回调
std::vector<std::function<bool(AVFrame*)>> m_videoCallbacks;
std::vector<std::function<bool(AVFrame*)>> m_audioCallbacks;
// 格式转化
SwsContext* m_swsContext = nullptr;
SwrContext* m_swrContext = nullptr; // 音频格式转换实例
// 解码
AVFormatContext* m_formatContext = nullptr;
int m_videoIndex = -1; // 视频流所在索引
int m_audioIndex = -1; // 音频流所在索引
AVCodecContext* m_videoCodecContext = nullptr; // 视频解码器实例
AVCodecContext* m_audioCodecContext = nullptr; // 音频解码器实例
// 硬解码
AVBufferRef* m_hwDeviceCtx = nullptr; // 硬件设备上下文
AVBufferRef* m_hwFramesCtx = nullptr; // 硬件帧上下文
AVHWDeviceType m_hwDeviceType = AV_HWDEVICE_TYPE_NONE; // 硬件设备类型（如NVDEC）
enum AVPixelFormat m_hwPixFmt = AV_PIX_FMT_NONE; // 硬件像素格式（如AV_PIX_FMT_CUDA）
bool m_bHardWare = false;
// 滤镜
AVFilterGraph* m_filterGraph = nullptr;
AVFilterContext* m_videoSrcFilterCtx = nullptr;
AVFilterContext* m_videoSinkFilterCtx = nullptr;
AVFilterContext* m_audioSrcFilterCtx = nullptr;
AVFilterContext* m_audioSinkFilterCtx = nullptr;
};
#endif // VIDEODECODER_HVideoDecoder.cpp 代码如下
#include "VideoDecoder.h"
#include <iostream>
#include <thread>
VideoDecoder::VideoDecoder() {
m_timer = new Timer;
}
VideoDecoder::~VideoDecoder()
{
delete m_timer;
}
bool VideoDecoder::open(const string &filePath)
{
close();
int ret = avformat_open_input(&m_formatContext, filePath.c_str(), NULL, NULL);
if (ret < 0) {
showError(ret);
close();
return false;
}
// 查找流信息
ret = avformat_find_stream_info(m_formatContext, NULL);
if (ret < 0) {
close();
return false;
}
// 查找视频流
m_videoIndex = av_find_best_stream(m_formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, nullptr, 0);
if (m_videoIndex == -1) {
close();
return false;
}
// 查找视频解码器
AVStream *videoStream = m_formatContext->streams[m_videoIndex];
const AVCodec *videoCodec = avcodec_find_decoder(videoStream->codecpar->codec_id);
if (videoCodec == nullptr) {
close();
return false;
}
// 创建视频解码器
m_videoCodecContext = avcodec_alloc_context3(videoCodec);
if (m_videoCodecContext == nullptr) {
close();
return false;
}
// 把视频流中的编解码参数复制给解码器的实例
avcodec_parameters_to_context(m_videoCodecContext, videoStream->codecpar);
// 打开视频解码器实例
ret = avcodec_open2(m_videoCodecContext, NULL, NULL);
if (ret < 0) {
close();
return ret;
}
if (m_videoCodecContext->time_base.num <= 0 || m_videoCodecContext->time_base.den <= 0) {
// 强制同步为流的 time_base
m_videoCodecContext->time_base = videoStream->time_base;
}
m_videoParam.bitRate = m_videoCodecContext->bit_rate;
m_videoParam.width = m_videoCodecContext->width;
m_videoParam.height = m_videoCodecContext->height;
m_videoParam.fps = m_videoCodecContext->framerate.num / m_videoCodecContext->framerate.den ;
m_videoParam.gopSize = m_videoCodecContext->gop_size;
m_videoParam.maxBFrames = m_videoCodecContext->max_b_frames;
/*
m_hwDeviceType = findSupportedHWDeviceType(videoCodec); // 查找支持的硬件类型
if (m_hwDeviceType != AV_HWDEVICE_TYPE_NONE) {
if (!initHWDevice()) { // 初始化硬件设备
m_bHardWare = false;
m_hwDeviceType = AV_HWDEVICE_TYPE_NONE;
m_hwPixFmt = AV_PIX_FMT_NONE;
// 硬解码失败，仍尝试软解码（释放已绑定的硬件上下文）
if (m_videoCodecContext->hw_device_ctx) {
av_buffer_unref(&m_videoCodecContext->hw_device_ctx);
m_videoCodecContext->hw_device_ctx = nullptr;
}
}
}
*/
// 打开音频解码器实例
m_audioIndex = av_find_best_stream(m_formatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
if (m_audioIndex >= 0) {
AVStream *audioStream = m_formatContext->streams[m_audioIndex];
const AVCodec *audioCodec = avcodec_find_decoder(audioStream->codecpar->codec_id);
if (audioCodec == nullptr) {
close();
return false;
}
m_audioCodecContext = avcodec_alloc_context3(audioCodec);
if (m_audioCodecContext == nullptr) {
close();
return false;
}
avcodec_parameters_to_context(m_audioCodecContext, audioStream->codecpar);
ret = avcodec_open2(m_audioCodecContext, audioCodec, NULL);
if (ret < 0) {
close();
return false;
}
m_audioParam.sampleRate = m_audioCodecContext->sample_rate;
m_audioParam.sampleFmt = m_audioCodecContext->sample_fmt;
m_audioParam.bitRate = m_audioCodecContext->bit_rate;
m_audioParam.channels = m_audioCodecContext->ch_layout.nb_channels;
m_audioParam.samples = m_audioCodecContext->frame_size;
}
initFilter();
//initFilterByDescribe();
return true;
}
void VideoDecoder::play()
{
if (m_stop) {
m_stop = false;
m_play = true;
m_timer->start();
std::thread([this](){
readPacket();
m_threadCount--;
}).detach();
std::thread([this](){
decodeVideo();
m_threadCount--;
}).detach();
std::thread([this](){
decodeAudio();
m_threadCount--;
}).detach();
m_threadCount = 3;
} else {
m_timer->start();
m_play = true;
m_playCondition.notify_all();
}
}
void VideoDecoder::pause()
{
m_play = false;
m_timer->pause();
}
void VideoDecoder::stop()
{
m_stop = true;
m_timer->stop();
// 唤醒所有条件变量，避免线程无法退出
m_queueFull.notify_all();
m_videoAvailable.notify_all();
m_audioAvailable.notify_all();
while (m_threadCount > 0) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
}
void VideoDecoder::close() {
if (m_swsContext != nullptr) {
sws_free_context(&m_swsContext);
}
if (m_swrContext != nullptr) {
swr_free(&m_swrContext);
}
if (m_formatContext != nullptr) {
avformat_close_input(&m_formatContext);
}
if (m_videoCodecContext != nullptr) {
avcodec_free_context(&m_videoCodecContext);
}
if (m_audioCodecContext != nullptr) {
avcodec_free_context(&m_audioCodecContext);
}
}
void VideoDecoder::showError(int ret)
{
char errorBuf[1024];
av_strerror(ret, errorBuf, sizeof(errorBuf));
std::cerr << errorBuf << std::endl;
}
void VideoDecoder::readPacket()
{
AVPacket* packet = av_packet_alloc();
while (!m_stop && av_read_frame(m_formatContext, packet) >= 0) { // 轮询数据包
if (packet->stream_index == m_videoIndex) {
m_videoMutex.lock();
m_videoPackets.emplace_back(*packet);
m_videoMutex.unlock();
m_videoAvailable.notify_all();
} else if (packet->stream_index == m_audioIndex) {
m_audioMutex.lock();
m_audioPackets.emplace_back(*packet);
m_audioMutex.unlock();
m_audioAvailable.notify_all();
}
{
std::unique_lock<std::mutex> lock(m_fullMutex);
m_queueFull.wait(lock, [this]{
return m_videoPackets.size() <= 10 && m_audioPackets.size() <= 10 || m_stop;
});
}
}
av_packet_free(&packet);
stop();
}
void VideoDecoder::decodeVideo()
{
AVFrame* frame = av_frame_alloc();
AVFrame* filterFrame = av_frame_alloc();
AVFrame* convertFrame = av_frame_alloc();
AVFrame* swFrame = nullptr;
if (m_bHardWare) {
swFrame = av_frame_alloc();
}
while (!m_stop) {
{
std::unique_lock<std::mutex> lock(m_videoMutex);
m_videoAvailable.wait(lock, [this]{
return !m_videoPackets.empty() || m_stop;
});
}
if (m_stop) {
break;
}
while (!m_play) {
std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
m_videoMutex.lock();
AVPacket packet = m_videoPackets.front();
m_videoPackets.erase(m_videoPackets.begin());
m_videoMutex.unlock();
m_queueFull.notify_all();
int ret = avcodec_send_packet(m_videoCodecContext, &packet);
av_packet_unref(&packet);
while ( ret >= 0) {
ret = avcodec_receive_frame(m_videoCodecContext, frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
break;
} else if (ret < 0) {
return;
}
if (m_bHardWare) {
hwFrameToSwFrame(frame, swFrame);
int ret = av_buffersrc_add_frame(m_videoSrcFilterCtx, swFrame);
if (ret < 0) {
std::cerr << "推送视频帧到源滤镜失败" << std::endl;
showError(ret);
continue;
}
} else {
int ret = av_buffersrc_add_frame(m_videoSrcFilterCtx, frame);
if (ret < 0) {
std::cerr << "推送视频帧到源滤镜失败" << std::endl;
showError(ret);
continue;
}
}
while (av_buffersink_get_frame(m_videoSinkFilterCtx, filterFrame) >= 0) {
if (!m_swsContext) {
initSwsContext(filterFrame);
}
convertFrame->format = AV_PIX_FMT_RGB24;
convertFrame->width = m_videoCodecContext->width;
convertFrame->height = m_videoCodecContext->height;
convertFrame->pts = filterFrame->pts;
int ret = av_frame_get_buffer(convertFrame, 0);
if (ret < 0) {
showError(ret);
return;
}
ret = sws_scale(
m_swsContext,
filterFrame->data, filterFrame->linesize,
0, filterFrame->height,
convertFrame->data, convertFrame->linesize);
av_frame_copy_props(convertFrame, filterFrame);
double pts_in_seconds = av_q2d(m_videoCodecContext->time_base) * convertFrame->pts;
double diff = pts_in_seconds - m_timer->elapsedtime();
//while (diff > 0.001 && !m_stop) {
//std::this_thread::sleep_for(std::chrono::milliseconds(5));
//diff = pts_in_seconds - m_timer->elapsedtime();
//}
for (const auto& callback : m_videoCallbacks) {
callback(convertFrame);
}
av_frame_unref(filterFrame);
av_frame_unref(convertFrame);
}
av_frame_unref(frame);
}
}
av_frame_free(&frame);
av_frame_free(&convertFrame);
av_frame_free(&filterFrame);
if (m_bHardWare) {
av_frame_free(&swFrame);
}
}
void VideoDecoder::decodeAudio()
{
AVFrame *frame = av_frame_alloc();
AVFrame *convertFrame = av_frame_alloc();
AVFrame *filterFrame = av_frame_alloc();
while (!m_stop) {
{
std::unique_lock<std::mutex> lock(m_audioMutex);
m_audioAvailable.wait(lock, [this]{
return !m_audioPackets.empty() || m_stop;
});
}
if (m_stop) {
break;
}
while (!m_play) {
std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
m_audioMutex.lock();
AVPacket packet = m_audioPackets.front();
m_audioPackets.erase(m_audioPackets.begin());
m_audioMutex.unlock();
m_queueFull.notify_all();
int ret = avcodec_send_packet(m_audioCodecContext, &packet);
av_packet_unref(&packet);
while (ret >= 0) {
ret = avcodec_receive_frame(m_audioCodecContext, frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
break;
} else if (ret < 0) {
return;
}
ret = av_buffersrc_add_frame(m_audioSrcFilterCtx, frame);
if (ret < 0) {
std::cerr << "推送音频帧到源滤镜失败" << std::endl;
showError(ret);
continue;
}
while (av_buffersink_get_frame(m_audioSinkFilterCtx, filterFrame) >= 0) {
if (!m_swrContext) {
initSwrContext(filterFrame);
}
int samples = swr_get_out_samples(m_swrContext, filterFrame->nb_samples);
if (samples < 0) {
av_frame_free(&convertFrame);
return;
}
convertFrame->format = AV_SAMPLE_FMT_S16;
int ret = av_channel_layout_copy(&convertFrame->ch_layout, &filterFrame->ch_layout);
if (ret < 0) {
showError(ret);
return;
}
convertFrame->nb_samples = samples;
ret = av_frame_get_buffer(convertFrame, 0);
if (ret < 0) {
showError(ret);
av_frame_free(&convertFrame);
return;
}
convertFrame->pts = filterFrame->pts;
samples = swr_convert(m_swrContext,
convertFrame->data, samples,
(const uint8_t **)filterFrame->data, filterFrame->nb_samples);
if (samples < 0) {
av_frame_free(&convertFrame);
showError(samples);
return;
}
double pts_in_seconds = av_q2d(m_audioCodecContext->time_base) * convertFrame->pts;
double diff = pts_in_seconds - m_timer->elapsedtime();
while (diff > 0.001 && !m_stop) {
std::this_thread::sleep_for(std::chrono::milliseconds(5));
diff = pts_in_seconds - m_timer->elapsedtime();
}
convertFrame->nb_samples = samples;
//av_frame_copy_props(convertFrame, filterFrame);
//convertFrame->sample_rate = filterFrame->sample_rate;
for (const auto& callback : m_audioCallbacks) {
callback(convertFrame);
}
av_frame_unref(filterFrame);
av_frame_unref(convertFrame);
}
av_frame_unref(frame);
}
}
av_frame_free(&frame);
av_frame_free(&filterFrame);
av_frame_free(&convertFrame);
}
bool VideoDecoder::initSwrContext(const AVFrame* frame)
{
if (!m_audioCodecContext) {
return false;
}
int ret = swr_alloc_set_opts2(&m_swrContext,
&m_audioCodecContext->ch_layout,
AV_SAMPLE_FMT_S16,
m_audioCodecContext->sample_rate,
&frame->ch_layout,
(AVSampleFormat)frame->format,
frame->sample_rate,
0, NULL);
if (ret < 0) {
showError(ret);
return false;
}
ret = swr_init(m_swrContext);
if (ret < 0) {
showError(ret);
swr_free(&m_swrContext);
m_swrContext = nullptr;
return false;
}
return true;
}
bool VideoDecoder::initSwsContext(const AVFrame* frame)
{
// 创建转换上下文并指定颜色空间
m_swsContext = sws_getContext(
frame->width, frame->height, (AVPixelFormat)frame->format,
m_videoCodecContext->width, m_videoCodecContext->height, AV_PIX_FMT_RGB24,
SWS_BILINEAR, nullptr, nullptr, nullptr);
return m_swsContext != nullptr;
}
bool VideoDecoder::initFilter()
{
m_filterGraph = avfilter_graph_alloc();
if (!m_filterGraph) {
return false;
}
// 创建视频滤镜
int ret = 0;
const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
AVRational time_base = m_videoCodecContext->time_base;
char args[512];
snprintf(args, sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
time_base.num, time_base.den,
m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);
ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in", args, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out", NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffersink 滤镜" << std::endl;
return false;
}
AVFilterContext* videoSpeedCtx = nullptr;
const AVFilter* setpts = avfilter_get_by_name("setpts");
if ((ret = avfilter_graph_create_filter(&videoSpeedCtx,
setpts, "speed",
"0.5*PTS", nullptr, m_filterGraph)) < 0) {
std::cout << "无法创建视频滤镜 " << std::endl;
return -1;
}
if (avfilter_link(m_videoSrcFilterCtx, 0, videoSpeedCtx, 0) < 0 ||
avfilter_link(videoSpeedCtx, 0, m_videoSinkFilterCtx, 0) < 0) {
std::cerr << "连接视频滤镜失败" << std::endl;
return false;
}
const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");
char ch_layout_str[256] = {0};
size_t buf_size = sizeof(ch_layout_str);
av_channel_layout_describe(&m_audioCodecContext->ch_layout, ch_layout_str, buf_size);
char aargs[512];
ret = snprintf(aargs, sizeof(aargs),
"time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=%s",
m_audioCodecContext->time_base.num, m_audioCodecContext->time_base.den, m_audioCodecContext->sample_rate,
av_get_sample_fmt_name(m_audioCodecContext->sample_fmt), ch_layout_str);
ret = avfilter_graph_create_filter(&m_audioSrcFilterCtx, abuffersrc, "in", aargs, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
ret = avfilter_graph_create_filter(&m_audioSinkFilterCtx, abuffersink, "out", NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 abuffersink 滤镜" << std::endl;
showError(ret);
return false;
}
// atempo 无法处理超过两倍的加速，但是可以通过叠加滤镜进行处理
AVFilterContext* audioSpeedCtx = nullptr;
const AVFilter* atempo = avfilter_get_by_name("atempo");
if (avfilter_graph_create_filter(&audioSpeedCtx, atempo, "atempo", "tempo=2.0", nullptr, m_filterGraph) < 0) {
std::cerr << "创建音频倍速滤镜失败 " << std::endl;
return false;
}
if (avfilter_link(m_audioSrcFilterCtx, 0, audioSpeedCtx, 0) < 0 ||
avfilter_link(audioSpeedCtx, 0, m_audioSinkFilterCtx, 0) < 0) {
std::cerr << "连接音频滤镜失败" << std::endl;
return false;
}
if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
std::cerr << "滤镜图配置失败 " << std::endl;
return -1;
}
return true;
}
bool VideoDecoder::initFilterByDescribe()
{
m_filterGraph = avfilter_graph_alloc();
if (!m_filterGraph) {
return false;
}
// 创建视频滤镜
int ret = 0;
const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
AVRational time_base = m_videoCodecContext->time_base;
char args[512];
snprintf(args, sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
time_base.num, time_base.den,
m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);
ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in",
args, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
// 设置buffersink参数
ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out",
NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffersink 滤镜" << std::endl;
return false;
}
// 这里需要注意的是 outputs 对应的是输入滤镜，inputs 对应的是输出滤镜
// 可以理解为数据从输入滤镜输出到 AVFilterInOut ，所以是 output,
AVFilterInOut* outputs = avfilter_inout_alloc();
AVFilterInOut* inputs = avfilter_inout_alloc();
if (!outputs || !inputs) {
return false;
}
outputs->name = av_strdup("in");
outputs->filter_ctx = m_videoSrcFilterCtx;
outputs->pad_idx = 0;
outputs->next = NULL;
inputs->name = av_strdup("out");
inputs->filter_ctx = m_videoSinkFilterCtx;
inputs->pad_idx = 0;
inputs->next = NULL;
if ((ret = avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2",
&inputs, &outputs, NULL)) < 0) {
std::cout << "无法解析滤镜描述" << std::endl;
return false;
}
avfilter_inout_free(&inputs);
avfilter_inout_free(&outputs);
const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");
char ch_layout_str[256] = {0};
size_t buf_size = sizeof(ch_layout_str);
av_channel_layout_describe(&m_audioCodecContext->ch_layout, ch_layout_str, buf_size);
char aargs[512];
ret = snprintf(aargs, sizeof(aargs),
"time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=%s",
m_audioCodecContext->time_base.num, m_audioCodecContext->time_base.den, m_audioCodecContext->sample_rate,
av_get_sample_fmt_name(m_audioCodecContext->sample_fmt), ch_layout_str);
ret = avfilter_graph_create_filter(&m_audioSrcFilterCtx, abuffersrc, "in",
aargs, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 buffer 滤镜" << std::endl;
return false;
}
ret = avfilter_graph_create_filter(&m_audioSinkFilterCtx, abuffersink, "out",
NULL, NULL, m_filterGraph);
if (ret < 0) {
std::cout << "无法创建 abuffersink 滤镜" << std::endl;
showError(ret);
return false;
}
AVFilterInOut* ainputs = avfilter_inout_alloc();
AVFilterInOut* aoutputs = avfilter_inout_alloc();
if (!aoutputs || !ainputs) {
return false;
}
aoutputs->name = av_strdup("in");
aoutputs->filter_ctx = m_audioSrcFilterCtx;
aoutputs->pad_idx = 0;
aoutputs->next = NULL;
ainputs->name = av_strdup("out");
ainputs->filter_ctx = m_audioSinkFilterCtx;
ainputs->pad_idx = 0;
ainputs->next = NULL;
ret = avfilter_graph_parse_ptr(m_filterGraph, "atempo=2", &ainputs, &aoutputs, NULL);
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Cannot parse audio filter string\n");
showError(ret);
return false;
}
if ((ret = avfilter_graph_config(m_filterGraph, NULL)) < 0) {
std::cout << "滤镜图配置失败" << std::endl;
return false;
}
avfilter_inout_free(&ainputs);
avfilter_inout_free(&aoutputs);
return true;
}
AVHWDeviceType VideoDecoder::findSupportedHWDeviceType(const AVCodec* codec) {
for (int i = 0;; i++) {
const AVCodecHWConfig* hwConfig = avcodec_get_hw_config(codec, i);
if (hwConfig != nullptr) {
m_hwPixFmt = hwConfig->pix_fmt; // 记录硬件像素格式
return hwConfig->device_type;
} else {
break;
}
}
return AV_HWDEVICE_TYPE_NONE;
}
bool VideoDecoder::initHWDevice()
{
if (m_hwDeviceType == AV_HWDEVICE_TYPE_NONE || !m_videoCodecContext) {
return false;
}
// 创建硬件设备上下文（NULL表示自动选择设备，如默认GPU）
int ret = av_hwdevice_ctx_create(&m_hwDeviceCtx, m_hwDeviceType,
NULL, NULL, 0);
if (ret < 0) {
showError(ret);
std::cerr << "创建硬件设备上下文失败（如GPU未识别）" << std::endl;
return false;
}
// 将硬件设备上下文绑定到解码器
m_videoCodecContext->hw_device_ctx = av_buffer_ref(m_hwDeviceCtx);
if (!m_videoCodecContext->hw_device_ctx) {
showError(AVERROR(ENOMEM));
av_buffer_unref(&m_hwDeviceCtx);
m_hwDeviceCtx = nullptr;
return false;
}
// 初始化硬件帧上下文（管理硬件帧池）
if (!initHWFramesContext()) {
av_buffer_unref(&m_hwDeviceCtx);
m_hwDeviceCtx = nullptr;
return false;
}
m_bHardWare = true;
return true;
}
bool VideoDecoder::initHWFramesContext()
{
if (!m_hwDeviceCtx || !m_videoCodecContext) return false;
// 分配硬件帧上下文
m_hwFramesCtx = av_hwframe_ctx_alloc(m_hwDeviceCtx);
if (!m_hwFramesCtx) {
std::cerr << "分配硬件帧上下文失败" << std::endl;
return false;
}
// 设置硬件帧参数
AVHWFramesContext* hwFramesCtx = (AVHWFramesContext*)m_hwFramesCtx->data;
hwFramesCtx->format = m_hwPixFmt; // 硬件像素格式
if (m_hwPixFmt == AV_PIX_FMT_CUDA || m_hwPixFmt == AV_PIX_FMT_VULKAN) {
hwFramesCtx->sw_format = AV_PIX_FMT_NV12;
} else if (m_hwPixFmt == AV_PIX_FMT_VAAPI) {
hwFramesCtx->sw_format = AV_PIX_FMT_YUV420P;
}
hwFramesCtx->width = m_videoCodecContext->width; // 视频宽度
hwFramesCtx->height = m_videoCodecContext->height; // 视频高度
hwFramesCtx->initial_pool_size = 16; // 帧池大小（建议 8-32）
// 3. 初始化硬件帧上下文
int ret = av_hwframe_ctx_init(m_hwFramesCtx);
if (ret < 0) {
showError(ret);
av_buffer_unref(&m_hwFramesCtx);
std::cerr << "初始化硬件帧上下文失败" << std::endl;
return false;
}
// 4. 将硬件帧上下文绑定到解码器
m_videoCodecContext->hw_frames_ctx = av_buffer_ref(m_hwFramesCtx);
if (!m_videoCodecContext->hw_frames_ctx) {
std::cerr << "绑定硬件帧上下文到解码器失败" << std::endl;
av_buffer_unref(&m_hwFramesCtx);
return false;
}
return true;
}
bool VideoDecoder::hwFrameToSwFrame(AVFrame *hwFrame, AVFrame *swFrame)
{
if (!hwFrame || !swFrame) {
return false;
}
AVHWFramesContext* hwFramesCtx = (AVHWFramesContext*)m_hwFramesCtx->data;
av_frame_unref(swFrame);
swFrame->format = hwFramesCtx->sw_format;
swFrame->width = hwFrame->width;
swFrame->height = hwFrame->height;
int ret = av_frame_get_buffer(swFrame, 0);
if (ret < 0) {
showError(ret);
std::cerr << "分配软件帧内存失败" << std::endl;
return false;
}
// 硬件帧数据传输到软件帧（GPU -> CPU）
ret = av_hwframe_transfer_data(swFrame, hwFrame, 0);
if (ret < 0) {
showError(ret);
std::cerr << "硬件帧转软件帧失败（可能是GPU资源不足）" << std::endl;
return false;
}
// 将 时间戳（pts） 音频的采样率（sample rate） 视频的像素宽高比（sample aspect ratio）等字段从源帧（src）复制到目标帧（dst）
av_frame_copy_props(swFrame, hwFrame);
return true;
}AudioPlayer.h 代码如下
#ifndef AUDIOPLAYER_H
#define AUDIOPLAYER_H
#include <SDL.h>
#include <vector>
#include <mutex>
#include <condition_variable>
#include "VideoStructs.h"
struct AVFrame;
using namespace std;
class AudioPlayer
{
public:
AudioPlayer();
~AudioPlayer();
bool initSdl(const AudioParams& param);
void audioCallbackImpl(Uint8* stream, int len);
static void audioCallback(void* userdata, Uint8* stream, int len);
bool displayAudio(AVFrame* frame);
private:
void closeSdl();
private:
vector<AVFrame*> m_frames;
std::mutex m_mutex;
std::condition_variable m_available;
std::condition_variable m_full;
SDL_AudioDeviceID m_deviceId;
bool m_stop = false;
AVFrame* m_currentFrame = nullptr;
int m_offset;
};
#endif // AUDIOPLAYER_HAudioPlayer.cpp 代码如下
#include "AudioPlayer.h"
#include <iostream>
#include <thread>
extern "C" {
#include "libavcodec/avcodec.h"
}
AudioPlayer::AudioPlayer() {
}
AudioPlayer::~AudioPlayer()
{
m_stop = true;
m_available.notify_all();
closeSdl();
}
bool AudioPlayer::initSdl(const AudioParams ¶m)
{
// 初始化SDL音频
if (SDL_Init(SDL_INIT_AUDIO) < 0) {
std::cerr << "SDL初始化失败: " << SDL_GetError() << std::endl;
return false;
}
SDL_AudioSpec desired_spec, obtained_spec;
desired_spec.freq = param.sampleRate; // 采样率
desired_spec.format = AUDIO_S16SYS; // 16位有符号整数
desired_spec.channels = 2;
desired_spec.silence = 0; // 静音值
desired_spec.samples = param.samples; // 缓冲区大小
if (desired_spec.samples <= 0) {
desired_spec.samples = 1024;
}
desired_spec.callback = audioCallback; // 回调函数
desired_spec.userdata = this; // 传递this指针
m_deviceId = SDL_OpenAudioDevice(nullptr, 0, &desired_spec, &obtained_spec, 0);
if (m_deviceId == 0) {
std::cerr << "无法打开音频设备: " << SDL_GetError() << std::endl;
return false;
}
SDL_PauseAudioDevice(m_deviceId, 0);
return true;
}
void AudioPlayer::audioCallbackImpl(Uint8 *stream, int len)
{
SDL_memset(stream, 0, len);
{
std::unique_lock<std::mutex> lock(m_mutex);
m_available.wait(lock, [this] {
return m_frames.size() > 4 || m_deviceId == 0 || m_stop;
});
}
if (m_currentFrame == nullptr) {
m_currentFrame= m_frames.front();
m_frames.erase(m_frames.begin());
m_offset = 0;
}
int offset = 0;
while (len > 0 && !m_stop) {
if (m_deviceId == 0 || m_stop) {
return;
}
// 计算本次回调需要复制的数据量
int bytes_to_copy = std::min(len, m_currentFrame->linesize[0] - m_offset);
// 复制数据到音频缓冲区
memcpy(stream + offset, m_currentFrame->data[0] + m_offset, bytes_to_copy);
// 更新位置
offset += bytes_to_copy;
m_offset += bytes_to_copy;
len -= bytes_to_copy;
if (m_currentFrame->linesize[0] == m_offset) {
std::unique_lock<std::mutex> lock(m_mutex);
av_frame_free(&m_currentFrame);
m_currentFrame = m_frames.front();
m_frames.erase(m_frames.begin());
m_offset = 0;
}
}
}
void AudioPlayer::audioCallback(void *userdata, Uint8 *stream, int len)
{
AudioPlayer* player = static_cast<AudioPlayer*>(userdata);
player->audioCallbackImpl(stream, len);
}
bool AudioPlayer::displayAudio(AVFrame *frame)
{
{
std::lock_guard<std::mutex> lock(m_mutex);
AVFrame* cloneFrame = av_frame_alloc();
if (av_frame_ref(cloneFrame, frame) != 0) {
av_frame_free(&cloneFrame);
return false;
}
m_frames.emplace_back(cloneFrame);
}
m_available.notify_one();
return true;
}
void AudioPlayer::closeSdl()
{
SDL_AudioStatus status = SDL_GetAudioDeviceStatus(m_deviceId);
if (status == SDL_AUDIO_PLAYING) {
SDL_PauseAudioDevice(m_deviceId, 1);
}
if (m_deviceId != 0) {
SDL_CloseAudioDevice(m_deviceId);
m_deviceId = 0;
}
SDL_QuitSubSystem(SDL_INIT_AUDIO);
}VideoWidget.h 代码如下
#include <QOpenGLWidget>
#include <QOpenGLFunctions>
#include <QOpenGLShaderProgram>
#include <QOpenGLTexture>
#include <QMutex>
#include <SDL.h>
struct AVFrame;
class VideoWidget : public QOpenGLWidget, protected QOpenGLFunctions {
Q_OBJECT
public:
explicit VideoWidget(QWidget *parent = nullptr);
~VideoWidget() override;
// 接收解码后的帧并更新显示
bool displayVideo(AVFrame* frame);
// 设置是否保持宽高比
void setKeepAspectRatio(bool keep);
// 清除显示内容
void clear();
protected:
void initializeGL() override;
void resizeGL(int w, int h) override;
void paintGL() override;
private:
bool initShader();
void createTextures(int width, int height);
void updateTextures(const AVFrame* frame);
// 重置顶点坐标（处理宽高比）
void resetVertices(int windowWidth, int windowHeight);
private:
QOpenGLShaderProgram *m_program = nullptr; // 着色器程序
QOpenGLTexture* m_texture = nullptr;
QMutex m_mutex; // 多线程同步锁
AVFrame* m_currentFrame = nullptr; // 当前待渲染的帧
int m_frameWidth = 0; // 帧宽度
int m_frameHeight = 0; // 帧高度
bool m_hasNewFrame = false; // 是否有新帧待渲染
bool m_keepAspectRatio = true; // 是否保持宽高比
GLfloat m_vertices[8] = {0}; // 顶点坐标数组
bool m_isPlaying = false; // 播放状态标记
QString m_errorMsg; // 错误信息
};VideoWidget.cpp 代码如下
#include "VideoWidget.h"
#include <QDebug>
#include <QPainter>
#include <QEvent>
extern "C" {
#include "libavcodec/avcodec.h"
}
VideoWidget::VideoWidget(QWidget *parent)
: QOpenGLWidget(parent) {
setAutoFillBackground(false);
setAttribute(Qt::WA_OpaquePaintEvent);
setAcceptDrops(true);
}
VideoWidget::~VideoWidget() {
makeCurrent();
// 释放纹理资源
if (m_texture) {
delete m_texture;
m_texture = nullptr;
}
delete m_program;
// 释放缓存的帧
if (m_currentFrame) {
av_frame_free(&m_currentFrame);
m_currentFrame = nullptr;
}
doneCurrent();
}
void VideoWidget::setKeepAspectRatio(bool keep) {
if (m_keepAspectRatio != keep) {
m_keepAspectRatio = keep;
resetVertices(width(), height());
update();
}
}
void VideoWidget::clear() {
QMutexLocker locker(&m_mutex);
if (m_currentFrame) {
av_frame_free(&m_currentFrame);
m_currentFrame = nullptr;
}
m_hasNewFrame = false;
m_isPlaying = false;
m_errorMsg.clear();
update();
}
bool VideoWidget::displayVideo(AVFrame *frame) {
QMutexLocker locker(&m_mutex); // 加锁保护共享资源
// 释放旧帧
if (m_currentFrame) {
av_frame_free(&m_currentFrame);
}
// 复制新帧（避免原帧被解码器释放）
m_currentFrame = av_frame_alloc();
if (av_frame_ref(m_currentFrame, frame) != 0) {
qWarning() << "无法复制视频帧";
av_frame_free(&m_currentFrame);
return false;
}
// 更新帧尺寸（如果有变化）
m_frameWidth = m_currentFrame->width;
m_frameHeight = m_currentFrame->height;
resetVertices(m_frameWidth, m_frameHeight);
m_hasNewFrame = true;
m_isPlaying = true;
m_errorMsg.clear();
// 触发重绘
update();
return true;
}
void VideoWidget::initializeGL() {
initializeOpenGLFunctions();
glClearColor(0.0f, 0.0f, 0.0f, 1.0f); // 黑色背景
glEnable(GL_TEXTURE_2D);
// 初始化着色器（核心修改4：RGB着色器替换YUV着色器）
if (!initShader()) {
m_errorMsg = "着色器初始化失败";
qFatal("%s", m_errorMsg.toUtf8().constData());
}
// 初始化顶点坐标
resetVertices(width(), height());
// 启用顶点属性（0：顶点坐标，1：纹理坐标）
glEnableVertexAttribArray(0);
glEnableVertexAttribArray(1);
}
void VideoWidget::resizeGL(int w, int h) {
glViewport(0, 0, w, h); // 设置视口为窗口大小
resetVertices(w, h); // 重新计算顶点坐标
}
void VideoWidget::paintGL() {
glClear(GL_COLOR_BUFFER_BIT); // 清除颜色缓冲区
QMutexLocker locker(&m_mutex);
if (!m_isPlaying && !m_hasNewFrame) {
return;
}
if (m_hasNewFrame && m_currentFrame) {
// 首次渲染或帧尺寸变化时，创建RGB纹理
if (!m_texture || m_frameWidth != m_currentFrame->width || m_frameHeight != m_currentFrame->height) {
createTextures(m_frameWidth, m_frameHeight);
}
// 更新RGB纹理数据
updateTextures(m_currentFrame);
m_hasNewFrame = false;
}
// 绑定着色器并绘制（仅绑定单个RGB纹理）
if (m_program && m_texture) {
m_program->bind();
m_texture->bind(0);
m_program->setUniformValue("u_texture", 0);
static const GLfloat texCoords[] = {
0.0f, 1.0f, // 左下
1.0f, 1.0f, // 右下
0.0f, 0.0f, // 左上
1.0f, 0.0f // 右上
};
// 传递顶点坐标和纹理坐标
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, m_vertices);
glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 0, texCoords);
// 绘制四边形（GL_TRIANGLE_STRIP 高效绘制4个顶点）
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
m_program->release();
m_texture->release();
}
}
bool VideoWidget::initShader() {
m_program = new QOpenGLShaderProgram;
// 顶点着色器：传递顶点坐标和纹理坐标
const char* vertexShader = R"(
#version 120
attribute vec2 vertexIn; // 顶点坐标（location 0）
attribute vec2 textureIn; // 纹理坐标（location 1）
varying vec2 textureOut; // 传递给片段着色器的纹理坐标
void main() {
gl_Position = vec4(vertexIn, 0.0, 1.0);
textureOut = textureIn;
}
)";
// 片段着色器：实现YUV420P转RGB（BT.601标准）
const char* fragmentShader = R"(
#version 120
#ifdef GL_ES
precision mediump float; // 移动设备精度兼容
#endif
varying vec2 textureOut; // 从顶点着色器接收的纹理坐标
uniform sampler2D u_texture; // RGB纹理（仅1个）
void main() {
// 直接采样RGB纹理（无需YUV转RGB计算）
vec3 rgbColor = texture2D(u_texture, textureOut).rgb;
// 输出RGB颜色（Alpha=1.0，不透明）
gl_FragColor = vec4(rgbColor, 1.0);
}
)";
// 编译并链接着色器
if (!m_program->addShaderFromSourceCode(QOpenGLShader::Vertex, vertexShader)) {
qWarning() << "顶点着色器错误：" << m_program->log();
return false;
}
if (!m_program->addShaderFromSourceCode(QOpenGLShader::Fragment, fragmentShader)) {
qWarning() << "片段着色器错误：" << m_program->log();
return false;
}
if (!m_program->link()) {
qWarning() << "着色器链接错误：" << m_program->log();
return false;
}
return m_program->bind();
}
void VideoWidget::createTextures(int width, int height) {
// 释放旧纹理（避免内存泄漏）
if (m_texture) {
delete m_texture;
m_texture = nullptr;
}
// 创建2D纹理（格式为 RGB888，对应 RGB24 每通道8位）
m_texture = new QOpenGLTexture(QOpenGLTexture::Target2D);
m_texture->create();
m_texture->setSize(width, height); // 纹理尺寸 = 帧尺寸
m_texture->setFormat(QOpenGLTexture::RGBAFormat); // RGB24 对应格式
// 过滤模式：线性插值（缩放时平滑）
m_texture->setMinMagFilters(QOpenGLTexture::Linear, QOpenGLTexture::Linear);
// 环绕模式：边缘像素延伸（避免纹理边缘失真）
m_texture->setWrapMode(QOpenGLTexture::ClampToEdge);
m_texture->allocateStorage(); // 分配纹理内存
}
void VideoWidget::updateTextures(const AVFrame* frame) {
if (!frame || frame->format != AV_PIX_FMT_RGB24) {
m_errorMsg = "不支持的帧格式（仅支持 RGB24）";
qWarning() << m_errorMsg;
return;
}
GLint prevRowLength;
glGetIntegerv(GL_UNPACK_ROW_LENGTH, &prevRowLength);
glBindTexture(GL_TEXTURE_2D, m_texture->textureId());
int pixelPerRow = frame->linesize[0] / 3;
glPixelStorei(GL_UNPACK_ROW_LENGTH, pixelPerRow); // 设置每行像素数
glTexSubImage2D(
GL_TEXTURE_2D, 0, // 纹理目标、多级纹理层级
0, 0, // 纹理偏移（x,y）
frame->width, frame->height,// 纹理更新区域尺寸
GL_RGB, // 像素数据格式（RGB顺序）
GL_UNSIGNED_BYTE, // 像素数据类型（无符号字节）
frame->data[0] // RGB24 数据指针
);
// 恢复原像素存储模式
glPixelStorei(GL_UNPACK_ROW_LENGTH, prevRowLength);
glBindTexture(GL_TEXTURE_2D, 0); // 解绑纹理
}
void VideoWidget::resetVertices(int windowWidth, int windowHeight) {
if (windowWidth <= 0 || windowHeight <= 0 || !m_keepAspectRatio ||
m_frameWidth <= 0 || m_frameHeight <= 0) {
// 铺满模式
const GLfloat fullScreenVertices[] = {
-1.0f, -1.0f,
1.0f, -1.0f,
-1.0f, 1.0f,
1.0f, 1.0f
};
memcpy(m_vertices, fullScreenVertices, sizeof(fullScreenVertices));
return;
}
// 保持宽高比计算
float videoAspect = static_cast<float>(m_frameWidth) / m_frameHeight;
float windowAspect = static_cast<float>(windowWidth) / windowHeight;
float scaleX = 1.0f;
float scaleY = 1.0f;
if (videoAspect > windowAspect) {
// 视频更宽，按宽度缩放
scaleX = windowAspect / videoAspect;
} else {
// 视频更高，按高度缩放
scaleY = videoAspect / windowAspect;
}
// 计算顶点坐标
const GLfloat vertices[] = {
-scaleX, -scaleY,
scaleX, -scaleY,
-scaleX, scaleY,
scaleX, scaleY
};
memcpy(m_vertices, vertices, sizeof(vertices));
}Timer.h 代码如下
#ifndef TIMER_H
#define TIMER_H
#include <mutex>
class Timer
{
public:
Timer();
void start();
void pause();
void stop();
double elapsedtime();
private:
double currentTimestamp();
private:
std::mutex m_timeMutex;
double m_lastclock; // 记录上次更新的时间戳
double m_elapsedtime;
};
#endif // TIMER_HTimer.cpp 代码如下
#include "Timer.h"
#include <chrono>
Timer::Timer() {
m_elapsedtime = 0;
}
void Timer::start()
{
m_lastclock = currentTimestamp();
}
void Timer::pause()
{
m_timeMutex.lock();
m_elapsedtime += currentTimestamp() - m_lastclock;
m_lastclock = currentTimestamp();
m_timeMutex.unlock();
}
void Timer::stop()
{
m_timeMutex.lock();
m_elapsedtime = 0;
m_timeMutex.unlock();
}
double Timer::elapsedtime()
{
m_timeMutex.lock();
m_elapsedtime += currentTimestamp() - m_lastclock;
m_lastclock = currentTimestamp();
m_timeMutex.unlock();
return m_elapsedtime;
}
double Timer::currentTimestamp()
{
auto now = std::chrono::system_clock::now();
auto duration = now.time_since_epoch();
return std::chrono::duration<double>(duration).count();
}
浙公网安备 33010602011771号