Android Camera 流程学习记录（四）—— Camera.startPreview() flow

简介
通过整理前几篇笔记，我已经对整个 Camera 的框架有比较清晰的认识，并且也已经渐渐摸清楚了整个控制流的逻辑。
这次，我打算通过追踪 Camera.startPreview() 方法，以加深对控制流的理解。同时尝试结合上一篇关于 module 的认识，将这个流程走到尽可能底部的位置。
Camera.startPreview() flow
1. Frameworks
1.1 Camera.java
位置：frameworks/base/core/java/android/hardware/Camera.java
startPreview()：
给上层 application 提供一个接口。
进入 Runtime 层。
/**
* Starts capturing and drawing preview frames to the screen.
* Preview will not actually start until a surface is supplied
* with {@link #setPreviewDisplay(SurfaceHolder)} or
* {@link #setPreviewTexture(SurfaceTexture)}.
*
* <p>If {@link #setPreviewCallback(Camera.PreviewCallback)},
* {@link #setOneShotPreviewCallback(Camera.PreviewCallback)}, or
* {@link #setPreviewCallbackWithBuffer(Camera.PreviewCallback)} were
* called, {@link Camera.PreviewCallback#onPreviewFrame(byte[], Camera)}
* will be called when preview data becomes available.
*/
public native final void startPreview();
1
2
3
4
5
6
7
8
9
10
11
12
13
2. Android Runtime
2.1 android_hardware_Camera.cpp
位置：frameworks/base/core/jni/android_hardware_Camera.cpp
android_hardware_Camera_startPreview()：
调用 get_native_camera() 函数获取一个 Camera 实例。
调用 Camera::startPreview()。
static void android_hardware_Camera_startPreview(JNIEnv *env, jobject thiz)
{
ALOGV("startPreview");
sp<Camera> camera = get_native_camera(env, thiz, NULL);
if (camera == 0) return;

if (camera->startPreview() != NO_ERROR) {
jniThrowRuntimeException(env, "startPreview failed");
return;
}
}
1
2
3
4
5
6
7
8
9
10
11
get_native_camera()：
从 DVM 中获取关于 Camera 的上下文。
从上下文信息中获取 Camera 实例。
sp<Camera> get_native_camera(JNIEnv *env, jobject thiz, JNICameraContext** pContext)
{
sp<Camera> camera;
Mutex::Autolock _l(sLock);
JNICameraContext* context = reinterpret_cast<JNICameraContext*>(env->GetLongField(thiz, fields.context));
if (context != NULL) {
camera = context->getCamera();
}
ALOGV("get_native_camera: context=%p, camera=%p", context, camera.get());
if (camera == 0) {
jniThrowRuntimeException(env,
"Camera is being used after Camera.release() was called");
}

if (pContext != NULL) *pContext = context;
return camera;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
3. Libraries
3.1 Camera.cpp
位置：frameworks/av/camera/Camera.cpp
startPreview()：
mCamera 即是在 connect 过程中返回的 CameraClient，它具体实现了 startPreview() 接口。
调用 CameraClient::startPreview()。
// start preview mode
status_t Camera::startPreview()
{
ALOGV("startPreview");
sp <::android::hardware::ICamera> c = mCamera;
if (c == 0) return NO_INIT;
return c->startPreview();
}
1
2
3
4
5
6
7
8
3.2 CameraClient.cpp
位置：frameworks/av/services/camera/libcameraservice/api1/CameraClient.cpp
startPreview()：
通过 startCameraMode 函数进入具体的实现逻辑。
// start preview mode
status_t CameraClient::startPreview() {
LOG1("startPreview (pid %d)", getCallingPid());
return startCameraMode(CAMERA_PREVIEW_MODE);
}
1
2
3
4
5
startCameraMode()：
根据传入的参数 CAMERA_PREVIEW_MODE 确定进入的分支。
调用 startPreviewMode() 。
// start preview or recording
status_t CameraClient::startCameraMode(camera_mode mode) {
LOG1("startCameraMode(%d)", mode);
Mutex::Autolock lock(mLock);
status_t result = checkPidAndHardware();
if (result != NO_ERROR) return result;

switch(mode) {
case CAMERA_PREVIEW_MODE:
if (mSurface == 0 && mPreviewWindow == 0) {
LOG1("mSurface is not set yet.");
// still able to start preview in this case.
}
return startPreviewMode();
case CAMERA_RECORDING_MODE:
if (mSurface == 0 && mPreviewWindow == 0) {
ALOGE("mSurface or mPreviewWindow must be set before startRecordingMode.");
return INVALID_OPERATION;
}
return startRecordingMode();
default:
return UNKNOWN_ERROR;
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
startPreviewMode()：
如果预览已经存在，则直接返回成功信息。
如果未存在，则继续往下走。
mHardware 是 CameraHardwareInterface 的实例，在 connect 过程的最后被初始化。
通过 mHardware 调用 setPreviewWindow() 和 startPreview() 接口。
进入 HAL 层。
status_t CameraClient::startPreviewMode() {
LOG1("startPreviewMode");
status_t result = NO_ERROR;

// if preview has been enabled, nothing needs to be done
if (mHardware->previewEnabled()) {
return NO_ERROR;
}

if (mPreviewWindow != 0) {
mHardware->setPreviewScalingMode(
NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
mHardware->setPreviewTransform(mOrientation);
}
mHardware->setPreviewWindow(mPreviewWindow);
result = mHardware->startPreview();
if (result == NO_ERROR) {
mCameraService->updateProxyDeviceState(
ICameraServiceProxy::CAMERA_STATE_ACTIVE,
String8::format("%d", mCameraId));
}
return result;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
4. HAL
4.1 CameraHardwareInterface.h
位置：frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h
previewEnable()：
通过 mDevice->ops 继续向下调用（不是我们主要追踪的）。
mDevice 即是通过 hw_get_module() 相关流程进行初始化的设备实例，它的类型是 camera_device_t 。
如果 preview 存在，则返回 true 。
/**
* Returns true if preview is enabled.
*/
int previewEnabled()
{
ALOGV("%s(%s)", __FUNCTION__, mName.string());
if (mDevice->ops->preview_enabled)
return mDevice->ops->preview_enabled(mDevice);
return false;
}
1
2
3
4
5
6
7
8
9
10
setPreviewWindow()：
如果 set_preview_window 函数指针为空，则返回失败信息。
若否，通过 mDevice->ops 继续向下调用（不是我们主要追踪的）。
/** Set the ANativeWindow to which preview frames are sent */
status_t setPreviewWindow(const sp<ANativeWindow>& buf)
{
ALOGV("%s(%s) buf %p", __FUNCTION__, mName.string(), buf.get());
if (mDevice->ops->set_preview_window) {
mPreviewWindow = buf;
if (buf != nullptr) {
if (mPreviewScalingMode != NOT_SET) {
setPreviewScalingMode(mPreviewScalingMode);
}
if (mPreviewTransform != NOT_SET) {
setPreviewTransform(mPreviewTransform);
}
}
mHalPreviewWindow.user = this;
ALOGV("%s &mHalPreviewWindow %p mHalPreviewWindow.user %p", __FUNCTION__,
&mHalPreviewWindow, mHalPreviewWindow.user);
return mDevice->ops->set_preview_window(mDevice,
buf.get() ? &mHalPreviewWindow.nw : 0);
}
return INVALID_OPERATION;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
startPreview()：
若 start_preview() 函数指针为空，则返回失败信息。
若否，则通过 mDevice 进行下一步操作。
关于 mDevice，我们结合 Camera.open() 流程与 hw_get_module() 相关逻辑，可以知道它的逻辑是这样的：
在 CameraService 启动时，会调用 onFirstRef() 对 module 进行初始化，获取 module 实例。
在 open 过程中，CameraClient 连接 CameraServer 成功时，会实例化 CameraHardwareInterface，并传入 module 实例对其初始化。
在初始化过程中，通过 module 实例对应的 open 方法，我们获得一个 device 实例，即 mDevice，这对应了具体的摄像头设备。
通过 mDevice，我们就可以将对应的指令传达到硬件设备。
通过对 camera_device_t 类型进行追踪，可以找到函数指针的一个具体指向。
/**
* Start preview mode.
*/
status_t startPreview()
{
ALOGV("%s(%s)", __FUNCTION__, mName.string());
if (mDevice->ops->start_preview)
return mDevice->ops->start_preview(mDevice);
return INVALID_OPERATION;
}
1
2
3
4
5
6
7
8
9
10
4.2 camera.h
位置：hardware/libhardware/include/hardware/camera.h
struct camera_device：
这里就声明了我们想要追踪的 camera_device_t 。
ops 对应的类型是 camera_device_ops_t ，这个结构中声明了函数指针。
typedef struct camera_device {
/**
* camera_device.common.version must be in the range
* HARDWARE_DEVICE_API_VERSION(0,0)-(1,FF). CAMERA_DEVICE_API_VERSION_1_0 is
* recommended.
*/
hw_device_t common;
camera_device_ops_t *ops;
void *priv;
} camera_device_t;
1
2
3
4
5
6
7
8
9
10
struct camera_device_ops：
由于注释过长，我把除了 start_preview 以外的注释都去掉了。
可以看到，所有关于 Camera 设备的操作，对应的函数指针都在这里声明了。
但是这里没法看出，函数指针具体指向哪里。
在 Linux 下用 find . -name "*.cpp" | xargs grep "start_preview =" 可以找到一些对应的文件，这些文件所处的位置与具体的设备商有关。
在这些文件中，就确定了函数指针的指向。
typedef struct camera_device_ops {
int (*set_preview_window)(struct camera_device *,
struct preview_stream_ops *window);

void (*set_callbacks)(struct camera_device *,
camera_notify_callback notify_cb,
camera_data_callback data_cb,
camera_data_timestamp_callback data_cb_timestamp,
camera_request_memory get_memory,
void *user);

void (*enable_msg_type)(struct camera_device *, int32_t msg_type);

void (*disable_msg_type)(struct camera_device *, int32_t msg_type);

int (*msg_type_enabled)(struct camera_device *, int32_t msg_type);

/**
* Start preview mode.
*/
int (*start_preview)(struct camera_device *);

void (*stop_preview)(struct camera_device *);

int (*preview_enabled)(struct camera_device *);

int (*store_meta_data_in_buffers)(struct camera_device *, int enable);

int (*start_recording)(struct camera_device *);

void (*stop_recording)(struct camera_device *);

int (*recording_enabled)(struct camera_device *);

void (*release_recording_frame)(struct camera_device *,
const void *opaque);

int (*auto_focus)(struct camera_device *);

int (*cancel_auto_focus)(struct camera_device *);

int (*take_picture)(struct camera_device *);

int (*cancel_picture)(struct camera_device *);

int (*set_parameters)(struct camera_device *, const char *parms);

char *(*get_parameters)(struct camera_device *);

void (*put_parameters)(struct camera_device *, char *);

int (*send_command)(struct camera_device *,
int32_t cmd, int32_t arg1, int32_t arg2);

void (*release)(struct camera_device *);

int (*dump)(struct camera_device *, int fd);
} camera_device_ops_t;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
4.3 hardware/ti/omap4-aah/camera
通过 find 指令，我找到了一些与函数指针指向有关的文件。
从这些文件的路径上看，它们与不同的设备供应商有关。
我决定往 ti/omap4-aah 子文件夹去一探究竟。
4.3.1 CameraHal_Module.cpp
位置：hardware/ti/omap4-aah/camera/CameraHal_Module.cpp
camera_device_open()：
在 open 流程中，就指定了 ops 中指针的对应关系。
memset(camera_device, 0, sizeof(*camera_device));
memset(camera_ops, 0, sizeof(*camera_ops));

camera_device->base.common.tag = HARDWARE_DEVICE_TAG;
camera_device->base.common.version = 0;
camera_device->base.common.module = (hw_module_t *)(module);
camera_device->base.common.close = camera_device_close;
camera_device->base.ops = camera_ops;

camera_ops->set_preview_window = camera_set_preview_window;
camera_ops->set_callbacks = camera_set_callbacks;
camera_ops->enable_msg_type = camera_enable_msg_type;
camera_ops->disable_msg_type = camera_disable_msg_type;
camera_ops->msg_type_enabled = camera_msg_type_enabled;
camera_ops->start_preview = camera_start_preview;
camera_ops->stop_preview = camera_stop_preview;
camera_ops->preview_enabled = camera_preview_enabled;
camera_ops->store_meta_data_in_buffers = camera_store_meta_data_in_buffers;
camera_ops->start_recording = camera_start_recording;
camera_ops->stop_recording = camera_stop_recording;
camera_ops->recording_enabled = camera_recording_enabled;
camera_ops->release_recording_frame = camera_release_recording_frame;
camera_ops->auto_focus = camera_auto_focus;
camera_ops->cancel_auto_focus = camera_cancel_auto_focus;
camera_ops->take_picture = camera_take_picture;
camera_ops->cancel_picture = camera_cancel_picture;
camera_ops->set_parameters = camera_set_parameters;
camera_ops->get_parameters = camera_get_parameters;
camera_ops->put_parameters = camera_put_parameters;
camera_ops->send_command = camera_send_command;
camera_ops->release = camera_release;
camera_ops->dump = camera_dump;

*device = &camera_device->base.common;

// -------- TI specific stuff --------

camera_device->cameraid = cameraid;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
camera_start_preview()：
注意 gCameraHals 是 CameraHal * 。
通过调用 CameraHal::startPreview() 完成业务逻辑。
int camera_start_preview(struct camera_device * device)
{
CAMHAL_LOG_MODULE_FUNCTION_NAME;

int rv = -EINVAL;
ti_camera_device_t* ti_dev = NULL;

if(!device)
return rv;

ti_dev = (ti_camera_device_t*) device;

rv = gCameraHals[ti_dev->cameraid]->startPreview();

return rv;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
4.3.2 CameraHal.cpp
位置：hardware/ti/omap4-aah/camera/CameraHal.cpp
这个文件对应的功能是，将 Camera Hardware Interface 映射到 V4L2。
注意两个声明：
extern "C" CameraAdapter* OMXCameraAdapter_Factory(size_t);
extern "C" CameraAdapter* V4LCameraAdapter_Factory(size_t);
分别对应 OMX 与 V4L 的适配器工厂，这里可能是将 Adapter 模式与 Factory 模式结合使用。
startPreview()：
源代码中带有大量注释，在这里我将其去掉，只关注调用逻辑。
首先调用了 cameraPreviewInitialization() 函数进行初始化。
通过 CameraAdapter 发送 CAMERA_START_PREVIEW 指令，若成功执行，则完成流程。
status_t CameraHal::startPreview() {
LOG_FUNCTION_NAME;

status_t ret = cameraPreviewInitialization();

if (!mPreviewInitializationDone) return ret;

mPreviewInitializationDone = false;

if(mDisplayAdapter.get() != NULL) {
CAMHAL_LOGDA("Enabling display");
int width, height;
mParameters.getPreviewSize(&width, &height);

#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
ret = mDisplayAdapter->enableDisplay(width, height, &mStartPreview);
#else
ret = mDisplayAdapter->enableDisplay(width, height, NULL);
#endif

if ( ret != NO_ERROR ) {
CAMHAL_LOGEA("Couldn't enable display");
CAMHAL_ASSERT_X(false,
"At this stage mCameraAdapter->mStateSwitchLock is still locked, "
"deadlock is guaranteed");

goto error;
}
}

CAMHAL_LOGDA("Starting CameraAdapter preview mode");

ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_START_PREVIEW);

if(ret!=NO_ERROR) {
CAMHAL_LOGEA("Couldn't start preview w/ CameraAdapter");
goto error;
}
CAMHAL_LOGDA("Started preview");

mPreviewEnabled = true;
mPreviewStartInProgress = false;
return ret;

error:

CAMHAL_LOGEA("Performing cleanup after error");

//Do all the cleanup
freePreviewBufs();
mCameraAdapter->sendCommand(CameraAdapter::CAMERA_STOP_PREVIEW);
if(mDisplayAdapter.get() != NULL) {
mDisplayAdapter->disableDisplay(false);
}
mAppCallbackNotifier->stop();
mPreviewStartInProgress = false;
mPreviewEnabled = false;
LOG_FUNCTION_NAME_EXIT;

return ret;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
cameraPreviewInitialization()：
这段代码比较长，但从注释看，它主要做三件事：
通过 Adapter 设置相关参数；
申请 Buffers 空间；
对 Buffers 进行相应设置以进行预览。
了解了大体思路，再看看具体代码：
mCameraAdapter->setParameters()：设置参数。
allocPreviewBufs() ：申请 Buffers。
desc：注意到这个变量对应着 CameraAdapter::BuffersDescriptor，在申请 Buffers 空间成功后，便对其进行相应的成员设置。
mAppCallbackNotifier->start()：开启回调通知。
mAppCallbackNotifier->startPreviewCallbacks()：将 Buffers 对应到相应的回调函数中，以供上层 APP 获取预览所需的数据。
NOTE：
代码中不断使用 mCameraAdapter->sendCommand() 来发送指令，并获取一些数据。
指令发送到对应的 Adapter （如 V4L Adapter），就会调用相应的函数进行处理。
////////////
/**
@brief Set preview mode related initialization
-> Camera Adapter set params
-> Allocate buffers
-> Set use buffers for preview
@param none
@return NO_ERROR
@todo Update function header with the different errors that are possible
*/
status_t CameraHal::cameraPreviewInitialization()
{

status_t ret = NO_ERROR;
CameraAdapter::BuffersDescriptor desc;
CameraFrame frame;
unsigned int required_buffer_count;
unsigned int max_queueble_buffers;

#if PPM_INSTRUMENTATION || PPM_INSTRUMENTATION_ABS
gettimeofday(&mStartPreview, NULL);
#endif

LOG_FUNCTION_NAME;

if (mPreviewInitializationDone) {
return NO_ERROR;
}

if ( mPreviewEnabled ){
CAMHAL_LOGDA("Preview already running");
LOG_FUNCTION_NAME_EXIT;
return ALREADY_EXISTS;
}

if ( NULL != mCameraAdapter ) {
ret = mCameraAdapter->setParameters(mParameters);
}

if ((mPreviewStartInProgress == false) && (mDisplayPaused == false)){
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_QUERY_RESOLUTION_PREVIEW,( int ) &frame);
if ( NO_ERROR != ret ){
CAMHAL_LOGEB("Error: CAMERA_QUERY_RESOLUTION_PREVIEW %d", ret);
return ret;
}

///Update the current preview width and height
mPreviewWidth = frame.mWidth;
mPreviewHeight = frame.mHeight;
}

///If we don't have the preview callback enabled and display adapter,
if(!mSetPreviewWindowCalled || (mDisplayAdapter.get() == NULL)){
CAMHAL_LOGD("Preview not started. Preview in progress flag set");
mPreviewStartInProgress = true;
ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_SWITCH_TO_EXECUTING);
if ( NO_ERROR != ret ){
CAMHAL_LOGEB("Error: CAMERA_SWITCH_TO_EXECUTING %d", ret);
return ret;
}
return NO_ERROR;
}

if( (mDisplayAdapter.get() != NULL) && ( !mPreviewEnabled ) && ( mDisplayPaused ) )
{
CAMHAL_LOGDA("Preview is in paused state");

mDisplayPaused = false;
mPreviewEnabled = true;
if ( NO_ERROR == ret )
{
ret = mDisplayAdapter->pauseDisplay(mDisplayPaused);

if ( NO_ERROR != ret )
{
CAMHAL_LOGEB("Display adapter resume failed %x", ret);
}
}
//restart preview callbacks
if(mMsgEnabled & CAMERA_MSG_PREVIEW_FRAME)
{
mAppCallbackNotifier->enableMsgType (CAMERA_MSG_PREVIEW_FRAME);
}

signalEndImageCapture();
return ret;
}

required_buffer_count = atoi(mCameraProperties->get(CameraProperties::REQUIRED_PREVIEW_BUFS));

///Allocate the preview buffers
ret = allocPreviewBufs(mPreviewWidth, mPreviewHeight, mParameters.getPreviewFormat(), required_buffer_count, max_queueble_buffers);

if ( NO_ERROR != ret )
{
CAMHAL_LOGEA("Couldn't allocate buffers for Preview");
goto error;
}

if ( mMeasurementEnabled )
{

ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_QUERY_BUFFER_SIZE_PREVIEW_DATA,
( int ) &frame,
required_buffer_count);
if ( NO_ERROR != ret )
{
return ret;
}

///Allocate the preview data buffers
ret = allocPreviewDataBufs(frame.mLength, required_buffer_count);
if ( NO_ERROR != ret ) {
CAMHAL_LOGEA("Couldn't allocate preview data buffers");
goto error;
}

if ( NO_ERROR == ret )
{
desc.mBuffers = mPreviewDataBuffers;
desc.mOffsets = mPreviewDataOffsets;
desc.mFd = mPreviewDataFd;
desc.mLength = mPreviewDataLength;
desc.mCount = ( size_t ) required_buffer_count;
desc.mMaxQueueable = (size_t) required_buffer_count;

mCameraAdapter->sendCommand(CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW_DATA,
( int ) &desc);
}

}

///Pass the buffers to Camera Adapter
desc.mBuffers = mPreviewBuffers;
desc.mOffsets = mPreviewOffsets;
desc.mFd = mPreviewFd;
desc.mLength = mPreviewLength;
desc.mCount = ( size_t ) required_buffer_count;
desc.mMaxQueueable = (size_t) max_queueble_buffers;

ret = mCameraAdapter->sendCommand(CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW,
( int ) &desc);

if ( NO_ERROR != ret )
{
CAMHAL_LOGEB("Failed to register preview buffers: 0x%x", ret);
freePreviewBufs();
return ret;
}

///Start the callback notifier
ret = mAppCallbackNotifier->start();

if( ALREADY_EXISTS == ret )
{
//Already running, do nothing
CAMHAL_LOGDA("AppCallbackNotifier already running");
ret = NO_ERROR;
}
else if ( NO_ERROR == ret ) {
CAMHAL_LOGDA("Started AppCallbackNotifier..");
mAppCallbackNotifier->setMeasurements(mMeasurementEnabled);
}
else
{
CAMHAL_LOGDA("Couldn't start AppCallbackNotifier");
goto error;
}

if (ret == NO_ERROR) mPreviewInitializationDone = true;

mAppCallbackNotifier->startPreviewCallbacks(mParameters, mPreviewBuffers, mPreviewOffsets, mPreviewFd, mPreviewLength, required_buffer_count);

return ret;

error:

CAMHAL_LOGEA("Performing cleanup after error");

//Do all the cleanup
freePreviewBufs();
mCameraAdapter->sendCommand(CameraAdapter::CAMERA_STOP_PREVIEW);
if(mDisplayAdapter.get() != NULL)
{
mDisplayAdapter->disableDisplay(false);
}
mAppCallbackNotifier->stop();
mPreviewStartInProgress = false;
mPreviewEnabled = false;
LOG_FUNCTION_NAME_EXIT;

return ret;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
4.3.3 BaseCameraAdapter.cpp
位置：hardware/ti/omap4-aah/camera/BaseCameraAdapter.cpp
注意到文件中定义的常量：
分别对应不同的命令。
const LUT cameraCommandsUserToHAL[] = {
{ "CAMERA_START_PREVIEW", CameraAdapter::CAMERA_START_PREVIEW },
{ "CAMERA_STOP_PREVIEW", CameraAdapter::CAMERA_STOP_PREVIEW },
{ "CAMERA_START_VIDEO", CameraAdapter::CAMERA_START_VIDEO },
{ "CAMERA_STOP_VIDEO", CameraAdapter::CAMERA_STOP_VIDEO },
{ "CAMERA_START_IMAGE_CAPTURE", CameraAdapter::CAMERA_START_IMAGE_CAPTURE },
{ "CAMERA_STOP_IMAGE_CAPTURE", CameraAdapter::CAMERA_STOP_IMAGE_CAPTURE },
{ "CAMERA_PERFORM_AUTOFOCUS", CameraAdapter::CAMERA_PERFORM_AUTOFOCUS },
{ "CAMERA_CANCEL_AUTOFOCUS", CameraAdapter::CAMERA_CANCEL_AUTOFOCUS },
{ "CAMERA_PREVIEW_FLUSH_BUFFERS", CameraAdapter::CAMERA_PREVIEW_FLUSH_BUFFERS },
{ "CAMERA_START_SMOOTH_ZOOM", CameraAdapter::CAMERA_START_SMOOTH_ZOOM },
{ "CAMERA_STOP_SMOOTH_ZOOM", CameraAdapter::CAMERA_STOP_SMOOTH_ZOOM },
{ "CAMERA_USE_BUFFERS_PREVIEW", CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW },
{ "CAMERA_SET_TIMEOUT", CameraAdapter::CAMERA_SET_TIMEOUT },
{ "CAMERA_CANCEL_TIMEOUT", CameraAdapter::CAMERA_CANCEL_TIMEOUT },
{ "CAMERA_START_BRACKET_CAPTURE", CameraAdapter::CAMERA_START_BRACKET_CAPTURE },
{ "CAMERA_STOP_BRACKET_CAPTURE", CameraAdapter::CAMERA_STOP_BRACKET_CAPTURE },
{ "CAMERA_QUERY_RESOLUTION_PREVIEW", CameraAdapter::CAMERA_QUERY_RESOLUTION_PREVIEW },
{ "CAMERA_QUERY_BUFFER_SIZE_IMAGE_CAPTURE", CameraAdapter::CAMERA_QUERY_BUFFER_SIZE_IMAGE_CAPTURE },
{ "CAMERA_QUERY_BUFFER_SIZE_PREVIEW_DATA", CameraAdapter::CAMERA_QUERY_BUFFER_SIZE_PREVIEW_DATA },
{ "CAMERA_USE_BUFFERS_IMAGE_CAPTURE", CameraAdapter::CAMERA_USE_BUFFERS_IMAGE_CAPTURE },
{ "CAMERA_USE_BUFFERS_PREVIEW_DATA", CameraAdapter::CAMERA_USE_BUFFERS_PREVIEW_DATA },
{ "CAMERA_TIMEOUT_EXPIRED", CameraAdapter::CAMERA_TIMEOUT_EXPIRED },
{ "CAMERA_START_FD", CameraAdapter::CAMERA_START_FD },
{ "CAMERA_STOP_FD", CameraAdapter::CAMERA_STOP_FD },
{ "CAMERA_SWITCH_TO_EXECUTING", CameraAdapter::CAMERA_SWITCH_TO_EXECUTING },
{ "CAMERA_USE_BUFFERS_VIDEO_CAPTURE", CameraAdapter::CAMERA_USE_BUFFERS_VIDEO_CAPTURE },
#ifdef OMAP_ENHANCEMENT_CPCAM
{ "CAMERA_USE_BUFFERS_REPROCESS", CameraAdapter::CAMERA_USE_BUFFERS_REPROCESS },
{ "CAMERA_START_REPROCESS", CameraAdapter::CAMERA_START_REPROCESS },
#endif
};
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
BaseCameraAdapter::sendCommand()：
利用 switch 将不同的命令对应到各自的逻辑中。
此处的 BaseCameraAdapter::startPreview() 实际上做什么操作，具体的实现是在其子类中，接下来选取一个子类 V4LCameraAdapter 继续深入。
case CameraAdapter::CAMERA_START_PREVIEW:
{

CAMHAL_LOGDA("Start Preview");

if ( ret == NO_ERROR )
{
ret = setState(operation);
}

if ( ret == NO_ERROR )
{
ret = startPreview();
}

if ( ret == NO_ERROR )
{
ret = commitState();
}
else
{
ret |= rollbackState();
}

break;

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
4.3.4 V4LCameraAdapter.h
位置：hardware/ti/omap4-aah/camera/inc/V4LCameraAdapter/V4LCameraAdapter.h
类 V4LCameraAdapter 继承了 BaseCameraAdapter。
注意它有一个内部私有类：
threadLoop() 应该是与线程的循环执行有关。
这个线程不断执行 Adapter 中的 previewThread() 函数。
private:

class PreviewThread : public android::Thread {
V4LCameraAdapter* mAdapter;
public:
PreviewThread(V4LCameraAdapter* hw) :
Thread(false), mAdapter(hw) { }
virtual void onFirstRef() {
run("CameraPreviewThread", android::PRIORITY_URGENT_DISPLAY);
}
virtual bool threadLoop() {
mAdapter->previewThread();
// loop until we need to quit
return true;
}
};

//Used for calculation of the average frame rate during preview
status_t recalculateFPS();

char * GetFrame(int &index);

int previewThread();
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
4.3.5 V4LCameraAdapter.cpp
位置：hardware/ti/omap4-aah/camera/V4LCameraAdapter/V4LCameraAdapter.cpp
startPreview()：
通过 v4lIoctl() 函数从硬件获取需要的数据，并存入 Buffers。
启动一个 PreviewThread，用于接收从 V4L 摄像头设备传回的数据。
最后设置一些 flag 表明预览功能已开启，startPreview 的控制流程就结束了。
status_t V4LCameraAdapter::startPreview()
{
status_t ret = NO_ERROR;

LOG_FUNCTION_NAME;
android::AutoMutex lock(mPreviewBufsLock);

if(mPreviewing) {
ret = BAD_VALUE;
goto EXIT;
}

for (int i = 0; i < mPreviewBufferCountQueueable; i++) {

mVideoInfo->buf.index = i;
mVideoInfo->buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
mVideoInfo->buf.memory = V4L2_MEMORY_MMAP;

ret = v4lIoctl(mCameraHandle, VIDIOC_QBUF, &mVideoInfo->buf);
if (ret < 0) {
CAMHAL_LOGEA("VIDIOC_QBUF Failed");
goto EXIT;
}
nQueued++;
}

ret = v4lStartStreaming();

// Create and start preview thread for receiving buffers from V4L Camera
if(!mCapturing) {
mPreviewThread = new PreviewThread(this);
CAMHAL_LOGDA("Created preview thread");
}

//Update the flag to indicate we are previewing
mPreviewing = true;
mCapturing = false;

EXIT:
LOG_FUNCTION_NAME_EXIT;
return ret;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
previewThread()：
在 PreviewThread 线程中不断被调用。
获取设备传回的数据，并进行一些格式转换操作：
convertYUV422ToNV12Tiler()
给帧数据进行一些必要的参数设置，如帧大小、时间戳等。
将帧数据发送给用户：
sendFrameToSubscribers(&frame)
int V4LCameraAdapter::previewThread()
{
status_t ret = NO_ERROR;
int width, height;
CameraFrame frame;
void *y_uv[2];
int index = 0;
int stride = 4096;
char *fp = NULL;

mParams.getPreviewSize(&width, &height);

if (mPreviewing) {

fp = this->GetFrame(index);
if(!fp) {
ret = BAD_VALUE;
goto EXIT;
}
CameraBuffer *buffer = mPreviewBufs.keyAt(index);
CameraFrame *lframe = (CameraFrame *)mFrameQueue.valueFor(buffer);
if (!lframe) {
ret = BAD_VALUE;
goto EXIT;
}

debugShowFPS();

if ( mFrameSubscribers.size() == 0 ) {
ret = BAD_VALUE;
goto EXIT;
}
y_uv[0] = (void*) lframe->mYuv[0];
//y_uv[1] = (void*) lframe->mYuv[1];
//y_uv[1] = (void*) (lframe->mYuv[0] + height*stride);
convertYUV422ToNV12Tiler ( (unsigned char*)fp, (unsigned char*)y_uv[0], width, height);
CAMHAL_LOGVB("##...index= %d.;camera buffer= 0x%x; y= 0x%x; UV= 0x%x.",index, buffer, y_uv[0], y_uv[1] );

#ifdef SAVE_RAW_FRAMES
unsigned char* nv12_buff = (unsigned char*) malloc(width*height*3/2);
//Convert yuv422i to yuv420sp(NV12) & dump the frame to a file
convertYUV422ToNV12 ( (unsigned char*)fp, nv12_buff, width, height);
saveFile( nv12_buff, ((width*height)*3/2) );
free (nv12_buff);
#endif

frame.mFrameType = CameraFrame::PREVIEW_FRAME_SYNC;
frame.mBuffer = buffer;
frame.mLength = width*height*3/2;
frame.mAlignment = stride;
frame.mOffset = 0;
frame.mTimestamp = systemTime(SYSTEM_TIME_MONOTONIC);
frame.mFrameMask = (unsigned int)CameraFrame::PREVIEW_FRAME_SYNC;

if (mRecording)
{
frame.mFrameMask |= (unsigned int)CameraFrame::VIDEO_FRAME_SYNC;
mFramesWithEncoder++;
}

ret = setInitFrameRefCount(frame.mBuffer, frame.mFrameMask);
if (ret != NO_ERROR) {
CAMHAL_LOGDB("Error in setInitFrameRefCount %d", ret);
} else {
ret = sendFrameToSubscribers(&frame);
}
}
EXIT:

return ret;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
4.3.6 * AppCallbackNotifier.cpp
位置：hardware/ti/omap4-aah/camera/AppCallbackNotifier.cpp
在 CameraHal.cpp 中，预览功能初始化的部分，调用到了 AppCallbackNotifier 类的函数。这个类应该是与各个回调函数有关，即与数据流有着密切的联系。
在这里，我打算简单地了解一些对应的逻辑。
startPreviewCallbacks()：
对 previewBuffers 进行一些必要的设置。
同步预览帧：
mFrameProvider->enableFrameNotification(CameraFrame::PREVIEW_FRAME_SYNC)
到这里以后，就不知道该往哪去深入了，但是回调函数在哪调用了？
status_t AppCallbackNotifier::startPreviewCallbacks(android::CameraParameters &params, CameraBuffer *buffers, uint32_t *offsets, int fd, size_t length, size_t count)
{
unsigned int *bufArr;
int size = 0;

LOG_FUNCTION_NAME;

android::AutoMutex lock(mLock);

if ( NULL == mFrameProvider )
{
CAMHAL_LOGEA("Trying to start video recording without FrameProvider");
return -EINVAL;
}

if ( mPreviewing )
{
CAMHAL_LOGDA("+Already previewing");
return NO_INIT;
}

int w,h;
///Get preview size
params.getPreviewSize(&w, &h);

// save preview pixel format, size and stride
mPreviewWidth = w;
mPreviewHeight = h;
mPreviewStride = 4096;
mPreviewPixelFormat = CameraHal::getPixelFormatConstant(params.getPreviewFormat());
size = CameraHal::calculateBufferSize(mPreviewPixelFormat, w, h);

mPreviewMemory = mRequestMemory(-1, size, AppCallbackNotifier::MAX_BUFFERS, NULL);
if (!mPreviewMemory) {
return NO_MEMORY;
}

for (int i=0; i < AppCallbackNotifier::MAX_BUFFERS; i++) {
mPreviewBuffers[i].type = CAMERA_BUFFER_MEMORY;
mPreviewBuffers[i].opaque = (unsigned char*) mPreviewMemory->data + (i*size);
mPreviewBuffers[i].mapped = mPreviewBuffers[i].opaque;
}

if ( mCameraHal->msgTypeEnabled(CAMERA_MSG_PREVIEW_FRAME ) ) {
mFrameProvider->enableFrameNotification(CameraFrame::PREVIEW_FRAME_SYNC);
}

if ( mCameraHal->msgTypeEnabled(CAMERA_MSG_POSTVIEW_FRAME) ) {
mFrameProvider->enableFrameNotification(CameraFrame::SNAPSHOT_FRAME);
}

mPreviewBufCount = 0;

mPreviewing = true;

LOG_FUNCTION_NAME_EXIT;

return NO_ERROR;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
setCallbacks()：
我想知道回调函数在哪调用，于是找到了这个设置回调的函数。
数据回调名为 mDataCb，于是搜索 mDataCb。
void AppCallbackNotifier::setCallbacks(CameraHal* cameraHal,
camera_notify_callback notify_cb,
camera_data_callback data_cb,
camera_data_timestamp_callback data_cb_timestamp,
camera_request_memory get_memory,
void *user)
{
android::AutoMutex lock(mLock);

LOG_FUNCTION_NAME;

mCameraHal = cameraHal;
mNotifyCb = notify_cb;
mDataCb = data_cb;
mDataCbTimestamp = data_cb_timestamp;
mRequestMemory = get_memory;
mCallbackCookie = user;

LOG_FUNCTION_NAME_EXIT;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
notifyEvent()：
在这个函数中发现了对回调函数 mDataCb 的调用。
下面选出的这个分支是与 PREVIEW_METADATA 有关，即预览元数据。
元数据在 evt->mEventData 中，看命名应该是与 Event 有关。
Event 相关的机制目前我还不太清楚，也还不会去深究，只需要知道它拿到了数据就行。
申请一个 camera_memory_t 对应的 Buffers 空间后，就调用回调函数将元数据往上层进行传输了。
case CameraHalEvent::EVENT_METADATA:

metaEvtData = evt->mEventData->metadataEvent;

if ( ( NULL != mCameraHal ) &&
( NULL != mNotifyCb) &&
( mCameraHal->msgTypeEnabled(CAMERA_MSG_PREVIEW_METADATA) ) )
{
// WA for an issue inside CameraService
camera_memory_t *tmpBuffer = mRequestMemory(-1, 1, 1, NULL);

mDataCb(CAMERA_MSG_PREVIEW_METADATA,
tmpBuffer,
0,
metaEvtData->getMetadataResult(),
mCallbackCookie);

metaEvtData.clear();

if ( NULL != tmpBuffer ) {
tmpBuffer->release(tmpBuffer);
}

}

break;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
流程简图
图中标明了控制流程的主要调用顺序，对于数据流暂不考究。
实际上，在 HAL 层与 Device 之间应该还有一层 Linux Kernel（drivers），但这目前已经超出我所需要了解的范围，所以就先忽略掉了。
注意 HAL 层中，CameraHardwareInterface 是通用的入口，而真正实现与驱动层的对接是与平台相关的，不同平台有不同的实现方案。

 

小结
本篇笔记中，我们选定 Camera.startPreview() 方法作为切入点，结合 hw_get_module() 相关内容，对其整个流程进行追踪分析，从而对之前已有初步了解的 Camera 控制流逻辑有一个更全面的理解。
在 HAL 层中，涉及到了不同平台的实现，而且这些具体实现还是有比较大的区别的。
目前还不够清楚的地方，就是 Event 、Binder 等 Android 内部机制的实现。
在下一篇，我开始整理 Camera 数据流的逻辑。由于目前所探究的数据流相对来说比较简单，主要就是几个 callback 函数的调用逻辑，所以我打算结合初始化与控制流的部分，将整个 Camera 流程整理、结合起来，作为这系列学习笔记的终结。
---------------------
作者：StoneDemo
来源：CSDN
原文：https://blog.csdn.net/qq_16775897/article/details/77863065
版权声明：本文为博主原创文章，转载请附上博文链接！