Sailfish OS Camera架构

1 关键词

Gstreamer

　　　　Gstreamer是一个非常强大且通用流媒体框架，其具有非常灵活的模块化特点，本文中的相机技术接触gstreamer其原理是编写了一个gstreamer插件。

HAL

　　　　Android HAL(Hardware Abstract Layer)硬件抽象层，从字面意思可以看出是对硬件设备的抽象和封装，为Android在不同硬件设备提供统一的访问接口。HAL处于Android framework和Linux kernel driver之间。这种屏蔽了不同设备差异，对于上层开发者来说可以不用关心硬件设备，只需要按照HAL提供的标准接口访问硬件即可。这也为Andriod和GUN Linux的 “嫁接” 提供非常方便的标准。　　

2 简介

　　本文主要调研Sailfish OS中jolla camera相机的工作原理，其框架层设计与安卓相机HAL抽象层交互，以控制相机的工作参数和状态，以插件的形式将其与gstreamer连接。本文的基础是假设内核、HAL层都已实现，libhybris将安卓的HAL层代码从bonnic libc转为glibc，方便我们在GUN Linux中访问，整体框架如下图1-1。

　　　　　　　　　　　　　　　　　　　　　　　　图 1-1 Sailfish OS 相机技术框架图

　　在libhybris的转换下，可以通过android-headers-23和android_hardware_libhardware访问安卓HAL硬件抽象接口，而android.hardware.camera.provider@2.4-service 是一个常驻的后台进程为系统提供相机底层服务，而使用gstreamer作为视频流框架使得相机预览和录制视频等功能更加方便。为了提供一个标准的、统一的相机API层，Sailfish OS在框架上融合这些功能，将功能融合在gst-droidcamsrc项目中，通过标准结构hw_module_t、hw_device_t、camera_device_t访问HAL层相机，控制开关状态和Flash、Focus、White balance、zoom、ISO、EV comp等相机的运行参数。

3 安卓HAL层相机

　　HAL层提供相机最基本底层控制和数据，其中实现的一些以下的功能：

获取相机个数
区分前置相机和后置相机
获取相机设备的角度，方便预览时修正显示
开启\关闭相机
设置\获取相机运行参数,Flash、Focus、White balance、zoom、ISO、EV comp等
相机硬件唯一标识,方便识别

　　3.1 hw_module_t

　　　　　　HLA使用hw_module_t结构体描述一类硬件抽象模块,具体对应在本文中用来描述相机硬件抽象。

/**
 * Every hardware module must have a data structure named HAL_MODULE_INFO_SYM
 * and the fields of this data structure must begin with hw_module_t
 * followed by module specific information.
 */
typedef struct hw_module_t {
    /** tag must be initialized to HARDWARE_MODULE_TAG */
    uint32_t tag;

    /**
     * The API version of the implemented module. The module owner is
     * responsible for updating the version when a module interface has
     * changed.
     *
     * The derived modules such as gralloc and audio own and manage this field.
     * The module user must interpret the version field to decide whether or
     * not to inter-operate with the supplied module implementation.
     * For example, SurfaceFlinger is responsible for making sure that
     * it knows how to manage different versions of the gralloc-module API,
     * and AudioFlinger must know how to do the same for audio-module API.
     *
     * The module API version should include a major and a minor component.
     * For example, version 1.0 could be represented as 0x0100. This format
     * implies that versions 0x0100-0x01ff are all API-compatible.
     *
     * In the future, libhardware will expose a hw_get_module_version()
     * (or equivalent) function that will take minimum/maximum supported
     * versions as arguments and would be able to reject modules with
     * versions outside of the supplied range.
     */
    uint16_t module_api_version;
#define version_major module_api_version
    /**
     * version_major/version_minor defines are supplied here for temporary
     * source code compatibility. They will be removed in the next version.
     * ALL clients must convert to the new version format.
     */

    /**
     * The API version of the HAL module interface. This is meant to
     * version the hw_module_t, hw_module_methods_t, and hw_device_t
     * structures and definitions.
     *
     * The HAL interface owns this field. Module users/implementations
     * must NOT rely on this value for version information.
     *
     * Presently, 0 is the only valid value.
     */
    uint16_t hal_api_version;
#define version_minor hal_api_version

    /** Identifier of module */
    const char *id;

    /** Name of this module */
    const char *name;

    /** Author/owner/implementor of the module */
    const char *author;

    /** Modules methods */
    struct hw_module_methods_t* methods;

    /** module's dso */
    void* dso;

#ifdef __LP64__
    uint64_t reserved[32-7];
#else
    /** padding to 128 bytes, reserved for future use */
    uint32_t reserved[32-7];
#endif

} hw_module_t;

　　其中的methods提供了打开相机的接口

typedef struct hw_module_methods_t {
    /** Open a specific device */
    int (*open)(const struct hw_module_t* module, const char* id,
            struct hw_device_t** device);

} hw_module_methods_t;

　　3.2 camera_module_t

　　　　　　获取相机硬件抽象的基本信息和主要功能

typedef struct camera_module {
    /**
     * Common methods of the camera module.  This *must* be the first member of
     * camera_module as users of this structure will cast a hw_module_t to
     * camera_module pointer in contexts where it's known the hw_module_t
     * references a camera_module.
     *
     * The return values for common.methods->open for camera_module are:
     *
     * 0:           On a successful open of the camera device.
     *
     * -ENODEV:     The camera device cannot be opened due to an internal
     *              error.
     *
     * -EINVAL:     The input arguments are invalid, i.e. the id is invalid,
     *              and/or the module is invalid.
     *
     * -EBUSY:      The camera device was already opened for this camera id
     *              (by using this method or open_legacy),
     *              regardless of the device HAL version it was opened as.
     *
     * -EUSERS:     The maximal number of camera devices that can be
     *              opened concurrently were opened already, either by
     *              this method or the open_legacy method.
     *
     * All other return values from common.methods->open will be treated as
     * -ENODEV.
     */
    hw_module_t common;

    /**
     * get_number_of_cameras:
     *
     * Returns the number of camera devices accessible through the camera
     * module.  The camera devices are numbered 0 through N-1, where N is the
     * value returned by this call. The name of the camera device for open() is
     * simply the number converted to a string. That is, "0" for camera ID 0,
     * "1" for camera ID 1.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     * CAMERA_MODULE_API_VERSION_2_3 or lower:
     *
     *   The value here must be static, and cannot change after the first call
     *   to this method.
     *
     * CAMERA_MODULE_API_VERSION_2_4 or higher:
     *
     *   The value here must be static, and must count only built-in cameras,
     *   which have CAMERA_FACING_BACK or CAMERA_FACING_FRONT camera facing values
     *   (camera_info.facing). The HAL must not include the external cameras
     *   (camera_info.facing == CAMERA_FACING_EXTERNAL) into the return value
     *   of this call. Frameworks will use camera_device_status_change callback
     *   to manage number of external cameras.
     */
    int (*get_number_of_cameras)(void);

    /**
     * get_camera_info:
     *
     * Return the static camera information for a given camera device. This
     * information may not change for a camera device.
     *
     * Return values:
     *
     * 0:           On a successful operation
     *
     * -ENODEV:     The information cannot be provided due to an internal
     *              error.
     *
     * -EINVAL:     The input arguments are invalid, i.e. the id is invalid,
     *              and/or the module is invalid.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     * CAMERA_MODULE_API_VERSION_2_4 or higher:
     *
     *   When a camera is disconnected, its camera id becomes invalid. Calling this
     *   this method with this invalid camera id will get -EINVAL and NULL camera
     *   static metadata (camera_info.static_camera_characteristics).
     */
    int (*get_camera_info)(int camera_id, struct camera_info *info);

    /**
     * set_callbacks:
     *
     * Provide callback function pointers to the HAL module to inform framework
     * of asynchronous camera module events. The framework will call this
     * function once after initial camera HAL module load, after the
     * get_number_of_cameras() method is called for the first time, and before
     * any other calls to the module.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     *  CAMERA_MODULE_API_VERSION_1_0, CAMERA_MODULE_API_VERSION_2_0:
     *
     *    Not provided by HAL module. Framework may not call this function.
     *
     *  CAMERA_MODULE_API_VERSION_2_1:
     *
     *    Valid to be called by the framework.
     *
     * Return values:
     *
     * 0:           On a successful operation
     *
     * -ENODEV:     The operation cannot be completed due to an internal
     *              error.
     *
     * -EINVAL:     The input arguments are invalid, i.e. the callbacks are
     *              null
     */
    int (*set_callbacks)(const camera_module_callbacks_t *callbacks);

    /**
     * get_vendor_tag_ops:
     *
     * Get methods to query for vendor extension metadata tag information. The
     * HAL should fill in all the vendor tag operation methods, or leave ops
     * unchanged if no vendor tags are defined.
     *
     * The vendor_tag_ops structure used here is defined in:
     * system/media/camera/include/system/vendor_tags.h
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     *  CAMERA_MODULE_API_VERSION_1_x/2_0/2_1:
     *    Not provided by HAL module. Framework may not call this function.
     *
     *  CAMERA_MODULE_API_VERSION_2_2:
     *    Valid to be called by the framework.
     */
    void (*get_vendor_tag_ops)(vendor_tag_ops_t* ops);

    /**
     * open_legacy:
     *
     * Open a specific legacy camera HAL device if multiple device HAL API
     * versions are supported by this camera HAL module. For example, if the
     * camera module supports both CAMERA_DEVICE_API_VERSION_1_0 and
     * CAMERA_DEVICE_API_VERSION_3_2 device API for the same camera id,
     * framework can call this function to open the camera device as
     * CAMERA_DEVICE_API_VERSION_1_0 device.
     *
     * This is an optional method. A Camera HAL module does not need to support
     * more than one device HAL version per device, and such modules may return
     * -ENOSYS for all calls to this method. For all older HAL device API
     * versions that are not supported, it may return -EOPNOTSUPP. When above
     * cases occur, The normal open() method (common.methods->open) will be
     * used by the framework instead.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     *  CAMERA_MODULE_API_VERSION_1_x/2_0/2_1/2_2:
     *    Not provided by HAL module. Framework will not call this function.
     *
     *  CAMERA_MODULE_API_VERSION_2_3:
     *    Valid to be called by the framework.
     *
     * Return values:
     *
     * 0:           On a successful open of the camera device.
     *
     * -ENOSYS      This method is not supported.
     *
     * -EOPNOTSUPP: The requested HAL version is not supported by this method.
     *
     * -EINVAL:     The input arguments are invalid, i.e. the id is invalid,
     *              and/or the module is invalid.
     *
     * -EBUSY:      The camera device was already opened for this camera id
     *              (by using this method or common.methods->open method),
     *              regardless of the device HAL version it was opened as.
     *
     * -EUSERS:     The maximal number of camera devices that can be
     *              opened concurrently were opened already, either by
     *              this method or common.methods->open method.
     */
    int (*open_legacy)(const struct hw_module_t* module, const char* id,
            uint32_t halVersion, struct hw_device_t** device);

    /**
     * set_torch_mode:
     *
     * Turn on or off the torch mode of the flash unit associated with a given
     * camera ID. If the operation is successful, HAL must notify the framework
     * torch state by invoking
     * camera_module_callbacks.torch_mode_status_change() with the new state.
     *
     * The camera device has a higher priority accessing the flash unit. When
     * there are any resource conflicts, such as open() is called to open a
     * camera device, HAL module must notify the framework through
     * camera_module_callbacks.torch_mode_status_change() that the
     * torch mode has been turned off and the torch mode state has become
     * TORCH_MODE_STATUS_NOT_AVAILABLE. When resources to turn on torch mode
     * become available again, HAL module must notify the framework through
     * camera_module_callbacks.torch_mode_status_change() that the torch mode
     * state has become TORCH_MODE_STATUS_AVAILABLE_OFF for set_torch_mode() to
     * be called.
     *
     * When the framework calls set_torch_mode() to turn on the torch mode of a
     * flash unit, if HAL cannot keep multiple torch modes on simultaneously,
     * HAL should turn off the torch mode that was turned on by
     * a previous set_torch_mode() call and notify the framework that the torch
     * mode state of that flash unit has become TORCH_MODE_STATUS_AVAILABLE_OFF.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     * CAMERA_MODULE_API_VERSION_1_x/2_0/2_1/2_2/2_3:
     *   Not provided by HAL module. Framework will not call this function.
     *
     * CAMERA_MODULE_API_VERSION_2_4:
     *   Valid to be called by the framework.
     *
     * Return values:
     *
     * 0:           On a successful operation.
     *
     * -ENOSYS:     The camera device does not support this operation. It is
     *              returned if and only if android.flash.info.available is
     *              false.
     *
     * -EBUSY:      The camera device is already in use.
     *
     * -EUSERS:     The resources needed to turn on the torch mode are not
     *              available, typically because other camera devices are
     *              holding the resources to make using the flash unit not
     *              possible.
     *
     * -EINVAL:     camera_id is invalid.
     *
     */
    int (*set_torch_mode)(const char* camera_id, bool enabled);

    /**
     * init:
     *
     * This method is called by the camera service before any other methods
     * are invoked, right after the camera HAL library has been successfully
     * loaded. It may be left as NULL by the HAL module, if no initialization
     * in needed.
     *
     * It can be used by HAL implementations to perform initialization and
     * other one-time operations.
     *
     * Version information (based on camera_module_t.common.module_api_version):
     *
     * CAMERA_MODULE_API_VERSION_1_x/2_0/2_1/2_2/2_3:
     *   Not provided by HAL module. Framework will not call this function.
     *
     * CAMERA_MODULE_API_VERSION_2_4:
     *   If not NULL, will always be called by the framework once after the HAL
     *   module is loaded, before any other HAL module method is called.
     *
     * Return values:
     *
     * 0:           On a successful operation.
     *
     * -ENODEV:     Initialization cannot be completed due to an internal
     *              error. The HAL must be assumed to be in a nonfunctional
     *              state.
     *
     */
    int (*init)();

    /* reserved for future use */
    void* reserved[5];
} camera_module_t;

　　3.3 hw_device_t

　　　　　　HAL使用hw_device_t结构体描述硬件模块中的独立硬件设备。具体对应在本文中用来描述相机硬件设备。提供关闭该设备的接口。

typedef struct hw_device_t {
    /** tag must be initialized to HARDWARE_DEVICE_TAG */
    uint32_t tag;

    /**
     * Version of the module-specific device API. This value is used by
     * the derived-module user to manage different device implementations.
     *
     * The module user is responsible for checking the module_api_version
     * and device version fields to ensure that the user is capable of
     * communicating with the specific module implementation.
     *
     * One module can support multiple devices with different versions. This
     * can be useful when a device interface changes in an incompatible way
     * but it is still necessary to support older implementations at the same
     * time. One such example is the Camera 2.0 API.
     *
     * This field is interpreted by the module user and is ignored by the
     * HAL interface itself.
     */
    uint32_t version;

    /** reference to the module this device belongs to */
    struct hw_module_t* module;

    /** padding reserved for future use */
#ifdef __LP64__
    uint64_t reserved[12];
#else
    uint32_t reserved[12];
#endif

    /** Close this device */
    int (*close)(struct hw_device_t* device);

} hw_device_t;

　　3.4 camera_device_t

　　　　　　ops提供了操作相机和获取相机参数的接口　，包括拍照、数据、Flash、Focus、White balance、zoom、ISO、EV comp等　　　

typedef struct camera_device {
    /**
     * camera_device.common.version must be in the range
     * HARDWARE_DEVICE_API_VERSION(0,0)-(1,FF). CAMERA_DEVICE_API_VERSION_1_0 is
     * recommended.
     */
    hw_device_t common;
    camera_device_ops_t *ops;
    void *priv;
} camera_device_t;

4 技术方案

　　Sailfish OS的技术方案为在HAL层之上对接gstreamer多媒体框架。QtmultiMedia基于gstreamer实现的多媒体库，而gstreamer提供模块化插件的形式对接，gst-droid、gst-droidcamsrc就是插件实现。

　　4.1 整体设计

　　4.2 关键技术

　　4.3 工作原理

　　4.4 工作流程

　　4.5 “0拷贝”

　　　　　　client自己渲染了一副图片，窗管要将桌面上的所有client包括桌面生成一副图片显示到屏幕上。所以dma buf就是client渲染的buffer和server给你的Fd是同一个。你通过server拿到fd，然后把你的窗口渲染到这个fd中去，就实现了0拷贝了。

　　　　参考文章：

　　　　 0拷贝基础

4 对接QtMultimedia

5 安卓应用使用Camera

posted @ 2021-08-11 16:48 つContent 阅读(429) 评论(0) 收藏举报

刷新页面返回顶部

つContent

Sailfish OS Camera架构

1 关键词

2 简介

3 安卓HAL层相机

3.1 hw_module_t

3.2 camera_module_t

3.3 hw_device_t

3.4 camera_device_t

4 技术方案

4.1 整体设计

4.2 关键技术

4.3 工作原理

4.4 工作流程

4.5 “0拷贝”

4 对接QtMultimedia

5 安卓应用使用Camera

公告

　　3.1 hw_module_t

　　3.2 camera_module_t

　　3.3 hw_device_t

　　3.4 camera_device_t

　　4.1 整体设计

　　4.2 关键技术

　　4.3 工作原理

　　4.4 工作流程

　　4.5 “0拷贝”