V4L2 调试之(四)
调试 V4l2
2023-09-11 14:15:47 时间
1.V4l2 架构
Linux的video设备驱动实现，驱动开发者需要按照V4L2的驱动模型进行设计，该驱动模型主要围绕核心数据结构struct video_device来展开设计，通过该数据结构来完成视频设备的分配、设置、注册等工作。驱动程序需要重点实现两个操作集：struct v4l2_file_operations和struct v4l2_ioctl_ops，V4L2架构最终会调用这两个操作集中的函数接口，来完成对视频设备硬件的控制。其实现流程如下图所示：
对于melis的porting来说，v4l2_file_operations有两个，一个是struct v4l2_file_operations vin_fops，另一个是struct v4l2_file_operations v4l2_subdev_fops。而v4l2_ioctl_ops只有一个，就是struct v4l2_ioctl_ops vin_ioctl_ops。subdev直接调用subdevice的IOCTL，没有独立的struct v4l2_ioctl_ops。
关于CCI/TWI/GPIO：
架构图中的CCI表示的是CSI-2接口中集成的控制接口CCI(Camera Control Interface)，CCI是一个能够支持400KHz传输速率的全双工主从设备通信控制接口，它能够兼容现有很多处理器的IIC标准接口，因此可以非常方便地实现Soc上CCI Master Module到 CSI-2 TX 端CCI Slave Module的控制，CSI-2物理接口框图如下图所示。
2.Linux 里面V4l2初始化首先注册class，并且注册设备号
这样，就会出现/sys/class/video4linux目录.
插入USB摄像头之后，video4linux目录中，马上就会出现新的device。
V4L2默认是编译为模块的
UVC摄像头挂载堆栈：
[ 37.796088] Call Trace:
[ 37.796096] dump_stack+0x6d/0x8b
[ 37.796110] __video_register_device+0x38/0x1110 [videodev]
[ 37.796126] uvc_register_video_device+0xf1/0x150 [uvcvideo]
[ 37.796131] uvc_probe+0x25e8/0x2a10 [uvcvideo]
[ 37.796137] usb_probe_interface+0x149/0x300
[ 37.796147] really_probe+0xf5/0x440
[ 37.796150] driver_probe_device+0x11b/0x130
[ 37.796153] __device_attach_driver+0x7b/0xe0
[ 37.796160] bus_for_each_drv+0x6e/0xb0
[ 37.796163] __device_attach+0xe4/0x160
[ 37.796166] device_initial_probe+0x13/0x20
[ 37.796168] bus_probe_device+0x92/0xa0
[ 37.796171] device_add+0x402/0x690
[ 37.796178] usb_set_configuration+0x3fd/0x8f0
[ 37.796187] generic_probe+0x2e/0x80
[ 37.796189] usb_probe_device+0x31/0x70
[ 37.796191] really_probe+0xf5/0x440
[ 37.796194] driver_probe_device+0x11b/0x130
[ 37.796196] __device_attach_driver+0x7b/0xe0
[ 37.796202] bus_for_each_drv+0x6e/0xb0
[ 37.796205] __device_attach+0xe4/0x160
[ 37.796208] device_initial_probe+0x13/0x20
[ 37.796210] bus_probe_device+0x92/0xa0
[ 37.796213] device_add+0x402/0x690
[ 37.796221] usb_new_device+0x218/0x4a0
[ 37.796224] hub_event+0x11b1/0x1760
[ 37.796231] process_one_work+0x20f/0x400
[ 37.796235] worker_thread+0x34/0x410
[ 37.796238] kthread+0x121/0x140
[ 37.796247] ret_from_fork+0x35/0x40
[ 37.796250] __video_register_device line 853, comm kworker/3:1
[ 37.796405] CPU: 3 PID: 100 Comm: kworker/3:1 Not tainted 5.4.129+ #20
[ 37.796407] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 37.796411] Workqueue: usb_hub_wq hub_event
[ 37.796412] Call Trace:
[ 37.796416] dump_stack+0x6d/0x8b
[ 37.796427] __video_register_device+0x38/0x1110 [videodev]
[ 37.796444] uvc_register_video_device+0xf1/0x150 [uvcvideo]
[ 37.796449] uvc_meta_register+0x46/0x50 [uvcvideo]
[ 37.796454] uvc_probe+0x25f9/0x2a10 [uvcvideo]
[ 37.796458] usb_probe_interface+0x149/0x300
[ 37.796468] really_probe+0xf5/0x440
[ 37.796472] driver_probe_device+0x11b/0x130
[ 37.796475] __device_attach_driver+0x7b/0xe0
[ 37.796482] bus_for_each_drv+0x6e/0xb0
[ 37.796484] __device_attach+0xe4/0x160
[ 37.796487] device_initial_probe+0x13/0x20
[ 37.796489] bus_probe_device+0x92/0xa0
[ 37.796492] device_add+0x402/0x690
[ 37.796498] usb_set_configuration+0x3fd/0x8f0
[ 37.796506] generic_probe+0x2e/0x80
[ 37.796509] usb_probe_device+0x31/0x70
[ 37.796511] really_probe+0xf5/0x440
[ 37.796514] driver_probe_device+0x11b/0x130
[ 37.796516] __device_attach_driver+0x7b/0xe0
[ 37.796523] bus_for_each_drv+0x6e/0xb0
[ 37.796526] __device_attach+0xe4/0x160
[ 37.796528] device_initial_probe+0x13/0x20
[ 37.796530] bus_probe_device+0x92/0xa0
[ 37.796533] device_add+0x402/0x690
[ 37.796541] usb_new_device+0x218/0x4a0
[ 37.796544] hub_event+0x11b1/0x1760
[ 37.796549] process_one_work+0x20f/0x400
[ 37.796553] worker_thread+0x34/0x410
[ 37.796556] kthread+0x121/0x140
[ 37.796567] ret_from_fork+0x35/0x40
[ 37.796570] __video_register_device line 853, comm kworker/3:1
VIDIOC_ENUM_FMT命令的调用堆栈：
[ 37.466546] uvc_ioctl_enum_fmt line 629, usbuvc comm a.out.
[ 37.466549] CPU: 0 PID: 2029 Comm: a.out Not tainted 5.4.129+ #20
[ 37.466550] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 37.466551] Call Trace:
[ 37.466554] dump_stack+0x6d/0x8b
[ 37.466559] uvc_ioctl_enum_fmt+0x3f/0xe0 [uvcvideo]
[ 37.466564] uvc_ioctl_enum_fmt_vid_cap+0x18/0x20 [uvcvideo]
[ 37.466573] v4l_enum_fmt+0x97/0x17f0 [videodev]
[ 37.466583] __video_do_ioctl+0x190/0x4c0 [videodev]
[ 37.466593] video_usercopy+0x1ae/0x6a0 [videodev]
[ 37.466602] ? v4l_s_fmt+0x6a0/0x6a0 [videodev]
[ 37.466613] video_ioctl2+0x15/0x20 [videodev]
[ 37.466622] v4l2_ioctl+0x49/0x50 [videodev]
[ 37.466624] do_vfs_ioctl+0xa9/0x640
[ 37.466628] ? vfs_write+0x16a/0x1a0
[ 37.466631] ksys_ioctl+0x75/0x80
[ 37.466634] __x64_sys_ioctl+0x1a/0x20
[ 37.466638] do_syscall_64+0x57/0x190
[ 37.466641] entry_SYSCALL_64_after_hwframe+0x44/0xa9
那么这个FMT是如何产生的呢？是在UVC设备被枚举的时候获取到的：
[ 50.176028] uvc_parse_format line 455, comm kworker/3:2.
[ 50.176030] CPU: 3 PID: 206 Comm: kworker/3:2 Not tainted 5.4.129+ #20
[ 50.176031] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 50.176034] Workqueue: usb_hub_wq hub_event
[ 50.176035] Call Trace:
[ 50.176038] dump_stack+0x6d/0x8b
[ 50.176043] uvc_probe+0x1613/0x2a40 [uvcvideo]
[ 50.176047] usb_probe_interface+0x149/0x300
[ 50.176052] ? uvc_register_video_device+0x150/0x150 [uvcvideo]
[ 50.176054] ? usb_probe_interface+0x149/0x300
[ 50.176057] really_probe+0xf5/0x440
[ 50.176059] driver_probe_device+0x11b/0x130
[ 50.176062] __device_attach_driver+0x7b/0xe0
[ 50.176064] ? driver_allows_async_probing+0x60/0x60
[ 50.176068] bus_for_each_drv+0x6e/0xb0
[ 50.176070] __device_attach+0xe4/0x160
[ 50.176073] device_initial_probe+0x13/0x20
[ 50.176075] bus_probe_device+0x92/0xa0
[ 50.176078] device_add+0x402/0x690
[ 50.176082] ? _cond_resched+0x19/0x40
[ 50.176084] usb_set_configuration+0x3fd/0x8f0
[ 50.176087] ? kernfs_activate+0x78/0x80
[ 50.176091] generic_probe+0x2e/0x80
[ 50.176094] usb_probe_device+0x31/0x70
[ 50.176096] really_probe+0xf5/0x440
[ 50.176099] driver_probe_device+0x11b/0x130
[ 50.176101] __device_attach_driver+0x7b/0xe0
[ 50.176104] ? driver_allows_async_probing+0x60/0x60
[ 50.176107] bus_for_each_drv+0x6e/0xb0
[ 50.176110] __device_attach+0xe4/0x160
[ 50.176112] device_initial_probe+0x13/0x20
[ 50.176114] bus_probe_device+0x92/0xa0
[ 50.176117] device_add+0x402/0x690
[ 50.176121] ? add_device_randomness+0x9d/0x1c0
[ 50.176124] usb_new_device+0x218/0x4a0
[ 50.176127] hub_event+0x11b1/0x1760
[ 50.176133] process_one_work+0x20f/0x400
[ 50.176136] worker_thread+0x34/0x410
[ 50.176139] kthread+0x121/0x140
[ 50.176143] ? process_one_work+0x400/0x400
[ 50.176145] ? kthread_park+0x90/0x90
[ 50.176148] ret_from_fork+0x35/0x40
set format 接口,可以看到是通过USB Contorl pipeline传递的参数数据。
[ 33.704038] __uvc_query_ctrl line 39.comm guvcview.
[ 33.704039] CPU: 0 PID: 2045 Comm: guvcview Not tainted 5.4.129+ #20
[ 33.704040] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 33.704040] Call Trace:
[ 33.704043] dump_stack+0x6d/0x8b
[ 33.704046] __uvc_query_ctrl+0x86/0xd0 [uvcvideo]
[ 33.704049] uvc_get_video_ctrl.constprop.19+0xa1/0x3e0 [uvcvideo]
[ 33.704051] uvc_probe_video+0x72/0x130 [uvcvideo]
[ 33.704053] uvc_ioctl_s_parm+0x1d1/0x2d0 [uvcvideo]
[ 33.704055] ? uvc_simplify_fraction+0xd6/0x100 [uvcvideo]
[ 33.704059] v4l_s_parm+0x70/0xa0 [videodev]
[ 33.704063] __video_do_ioctl+0x190/0x4c0 [videodev]
[ 33.704064] ? __account_locked_vm+0xb0/0x140
[ 33.704068] video_usercopy+0x1ae/0x6a0 [videodev]
[ 33.704072] ? v4l_s_fmt+0x6a0/0x6a0 [videodev]
[ 33.704073] ? unmap_region+0xf7/0x130
[ 33.704074] ? vm_area_free+0x18/0x20
[ 33.704076] ? kmem_cache_free+0x294/0x2b0
[ 33.704076] ? kmem_cache_free+0x294/0x2b0
[ 33.704080] video_ioctl2+0x15/0x20 [videodev]
[ 33.704083] v4l2_ioctl+0x49/0x50 [videodev]
[ 33.704085] do_vfs_ioctl+0xa9/0x640
[ 33.704086] ksys_ioctl+0x75/0x80
[ 33.704087] __x64_sys_ioctl+0x1a/0x20
[ 33.704089] do_syscall_64+0x57/0x190
[ 33.704090] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 33.704091] RIP: 0033:0x7fcb0c64b639
streaming on接口：
[ 33.704300] __uvc_query_ctrl line 39.comm guvcview.
[ 33.704301] CPU: 0 PID: 2045 Comm: guvcview Not tainted 5.4.129+ #20
[ 33.704302] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 33.704302] Call Trace:
[ 33.704304] dump_stack+0x6d/0x8b
[ 33.704307] __uvc_query_ctrl+0x86/0xd0 [uvcvideo]
[ 33.704310] uvc_set_video_ctrl+0x13b/0x1d0 [uvcvideo]
[ 33.704312] uvc_video_start_streaming+0x6b/0xd0 [uvcvideo]
[ 33.704314] uvc_start_streaming+0x28/0x70 [uvcvideo]
[ 33.704316] vb2_start_streaming+0x6d/0x130 [videobuf2_common]
[ 33.704318] vb2_core_streamon+0xfc/0x130 [videobuf2_common]
[ 33.704320] vb2_streamon+0x29/0x50 [videobuf2_v4l2]
[ 33.704322] uvc_queue_streamon+0x2e/0x50 [uvcvideo]
[ 33.704324] uvc_ioctl_streamon+0x45/0x60 [uvcvideo]
[ 33.704328] v4l_streamon+0x20/0x30 [videodev]
[ 33.704331] __video_do_ioctl+0x190/0x4c0 [videodev]
[ 33.704333] ? vb2_vmalloc_num_users+0x1/0x10 [videobuf2_vmalloc]
[ 33.704334] ? __rb_insert_augmented+0x1ab/0x240
[ 33.704338] video_usercopy+0x1ae/0x6a0 [videodev]
[ 33.704341] ? v4l_s_fmt+0x6a0/0x6a0 [videodev]
[ 33.704343] ? do_mmap+0x3e1/0x5b0
[ 33.704346] video_ioctl2+0x15/0x20 [videodev]
[ 33.704349] v4l2_ioctl+0x49/0x50 [videodev]
[ 33.704350] do_vfs_ioctl+0xa9/0x640
[ 33.704351] ksys_ioctl+0x75/0x80
[ 33.704352] __x64_sys_ioctl+0x1a/0x20
[ 33.704354] do_syscall_64+0x57/0x190
[ 33.704355] entry_SYSCALL_64_after_hwframe+0x44/0xa9
USB 枚举过程：
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
|__ Port 1: Dev 8, If 3, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 1: Dev 8, If 1, Class=Video, Driver=uvcvideo, 480M
|__ Port 1: Dev 8, If 2, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 1: Dev 8, If 0, Class=Video, Driver=uvcvideo, 480M
[ 37.213832] usb_set_configuration line 2023, nintf = 4.adding 1-1:1.0 (config #1, interface 0)
[ 37.214101] uvcvideo: Found UVC 1.00 device GENERAL WEBCAM (1b3f:2247)
[ 37.214151] uvcvideo: Failed to query (GET_INFO) UVC control 2 on unit 1: -32 (exp. 1).
[ 37.216965] uvcvideo: UVC non compliance - GET_DEF(PROBE) not supported. Enabling workaround.
[ 37.217584] input: GENERAL WEBCAM: GENERAL WEBCAM as /devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/input/input20
[ 37.217771] usb_set_configuration line 2023, nintf = 4.adding 1-1:1.1 (config #1, interface 1)
[ 37.217826] usb_set_configuration line 2023, nintf = 4.adding 1-1:1.2 (config #1, interface 2)
[ 37.223884] usb 1-1: Warning! Unlikely big volume range (=5120), cval->res is probably wrong.
[ 37.223888] usb 1-1: [5] FU [Mic Capture Volume] ch = 1, val = 7680/12800/1
[ 37.224277] usb_set_configuration line 2023, nintf = 4.adding 1-1:1.3 (config #1, interface 3)
[ 37.268340] uvcvideo: Failed to query (GET_DEF) UVC control 2 on unit 1: -32 (exp. 1).
[ 37.268996] uvcvideo: Failed to query (GET_DEF) UVC control 2 on unit 1: -32 (exp. 1).
确实有四个interfaces
UVC帧数据通路：
中断中获取数据：
[ 81.989226] CPU: 3 PID: 2098 Comm: dmesg Not tainted 5.4.129+ #21
[ 81.989227] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 81.989228] Call Trace:
[ 81.989229] <IRQ>
[ 81.989233] dump_stack+0x6d/0x8b
[ 81.989237] uvc_video_complete+0x154/0x180 [uvcvideo]
[ 81.989240] __usb_hcd_giveback_urb+0x93/0x120
[ 81.989241] usb_hcd_giveback_urb+0xb8/0xe0
[ 81.989243] xhci_giveback_urb_in_irq.isra.46+0x84/0xf0
[ 81.989244] xhci_td_cleanup+0xe2/0x180
[ 81.989246] finish_td+0x13f/0x180
[ 81.989247] xhci_irq+0x13fa/0x2380
[ 81.989249] ? handle_irq_event+0x46/0x60
[ 81.989251] xhci_msi_irq+0x11/0x20
[ 81.989252] __handle_irq_event_percpu+0x44/0x1a0
[ 81.989253] handle_irq_event_percpu+0x32/0x80
[ 81.989254] handle_irq_event+0x3b/0x60
[ 81.989255] handle_edge_irq+0x83/0x1a0
[ 81.989257] do_IRQ+0x54/0xe0
[ 81.989258] common_interrupt+0xf/0xf
[ 81.989259] </IRQ>
最后由queue_work将数据通过一个工作队列交给vb2_buffer_done队列：
vb2_buffer_done:
[ 81.989343] CPU: 0 PID: 124 Comm: kworker/u9:0 Not tainted 5.4.129+ #21
[ 81.989344] Hardware name: Dell Inc. Vostro 3268/0TJYKK, BIOS 1.11.1 12/11/2018
[ 81.989348] Workqueue: uvcvideo uvc_video_copy_data_work [uvcvideo]
[ 81.989348] Call Trace:
[ 81.989351] dump_stack+0x6d/0x8b
[ 81.989353] vb2_buffer_done+0x21/0x210 [videobuf2_common]
[ 81.989355] ? __switch_to+0x85/0x490
[ 81.989357] uvc_queue_buffer_complete+0x83/0x90 [uvcvideo]
[ 81.989360] uvc_queue_buffer_release+0x24/0x30 [uvcvideo]
[ 81.989361] uvc_video_copy_data_work+0x47/0x80 [uvcvideo]
[ 81.989364] process_one_work+0x20f/0x400
[ 81.989365] worker_thread+0x34/0x410
[ 81.989367] kthread+0x121/0x140
[ 81.989368] ? process_one_work+0x400/0x400
[ 81.989369] ? kthread_park+0x90/0x90
[ 81.989371] ret_from_fork+0x35/0x40
回调调用的地方：
3.query capabilities:
4.设置翻转，到底是VIPP设置的还是sensor设置的？看下图，是通过twi设置的sensor.
FLIP的实现有两种，一种是通过sensor去实现，这就需要通过cci(i2c兼容）总线去配置外挂sensor去支持，另一种则是设置VIPP CSI DMA控制器，通过设置DMA使能FLIP，vin_s_ctrl默认选择的是第一种，通过宏控制。
flip:
unflip:
注意图像中显示器边缘的位置，明显是做了FLIP了。
STREAM ON堆栈：
__vin_pipeline_s_stream是steam on的关键函数,代码中按照sensor,mipi,csi,tdm,isp,scaler,capture的顺序逐个打开subdev设备。
buffer的管理分为两种，FIFO和register模式，当s_stream设置到vin subdev的时候，会设置对应的Buffer模式。
打开和关闭parser的过程，启动和关闭.
其它几个subdevice 的上下电打开关闭clk的调用堆栈:
Pipeline创建：
