Automatic and accurate collection of fishery catch data is an important part of the electronic observer system. However, automated tuna longline catch estimation remains challenging to deploy in practice due to the complexity of the working environment and the instability of tracking. In this study, a lightweight counting network was designed to automate the processing of real-time video data from fishing vessels in order to enable real-time tracking and counting of tuna catches on board fishing vessels. YOLOv5s was selected as the benchmark network in this study. The channel pruning algorithm was first used to prune the backbone network of YOLOv5s, and the results showed that the detection accuracy of the pruned model mAP_0.5~0.95 reached 68.8%, and the detection speed was 16.5 frames per second （FPS） under CPU, which was basically unchanged compared with the original model. The number of parameters, model size and computation of the model were reduced by 67.2%, 66.4% and 42.5% respectively, and the detection speed was increased by 33.1%. Secondly, the ByteTrack algorithm was used to achieve real-time tracking of multiple targets, optimize the shape of the counting area and solve the problem of counting deviation caused by the jump in the identity （ID） of the tuna being tracked. The test results of 10 videos showed that the average counting accuracy of the method was 80% and the video processing speed was 50.7 FPS, which meets the requirements of industrial-grade real-time detection. In summary, the model had the advantages of light weight, high accuracy and real-time, which could complete the real-time monitoring of longline catch in complex working environments and provide a solution to realizing fisheries automation.