The accumulation of oceanic debris is a matter of increasing concern in today’s world. Thus, it is crucial to develop an autonomous solution that can segment debris in real-time and at high speeds while maintaining cost-effectiveness, low power consumption, and scalability. FPGAs are well-known to be low-power, high-speed alternatives to CPUs and GPUs that can also be reprogrammed at the user’s convenience, allowing for improvements to be made to the system easily. However, segmentation algorithms typically utilize complex machine learning or image processing algorithms that are difficult to implement effectively on hardware. In this work, an algorithm using thresholding in the YCbCr colour space is proposed and its accuracy is evaluated using Entropy, True Positive Rate, and True Negative Rate. The segmented regions in all the test images are found to have an entropy greater than 7 while the average True Positive Rate is 91.14%. The average True Negative Rate of the segmented regions is found to be 93.8%. The algorithm was deployed on a Cyclone IV-E FPGA wherein live camera feed was processed on the FPGA and the output was visualized on a VGA monitor. The overall performance was satisfactory, although some glitches and jitters were observed, likely due to post-processing steps required for integrating the camera feed with the VGA monitor. The algorithm proved to be robust in segmenting ocean debris under good lighting conditions. However, accuracy decreased when the ocean colour significantly deviated from its expected blue or when the image was not an aerial view.