This research focuses on developing an automated framework for evaluating distress on flexible and rigid pavement surfaces through deep learning and algorithms, enhancing infrastructure monitoring by efficiently identifying, assessing, and measuring road distresses. The methodology begins with identifying road stretches from ground-level images, followed by capturing photos of distresses and applying algorithms to measure their dimensions accurately. A YOLOv5 model is developed to evaluate the length and width of identified distresses, with an exploration of the relationship between camera position and measurement accuracy. Physical measurements using tape are employed for validation, ensuring that the automated results align with real-world dimensions. Results indicate that the average errors of 26.1% for length and 26.9% for width for flexible pavement and the average percentage error in length is about 29% and average percentage error in width is about 1% for rigid pavement. This highlights the importance of precise measurements for effective road rehabilitation. The integration of computer vision in road maintenance, validated through physical measurements, promises significant improvements in the accuracy, efficiency, and resilience of road networks.