It is still hard to accurately find bone fractures and figure out how bad osteoarthritis (OA) is from X-ray pictures because of the complicated anatomical differences and the lack of contextual modelling in standard deep learning methods. This research presents XrayBoneNet, a hybrid deep learning system that combines Convolutional Neural Networks (CNNs), Transformer-based global feature modelling, and attention mechanisms for concurrent fracture identification, OA grading, and localization. The model has two heads: one for binary fracture classification and one for multi-class OA staging. It also has a bounding box regression head for accurate localization. To improve training efficiency and performance, a hybrid optimization technique that uses Bighorn Sheep Optimization (BSO) for global exploration and Logarithmic Mean Optimization (LMO) for fine-tuning is developed. The Bone Fracture Multi-Region X-ray dataset shows that XrayBoneNet works better than state-of-the-art models like ResNet50, DenseNet121, and Vision Transformer. It has 96.8% accuracy in fracture detection, 94.5% accuracy in OA classification, and an Intersection-over-Union (IoU) of 0.87 for localization. The suggested system offers an effective and comprehensible alternative for automated radiological diagnosis.