Despite the growing integration of programming education within STEM curricula worldwide, there remains a significant gap in the availability of comprehensive and empirically validated assessment tools, particularly those capable of evaluating students’ broader computational thinking competencies. Addressing this gap, this study introduces a novel rubric grounded in the framework of algorithmic literacy—defined by Kalaš (2011) as a set of skills that enable individuals to manage behavior and attain goals within dynamic, interactive environments. The rubric, comprising four key dimensions—Subject Knowledge, Programming Design, Programming Techniques, and the Learning Process—was developed to assess students’ algorithmic literacy in the context of Scratch-based STEM learning activities. The study was guided by two primary research questions: ⑴ To what extent can the developed rubric effectively assess students' algorithmic literacy in Scratch-based STEM education? and ⑵ What similarities and differences exist between Japanese and Indonesian students in their programming outcomes, and what contextual factors contribute to these patterns? A total of 228 students—132 from Japan and 96 from Indonesia—participated in three intervention lessons focused on programming regular polygons using Scratch. Evaluation of their work revealed high proficiency in Programming Techniques, particularly in utilizing sequential commands and loops, while students exhibited difficulties in integrating complex mathematical concepts and multi-shape design tasks. Comparative analysis highlighted the influence of curriculum structure, access to technological resources, and pedagogical approaches on student performance across the two contexts. The findings contribute a validated and practical tool for assessing algorithmic literacy, offering critical insights to inform the design of culturally responsive and pedagogically effective programming education initiatives.