The Abrolhos Bank, encompassing a wide portion of the Brazilian continental shelf, harbors the world's largest rhodolith bed, which plays an important role in calcium carbonate production in the South Atlantic Ocean. Little is known about the community structure and species composition of this habitat. The aim of this study was to test the hypothesis that latitudinal differences exist in the community structure of rhodolith beds and their associated flora along the Abrolhos Bank. We sampled a total of 33 sites of rhodolith beds located in three regions—northern, central, and southern—of the Abrolhos Bank between depths of 20 and 70 m. Rhodolith density (rhod. m−2) within the beds ranged from 990 ± 347.6 to 57 ± 18.7, with mean diameter ranging from 9.4 ± 3.2 to 3.1 ± 1.4 cm. A total of 146 macroalgae species were identified, including 14 rhodolith-forming species of crustose coralline algae. Abrolhos Bank supports the world’s greatest species richness of rhodolith-forming CCA, with regional distinctiveness. Observed differences in bed structure among regions can be related to differences in shelf width, slope and depth. The studied rhodolith beds constitute a unique habitat supporting a distinctive diversity of associated organisms, and thus require special attention. We highlight the importance of local and regional differences for defining appropriate conservation strategies to protect the rhodolith bed diversity of Abrolhos Bank.