To make the bread: Stir warm water and yeast together in a large bowl; let stand until yeast softens and begins to form a creamy foam, about 5 minutes.
Mix in 2 cups flour, milk, sugar, melted butter, egg, and salt until combined.
Gradually mix in remaining 2 cups flour and cinnamon until dough comes together.
Turn dough out onto a floured surface and knead until smooth and elastic, 6 to 8 minutes. Place in a large greased bowl; turn dough to coat.
Cover and let rise in a warm place until doubled, about 1 hour.
Meanwhile, make the topping: Beat sugar and butter in a medium bowl with an electric mixer until light and fluffy.
Stir in flour until a thick paste forms.
Transfer 1/2 of the paste to a separate bowl. Mix cinnamon into one half and vanilla into the other half.
Divide dough into 12 equal pieces. Shape into balls and place 3 inches apart on a greased cookie sheet.
Divide each bowl of topping into 6 balls; pat flat. Place circles of topping on top of dough balls, patting down lightly. Use a knife to cut grooves in the topping like a seashell.
Cover and let rise until doubled, about 45 minutes.
Preheat the oven to 375 degrees F (190 degrees C).
Bake in the preheated oven until lightly golden brown, about 20 minutes.