Preheat oven to 350 degrees F (175 degrees C). Grease a 7x5x2-inch baking dish with 1 teaspoon butter.
Whisk sugar, brown rice flour, sorghum flour, potato starch, tapioca starch, powdered milk, baking powder, and nutmeg together in a bowl. Stir in milk until batter is the consistency of thin pancake batter. Fold in 2 tablespoons softened butter until just combined.
Pour batter into the prepared baking dish. Place peaches gently on top in a single layer. Sprinkle vanilla sugar over the peaches.
Bake in the preheated oven until top and edges are browned, about 45 minutes. Cool before serving, 4 to 5 minutes.