Preheat oven to 350 degrees F (175 degrees C). Grease a 2-quart baking dish, and set aside.
Mix together the sweet potatoes, white sugar, 1/2 cup melted butter, milk, eggs, vanilla extract, cinnamon, and nutmeg in a bowl.
In another bowl, stir together the brown sugar, flour, 2 1/2 tablespoons of butter, and the pecans.
Spoon the sweet potato mixture into the greased baking dish, spread the pecan mixture evenly over the top, and bake in the preheated oven until the topping is browned and bubbling, about 25 minutes.