Preheat the oven to 400 degrees F (200 degrees C). Spray 15 standard-sized muffin cups with nonstick spray.
Combine flour, cinnamon, baking soda, ginger, nutmeg, allspice, baking powder, salt, and pumpkin pie spice in a bowl.
Beat pumpkin puree, sugar, oil, eggs, and vanilla in a large bowl until well combined. Slowly mix in dry ingredients until well blended and smooth. Distribute the batter evenly among the prepared muffin cups.
Bake in the preheated oven until a toothpick inserted in the centers comes out clean, 15 to 18 minutes. Cool in the tin for 5 to 7 minutes.
While the muffins are cooling, blend powdered sugar, half-and-half, pumpkin pie spice, vanilla, and cinnamon for glaze until well combined.
Remove muffins from the tin and dip the tops into the glaze. Set on a wire rack until glaze sets, about 30 minutes.