Preheat the oven to 375 degrees F (190 degrees C). Spray a 10x15-inch cake pan thoroughly with nonstick cooking spray.
Bring water to a boil in a microwave-safe bowl in a microwave oven or in a pot on the stovetop. Remove from heat and add chai tea bag. Let steep for 10 minutes; remove bag.
Combine chai tea, applesauce, and butter in a large saucepan. Bring to a boil. Remove from heat and add flour, sugar, sour cream, eggs, baking soda, vanilla extract, and salt and whisk until smooth. Pour batter evenly into the prepared pan.
Bake until a toothpick inserted into the center comes out clean, 22 to 25 minutes.
Remove cake from the oven.
Combine butter and milk in a large saucepan. Bring to a light boil and remove from heat. Add confectioners' sugar, vanilla extract, and cinnamon; whisk icing until smooth.
Pour icing over warm cake and smooth across the surface. Let cool thoroughly before cutting into 24 pieces.