Preheat an oven to 350 degrees F (175 degrees C). Prepare a 9x12-inch baking dish with cooking spray.
Place the bread cubes in the prepared dish. Sprinkle the pecans over the bread.
Beat the eggs, 1 1/2 cup milk, almond extract, 1 tablespoon cardamom, cinnamon, and brown sugar together in a bowl until thoroughly combined; pour the mixture evenly over the bread cubes and pecans. Cover the dish with aluminum foil and allow the casserole to sit 15 minutes to allow the flavors to combine.
Bake in the preheated oven for 20 minutes; remove the aluminum foil and continue baking until the top is golden brown, another 20 to 25 minutes. Remove from oven and allow to cool slightly, about 10 minutes.
Mix the confectioners' sugar, 1/2 teaspoon cardamom, and 3 tablespoons milk in a bowl with a fork until smooth; pour over the slightly cooled casserole to serve.