Bring almond milk to a boil in a saucepan; add tea bag. Remove saucepan from heat and allow tea to steep, about 5 minutes. Remove and discard tea bag.
Mix chia seeds, vanilla extract, and brown sugar together in a bowl; stir into the almond milk mixture. Pour almond milk mixture into small bowls and top each with maple syrup. Cover and refrigerate until set, 8 hours to overnight.