Preheat oven to 425 degrees F (220 degrees C). Lightly grease a baking sheet.
Coat a large skillet with nonstick cooking spray and heat over medium heat; cook and stir carrots, bean sprouts, water chestnuts, green pepper, green onions, and garlic until vegetables are crisp, about 3 minutes. Stir in chicken until heated through, 3 to 5 minutes.
Combine cornstarch, water, soy sauce, 1 teaspoon oil, brown sugar, and cayenne in a small bowl; stir into chicken mixture. Bring to a boil over high heat and stir, cooking until sauce is thickened, about 2 minutes; remove from heat.
Spoon 1/4 cup chicken mixture on the bottom third of one egg roll wrapper. Fold sides toward center and roll tightly; place seam side down on prepared baking sheet. Repeat with remaining filling and wrappers. Spray top of egg rolls with nonstick cooking spray.
Bake in preheated oven until lightly browned, 10 to 15 minutes.