Preheat the oven to 350 degrees F (175 degrees C). Spray 12 muffin cups with cooking spray.
Meanwhile, heat olive oil in a skillet over medium heat. Add asparagus, green bell pepper, and onion; cook and stir until vegetables are softened, 5 to 10 minutes.
Whisk eggs, milk, salt, and black pepper together in a bowl. Stir cooked vegetables and Cheddar cheese into egg mixture; spoon about 1/4 cup of mixture into each muffin cup.
Bake in the preheated oven until frittatas are set in the middle and lightly browned, about 20 minutes.