Combine potatoes, celery, onion, 1/2 of the Cheddar cheese, green chiles, and pepper in a soup pot and cover with water. Cover the pot and bring to a boil. Reduce heat and simmer until celery is tender, 30 to 40 minutes. Add evaporated milk and butter. Stir in bouillon; taste and adjust if needed.
Pour into serving bowls and top with remaining Cheddar cheese.