Place 2 cups of ice cubes into a 2-quart pitcher; slowly pour coffee over ice and stir until ice is melted. Stir sugar into coffee and mix thoroughly until sugar is completely dissolved. Add milk, stirring just until combined. Stir in remaining 2 cups ice, adding more as needed; refrigerate until well chilled, about 30 minutes.
Stir before serving. Pour over ice in tall glasses or into recycled serving-size glass bottles or insulated tumblers.