Mix sweet potatoes and milk together in a bowl. Mix flour, 1 tablespoon sugar, 1 teaspoon cinnamon, ginger, and 1/2 teaspoon salt together in a separate bowl. Remove a few sweet potato slices at a time, allowing excess milk to drip offl; dip the slices into the flour mixture. Place the coated sweet potatoes on a plate. Repeat with remaining sweet potatoes.
Heat vegetable oil in a large skillet over medium-high heat.
Combine 1 teaspoon sugar, 1/2 teaspoon cinnamon, and 1/4 teaspoon salt in a small bowl.
Fry the coated sweet potatoes, working in 2 to 3 batches, until crisp, 5 to 7 minutes. Drain chips on a paper towel-lined plate. Sprinkle with cinnamon-sugar mixture.