Preheat the oven to 425 degrees F (220 degrees C).
Thinly slice each potato several times, stopping just short of cutting all the way through.
Whisk olive oil, garlic, paprika, oregano, and celery salt together in a bowl; pour into a resealable plastic bag. Add potatoes, seal the bag, and shake to coat completely.
Transfer potatoes to a baking dish and reserve oil mixture in a small bowl. Sprinkle Parmesan cheese over potatoes.
Bake potatoes in the preheated oven until layers start to separate and potatoes are slightly tender, about 30 minutes. Brush potatoes with reserved oil mixture, pulling layers apart if they are sticking together. Continue baking until potatoes are crispy on the edges and easily pierced with a knife, 30 to 40 minutes more.