Place feta cheese in a glass jar. Add chile pepper, herbes de Provence, and lemon zest. Fill jar with olive oil so feta is completely covered and seal.
Marinate in a cool place, but not in the refrigerator, for 3 to 4 days. Remove chile pepper before serving.