Preheat the oven to 350 degrees F (175 degrees C). Grease and flour a fluted tube pan (such as Bundt).
Mix butter, eggs, and vanilla extract together in a large bowl with an electric mixer until light and fluffy; gradually add sugar until well blended.
Add flour, cinnamon, baking powder, baking soda, salt, and nutmeg to a separate medium bowl. Mix lightly with a fork. Add approximately 1/3 of the flour mixture to the egg mixture; do not overbeat. Add 1/2 of the buttermilk and alternate between flour mixture and buttermilk, ending with flour mixture, until blended. Add bananas and walnuts and mix until just blended. Pour batter into the prepared pan.
Bake in the preheated oven until a toothpick inserted into the center of the cake comes out clean, about 45 minutes. Remove cake carefully while still hot to a separate plate and let cool.