Preheat your oven to 350°F (175°C). Grease and flour a 9x13-inch baking pan or line it with parchment paper.
In a medium bowl, whisk together the flour, baking powder, baking soda, cinnamon, nutmeg, ginger, and salt. Set aside.
In a large mixing bowl, cream together the softened butter, granulated sugar, and brown sugar until light and fluffy, about 3-4 minutes.
Add the eggs one at a time, mixing well after each addition. Stir in the vanilla extract.
Gradually add the dry ingredients to the butter mixture, alternating with the buttermilk. Begin and end with the dry ingredients, mixing until just combined.
Gently fold in the diced apples and nuts (if using) until evenly distributed in the batter.
Pour the batter into the prepared baking pan and spread it evenly. Bake for 35-40 minutes, or until a toothpick inserted in the center comes out clean.
Allow the cake to cool in the pan for about 10 minutes before transferring it to a wire rack to cool completely. Dust with powdered sugar before serving, if desired.