Preheat your oven to 350°F and grease a loaf pan really well. I like using a nonstick spray, but butter works too. Line the bottom with parchment paper so nothing sticks.
Cream your softened butter and sugar together for about 3 minutes until it looks pale and fluffy. Use an electric mixer on medium speed. This step is worth the time—don't skip it.
Add eggs one at a time, beating well after each addition so everything mixes smoothly. Scrape down the sides of your bowl between each egg. This takes about 4-5 minutes total but it matters.
Mix your flour, baking powder, salt, cinnamon, and orange zest in a separate bowl. Alternate adding the dry mix and the milk mixture to the butter mixture. Start and end with the dry ingredients, then fold in the yogurt gently.
Toss your fresh peach chunks in a tiny bit of flour so they don't sink. Fold them into the batter carefully—try not to break them up too much. The batter should look thick but spreadable.
Pour the batter into your prepared pan and smooth the top with a spatula. Bake for 50-55 minutes until a toothpick inserted in the center comes out clean. The top should be golden brown around the edges.
Let the fresh peach pound cake cozy cool in the pan for about 10 minutes. Then turn it out onto a wire rack to cool completely. Don't slice it while it's still warm or the crumb structure falls apart.