Instructions

for the cookies

Combine flour, cocoa, baking powder, and salt in bowl. Using stand mixer fitted with paddle, beat butter, peanut butter, and sugar on medium-high speed until fluffy, about 3 minutes. Add eggs, one at a time, and beat until combined. Add vanilla and beat until incorporated. Reduce speed to low and add flour mixture in 3 additions, alternating with milk in 2 additions, scraping down bowl as needed. Refrigerate until dough is firm, about 1 hour.
Adjust oven racks to upper-middle and lower-middle positions and heat oven to 350 degrees. Line 2 baking sheets with parchment paper. Using half of dough and working with 2 teaspoons dough at a time, roll into balls and space them 2 inches apart on prepared sheets. Bake until edges appear dry, 8 to 10 minutes, switching and rotating sheets halfway through. Remove cookies from oven and press 1 marshmallow half into center of each cookie. Return to oven and bake for 2 minutes. Let cookies cool on sheets for 5 minutes. Transfer cookies to wire rack to cool completely, about 15 minutes. Repeat with remaining dough and marshmallows.

for the frosting

Whisk sugar, butter, cocoa, water, and vanilla in bowl until smooth. Working with 1 cookie at a time, spread 1 heaping teaspoon frosting over top of cookie, covering marshmallow completely. Sprinkle nonpareils on top of cookies and let sit for at least 30 minutes before serving. (Cookies can be stored at room temperature for up to 2 days.)