1. Preheat oven to 350F. For cupcakes, grease and flour or line with paper liners 24 cups in muffin pans. For cakes, grease and flour two 8- or 9-inch round pans or one 9×13-inch rectangular pan and line the bottom with parchment paper.
2. Whisk together the milk and 1/4 cup condensed milk in a medium pot. Heat over medium or medium-high heat until scalded – steaming with bubbles forming around the edge – stirring occasionally. Add the tea mix, remove from heat, cover, and let sit 10 minutes. Pour through a sieve, coffee filter, or tea screen to strain out the tea. Let the mixture cool to somewhere between room temperature, using the refrigerator or freezer to cool down quickly if necessary.
3. Whisk together the flour, baking powder, and salt in a large bowl. Set aside.
4. Using an electric mixer on medium speed, cream together the butter and sugar until light and fluffy (4-6 minutes). Beat in the eggs one at a time, making sure each egg is fully incorporated before adding the next. Beat in the vanilla. Scrape down the beaters and the sides of the bowl as needed.
5. Reduce speed to low and beat in one-third of the flour mixture. As soon as the flour is almost incorporated, add half the milk mixture. Repeat with a second third of the flour mixture, the rest of the milk, then the rest of the flour, making sure not to overmix after each addition.
6. Use a rubber spatula to give the batter a final stir, scraping along the bottom and sides of the bowl to make sure all lumps are incorporated and that there are no pockets of dry ingredients hiding in the batter.
7. Pour the batter into the prepared pan(s), filling cupcake cups about 3/4 full. Bake 20-30 minutes (shorter end for cupcakes and rounds, longer end for sheet cake), rotating pans halfway through baking. When done, the middle of the cake will spring back when lightly pressed with a fingertip and a toothpick or wooden skewer inserted into the center of the cake should come back with a few crumbs attached.
8. Let the cakes cool in the pans on wire racks for 10 minutes. For cupcakes, remove each one and let cool on wire racks until completely cool. For cakes, run a small, sharp knife around the edge to loosen the cake from the pan, then flip onto wire racks. Remove the parchment paper and flip upright. Let cool on wire racks until completely cool.