Stir 2 cups whipping cream, milk, lemon pudding mixes, and lemon juice in a large bowl until creamy and smooth. Cover and refrigerate until set, about 1 hour.
Beat remaining 2 cups whipping cream in a bowl until foamy. Gradually add confectioners' sugar and vanilla extract, beating until stiff peaks form. Lift your beater or whisk straight up: the whipped cream should form sharp peaks.
Layer 1/2 of the angel food cake in a trifle bowl. Top with 1/2 of the lemon pudding mixture, 1/2 of the strawberries, and 1/2 of the blueberries. Spread 1/2 of the whipped cream over berries. Repeat layers. Cover and refrigerate until ready to serve.