Make the dressing by whisking the mayonnaise, sour cream, lemon juice, lemon zest, sugar, ginger, and salt in a mixing bowl. Add the chicken, blueberries, celery, grapes, and almonds; stir gently until evenly mixed. Cover the bowl and refrigerate for 1 hour.