Fusion of Perturbed Defects in Conformal Field Theory