Relatives have reacted with anger after some elderly residents were temporarily moved out of a care complex following the discovery of bed bugs.