It is backbiting if you say something derogatory about another person in their absence, irrespective of whether you meant harm or not. The only way it is not is when the person does the bad thing being spoken about openly or you are trying to protect others from their bad actions