If I don't talk to the guardian will i keep custody?

Prepare to loose custody of your child. You are a TERRIBLE mother! Making false accusations of abuse and withholding visitation are legitimate reasons to change custody. It is probably too late to fix it but YES you should cooperate with the guardian. At least that way you will get some visitation with your child. But that said, I think you are a TROLL.