Bill Cosby and Beverly Johnson  12/17/2014 AT 12:45 PM EST  last week to claim she was drugged by  , cites her daughter as one of the reasons she finally spoke out.  It was her daughter, Anansa Sims, now 35, who first accompanied her to Cosby's Manhattan brownstone for brunch in the mid '80s, a few days before the comedian invited her back for an alleged audition.  "My daughter loved him," Johnson  It was only recently, after a series of women came forward claiming they had been drugged and assaulted by Cosby, that Johnson, 62, told her daughter she had also been drugged.  "When I called my daughter to tell her, I said, 'Remember when we went to Bill Cosby's house?' " Johnson tells PEOPLE. "And she said, 'Yeah, and I knocked over the orange juice.' 'Well,' I said, 'Bill Cosby drugged me, and I think I'm going to speak out about it.' "  , first replied, "Mom be careful, a lot of people like Bill Cosby and I don't want you to get hurt.' "  But once Johnson asked Sims if, in her place, she would tell her own daughter, Sims said, "Go, Mom. You are doing the right thing and I support you."  "She said I'm doing the right thing for women," says Johnson, "because you see things differently when you have child – because if anything you have to take action."  In recent days, Cosby's wife Camille  of her longtime husband. "There appears to be no vetting of my husband's accusers," she said in a statement.  When reached for comment about Camille's statement, Beverly Johnson's manager said she's done speaking for now, noting, "I think the facts speak for themselves."  While representatives for Cosby have not commented specifically on Johnson's claims, the actor's lawyer, Martin Singer, has continued to  "The new, never-before-heard claims from women who have come forward in the past two weeks with unsubstantiated, fantastical stories about things they say occurred 30, 40 or even 50 years ago have escalated past the point of absurdity," Singer said in a statement on Nov. 21.
The supermodel says her daughter was a primary reason she has come forward with claims that she was drugged by the comedian