The feeders are spreading a disease which is deadly to the birds.
Since 2005 around one in five greenfinches in Britain has been killed by a disease called Trichomoniasis, or "trich", which causes their throats to swell, leaving them to starve to death.
Around 500,000 greenfinches were killed by trich in 2007 alone, while other types of birds, including other finches as well as pigeons and house sparrows are susceptible to the disease
Caused by a parasite, it is spread when birds feed each other, and when they visit tables and feeders which have been left contaminated by other birds, a conference in London next month will hear.
In America dirty bird feeders are also thought to be responsible for spreading an eye disease called mycoplasmal conjunctivitis, the conference will be told.
Dr Jim Reynolds, from the University of Birmingham, said that not enough was known about the larger impact of bird feeders, which are used by millions of households across the country.
"We should have a huge amount of data, but we don't," he told New Scientist.
But the Royal Society for the Protection of Birds (RSPB) defended the role of bird feeders and said that they had saved the lives of many birds during harsh winters.
Nik Shelton, from the RSPB, said: 'We don't want people to stop putting out food for birds.
“The benefits of feeding birds are enormous.
“This winter we had birds like field fares coming into gardens because they couldn't find food in the countryside.
“It was a real lifeline for them.”
He added” 'However, it's important that feeders and bird baths are hygienic to stop the spread of disease.”
He said that feeders of bird baths should be cleaned, using weak bleach, once every two weeks.
Bird lovers who see greenfinches showing signs of disease should stop feeding them for a few week
Rob Robinson, from the British Trust for Ornithology, added: "This is the first big mortality effect but it's only in one or two species.
"We're safely feeding another 30 or 40."