The first time Michael Roemer set foot in the American south, something pinged in his brain. He had never been there before; he grew up in Germany and Britain, but that day in segregated Alabama in the early 1960s, "I recognised everything. It was immediate. I said, 'Oh, I know this. I know what this feels like.'"  In the last 10 days, I have seen three films by Roemer: two documentaries and Nothing But a Man, his first feature, shot in 1963. The documentaries – Dying, a short piece following three people in the last few months of their lives; and Cortile Cascino, a study of a slum in Palermo, Sicily – are 40 years old and hard to get hold of. Nothing But a Man is rereleased this week by the BFI, and if I could reach out and grab your collar I would: do whatever you can to see these films.  At 84, Roemer still teaches at the Yale School of Art, where we meet in a sunny conference room, interrupted now and then by colleagues coming in to use the microwave. He has a look of Samuel Beckett about him and a slight German accent. Roemer left Berlin aged 11 as part of the Kindertransport and spent his teenage years in Britain, before joining his mother in the US. His sensibility was probably formed by three things, he says: fascism, Brecht and boarding school in Britain. The result is a rare and scalding humanity. Cortile Cascino, which Roemer calls "a very visceral film", was judged by a director at the Cinémathèque in Paris to be "better than Fellini".  The question of why Roemer isn't better known comes down to a choice of subject matter – and the eye he turned on it – years before anyone else was making such films. Nothing But a Man, which has an almost entirely African-American cast, tells the story of a young black couple in Alabama struggling to keep a family together. It is at once a period piece and, thanks to the startlingly natural performances, as modern a film as you'll see.  The lead actors – Ivan Dixon as Duff, Julius Harris as his father Will, and Abbey Lincoln as Josie, his wife – had to film in New Jersey since filming with a black cast in the south was too incendiary in 1963. Roemer and Robert Young, his long-time collaborator, had spent two and a half months touring the southern states, looking for stories that picked out the terrible things going on there in the early 60s, and out of this wrought a film that Roemer hoped would say something about the way larger political movements can undermine people at the most basic levels.  "It was showing how the economic system, the social system, destroyed the most intimate relationships. I saw it happen. It happened to Jews. It happened to my grandfather. He came from a very assimilated family, and they were interrelated with the Prussian aristocracy. Nonetheless, my grandfather was destroyed by everything he had taken away from him. He was a wonderful man. He shrivelled up and lost his identity."  Likewise Nothing But a Man shows Duff, a railroad worker, being systematically stripped of every piece of selfhood. While he fights against the system, his father-in-law, a preacher, negotiates with "the white power structure", so that his son-in-law yells at him: "You been stoopin' for so long, you don't know how to stand up."  Roemer's own early years were a mix of privilege – his extended family owned one of the biggest shoe manufacturers in central Europe – and neglect. His parents divorced when he was an infant and he hardly knew them. "I was raised by a governess. I had no sense of what family life was like. So when I was without a family, it wasn't a big change. And I was free of the governess, which was marvellous. I was terrified of her."  She was, he says, "a sadist", but one with an awareness of her own motivation. "She had a story about why she was the way she was, and, oddly, she shared it with me – perhaps in the hope of my understanding her. It was about her childhood: she would watch her mother being beaten by her father. She would say, 'I was at the bottom of the stairs, and he was beating her upstairs and she was screaming and crying.' And this was the woman that beat a child." He smiles. "Everybody has a story and a justification for who they are and what they do."  So it was with some relief that Roemer was one day told to pack up and prepare to leave Germany. Looking back now, he marvels both at his acquiescence – "I was very obedient, because I'd been scared a lot. I did what I was told" – and the surrealism of what came next. The children arrived in Kent and were taken to a local hall, where they were put on stage and their names called out one by one. "And you'd go down the steps to someone you didn't know. It was bizarre."  Roemer was lucky. The Bunce Court School, for which he was destined, had relocated to England from Germany after the Nazis seized power, and it now catered to large numbers of German Jewish refugees. The British pupils were largely the children of communists or artists. Unusually for the time, it was co-educational. There was almost no discipline. "We were outside a lot; if we got out of bed at night and started roaming around, nobody ever noticed. We never had a pregnant girl, which is amazing. We were totally unsupervised." It suited Roemer, as it did his new and enduring best friend, Frank Auerbach. "Frank and I both feel that it was the really big, formative experience. The encouragement to go your own way. The teachers were good parents to us. I loved it and didn't want to leave."  After the war, however, at the age of 17, Roemer was summoned to Boston, where his mother had been living with her sister. While in England, he had obsessively read German literature. Now, in the US, he started to read the English classics. "I was always one country behind."  He was also at a loss as to what to do next. His aunt told him there was a rather good college across the river from where she lived and he should pull his socks up and apply. "And that's how I got to Harvard. I mean, it was insane."  It was at college that Roemer started getting interested in film. When Charlie Chaplin's Gold Rush came out, he spent a week watching it over and over at the cinema. When the Harvard Crimson advertised for students interested in film-making, he applied and was told to turn in a script by the end of the summer. No one else finished theirs and so, by default, Roemer's got made. "It was a terrible script – nobody liked it – but it got made!" He directed it, and somehow a film reviewer at Life magazine saw it and made it movie of the week. "Amazing! But if you look at anybody's life, it's pretty crazy. That your life doesn't end up on the rocks somewhere is just a miracle."  Reading on mobile? Click here to watch Nothing But a Man trailer  This has always been his outlook. Whereas Young, his producing partner, has what Roemer thinks of as the American birthright – "that sense of 'things are going to be all right'"– Roemer has the opposite. "Things are not going to be all right," he tends to think, and in the case of their 1962 documentary, Cortile Cascino, he was right. The TV network, after commissioning the film, refused to show it. The film-making was too stark, too shocking. They even destroyed the prints, but a producer at NBC smuggled out a copy – not something Roemer himself would ever have done, he says. "Even if it had been a Leonardo or a Rembrandt, I wouldn't have been able to do it: to go against the people who were paying me. I can't do that."  Nothing But a Man suffered from distribution problems, too. None of the big movie theatres wanted to show it. "In Philadelphia, I asked the exhibitor why it was playing in the suburbs, and how are black people supposed to get there to see it? And he said, 'We don't want them.' So, you see, the audience was not supposed to be the black audience. And that's where the real life of the film would be."  The film did nothing on release; but over the next 40 years it became a staple on video, shown in schools, churches and youth centres to largely black audiences. One day Malcolm X "ran into the man who played the father, on the street, three days before he died. And he said, 'You're Julius Harris. I saw Nothing But a Man, I really liked it.' Juli was very pleased."  If Roemer has a regret about the film, it is that his own story is too nakedly visible in it. The difficult relationship between Duff and his father is his own. (Roemer's father survived the war in Spain, but was a "very unhappy man," about whom his son knew almost nothing). "I see the graft too clearly. Ja, I do. People know that the heart of the film is in the right place. But a few people, mostly African-American, can see the graft, and I have to agree with them."  Roemer went on to make a few more films, but commercial success never troubled him. "I can't make a film to get to do something else. I just have to do it for itself, and if it costs me … " He worries now that he jeopardised his family's welfare to make the films he wanted to make but that no one, at the time, wanted to see. "It's a little crazy, you know? You put everything into this thing. It's this and nothing else. And whatever happens next, I can't think about."  But he was determined – not that the word "determined" makes much sense, he says, laughing. If his life has taught him anything, it is the limits to which the individual determines anything.  • Nothing But a Man is screening at the BFI Southbank, London SE1, until 10 November.
He fled the Nazis for a British boarding school – then made a shocking drama about segregation in the deep south that was way ahead of its time. Michael Roemer talks fate, family and sadistic governesses with Emma Brockes