There is a film in this riveting show by the American artist Kerry Tribe that works in the most mysterious ways. It shows an elderly gentleman – imagine Charlton Heston – finalising the table settings for an elaborate social occasion. Around him stand the members of his family, tense and silent as in a scene from Ingmar Bergman, watching as he lights the candles. Whereupon, after a brief deliberation, the old man suddenly sweeps the cloth from the table, dashing plates, glasses and all to the floor.  A moment later the film begins again.  You watch it once. You watch it twice. At first, it strikes as nothing more than a bizarre conflation of slapstick and Swedish tragedy. Then you begin to search for minuscule differences. The performance of the paterfamilias never changes, but does his daughter shudder slightly more each time? Does his son-in-law start to wince? To be certain, the viewer has to keep tight hold of the previous iteration, which proves strangely elusive, even though it lasts barely two minutes.  The table is laid out; the table is swept clean. It stands as a tabula rasa. Whatever else The Procedure might signify – and it will amount to much more by the end of the show – with its supporting cast of anxious relatives, it is as good a portrait of violent dementia as anything in contemporary art.  Kerry Tribe was born in Boston in 1973. She has shown at most of America's prestigious art museums, but this is her first British show. It is very skilfully designed to unfold as a journey up through the various floors of the Arnolfini, from darkness to light. It has films, images and installations, all interconnected like characters in a novel or clues in a mystery. All of its subjects are enthralling.  Take The Last Soviet, which centres on the story of two Russian cosmonauts. What you see on screen is a sequence of footage, dovetailing, merging, cross-fading, drifting, in which the sensation of being in outer space – and thus in freefall – becomes vividly apparent.  The tale is told of Sergei Krikalev, who manned the Mir expedition of 1991, during the collapse of the USSR. On Earth, the political crisis was so acute that Krikalev was almost forgotten, his return constantly delayed by events below.  It was rumoured that he was a KGB spy, abandoned forever to his mission. Or perhaps he had simply disappeared. In the end, losing all sense of diurnal rhythm – there is no night or day in space – he whiled away 311 days before being allowed back to Earth, by which time the USSR no longer existed and Leningrad was no longer the name of his birthplace.  The tale of The Last Soviet, as he became known, is entwined with that of a pioneering 60s cosmonaut whose name was struck from the record on his return. What had gone wrong? Failed missions were deleted from history, failed cosmonauts erased. Just as television would break into black-and-white footage of Swan Lake during the collapse of communism, this man was simply blanked from the headlines.  Everything censored, rumoured or faked in these cases has its analogies in Tribe's film, where images of outer space crackle and fade like low-fi video feed and birds appear to be trying to fly inside the capsules. Most poignantly, she unites these strange tales with a view of the Earth's curvature disappearing as the atmosphere thins: a mystery shimmering even as it fades.  Tribe's theme for the past decade or so has been time and how it insidiously turns into memory. She generally works by counterposing the personal and intimate with the factual and neurological, and not just to show how distant they might be. She has made family portraits in which every member's recollection diverges from the next, as much as from fact; she has devised a chat show in which she and her old school friends turn over an extremely ambiguous experience.  In this show, she has found ways to turn an American pilot's UFO sighting in the 1950s into a vision of memory in action. His recollections are communicated through 16mm film that spools all round the gallery, a fragile, shivering ribbon that somehow transmits continuous images, bringing to mind the characteristics of memory itself.  She also has a beautiful black-and-white film of a butterfly's wing seen flickering beneath a microscope. It looks like shifting sand, rippling water, bright metal lozenges on an art deco dress. It even looks, momentarily, like a futurist painting. A spool of ever-changing images, unstable yet looped like a Möbius strip: it might be the emblem of Tribe's work as much as memory itself.  A spooling reel is a simple enough metaphor for time, as well as memory. But this artist goes much further. Her films put the viewer on the spot, in the subject's position, so that even as you are learning about his or her experiences you are experiencing something of them yourself.  And by the same token, many of the questions she raises – Does memory live in the past or the present? Do photographs show what actually happened? Can they only contain the past, not the future? – are asked directly of a prodigiously gifted 10-year-old girl. Audrey's answers are so stimulating, it would be worth watching this quizzing film in the company of a child.  The finest example of Tribe's approach is her portrait of one of the most celebrated patients in neurological history, HM, a man whose epileptic seizures were so severe that a surgeon attempted to remove the hippocampus from his brain in "a frankly experimental procedure" in 1953. The damage was devastating. HM lost the ability to make new memories.  In Tribe's film, he has no idea how old he is, cannot recognise anyone he meets and has no memory of anything after 1953 except for whatever happened in the last 20 seconds; the table is repeatedly swept bare. You see him interviewed, and his attempts at recollection recreated with documentary footage that's intercut with little Klee-like animations of chain reactions, fishes, spools and geometric forms evoking his state with subtle clarity.  Few artists have used the viewing conditions of art film to greater effect than Tribe. The circularity of the film loop becomes a metaphor of the mind going round and round and the incessant repetitions that HM suffers, but also of the gallery-goer's experience of always arriving without the slightest knowledge of where he or she is in the proceedings.  Tribe also projects the 16mm film twice, on a split-screen, with a 20-second delay. This means that one sequence is always spooling away as the other is arriving, a sensation that presumably mimics to some degree the neurological events in HM's brain.  And here is where the film disturbs all sense of oneself as an observer. HM is shown photographs of famous people he cannot recognise – but perhaps we don't either. He is asked to remember what it was like to go diving decades ago and can't describe – any more than we could – the characteristics of the sea water. Eventually, just looking at the film becomes confusing in itself. One set of images looks hazier, darker, less familiar; it ceases to look the same the second time round. HM's amnesia, no matter how extreme, turns out to be in all of us.
American Kerry Tribe's first British show is a beguiling exploration of memory and what happens when we lose it, writes Laura Cumming