There is a lovely scene in Glasgow-born film maker Lynne Ramsay's debut feature, 'Ratcatcher' (1999), in which its 12-year-old protagonist James pulls his sleeping mother's laddered tights over her toes. This gesture is repeated later in the film, and its ripple effect is felt nearly 20 years later in Ramsay's new film, 'You were never really there,' when its protagonist, ex-marine and FBI agent Joe (Joaquin Phoenix), returns home from a killing spree and gently holds the stocking-soled foot of his elderly sleeping mum (Judith Roberts).

A close-up image of Joe's head wrapped in a plastic bag, accompanied by muffled sound, opens the new film – an attempted suicide that is repeated several times. It is an image that eerily evokes the opening shot of 'Ratcatcher': a boy's head swathed in net curtains, accompanied by the faint sounds of children playing. Such tactile details play a distinctive role in Ramsay's work. Some of her most striking images are completely still, punctuating a medium whose defining characteristics are motion and sound.

As mentioned in the BFI Classic, it is this play of stillness and movement that makes 'Ratcatcher' such an extraordinarily cinematic film. In her new feature Ramsay once again creates fresh ways of using all of the expressive means that make cinema different from every other medium. As with her other two features, 'Morvern Callar' (2002) and 'We Need to Talk About Kevin' (2011), the film is loosely based on a book, this time a 2013 novella by Jonathan Ames.

Since the film refuses any straightforward linear narrative, watching it for its story alone is pointless. There is a story – one dictated by Joe's point of view and refracted through Ramsay's cinematic vision. Joe's mind is a tangle of memories, flash-forwards and hallucinations. He specialises in saving children from sex rings by whatever means necessary, leaving no trace of himself. In between jobs he looks after his ailing mother. A scene in which they clean cutlery together, singing '"A" You're Adorable,' is tender and telling, whilst a shared musical joke about 'Psycho' deftly distinguishes Joe's relationship from Norman Bates' sick relationship with his mother.

Again, when 'The Shawshank Redemption' appears on TV this neatly signposts the present film's own journey of redemption. Joe's only hope of ridding himself of his ghastly past is suicide, a desire gorgeously imagined in a slow-motion lake immersion scene reminiscent of Jane Campion's 'The Piano.' All such filmic references are there for a reason; they are never gratuitous.

Ramsay distils and distils the material she has shot to make every second count, the pauses just as much as the action. Few words are spoken. Instead of dialogue, her preference is to use silence and physical space within a frame to indicate how people feel about each other. She allows us glimpses of Joe's mental instability and its formation in childhood trauma, including harrowing flashbacks of a child in a cupboard beside a plastic bag and a father with a hammer ('I'll be better daddy').

A hammer is now Joe's weapon of choice. Haunted by his abusive father whose awful beatings of himself and his mother he was powerless to oppose; haunted too by his experience as a combat-shocked Gulf War veteran, Joe sees ghosts everywhere: a child killed in a war zone, a container full of dead bodies – images that fill the screen as he encounters people on the street in his everyday life. Some of the quickest shots are the most disturbing.

The plot concerns Joe's search for 14-year-old Nina (Ekaterina Samsonov), the daughter of a local politician, Senator Votto (Alex Manette). It soon emerges that the kidnapping is not what it seems and in the course of getting the job done our hammer-wielding avenger exposes a world that the powerful – especially one powerful politician whose 'favourite' Nina is – do not want revealed. As a result, having rescued Nina – whom we witness counting down from 100 so as to absent herself from the unspeakable acts being visited upon her – she is immediately recaptured and Joe left for dead. But since Ramsay refuses to hold our hand in terms of narrative development we should not strain too hard to work out exactly what is going on; otherwise, we'll fail to appreciate the viscerally unnerving journey to the dark side that is Ramsay's film.

'You were never really there' weaves together several realities through a complex mix of image and sound. This means that the question of 'what happens' is strictly unanswerable and in any case redundant. The film is not comparable with Jonathan Ames's hard-boiled novella, which clearly delineates what is going on inside Joe's head from what is happening in the 'real' world. I read it after seeing the film and discovered that the last half hour of Ramsay's film extrapolates from Ames's story in a way that is entirely of her own making qua filmmaker.

By stripping out explicit exposition and yet extending the book's 'plot' to include a second rescue (maybe), Ramsay and cinematographer Thomas Townsend place us right inside Joe's pained body and fractured mind, alternating passages of poetic beauty with episodes of extreme violence. And at the very end, we (and Joe) are allowed (just maybe) a glimpse of redemption when Nina prompts Joe: 'Come on...it's a lovely day.'

The soundtrack cleverly signals changes of register and shifting realities in the film's world and in the protagonist's relation to his world. Jonny Greenwood's unsettling musical score informs every cut, meshing seamlessly with Paul Davies's immersive sound design. Throughout, astutely selected pop songs on the radio offer an ironic counterpoint to the violence. Nothing is superfluous in this lean 90-minute film.

Most of the shocking violence and sexual abuse is not directly seen. Ramsay and editor Joe Bini intimate Joe's brutal work in myriad ways without voyeuristically dwelling on it. We see its effects in a garishly slit throat or as filtered through the black-and-white footage of surveillance cameras or as reflected in the broken glass of a mirror. And when Joe is in a position to torture the killer of someone he loves, he lies down beside him, offers him a painkiller and sings to him softly as he dies.

Only Joaquin Phoenix could be Joe. He is Joe, grounding the film through his bulky physicality and extraordinary interiorised performance. This is the performance of his career. But as Mark Kermode says, from the unsettling opening scene to the enigmatic finale, Lynne Ramsay is always really there, her commanding vision shining through every frame.