In March, Ted Chiang, whose 1998 science fiction novella, Story of Your Life, about alien communication became the basis of the Oscar-nominated film Arrival, was sitting at his computer at home in Seattle, when he came across a news story that surprised him.

The story was about how Jibo, a manufacturer of artificially intelligent ‘companion’ robots for the home, was now shutting down its servers, causing distress to Jibo owners who had come to think of their little robots as part of the family. Chiang had read this story before. In fact, he had written it.

‘The Lifecycle of Software Objects’ – the novella-length centrepiece of a new short story collection, Exhalation, his first since his 2002 debut, Stories of Your Life and Others – details the efforts of a fictional company, Blue Gamma, to maintain socially conscious virtual ‘digients’, which owners ‘raise’ for up to 20 years. When sales stall, Blue Gamma plans to shut down their servers, leaving digient owners distraught.

The Jibo story was a sharp example of life imitating art, and a reminder of why Chiang, who until recently was a technical writer for Microsoft, is revered in the sci-fi community as one of the most exciting thinkers of his generation. More impressive, the 51-year-old, who has won 27 science fiction awards, wrote the story in 2010, a whole year before Apple launched Siri.

‘It was an unexpected similarity,’ the modest Chiang tells me over the phone, his precise sentences punctuated by long periods of considered silence. ‘I did not imagine these [technological] advancements would happen.’ He explains that his story was a ‘thought experiment’ taken to its logical conclusion, one that asks: ‘What happens when you develop an emotional connection to a product and the product is taken away?’ and: ‘What is the business case for giving tech consciousness?’ (There isn’t one, he says, although he believes it will happen regardless.)

The eight other thought experiments in Exhalation, all written during the past 17 years, are equally provocative future scenarios that tread high above the cliched dystopias of so much modern sci-fi. ‘Dacey’s Patent Automatic Nanny’, for instance, follows the crazed ambition of a mathematician who, frustrated with conventional childcare, invents a machine capable of raising his son. The title story, ‘Exhalation’, imagines humans breathing from metal lungs which they must constantly refill, until the ‘master lung’ eventually runs out of air. ‘What’s Expected of Us’ explores a world without free will, while ‘The Merchant and the Alchemist’s Gate’ is set in the past and draws on Kip Thorne’s theory of wormholes to reimagine time travel.

[How to become a filmmaker]

Although dark in premise, these parables – threaded through with references to ancient mythology and folklore – are filled with hope and humanism: a balm for anxious souls. ‘I hope that you were motivated by a desire for knowledge, a yearning to see what can arise from a universe’s exhalation... Contemplate the marvel that is existence, and rejoice that you are able to do so,’’ reads a typical passage towards the end of ‘Exhalation’, striking a celebratory note that recalls Ray Bradbury. ‘I don’t know how anyone can look at the current political landscape and not be cynical about people’s motives,’ says Chiang. ‘In my fiction, I try to resist pessimism and cynicism because I don’t want to spend more time in that frame of mind than I already do.’

The son of an engineering professor and a librarian who fled the Chinese Communist Revolution for Long Island, Chiang began writing about intergalactic war and space adventure at the age of 15. He’d grown up an avid reader of physics and astronomy, as well as science fiction by his heroes Isaac Asimov and Arthur C Clarke. But when he tried to emulate them, he struggled to finish his stories. His solution was to begin each new tale by coming up with an ending – a technique he still uses. For years, Chiang would submit his work to sci-fi magazines; by the time he graduated from Brown University in Computer Science, he had accumulated over 100 rejection slips. ‘I can safely say that my stories were terrible,’ he says.

A year later, at 23, his determination paid off. While Chiang was working full time as a technical writer, the science fiction magazine Omni published his story The Tower of Babylon. A reworking of the Babel myth. It won a Nebula Award, one of the highest honours in science fiction.

Chiang began to amass a cult following, and the rejection letters slowed down. By 2002, he had published seven further stories in a variety of magazines: These were assembled into his debut collection by the American fantasy imprint Tor Books. ‘And yet I was still in the genre ghetto,’ Chiang says. ‘I was only published in science fiction [publications] and only read by science fiction writers. Even when I was at college, a tutor at a writing class told me she was only accepting ‘original writing’, and no genre fiction, so I took that as my cue to leave. I’ve experienced that kind of snobbery for the vast majority of my career.’

Chiang had to wait until 2016 for his breakthrough, when Vintage republished his debut collection in over 21 languages, finally enabling him to reach what he calls ‘a readership outside science fiction. I never expected it would happen’.

Since 2011, Chiang had also been in talks with the screenwriter Eric Heisserer (who wrote the 2010 remake of A Nightmare on Elm Street and Final Destination 5) about a film adaptation of the collection’s title tale ‘Story of Your Life’. Heisserer pitched his idea to various studios, and was turned down by all – until Sicario director Denis Villeneuve signed on. In 2016, the film of Story of Your Life was released as Arrival, starring Amy Adams as a linguistics professor hired by the US government to act as the interpreter for alien visitors to earth. It was nominated for eight Oscars. Chiang has now seen the film five times. Were Arrival’s seven-footed aliens, the ‘Heptapods’, as he had imagined? ‘They are similar to the story in two important respects,’ he says. ‘They are not bilaterally symmetrical and they don’t have a face you can look at, which are traits we’ve rarely – if ever – seen in movies and TV.’

Debunking sci-fi cliches with fresh ideas has always been vital to him. ‘I am more frustrated than most by depictions of AI artificial intelligence,’ he says, citing examples such as Alex Garland’s Ex Machina and Spike Jonze’s Her. ‘Her combines two familiar tropes: the idealised robot female, and the manic pixie dream girl trope, which sees an effervescent woman renew a morose man’s zest for life. I thought these tropes would be confronted when the character played by Amy Adams becomes friends with the software girlfriend her husband left her for,’ says Chiang. ‘I thought, that would be a super interesting movie; that’s not a story we’ve ever seen before. That was the most original thing in the whole film, but it was just a throwaway line.’

Perhaps, I suggest, it could provide the basis for Chiang’s next story? ‘I have not thought about it,’ he replies, embarrassed, before reluctantly allowing me to quiz him on his personal interests. ‘I consume plenty of modern science fiction, TV and film,’ he says. He used to be a Star Wars fan, but not any more; he enjoyed the first series of the television drama Westworld, but not the second, and does not read Stephen King.

Chiang says he is not active on social media – ‘I don’t have the personality for it. I wouldn’t know what to say’ – and refuses to have an Alexa in his home. ‘If the capabilities of an Alexa were done locally, rather than on Amazon servers somewhere, I might try that as a lark,’ he says.

Does his personal life influence his stories? ‘No,’ he says. Is he inspired by dreams or by the news? ‘Neither, my stories come from general abstract musings.’

After much cajoling, Chiang ‘identifies’ the motivation for one story, ‘Liking What You See: a Documentary’, from his debut collection. ‘I was sitting in my car in the parking lot, and an attractive woman walked by. My head turned to follow her, and I thought, ‘why did I do that?’ I had been thinking about something totally different. So I asked myself whether it would have been better to not have done that, or whether it was good that I had.’

What, I ask, was his answer? Chiang disappears into another, considerable silence. And then, logical as ever, says: ‘My answer was that story.’