On Tuesday evening, West 20th Street in New York was impossible to ignore. A lengthy queue stretched down the sidewalk toward St Peter’s Episcopal Church in Chelsea.

One passer-by stopped to ask who had drawn the crowd. “Who is this Earl?” he asked. “His name is Charles,” came the reply. “Then who is Earl?” “That’s his title.”

The man attracting the attention was Charles Spencer, who had arrived to discuss his new book, Swan Song, with journalist Tina Brown. Tickets, priced at $20, had completely sold out.

The New Yorker appeared unconvinced. He shrugged, continued down the street and remained no closer to understanding the fascination surrounding the English aristocrat.

Yet Spencer’s public standing has changed dramatically. During my years reporting on Diana for Fleet Street, he was a figure of considerable prominence.

At just 33, he took the pulpit at Westminster Abbey to deliver a powerful and fiercely critical eulogy for his sister.

The Queen, who was also his godmother, sat alongside other members of the royal family as the service unfolded, their expressions tense and unsmiling.

Today, the tall aristocrat is known chiefly for one defining connection: he is the brother of the late Princess Diana.

Nearly three decades later, Spencer was once again the evening’s main attraction. The venue was far smaller than Westminster Abbey, but not a single pew appeared to be empty.

He also had a new book to promote. Holding the microphone, he appeared slightly nervous, though he relaxed when the audience applauded his claim that the Palace’s reaction to the publication amounted not to a denial, but to gaslighting.

Brown, chosen to conduct his only live conversation in the United States, questioned him closely about his childhood with Diana. Spencer maintained that their family life had been largely happy, despite his parents’ separation.

Diana had previously offered Brown a very different account. Spencer responded sharply: “I was there.”

He was not entirely without justification. Brown made a point of noting that she had lunch with Diana only weeks before the princess died, then reminded Spencer that she was three years younger than him.

For a moment, the exchange raised the curious question of whether the two were competing over who had understood Diana best.

In the end, the conversation remained civil. Spencer’s argument for telling this story is simple: he was present at the centre of it, and he has every right to draw on that experience.

He reinforced the point by imagining how valuable it would have been had Anne Boleyn’s brother, the poet George Boleyn, Viscount Rochford, left behind a memoir. It was an unfortunate example: George was executed for treason after being accused of having an affair with his sister. The allegation was untrue and formed part of the charges manufactured against Anne by Henry VIII, who wanted to end the marriage and take another wife.

Spencer describes himself as both a historian and a journalist. His previous books on Charles I and Charles II were well received, while some critics view his latest work as an attack on Charles III’s character.

He studied Modern History at Magdalen College, Oxford, with the qualification later converted into an MA. His journalism career included almost ten years as an NBC News correspondent beginning in 1986. In both fields, however, the essential responsibility remains the same: accuracy matters.

He insists he remembers his phone calls with Charles clearly, and on the night, he recounted the episode clearly. He even joked about the pen that tried the King’s temper after his accession. Brown did not give him an easy ride.

Still, I do not recognize the King he describes, a man so cruel about his ex-wife, the mother of his sons, that Spencer says he sounded ‘giddily elated,’ ‘like a lottery winner,’ when speaking after her death. In the heat of a row over whether William and Harry should walk behind her coffin, he may have said ‘we’ll forget her soon enough.’

The elation, we are told, was because Charles would be free to marry his now Queen, Camilla. Spencer seems to have forgotten that Charles was already divorced from Diana. He was free to marry Camilla anyway.

The book will sell. It will be read for years and some of it will be taken seriously.

Near the end came the question that mattered. What would Diana have been if she had not married the heir to the throne?

He paused a long while. ‘Happy.’

I am not so sure she wasn’t happy anyway.

The Diana being written about by commentators now is a sad, troubled woman.

Her brother believes she had undiagnosed ADHD. Others have said she suffered from paranoia.

Some of it is true and some is pure guesswork. It is not how I remember her.

This week I recorded an interview with my old friend Inspector Ken Wharfe MVO for my podcast, The Royal Editor. He reminded me she had the most infectious giggle and wicked sense of humor. 

Once, in the back of a car, she decided to have some fun with her hairdresser, Richard Dalton. Posing as an American correspondent, Ken rang him. He had seen the Princess’s new flicked hairstyle at an event in the US, he said, and it looked marvelous, though there was some concern about her roots.

Diana stuffed a handkerchief in her mouth so she would not be heard. Her sister, Lady Sarah McCorquodale, helpless with laughter, sat beside her and the driver could barely keep his eyes on the road. They had to call it off.

She found her own happiness. She learned to live with a husband who loved Camilla, and she had affairs of her own: with James Hewitt, who has admitted it, and with the late art dealer Oliver Hoare. James Gilbey may not have been a lover, but he was clearly a very intimate friend. He was the man heard calling her ‘Squidgy’ on the Squidgygate tapes.

She was perhaps happiest with Hasnat Khan, the heart surgeon. He would not commit because his family wanted him to marry a woman of the Muslim faith. Diana went to Pakistan to show them she was serious. They did not budge. Hasnat could not cope with the drama around her either, and the relationship ended weeks before her death.

Her worst mistake was giving up her police protection. I get a mention in Spencer’s book, on page 251, for saying so. He quotes an article I wrote for the Daily Express, where I was then royal correspondent, about her decision to disband her team of royal bodyguards. At the time I said it ‘speaks volumes for her present state of mind, it is very sad.’

Ken, loyal to the core, warned her against doing it before he left her side. She did not listen.

In Paris in August 1997, there was no Scotland Yard officer beside her. There was Trevor Rees-Jones, military trained and on Mohamed Al Fayed’s payroll. She was 36.

Spencer reveals that after she died, the Queen’s private secretary, Sir Robert Fellowes, later Lord Fellowes and Spencer’s own brother-in-law, brought word that Her Majesty wished to restore Diana’s HRH.

Losing it in the divorce had upset her deeply. She could not believe they had done it. She was the mother of a future king and she thought it was hers by right.

Spencer turned it down. She was dead. What would be the point?

He has wondered ever since whether he was right.

Brown reminded him that William had reportedly promised his mother he would give it back when he was king. Surely that was for William and Harry to decide, not him? Whatever was promised as a boy, I do not expect William to reopen it when he is king.

Diana had made her brother the boys’ guardian, a role with no real clout while their father was alive. He was careful about his nephews all evening. ‘Is it astounding that an Englishman wants to move back to England?’ he asked, defending Harry’s return.

Harry may well be making a film about his mother, Spencer said, but he did not know. He was mindful of William throughout. He does not want to anger the future king, though he may already have done so.

What would grieve Diana now is the silence between her sons. 

You need to be in your mid-forties to remember her as a living public figure. She has passed into history, and history needs the people who were in the room. Her brother has written his memoir. Ken Wharfe has written his, with me. Patrick Jephson, her private secretary, and Paul Burrell, her butler, have written theirs. The historian Dr David Starkey has made the point that first-hand witnesses are what historians will rely on.

Spencer told New York his sister would have been ‘happy’ had she never been a princess.

By implication, he was suggesting that if she hadn’t caught Charles’s eye, her life would have been far better.

Maybe. He was living in Cape Town when she died.

Sometimes she was happy anyway, despite it all.

Ken and I and many others close to her heard that infectious laugh often.

Robert Jobson is a No. 1 New York Times bestselling author. His latest book is The Windsor Legacy, published by Pegasus (USA) and Blink (UK).