A Sussex spokesman confirmed to ABC News that the meeting had taken place, although Buckingham Palace had not.
It is believed the Sussexes arrived in the UK earlier this week, and may also meet Prince Charles and Duchess Camilla.
Harry and Meghan, who sensationally stepped down from royal duties in January 2020, will travel to the Netherlands this weekend to kick off the Invictus Games.
Harry is currently seeking a judicial review of the Home Office’s decision not to allow him to pay for police protection while on British soil.
This legal battle stopped Harry from visiting Britain’s Prince Philip memorial service on March 29.
A former spokeswoman for the Queen, Dickie Arbiter, said the meeting may represent an “olive branch”.
“It does raise questions, however, that Harry stopped by because he was on his way to the Netherlands for the Invictus Games,” the Arbiter told Today.
“If he wasn’t on his way to Holland for the Invictus Games, would he have visited? We don’t know.”
Queen’s special flood message
Mr Arbiter also said there would be a long way to go before the Sussexes were welcomed back into the kingdom.
“We can’t forget the interview they had with Oprah Winfrey where, frankly, both insulted the Royal Family, insulted the Queen, insulted the Prince of Wales,” she said.
“And it really hurts them.”