Hatem Ben Arfa’s late penalty earned the points after Jermain Defoe replied to Demba Ba’s curling strike.
But Pardew was sent to the stands for pushing linesman Peter Kirkup while complaining about the ball going out for a throw-in during a Spurs attack.
“We’re overjoyed about the result but I hold my head in shame,” he said.
“I was just saying to the players [beforehand] about the great Olympics and how to set an example and there’s me pushing the linesman over. It was comical, he just happened to be right in front of me.
“I went in and said I’m sorry [to the referees] and said I would have to apologise publicly too because it was ridiculous.”
Should the incident be mentioned in the match report by referee Martin Atkinson, Pardew could face a fine or a ban.
The goal for Ba was his first in 15 games, and showed evidence that Newcastle can again challenge at the top of the Premier League table after a fine fifth-place finish last season.
And Pardew gave credit to his attackers as they continued where they left off from last term.
“It was a great result,” Pardew told BBC Sport. “It was a tight game tactically. We kept our two strikers on the pitch and luckily we did because it was a real striker’s goal [by Ba] and that gave us the edge. I thought we just shaded it and the crowd helped us, they were brilliant.
“Spurs dictated the play in the first half and we changed it at half-time and that helped us. We had two or three players who were carrying knocks and we didn’t have our full capacity in terms of our energy. But we stuck at it, were diligent and we have goalscorers – and that makes a difference at this level.
“Hatem has got fantastic ability, as we all know, and he will create moments for us but it was the other side of his game that was impressive today, he defended really well and kept his shape.”
Tottenham boss Andre Villas-Boas said he was impressed by his team’s display but that Newcastle were more “efficient”.
He also praised Defoe, and stated his desire to add more strikers to his squad before the end of the transfer window.
“It was a good performance from the team,” he told BBC Sport. “In the first half we had the larger share of opportunities and created lots of problems and looked to be the first team to score.
“The split second that Demba Ba escaped us, he managed to score, and we were running against the result for the rest of the second half and when we had the momentum we conceded a penalty which blocked us a little bit.
“There were good positive signs that we came to Newcastle, a difficult ground to play and against a team that did so well last season, and we showed that we are the better team.
“But they came out with the three points and we wanted to obviously avoid that but I have to praise my team for what they did.”