He greeted survivors, fire and police workers, and relatives of some of the 2,749 people who died there.
The Pope prayed for the rescuers and victims, as well as "those whose hearts and minds are consumed with hatred".
He later celebrated Mass at New York's Yankee stadium, concluding what analysts describe as a successful trip.
His visit has dominated the American media, and demand for the 55,000 tickets for Sunday's Mass far outstripped supply, reported the BBC's David Willey in New York.
There was a party atmosphere among the tens of thousands gathered at the Yankee Stadium, with clergy and lay members of the Church alike taking part in the Mexican waves which rippled through the crowd ahead of the pontiff's arrival.
Congregants stood and clapped as the song Lean on Me was performed by guitarist Jose Feliciano on the white, purple and yellow platform in the middle of the baseball pitch.
As the Pope arrived the crowds cheered and waved yellow and white handkerchiefs, the official papal colors.
"The visit by his Holiness Pope Benedict XVI is wonderful for New York, our nation and indeed the world," George Steinbrenner, the owner of the Yankees baseball team, said in
Earlier, after arriving in the popemobile at Ground Zero, Pope Benedict knelt in silent prayer, and rose to light a memorial candle, and blessed with holy water what he called "the scene of incredible violence and pain".
He requested "eternal light and peace" for those who died, not only in New York but at the Pentagon in Washington DC and in a Pennsylvania field on 11 September 2001.
"God of peace, bring your peace to our violent world," he said. "Turn to your way of love those who’s hearts and minds are consumed with hatred."
The Pope then met 24 people with ties to the tragedy, exchanging a few words with each.
In a personal address on Saturday, Benedict told youths at a New York rally about growing up under the "monster" of Nazism.
With a candor that correspondents say has been a hallmark of this visit, he spoke publicly for the first time about being forced to join the Hitler Youth and being conscripted into the Nazi army.
The Pope has also repeatedly condemned pedophile Catholic priests. More than 4,000 US Catholic clergy have been accused of sexually abusing minors since 1950.
The Church has paid out more than $2bn (£1bn) in compensation and legal fees, most of it since the scandal erupted in 2002.

Source: BBC