Mattis, who was joined by Secretary of State Mike Pompeo at the private session, said in remarks released by the Defense Department that overall USA national security interests must take precedence over Saudi culpability for the murder of Khashoggi, a Washington Post contributor who disappeared after visiting the Saudi consulate in Istanbul last month.
Graham spoke after senators were briefed by Secretary of State Mike Pompeo and Defense Secretary Jim Mattis.
Pompeo told reporters after the briefing that there was no direct evidence connecting Crown Prince Mohammed to Khashoggi's murder.
Meanwhile, Pompeo told senators that America's national security interests are at stake as they consider a vote to halt US involvement in the Saudi-led war in Yemen.
In his remarks, which were released as the meeting with lawmakers began, Mattis said pulling back USA military support in Yemen and stopping weapons sales to important partners would be misguided.
The briefing could determine how far Congress goes in punishing the longtime US ally in the Mideast. A vote could happen later Wednesday but was not certain.
Senate Majority Leader Mitch McConnell says "some kind of response" is needed from the United States for the Saudis' role in the gruesome death.
Last week, the Central Intelligence Agency issued an assessment concluding with "high confidence" that Saudi Crown Prince Mohammed Bin Salman was involved in the plot to against Khashoggi.
Two-thirds of Yemen's 27 million population rely on aid and more than 8 million are at risk of starvation. Senators voted 63-37 to advance the measure.
Saudi crown prince arrives in Argentina for G20 amid Khashoggi murder furor
"Our security interests can not be dismissed, even as we seek accountability for what President [Donald] Trump described as the "unacceptable and awful crime" of Jamal Khashoggi's murder, a crime which "our country does not condone", Mattis said in his prepared remarks.
"It is outrageous that we are willing to turn our eye away from such a murder because we have 'interests, '" he said.
Bill and Hillary Clinton kicked off their speaking tour in Toronto on Tuesday and took plenty of shots at President Trump, including saying he was "part of the cover up" in the murder of Saudi journalist Jamal Khashoggi. "But degrading U.S. -Saudi ties would be a grave mistake for the national security of the U.S. and its allies".
Independent senator Bernie Sanders, who sponsored the Yemen resolution, called Saudi Arabia's government a "despotic, dishonest dictatorship" and said the situation in Yemen is the "worst humanitarian crisis on earth". Chris Murphy (D-Conn.) and Mike Lee (R-Utah) that demands the policy be ended within 30 days. It drew a mix of Democrats and Republicans who have grown uneasy with USA involvement in the Saudi-led campaign against the Houthis in a war that human rights advocates say is subjecting civilians to indiscriminate bombing and wreaking havoc on the country.
Several senators said they were disappointed that CIA Director Gina Haspel was not present during Wednesday's briefing.
Democrats blasted her absence as a "cover-up", with Sen. "The whole thing with Khashoggi is very much concerning".
Democratic Senator Bob Menendez said Washington was basically telling an ally "you can kill with impunity".
The BBC's Barbara Plett Usher in Washington says opposition to the war in Yemen has gained momentum because of outrage at the administration's response to the killing and many in Congress want a tougher stance from the White House.More news: Google Pixel 3 Lite leaks once again, poses for the camera
More news: Baker Mayfield has career day for Cleveland Browns, highlights, Twitter reacts
More news: Trump Fires Barrage of Angry Tweets at Mueller: ‘Conflicted Prosecutor Gone Rogue’