(CNN) — After seeing his three-decade rule come to an end, Yemen’s former president, Ali Abdullah Saleh, will leave the country for Ethiopia this week, ruling party officials said Monday.

Saleh returned to Yemen just days ago. However, he was under increased pressure to leave, as Yemenis worry his presence will undermine efforts of the new president, Abdurabu Mansur Hadi.

Saleh and Hadi appeared at the presidential palace Monday for a former handover of power ceremony amid cautious optimism and ongoing threats of violence in the country.

Hadi, who served as Saleh’s vice president, was elected to the post last week and was sworn in Saturday in the capital, Sanaa.

The event cemented a power transfer deal reached in November to end months of protests and violence over Saleh’s longtime rule. Before Hadi became acting president in November, Yemen had been engulfed in anti-Saleh protests. The agreement was brokered by the Gulf Cooperation Council and backed by the United States.

Officials with Hadi’s office confirmed Saleh will depart Yemen this week, but would not divulge his destination.

There have been discussions about Saleh settling in either Oman or Ethiopia, but the former president has not made a decision, according to a Yemeni government official who asked for anonymity as the official is not authorized to speak to the media.

Saleh expressed his support for Hadi at the ceremony Monday. “I would like to congratulate my brother and colleague, His Excellency President Abdurabbu Mansur Hadi, for being elected as president of the republic.”

Hadi spoke next, saying, “Today, we welcome and bid farewell. … Welcome a new leadership and we bid farewell to the leadership. … This means that we lay new rules for the exchange of peaceful transfer of power in Yemen, because security and stability is the basis of development.”

Acknowledging the numerous challenges faced by Yemen, Hadi added, “We are in front of a difficult and complex time and the Yemeni people, who in millions participated in the early elections, gave a clear message for standing with security and stability in order for better change.”

At the end of the ceremony, Saleh handed a Yemeni flag to Hadi.

While the handover was taking place, and in a sign of just how much anger is still directed at Saleh, more than 1,000 protesters marched from Change Square toward Hadi’s house, demanding Saleh’s prosecution and condemning Hadi’s appearance with the former president.

On Saturday, Hadi promised to continue the fight against al Qaeda, calling it “a religious and national duty” for citizens of his nation.

But as he was being sworn in, a deadly car bomb killed more than 20 security force members in another part of the country.

The explosion occurred near the presidential palace in Mukalla, the provincial seat of Hadramout province, a senior security official and an eyewitness said.

The Mukalla palace is one of several presidential palaces throughout the country. Hadramout province is in the country’s east; Sanaa is in the west.

“Yemen is a country where such attacks take place, so this is not a surprise,” said Ali Saeed Obaid, a government official.

No one immediately claimed responsibility.

Speaking about the fight against al Qaeda, Hadi said he hoped an effort to eradicate militants would lead to the return of displaced persons to their towns and villages.

“The country does not need additional crises to deepen its wounds, as the next period requires serious dialogue and clear outlines through a new constitution that meets the national expectations,” Hadi said.

U.S. President Barack Obama called Hadi to congratulate him and to say that the United States “will stand with the people of Yemen as they continue their efforts to forge a brighter future for their country,” according to a White House statement.

“Under President Hadi’s leadership, Yemen has the potential to serve as a model for how peaceful transitions can occur when people resist violence and unite under a common cause,” Obama said, noting that much work still lies ahead.

Hadi received 99.8% of the 6.6 million votes cast in Tuesday’s election, according to Mohmmad Hassan al-Hakimi, chairman of the Supreme Committee for Elections and Referendum.

Saleh, who led Yemen for 33 years, was wounded in a June assassination attempt at his presidential palace during battles between government troops and tribal fighters.

“Saleh has the option to continue involvement in politics, and the power transfer deal will not force him to step aside,” said Abdu Ganadi, his senior aide. “He is the leader of the GPC, and his voice and support will continue being heard in the GPC.”

But the U.N. envoy to Yemen, Jamal bin Omar, said the Arab nation “is turning the page.”

He said a coming “national dialogue conference” will be an “opportunity to draft a new constitution that will be a new social pact among the Yemenis.”

Human Rights Watch has called on Hadi to make changes without delay.

“Yemen’s potentially historic transition will be off to a shaky start unless Hadi makes an immediate break with the abuses of the past,” said Letta Tayler, the group’s Yemen researcher. “Yemen’s new leader needs to move decisively to usher in promised reforms that uphold human rights and the rule of law.”

The 65-year-old Hadi is a British-, Egyptian- and Soviet-trained army officer, recently promoted to field marshal. He had been vice president since 1994 and ran for a two-year term as president on pledges of improving security and creating jobs.

But he’s never had much of a power base, and Yemen’s problems are expected to take longer than two years to fix.

Yemen is the poorest country in the Middle East, with a severe shortage of water and rising levels of malnutrition among its population of about 25 million.

Saleh faced a separatist movement in the south, sectarian tensions in the north and the growing presence of what Western officials describe as al Qaeda’s most dangerous affiliate, al Qaeda in the Arabian Peninsula.

Click on this link to Watch Video: CNN