Speaking to media in Perth today, Albanese said Australia needed to revitalise its diplomatic relations with Indonesia.
"We have strategic competition in the region," he said.
"What we need to do is to make sure that we have competition, recognise that it's there without catastrophe.
"And I certainly seek peaceful relations with all of our neighbours."
Albanese said he wanted to enhance ties with the South-East Asian ally and said it was a priority of his government's renewed focus on foreign policy in the region.
He said his early discussions with Indonesia's President Joko Widodo have been positive.
READ MORE: Journalist detained in China 'denied calls'
"My discussions with the President Widodo have been very cordial and positive. We had a good discussion on Friday. We've met, of course, in the past. I look forward to the next couple of days," he said.
"I thank him for going out of his way, and his government, to welcome us so soon in the government's period in office.
"They have gone out of their way to put this in place. I look forward to our bilateral leaders' meeting tomorrow."
The prime minister said he would also be having discussions with the secretary-general of Association of Southeast Asian Nations (ASEAN).
Labor cabinet minister makes history, with a small pink book
He said he was determined to have better relations across the Indo-Pacific region.
"Those discussions reflect the priority that we have on South-East Asia," he said.
"We announced during the election campaign additional aid for South-East Asia and we also announced a particular envoy and other measures to assist our relations.
"From very early on, we have done two visits from Foreign Minister Wong into the Pacific.
"That's why I attended the Quad leaders' meeting on the day that we were sworn in as prime minister and foreign minister, but also why this early visit with a very high-level delegation from Australia indicates to our Indonesian friends the importance that we place on that relationship."