The Kremlin is asking for clarification from Armenia that it had frozen its participation in the Collective Security Treaty Organization (CSTO), a group of post-Soviet states considered to be Moscow's answer to NATO.

A military alliance formed in 2022, the CSTO also consists of Belarus, Kazakhstan, Kyrgyzstan and Tajikistan and, akin to Article 5 of NATO's North Atlantic Treaty, an act of aggression against one signatory is considered an attack on all members.

Armenian Prime Minister Nikol Pashinyan said on Thursday that the pact had failed the country, after months of expressing discontent over its longstanding ties with Moscow, which he said Armenia can no longer rely on for its defense needs.

He told France 24 television that the CSTO bloc "has not fulfilled its objectives as far as Armenia is concerned, particularly in 2021 and 2022, and we could not let that happen without taking notice."

This was reiterated by Andranik Kocharyan, an MP from the ruling Civil Contract Faction and chair of the government's defense and security committee, who told reporters on Monday, "the word 'frozen' means that it is frozen," Armenian outlet News AM reported.

Kremlin spokesman Dmitry Peskov said on Monday that there has been no "clarity" about Armenia's position on leaving the bloc, according to state media. "We have frequent contacts with our Armenian partners in this area. Of course, we will clarify," Peskov said, "we will find out."

Speaking to Newsweek from Yerevan, Olesya Vartanyan, the International Crisis Group's (ICG) senior analyst for the South Caucasus region, said as yet there is no formal procedure under way for Armenia to exit the bloc.

"It's clear that they have been discussing it for quite some time, including with their western partners," she said. "The list of grievances on the Armenian side has been growing over the last couple of years.

"Since Russia started its invasion of Ukraine, Armenia stopped profiting from the CSTO because before they used to get discounts for weaponry that they were receiving from Russia, and Russia doesn't have any weapons to sell anymore.

"Russia was not able to prevent continuous attacks from the Azerbaijani side of the border and Russia basically telling everyone that it's busy in Ukraine."

Despite the region being an area of Russian influence, Moscow has been unable to defuse tensions between Armenia and its neighbor Azerbaijan.

Last September, Baku launched an offensive in the breakaway Nagorno-Karabakh region, recognized internationally as part of Azerbaijan which was home to a majority ethnic Armenian population.

A swift Azerbaijani victory was followed by the exodus of almost all the Armenian population and the dissolution of what was known in Armenian as the Republic of Artsakh. Armenia has raised concerns about Russia not intervening in the conflict and Baku was accused of ethnic cleansing

But Vartanyan said this was not among the main grievances that Yerevan had with Moscow. "Armenia is still having enormous security issues with Azerbaijan," said Vartanyan, which could see it "attack and cut the country into two."

Even with ever-dwindling military benefits from its membership of the alliance, leaving the CSTO would be a big wrench for Armenia politically and economically.

"If Armenia leaves or freezes its membership with CSTO, it will not automatically affect the economic alliances," Vartanyan said, because Yerevan has a separate deal with Moscow through the Customs Union of the Eurasian Economic Union.

"Armenia's economy is very much dependent on Russia and it will take the country a lot of time to diversify its markets," she said.

Newsweek has contacted the Armenian and Russian foreign ministries for comment.

Update 02/26/24, 7 a.m. ET: This article has been updated with further information and comment from Olesya Vartanyan.