There are a few nuances here, my friend
Of the almost half a million registered refugees from the former J&K, only about 30,000 are from the Kashmir Valley. Around 450,000 are from Jammu Division, most of whom settled in 3 or 4 of districts in central Punjab following partition.... While they are still locally referred to as Riyasati, culturally and socially they are as Punjabi as they come, with no connection to Kashmir...
Yet they continue to elect representatives for Azad Kashmir and help decide the political fate of a land many have never even seen... !!
Also, if GB can be administratively separated from J&K and its people given the right to choose their own representatives, why should Kashmiris living in Azad Kashmir be denied the same principle?
And if preserving the political rights of Jammu refugees is the concern, then create a separate representative body for them in Islamabad or Lahore... Why should the politics of Azad Kashmir be shaped by people who neither live there nor have any meaningful stake in its affairs?