To further complicate things, the CTA isn't a legal Act or Treaty, it's more an informal arrangement. To withdraw it would cause outrage, as it's estimated that there's up to 40% crossover wrt to British/Irish family relationships. I can't imagine the EU has even considered this, given it's not in any statute book, and most likely why UK and Ireland joined the EU at exactly the same time (thanks TridentTested).
To further complicate things there are actually more British living in Ireland, than Irish living in Britain - when you look at it in percentage terms. So anything which hurts the CTA will hurt a greater number of Irish living in Britain but will also hurt a higher percentage of British living in Ireland.
The CTA came into existence because one hundred years ago (exactly) Ireland was separating from England and the negotiators at the time already understood that the citizens of both countries were complicatedly intertwined. To have insisted on visas and residency permits would have been chaotic and cut deeply across both countries. Don't forget a large proportion of the British in Ireland at the time were the land-owning class, they didn't want this disruption to their lives; they had influence in Westminster and shaped the settlement.
It seems to me that this successful example - and let's face it, it has been successful, it endured even through the worst years of the civil war we call the Troubles - should have been followed for the EU26 (27 minus Ireland) rights. It was promised by the Leave campaign that their rights "would continue exactly as before". This hasn't happened, and is shameful. A hundred years ago Ireland and England were locked in violence and we still managed to reach an honourable settlement. It's such a shame this lesson has been forgotten.
As to what happens to the CTA now. Who knows. Imagine you are doing business in Belfast, if the person you are contracting with identifies as Irish they have the right to the ECJ, if they identify as British they don't. It is going to be a legal mess for decades. If we have any international law barristers on the forum, time to start specialising in this area, it is going to be very lucrative.
Ending red tape? - kiss my royal Irish ar5e. (Kiss my RIA was a common expression in Dublin a hundred years ago).