Thanks for the help, in the end succeeded.
Here are the step which worked for me (might not be the best):
The console is held by 5 screws.
First remove the wood trim around the radio / AC: remove the two small screw holding the wood trim, pull out the button bar, tilt out the top of the wood trim and pull it up.
This reveals the first screw
Next remove the ash tray and the box under the arm rest, both have to be pulled up. The box might be harder to remove, use a plastic scraper or similar (I've used a pancake flipper) and pull on the lip, while pushing against from the inside with the other hand.
Two screws under the box
Two more screws at the front where it joins the console of the radio / AC. To access them the side carpets might have to be removed.
Then remove the gear lever knob, 3 hex screws
And the parking brake knob, 1 hex screw
Now comes the hard part. I couldn't remove the console, since on the front the radio / AC console was in the way, and in the back the parking brake was blocking the way. After wasting a lot of time in the end I unscrewed the two screws holding the brake which gave enough wiggle room to free the console.
Disconnecting the cables is easier if the panel around is separated. For this bend the biggest metal tab found at the rear part of the panel.
The Koni selector is tricky, utilizes some custom made **** solution to hold it in place.
The result
Console and arm rest now off for re-trimming. Only the two skirts and the arm rest were in a bad shape, but I though let's do the whole then.