Your assumption is correct. Lever off the insert with a plastic spatula or similar, and you will find 3 screws underneath. Lots of wiggling/flexing to get the panel free where it fits under the B pillar trim. You may need to enlarge the hole in the metalwork slightly as there are not many speakers which fit the holes as is, and even less which are much good. Stick some dynomat on the inside of the wing and fill the cavity with fibreglass insulation for best damping of resonances.