That won't work as an inverter, and the switch will only kill the signal. For a op amp buffer to be inverting, you need to use the inverted (-) input, with an input resistor and a feedback resistor of the same value.
R1 and R2 can both be 10K.
You could have a non-inverting buffer (as the one you posted) and a inverting buffer in series (from the same dual op amp), and simply have the switch select output from one or the other
I hope that helps
/ Fredrik
check out my building blog at www.parasitstudio.se