If you want to make planet orbit in retrograde, you can simply put a negative sign ( - ) in front of the planet's velocity and that should do it.
If you're wanting to make a moon orbit in retrograde, that's a bit more tricky. There should be a value called "relative velocity" or something like that, by the top of the second section of the properties panel. Putting a negative sign there could work, or you may need to press the edit object button by the add button, and drag the moon around until it orbits like you want, or set the inclination high enough that orbits backwards. It's pretty hard to make a moon do this, some trial and error will probably be needed.