If I create a fleet and want it to STAY within a star system, exactly how would I go about setting that up?
Assign the fleet a home base within the system, put the fleet into Defensive Posture, set the engagement range to 'system' or 'home base,' and turn the fleet's automation on. Make sure that under Empire Policies you have Fleet Formation on manual, or the computer will take your automated fleets and do whatever it wants with them.
Assign the fleet a home base within the system and automate the fleet, and while the fleet may not stay within the system it will at least return there when idle, as long as you have set the Fleet Formation policy to manual.
Alternatively, you can just turn off the fleet's automation and it should stay near the last location to which you ordered the fleet, though ships of the fleet may jump a short distance into deep space in pursuit of fleeing hostiles. It may still be useful to give the fleet a home base in the area in which you want the fleet to stay.
Also, which is better? Robotic Troops or Clone Troops?
Depends on what you want. Robotic Troops have very low unit upkeep (you can maintain four units of them for the same cost as a single regular infantry unit of most species), and have fairly decent combat strength for the upkeep despite having low combat strength per unit. They make acceptable garrison forces, and can be used as filler units to draw fire for more valuable units during invasions. Clone troops are as good as your best regular infantry units, so if you have highly-experienced infantry you can use clones to make a large, elite army, but if you don't have highly-experienced infantry they aren't much better than normal recruits.
Note that Robotic Troops cannot gain experience and do not benefit from the attack and defense bonuses you can give to regular infantry, armored, planetary defense, and special forces units, but do benefit from the reduced unit upkeep technologies. The robotic troops will still retain a slight advantage in strength per upkeep compared to the basic infantry of most species even with all the attack and defense bonuses researched, but the difference becomes much smaller and the discrepancy in strength per unit becomes much larger; if using the advanced infantry which can be recruited at an Academy or using Clone Troopers, robotic troops may well lose the advantage in strength per upkeep by the late game (and potentially even earlier against clone troops, or if you can recruit troops at the Fortress of Torak) but will nevertheless still offer a very low upkeep per unit.