I wouldn't recommend it. Battery cable insulation is tougher, more abrasion resistant, higher temp, and usually has better solvent resistance to oil, etc.
Make sure you put a fuse or breaker at the battery end, so if the cable gets cut, chafed, pinched, whatever, you don't have full unprotected battery amperage ready to catch the cable on fire and burn your car to the ground. You may laugh, but you can never tell what's going to happen when you get into an accident and sheet metal starts moving around.

The alternator end isn't as important because the alternator can't supply anywhere near the current that a battery can, and if you really load the alternator it will usually stall the engine at light throttle. No engine, no alternator output.

4 gauge is fine for a CRX; they're short enough and Honda starters are small. 2 gauge is fine too, but not really necessary. Lots of relocation kits come with 2 gauge, so if you get a good deal go for it, but if you get everything separately you'll enjoy running 4 gauge much more (it's smaller).
There isn't any need to go larger unless you turn it into a limo or put a V8 in it. Lots of people are going to tell you to use 00 or 1/0 gauge, but you shouldn't listen to them.