It's supposed to.  The owner's manual is specific about this feature.
The idea is that whenever you need the windshield cleared, the air
conditioner will come on to help get the job done.  The A/C not only
provides cool air when it's hot outside, but it also dehumidifes the
air.  Dehumidified air is much less likely to cause the windshield to
fog up.  Many cars have this feature without their drivers even knowing
about it.  Some people even contend that the primary purpose of the
A/C is to dehumidify the air, and cooling is a happy side effect.  In
any case, if the conditioned air is too cold for you, just add a little
heat by sliding the temperature lever to the right a bit.

That said, use of the A/C does cause a reduction in both available
power and gas mileage.  Most estimates I've seen are in the 10% 
range, but this kind of drop is especially noticable in a vehicle 
like the RAV4, with a small engine and relatively low horsepower.  
Some drivers would prefer to have complete control over
whether or when the A/C comes on because of the penalties.  The
easiest way is to disable the A/C in defrost mode is to move the
control lever all the way to the right (full defrost), then move it
slightly to the left until the A/C turns off.

Update: This "feature" was removed in the middle of the 1999 model
year.  In the newest RAV4s, the A/C no longer comes on when the
defroster is engaged.

A more permanent way to disable the automatic A/C feature 
requires a minor modification to the vehicle.  

According to Mark in Vegas <ranyhyn@wizard.com>:

I just did it, probably took about 10 minutes.  If your Rav is the 
same as my '97, it's pretty straightforward.  If you aren't 
mechanically inclined, maybe have someone at a stereo shop 
do it for you.  Start with removing the fascade for the stereo, 
remove the two phillips screws on the bottom of the fascade, 
there are two tabs on the top that will come loose once you swing
the bottom of the radio fascade out.  Then there are 4 phillips 
screws holding the radio in place, two on the sides and two on 
the bottom.  Remove those and pull the radio out, if yours is like 
mine, it should pull out enough so that you can get your hands 
in the opening and not have to pull any radio wires loose.  
Anyway, take a look in the opening and you'll see the bottom 
of the temperature control unit, on the back of this, towards
the center pointing down is a white plastic plug with a yellow 
wire and a black wire with a blue stripe.  There is a little tab 
on the plug that you can push in and the plug will pull out (down).  
Do this and voila, you can now control the A/C manually when 
in defrost.  If you want the A/C on to defrost the windshield, 
push the A/C button, if not, don't push it.  

A comment from another reader:

The wire colors have changed on my '99.  Instead of yellow and 
black/blue to the plug in the procedure, my colors were yellow 
and white with a blue tracer.