The SR5 is plenty capable and the base 4wd system is excellent. ATRAC has been integrated into the 4wd low and really is perfectly capable for dirt/gravel roads/Forest service trails etc.
That being said, If your budget allows for it, I would go with the TRD off-road. The big advantage is it...