I personally like standing by the 3rd fence as the race begins, then after the horses jump it for the 2nd time getting back over to the finish line. I advise binoculars! I’ve also stood next to the first/last fence and liked that as you have a good finish line spot all staked out.
The whole valley is beautiful and you will want to spend some time driving around. There are 2 tack shops (MD Saddlery and Dover). Ladew Topiary Gardens is a cool and the whole Manor area is worth a drive as well. Gunpowder Falls state park has a number of pretty trails along the river, though unmarked. There is hiking around the reservoirs too (Prettyboy and Loch Raven) but again unmarked, I sort of pick a parking area and see what I get.
Baltimore has 2 great art museums (Baltimore Art and Walters). Rocks State Park is small but has two nice features in the waterfall and the King and Queen Seat rock formation. If you don’t want to go as far as DC, Annapolis is charming as well.