Upon further research, it appears the first section I described is actually IN Waples Mill Rd. Park. I've never actually been in the park, but maybe the buildings were park facilities? I don't think they're there anymore.
https://www.openstreetmap.org/way/121176222/history#map=17/38.87341/-77.33714
I still don't know about the part next to Route 66. I think that part had graffiti on it last I saw it.