Still walking - trying to find the shrine

 I looked around the graveyard a bit more and then started back down the hill.

 Part way down, I found a stairway leading down into the valley. It was very well tended and the buildings at the foot of the stairs looked promising as a possible site for the shrine.

 At the bottom of the stairs, I found these irises and an extremely well kept landscape.

 At one point I thought that this might be the back of a shrine but it turned out to be a private home.

 On closer inspection, I discovered that I was wandering around in someone's private yard. There was not a shrine in sight. I quickly walked down to the nearest street, what I was on was a wide driveway. Just before reaching the street I saw this rather picturesque house.

After wandering around the area for another 15 or 20 minutes, during which I did not see any people, I finally found a map of the area. The shrine was marked on the map and was actually just a little ways behind me.