A short drive and hike east of Copper Harbor is Horseshoe Harbor, a rock beach-turned-bay by prominent outcrops of conglomerate rock. The beach area is beautiful in its own right, but walking along the conglomerate mounds offers a fantastic view of Lake Superior, and if you're lucky, the Northern Lights. The south side (near the beach) of this outcrop offers great wind protection and easy access to firewood makes this location a fantastic camping spot, particularly if you're not equipped to hike miles to your campsite.
Continuing west down the outcrop, you'll find a couple crevasses with water rushing underneath that you'll need to jump over. Once you get towards the end of the outcrop, you can crawl down the rock to a small cove area. If you're into rocks, there are some extremely old stromatolite fossils (over 1 billion years old) that can be found near the base of the conglomerate outcrop here. Climbing and following the next outcrop to the south will take you to another beach area that makes for a great campsite as well, though with a little less wind protection.
You can actually follow the outcrops pretty far out into the lake, especially if you don't mind swimming. These also make for some pretty awesome sunset pictures. There is a trail behind the rocks that goes from the beach to the southern outcrop if you plan on camping the second campsite. At the western end of the first outcrop, you might be able to do some cliff jumping but I wouldn't recommend it. Currents can be bad and the water depth is questionable. In my experience, you can almost always see a faint glow on a clear night between 11:00 to 2:00 am here, even if the Northern Lights aren't supposed to be out.
To get here from Houghton, head north across the lift bridge and take US-41 north through Copper Harbor. Continue until you reach the turn around/parking area where US-41 ends and continue down Mandan Road (dirt road). Drive for 0.9 miles and take a left onto Highrock Bay road. Follow this road for about 1.2 miles and you will see areas to park on your left. The trailhead is just a quick walk down the road to your right. Follow the trail and in about 1/4 mile you'll reach the main beach.
Thanks to Andrew Dubay for his photo!