Entrance to Crypt of the Exiles is inside this stone structure build into a cliff.
Crypt of the Exiles entrance map location
Shard is in the large cave at the end of the dungeon. Take a left turn as you enter the cave. On the screenshot above you can clearly see that the collectible is to the left in a dead end part of the room.