The entrance to this public dungeon is located a bit further in the mountain. From the south, there is a small passage, under the stone boulder. Follow this path, find the stone stairs and follow them up the hill. In general just follow the mountain line to your left.
Once inside you need to speak to Nashruth. He will give you the quest called “Flames of Forge and Fallen”. This quest is essential for going through one of the dungeon’s door. You can’t progress without it, and eventually can’t reach the skyshard’s location.
Once you take the quest and complete its first step, you’ll get the next step “Honor Orsinium’s Founding Clans”. This is where the tricky part comes. Once you honor the fallen, you’ll need to use the specific signed pressure plates to unlock the door and continue further. Small tip for solving (if you don’t want to read on further to avoid spoilers) is: every time you use the correct sign, it will raise a small door weight on your left. You can just keep finding the right sequence, once it is risen four times. The sign order, that unlocked the Hammer’s Bypass for me is: Tumnosh, Morkul, Shatul and Igrun.
At last you’ve unlocked the door and you are in the area. Keep to your right all the time. There is another entrance you have to go through. Luckily for us, it doesn’t require any unlocking, you can just go through it. This is the most eastern part of the dungeon. From its entrance, on the first crossroad turn right. In the next larger cave, you’ll find the Old Orsinium Skyshard.
Old Orsinium entrance location.
Accept the quest.
Solve the puzzle to open the door.