If you go by the gazetteer that Sherlock Holmes pulled down from the shelf in The Sign of the Four, you'd have to go to the Andaman Islands to find an actual shark in Holmes-world. But London was not without its human sharks, as with any metropolis, and foremost among them might seem to have been Moriarty, of course. But Holmes only used that parallel to explain Porlock's relationship to Moriarty, in The Valley of Fear, also citing the lion and the jackal.
The real shark of the Sherlockian Canon, the man whom Holmes can't help but compare to that killer of the sea a full four times in one short story, is the man they call Count Negretto Sylvius.
They call him "Count Negretto Sylvius," of course, because he seems a lot like Colonel Sebastian Moran with a pseudonym. And like Moran, he's a killer that Sherlock Holmes actually appears to be afraid of. In Moran's case, Holmes seemed to stay away from London for years to avoid the Colonel's vengeance. In the matter of Silvius, Holmes is just giving Watson the Count's address so the good doctor can send the police there should Holmes get killed by Silvius . . . a route Holmes takes on no other occasion.
If Sylvius and Moran are truly one in the same, which seems likely as they look alike, share the same talents, and seem to have the same habits, this opens up several possibilities. Did Holmes encounter Moran once when the Colonel was running a scheme under a false identity, prior to seeing the whole Moriartian big picture? Or was the name change a sign that the anonymous third-person writer of "The Mazarin Stone" still felt a great fear of Moran, even decades later? (And who was in a position to both write that tale and be very afraid of the very thought of Moran so long after? Billy the page, definitely.)
It's hard to imagine that the police grabbing Silvius/Moran at the climax of "The Mazarin Stone" went off in as civilized a manner as that recounting seems to tell, and one has to wonder if "Silvius" escaped to vex Holmes another day in another country, then returned to London when it was clear Holmes was wandering. His capture at the climax of "The Empty House" seems much more appropriate for the killer shark Holmes knew him to be.
Whoever "Count Negretto Sylvius" actually was, he was definitely the one true shark in Sherlock Holmes's rogues gallery. And one worthy of his own Asylum Films movie with Sherlock as well.
"I rather want to see my shark without his seeing me . . ."
-- Wise words from Sherlock Holmes