I don't have the patience to wade all the way into Mossel Bay during the holiday season, so why not hit Catchalot which is right next door to the bike shop at the same time? I was there shortly after 8:00 a.m. and no one else was there - perfect! Incidentally, they now have a location on York Street (at Wellington) in George.
If do want to join the crowds, then of course, there's also "the original", Viking Fisheries on Bland Street in Mossel Bay proper... see these older posts from 2013 and 2015 (of course I go more regularly than that!). But Catchalot is giving them an honest run for their money in my books - both are great options for the freshest fish you'll find anywhere other than on the end of your own line!