Those who witnessed the incident called triple zero but lost sight of the man in the sea before emergency crews arrived.
A police helicopter and dinghies were used to try and find the man on Sunday afternoon.
Green dye was put in the water to try and determine the direction of the current.
Paramedics pulled the 45-year-old father from the water and performed CPR, however he could not be revived and was declared dead at the scene.
His son, a 20-year-old man, was also pulled from the water and died at the scene.
Simone is a breaking news reporter for The Age. Most recently she covered breaking news for The Australian in Melbourne.