Public Holiday falling on Saturday gets no replacement. The following Section 60 is quite clear about it. It says only the last of the two days is treated as rest day:
"...59. Rest days.
(1) Every employee shall be allowed in each week a rest day of one whole day as may be determined from time to time by the employer, and where an employee is allowed more than one rest day in a week the last of the rest days shall be the rest day for the purposes of this Part:..."
Anywhere, if you following the Government service, you wouldn't go wrong. When PH falls on Saturday, government servants don't get a holiday on the following Monday.