According to World Pool-Billiard Association rules (under "Definitions" heading), if the cue ball touches an already pocketed ball, the cue ball will be considered "pocketed", even if it rebounds back onto the pool table's surface. This applies to all games covered by the WPA.
For your scenario, that means that you have scratched while pocketing the 8 ball, which is a "serious foul" resulting in loss of game.
This definition also tells us that it is the shooter's fault if too many pocketed balls in the pocket causes the pocket to reject your shot. You could have removed balls. Your opponent also could have removed balls. You failing to do in this scenario came back to haunt you.
Here's how the World Pool-Billiard Association rules apply to your scenario:
8.3 Ball Pocketed
A ball is pocketed if it comes to rest in a pocket below the playing surface or enters the ball return system. A ball near the brink of a pocket partly supported by another ball is considered pocketed if removal of the supporting ball would cause the ball to fall into the pocket.
If a ball stops near the edge of a pocket, and remains apparently motionless for five seconds, it is not considered pocketed if it later falls into the pocket by itself. See 1.7 Balls Settling for other details. During that five second period, the referee should ensure that no other shot is taken.
An object ball that rebounds from a pocket back onto the playing surface is not a pocketed ball.
If the cue ball contacts an already pocketed ball, the cue ball will be considered pocketed whether it rebounds from the pocket or not.
The referee will remove pocketed object balls from full or nearly full pockets, but it is the shooter’s responsibility to see that this duty is performed.
WPA 8 ball rule 3.9 tells us that a cue ball scratch is indeed a foul in 8 ball:
3.9 Standard Fouls (8 ball)
The following are standard fouls at eight ball:
- 6.1 Cue Ball Scratch or off the Table
WPA 8 ball rule 6.14 further defines the scratch foul:
6.1 Cue Ball Scratch or off the Table (8 ball)
If the cue ball is pocketed or driven off the table, the shot is a foul. See 8.3 Ball Pocketed and 8.5 Driven off the Table.
WPA 8 ball rule 3.8 Losing the Rack tells us that it's a loss of game to commit a standard foul while pocketing the 8 ball:
3.8 Losing the Rack (8 ball)
The shooter loses if he:
- (a) pockets the eight ball and fouls.;
- (b) pockets the eight ball before his group is cleared;
- (c) pockets the eight ball in an uncalled pocket; or
- (d) drives the eight ball off the table.
These do not apply to the break shot. (See 3.3 Break Shot.)