I agree 100% with you, and perhaps take an even stricter point of view on the subject.
I see so many folks saying it's not an issue, but this is ridiculous. If you disturb the balls as they lay on the table (whether or not you have the cue ball in your hand), that is a huge issue.
The rule you quoted is correct... It means you can't touch an object ball with the cue ball while you are placing it in the desired spot. This is true and correct, but it isn't the only rule that applies to the situation you described.
If you touch an object ball in any way (with hand, clothing etc.), it is a foul. Period.
Foul By Touching Balls (Regulation 3.21)
It is a foul to strike, touch, or in any way make contact with the cue ball in play or any object balls in play. This includes contact with anything, including the body, clothing, chalk, mechanical bridge, cue shaft, etc, except the cue tip that is attached to the cue shaft. This, and only this may contact the cue ball in the execution of a legal shot.
Whenever a referee is presiding over a billiard match, any object ball moved during a standard foul must be returned as closely as possible to its original position as judged by the referee. The incoming player does not have the option of restoration. For more details on this regulation, you can reference regulation 1.16.1 of the general rules.
Note: some players will argue that this rule only applies while a shot is literally in progress, but this is not the case. In the rule's verbiage, the term "object balls in play" means balls which are not yet pocketed.
When someone tries to argue this with you next time, point them toward this rule quoted above from the BCA General Rules of Pocket Billiards.
If this is not enough to convince them, point them at the World Pool-Billiard Association's 8-Ball Rules:
6.6 Touched Ball
It is a foul to touch, move or change the path of any object ball except by the normal ball-to-ball contacts during shots. It is a foul to touch, move or change the path of the cue ball except when it is in hand or by the normal tip-to-ball forward stroke contact of a shot. The shooter is responsible for the equipment he controls at the table, such as chalk, bridges, clothing, his hair, parts of his body, and the cue ball when it is in hand, that may be involved in such fouls. If such a foul is accidental, it is a standard foul, but if it is intentional, it is a foul under rule 6.17 - Unsportsmanlike Conduct.