Can be confusing for sure.
Here is how i keep the whistle signals straight:
VITAL PRINCIPLE: The signals are "self-centered", by this I mean that whoever initiates the signal is whose port or starboard is the subject.
For example, when I intend to overtake a vessel and give 2 blasts, meaning...