Correct. As I understand it, the driver changing lanes should determine whether lane changing is possible/safe, signal accordingly, and then change lanes.
I believe it is also technically a moving violation to cross multiple lanes at once, even if signalling.
Yielding in this case is entirely a matter of courtesy. If you would have needed to slow down substantially or change lanes yourself to avoid him, you are certainly not at all obliged to allow the other car to pass.
Speeding up to prevent the driver from lane change (while staying within the speed limit) would be kind of discourteous, but also well within your right, at least to the best of my knowledge -- you do not have to "allow" a driver to change lanes if they otherwise could not do so. It is the other drier's responsibility to determine if changing lanes can be done safely or not.
If I am understanding the situation correctly: Sounds like he just plowed across a bunch of lanes both illegally and discourteously. Even worse if he failed to signal for each change. Following all that with telling you that you need to learn the "rules of the road" is like icing on the cake...
If it's any consolation, in FL its common to have someone not signal, cut you off, almost causing an accident, then if you honk the horn to warn (or in fury, which technically you are not supposed to do...), the other driver, as likely as not, will give you the finger and scream out the window.
How dare you be outraged at someone who almost killed you with absolutely no concern for anyone else on the road except him/herself. Hey, that person has got places to go, so screw everyone else, right?
The driving norms here are like something out of Mad Max
