Should 'track' functionality apply to 'followed' Twitters?

I was just wondering: the initial intent of Twitter was to be able to 'follow' others. You knew whose information you were interested in and you 'subscribed' to follow them. There was a one-to-one relationship between you and everyone you followed.
However, with the implementation of the 'track' command, there is no longer a deterministic relationship between you and the people you follow. In fact, when you decide to "track" something, you do not know who you are following because anyone who is registered with Twitter can post an update that includes the things you're tracking and you will get the update, even if you were not interested in following that person. So what that they posted something that you tracked. The match may be purely coincidental.
Wouldn't it be better to apply 'track' filters only to the twitters you follow?