You can hide the Twitter Follow button only by adding the following lines to the panel.ini configuration file.
To hide the Follow button:
[twitter]
showFollowLink = false
To undo the change and return to the default values, remove these lines from panel.ini.
