In general, it's a very bad idea to remove or rename a field -- if you are not positive that it's one you added yourself, do not change it because it may be used for something. To remove something you do not want to use, you should simply edit it out of your templates.

If you're absolutely certain that the field isn't being used anywhere, you can remove it through your 'add fields' page by typing in its name. Removing it DOES NOT remove all of the associated HTML from your templates -- it will attempt to take out the very most obvious part, but since it's presumed that people will have customized the area it does not take the risk of guessing how much is related to the field. You will have to manually edit your templates to remove indications.

You can rename a field on your 'add fields' page as well. If you rename a field, be certain that you edit all templates to use the new name of the field. You may wish to use the 'advanced' page to do a mass replace of all instances of the old field name in all templates with the new name of the field. In doing this be sure to replace both the field itself and the associated template variable -- in you rename field1 to field2, you must mass-replace field1 with field2 AND mass replace {LINKFIELD1} with {LINKFIELD2}. Simply renaming the field DOES NOT do these changes for you automatically.