There are innumerable different species of Caterpillar. So many different shapes, sizes, colours and textures that there is no such thing as ‘your average caterpillar’. And not all of them are particularly harmful - yes they will eat through fruit leaves, stems and even roots but one day the very hungry, creepy crawly before you may become a beautiful butterfly.
Treatment and control
General tips: Try picking them off the leaves (they often feed at night so a torch might come in handy) and removing them to a site where any damage will be less harmful. If this is not possible then the answer is to spray. For best results, spray at the first sign of infestation.
Spray with a natural-based insecticide: Sprays based on fatty acids and sulphur are effective. An example is BugClear Gun! for Fruit & Veg. Spray with a broad spectrum insecticide
Examples are BugClear Ultra Gun! for flowering ornamental plants (contains acetamiprid).