In one word: Yes. But with more and more people raised and living in urban areas, there is a disconnect with nature. A simple rule then: if you don't know for sure, Hands Off!
Stinging caterpillars--some of the most widespread in North America are listed below--have urticating, hollow hairs containing toxins. When touched, these break off, enter the skin and release their poisons. This may result in anything from a discomforting itch to excruciating pain. Top contenders include:
Puss Caterpillars, Megalopyge opercularis
The larvae of Southern Flannel Moth is one of the most common stinging caterpillars. These may appear insidious but their soft coif--it has been compared to that of Donald Trump--covers a plethora tiny, extremely toxic spines. Upon piercing the skin, they cause extreme pain. Its extensive distribution is recognized in a wide selection of alternate names, including asp, woolly slug, Italian asp, tree asp, asp caterpillar, and opossum bug. Variable in colour it makes its habitat from eastern Canada and the United States down into Mexico and parts of Central America. It feeds on oak, elm, sycamore, wild plum, citrus, rose and ivy.
Hag Moth Caterpillar, Phobetron pithecium
The name Monkey Slug describes the larvae's weird appearance. Strictly speak, this caterpillar does not sting but the hairs may cause an allergic reaction in susceptible people. Common to North American orchards it feeds on a wide range of deciduous trees and shrubs, including ash, birch, chestnut, dogwood, hickory, oak, persimmon, walnut, and willow
Saddleback Caterpillar, Acharia stimulea
Native to eastern North America, Mexico and Central America, they feed on a wide selection of plants, including ornamentals. The venomous, urticating spines cause a painful rash;
Hickory Tussock Moth, Lophocampa caryae
Native to the eastern half of North America. The caterpillars are venomous and may cause reddening of skin, itching and even nausea if touched. They feed on hickory, pecan and walnut but will also eat ash, elm, oak, willow and other plants
Elsewhere
Foreign travel, particularly ecological vacations--can expose the unwary to a vast number of other potentially poisonous, possibly fatal insects and other arthropods. Such are rarely advertised by promoters of such trips.