Those small hairs that cover the plant are the nasty part. These hairs are hollow, and mostly made up of silica. They are filled with a mixture of very irritating chemicals. Imagine a plant that is covered with tiny glass needles that are filled with wasp venom.
When you brush up against the plant, the sharp hairs penetrate the skin and then break, injecting the chemicals and causing a strong, burning, itching sensation. The hairs are thin and strong enough to penetrate normal garden gloves, so you have to be very careful when you remove them from your flowerbed.