Nonvisual properties, such as conductivity, specific heat, specific gravity, density, texture, magneticity, etc.
You can probably tell identical gold-plated / silver-plated mugs apart by touch, because there will be less gold, and the gold one will be lighter. If you have identical gold / silver lumps, the gold will be heavier, because gold is more dense and natural lumps don't care about saving money on production.
Two cups, one filled with water, and the other with orange juice, will feel different when grasped, if they start at the same temperature, because they will change temperatures at different rates. The same can apply to minerals: given two crystals with different thermal properties, you can tell them apart by holding one in each hand, and feeling which changes temperature fastest, and how quickly it reaches the new equillibrium.
If you're dealing with loose particles and dusts, consider how different minerals react to static electricity. The simplest demonstration of this is to charge a balloon or sheet of plastic or whathaveyou, and move it over a mix of salt and pepper. One should be lifted due to the charge difference, and the other is left behind.
The limits depend on the available technology, or if visual sorting post-collecting is an option. I would assume that cupric rocks will respond differently to an electric current than other random rocks of similar texture, but I don't know the details, having only really heard about identifying copper ores by color. The examples I gave might be difficult to distinguish without finer measurements than human-like senses can perceive, for certain minerals.