The short of it is 'yes, asteroids contain the elements needed to make concrete.'
A typical asteroid contains
Oxygen 36%
Iron 26%
Silicon 18%
Magnesium 14%
Aluminum 1.5%
Nickel 1.4%
Calcium 1.3%
And hydraulic cement is made from any of
Belite (2CaO·SiO2);
Alite (3CaO·SiO2);
Tricalcium aluminate (3CaO·Al2O3) (historically, and still occasionally, called 'celite');
Brownmillerite (4CaO·Al2O3·Fe2O3).
And there is overlap between what is needed and what is available.
The long of it is, just because the elements are there, does not mean they can be extracted and then combined into the necessary chemical form.
Cement is a binder, that binds aggregate into a solid structure. We use lime based cements and the above hydraulic cements, mainly because of their abundant availability and ease of extraction. Calcium, in fact, is widely available in the form of limestone throughout the world. But that is not to say that exotic cements could not be formulated using other elements.
The problem, I posit, will not be in the materials, but in the curing. Concrete does not dry, it cures. That is, it depends on oxygen in the atmosphere to complete the chemical reaction. Hydraulic cement depends on water. So building concrete structures in a vacuum presents a considerable challenge.
However, engineers HAVE considered using concrete as a building material on a space station using raw materials shipped up from the moon.